a study of the effects of instructional analysis and
TRANSCRIPT
University of Massachusetts Amherst University of Massachusetts Amherst
ScholarWorks@UMass Amherst ScholarWorks@UMass Amherst
Doctoral Dissertations 1896 - February 2014
1-1-1974
A study of the effects of instructional analysis and feedback on A study of the effects of instructional analysis and feedback on
the classroom behavior and student achievement of university the classroom behavior and student achievement of university
teaching assistants. teaching assistants.
Michael L. Koffman University of Massachusetts Amherst
Follow this and additional works at: https://scholarworks.umass.edu/dissertations_1
Recommended Citation Recommended Citation Koffman, Michael L., "A study of the effects of instructional analysis and feedback on the classroom behavior and student achievement of university teaching assistants." (1974). Doctoral Dissertations 1896 - February 2014. 2864. https://doi.org/10.7275/c0r1-bg06 https://scholarworks.umass.edu/dissertations_1/2864
This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Doctoral Dissertations 1896 - February 2014 by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected].
A STUDY OF THE EFFECTS OF INSTRUCTIONAL ANALYSIS AND FEEDBACK ON
THE CLASSROOM BEHAVIOR AND STUDENT ACHIEVEMENT
OF UNIVERSITY TEACHING ASSISTANTS
A Dissertation Presented
by
MICHAEL L. KOFFMAN
Submitted to the Graduate School of the
University of Massachusetts in partial
fulfillment of the requirements for the degree of
DOCTOR OF EDUCATION
February, 1974
Major Subject: Education
u*»*»
(c) Michael L. Koffman 1974
All Rights Reserved
11
A STUDY OF THE EFFECTS OF INSTRUCTIONAL ANALYSIS AND FEEDBACK ON THE
CLASSROOM BEHAVIOR AND STUDENT ACHIEVEMENT OF UNIVERSITY TEACHING ASSISTANTS
A Dissertation
by
MICHAEL Lo KOFFMAN
Approved as to style and content by:
Dr. Robert Miltz, Chairman
Dr. William Lauroesch, Member*
Dr. John Louis, Member
Dr. Patrick Sullivan, Member
£uv,ft LO Qlk-v-.
Dwight W. Allen
February, 1974
iii
I would like to dedicate this completed work to my parents,
Elizabeth and David Koffman, as a small measure
of the love and esteem and gratitude that I hold for them.
iv
ACKNOWLEDGMENTS
This study owes a great deal to many persons. First, I
would like to thank my advisor, Dr. R.J. Miltz, who introduced
me to the field of teacher education, and who provided me with
constant support and stimulation.
I would also like to thank Dr. John Louis, who initiated
and supervised many aspects of this project, and Dr. William
Lauroesch and Dr. Patrick Sullivan who contributed to my progress
in so many ways
.
This study would not have been possible without the gener-
ous assistance of the staff and directors of the Clinic to Im-
prove University Teaching project at the University of Massachu-
setts, particularly Dr. Glen Erickson, Bette Erickson and Michael
Jackson.
I would especially like to thank Ric deFreisse, formerly of
that project, for the invaluable time and expertise he devoted to
organizing and processing the data that was collected. His energy
and patience in this regard were truly outstanding.
Finally, I would like to thank the finest teacher I know, my
wife Bobbie. Her wisdom, insight and good judgment have guided
every aspect of my work.
v
A Study of the Effects of Instructional Analysis and Feedback on
the Classroom Behavior and Student Achievement
of University Teaching Assistants (1974)
Michael L. Koffman, B.A. University of Massachusetts
Directed by: Dr. Robert Miltz
The purpose of this study was to evaluate an in-service
teacher education model designed for higher education.
Basically, this model presumes that effective instruc-
tion is related to the level of performance by teachers of
certain recurrent behaviors such as asking questions, pacing,
elaborating, emphasizing, bringing closure, facilitating stu-
dent participation, creating challenge, evaluating, etc. These
behaviors are referred to as "technical skills of teaching.
They are incorporated into a questionnaire and are rated
by the instructor, his students and an instructional specialist
Through analysis of these results as well as a videotape of the
instructor's classroom teaching, certain skills are identified
as "problematic" and in need of change. The desired changes
are sought by various means including successive videotaping
and analysis, successive student evaluations, micro-teaching,
instructor discussions with students or other instructors,
viewing instructional models, referring to specific readings,
etc
.
vi
To test this model a group of 13 in-service graduate
teaching assistants were divided into an experimental group,
a quasi-experimental group and a control group. All subjects
instructed their own section of a required freshman Rhetoric
course designed to improve student writing and expression.
As a pretest the subjects completed self-evaluations,
student evaluations, a 30-minute videotape of their classroom
instruction and an essay-type student achievement test.
Members of the experimental and quasi-experimental groups
individually reviewed their data with instructional specialists.
The experimental group continued to meet with instructional
specialists who provided additional experiences designed to
facilitate changes in instruction.
After approximately eight weeks all subjects completed
a posttest consisting of another 30-minute videotape of their
classroom instruction, student evaluations (using the same
form) and student achievement tests (using a parallel form).
Changes in classroom instructional behavior were analyzed
by means of an adaptation of Flanders interaction analysis.
The results showed observable change in all three groups. The
experimental and quasi-experimental group instructors increased
their "using student ideas," "focusing, summarizing, introduc-
ing or orienting statements" and "lecturing." The percentage
of class time occupied by teacher talking increased and the per-
vii
cent occupied by the students talking decreased. The control
group instructors showed a marked increase in silence in their
posttest lessons. Their "using student ideas" increased only
slightly and their use of "focusing, summarizing, introducing
or orienting statements" and "lecturing" decreased.
In the student evaluations,the experimental group showed
positive patterns of change in the categories of "clarity,"
"evaluation and feedback" and "relating to student responses."
The quasi-experimental group regressed in one category, "re-
lating to student responses," and the control group improved
in one category, "relating to student responses."
The achievement rating indicated that the mean score of
the students in the experimental group improved by 3.33 points
The mean score of the quasi-experimental group remained un-
changed, and the mean score of the control group students
dropped 2 points.
The results tended to support the usefulness of this in-
structional improvement model. The classroom instructional
behavior, student evaluations and student achievement of the
instructors in the experimental group appeared to change in
desirable ways. The classroom behavior of the instructors
in the quasi-experimental and control groups changed less,
and their student evaluations and student achievement for the
most part did not change in desirable ways.
viii
TABLE OF CONTENTS
Page
LIST OF TABLES .
LIST OF FIGURES
Chapter
I. INTRODUCTION 1
Statement of the Problem and Hypotheses. ... 1
In-service Teacher Education 1
The Training of Higher Education Faculty in
Instruction 5
The Role of Teaching in the University .... 15
II. REVIEW OF THE LITERATURE 18
Teacher Effectiveness Research 18
The Technical Skills of Teaching 24
Micro-teaching ..... 25
Learning Principles 30
Student Evaluations 32
The Inductive - Analytical Approach 34
III. PROCEDURES 43
Treatment. 44
SubjectsSettingInitial InterviewObservationVideotape I
Student EvaluationsInterpretation and Feedback
Follow-up
Additional Data Collection
Videotape II, Student Evaluations
Student Achievement Tests
ix
TABLE OF CONTENTS (continued)
Chapter Page
Data Analysis 55
Interaction AnalysisStudent EvaluationsStudent Achievement
IV. RESULTS
Changes in Instructional Behavior: InteractionAnalysis 63
Difference MatricesComparison of Difference Matrices Along
Selected DimensionsFlow ChartsSignificance of ChangesInter-rater Reliability
Student Evaluations 94
Student Achievement 104
Correspondance Among Measures 106
V. DISCUSSION AND CONCLUSIONS 110
Purpose and Procedures 110
Summary of Results 112
Change in Instructional Behavior
Change in Student Evaluations
Change in Achievement
Conclusions 114-
Limitations
Implications for Future Research 119
Summary 121
REFERENCES 122
APPENDICES
A. COURSE INFORMATION AND PRE -VIDEOTAPING IN-
FORMATION FORMS 131
B. INTERACTION ANALYSIS MATRICES 135
C. STUDENT EVALUATIONS 160
D. STUDENT ACHIEVEMENT TESTS 2 °1
x
LIST OF TABLES
Table Page
I. Research Abilities Judged "Essential" Pre-sently and "Acquired" In Graduate School. ... 9
II. Teaching Skills Judged "Essential" Presentlyand "Acquired" in Graduate School 9
III. Teaching Assistant Survey - Maryland 10
IV. Graduate Teaching Assistants Rating of TrainingProgram 31
V. Experimental Group Difference Matrix 64
VI. Control Group Difference Matrix ........ 66
VII. Quasi-Experimental Group Difference Matrix. . . 67
VIII. Comparison of Interaction Analysis Dimensions . 69
IX. Direction of Change on Student Evaluations for
Each Sub-group 80
X. Rater Scores for One Fifteen-Minute Lesson Using
the Flanders 10-Category System 95
XI. Comparison of Student Evaluation Pretest and
Posttest Scores For Individual Items and
Factors 96
XII. Quasi-Experimental Group Student Evaluation
Items and Factors Indicating Change 99
XIII. Control Group Student Evaluation Items and Fac-
tors Indicating Change 101
XIV. Experimental Group Student Evaluation Items and
Factors Indicating Change 102
XV. Student Achievement Scores 105
XVI. Instructional Problems and Improvement Goals of
Experimental and Quasi-Experimental Groups. . . 107
XVII. Correspondance Between Improvement Goals. Behav-
ior Changes, and Student Evaluation Scores for
the Experimental Group 108
xi
LIST OF FIGURES
Figure Page
1. In-service Teacher Education Components ... 3
2. Biddle's Seven-Variable-Class Model forTeacher Effectiveness 19
3. Relationship of Teacher Indirectness to Pu-
pil Growth 38
4. Relationship of Teacher Criticism to Pupil
Growth 38
5. Student Evaluation Form of the Clinic to Im-
prove University Teaching - Spring 1973 ... 48
6. Twenty-One Technical Skills of Teaching and
Related Student Evaluation Items 52
7. Sequence of Data Collection and Treatment . . 56
8. Seventeen-Category Interaction Analysis
System 57
9. Flanders Ten-Category Interaction Analysis
System ^8
10. Student Evaluation Factors 61
11. Experimental Group Pretest Flow Chart .... 84
12. Experimental Group Posttest Flow Chart. ... 85
13. Quasi-Experimental Group Pretest Flow Chart . 87
14. Quasi-Experimental Group Posttest 1 low Chart.
15. Control Group Pretest Flow Chart 90
16. Control Group Posttest Flow Chart 92
xii
CHAPTER I
INTRODUCTION
Statement of the Problem and Hypotheses
The present study is a contribution to the field of in-
service teacher education at the higher education level. The
purpose of the study was to measure the effects of particular
instructional analysis and feedback procedures on the classroom
behavior, student evaluations and student achievement of univer-
sity teaching assistants.
This purpose may be stated in terms of the following hy-
potheses :
(1) There will be significant differences between the
pretest and posttest classroom behavior of in-
structors who have undergone the instructional
analysis and feedback procedures when measured
by changes in Flanders interaction analysis cate-
gories.
(2) Differences in instructional behavior for this
group will be accompanied by corresponding posi-
tive changes in student evaluations of the in-
structors when measured on student evaluation
questionnaires
.
(3) Differences in instructional behavior for this
group will be accompanied by positive changes
in student achievement as measured on student
achievement tests.
In-Service Teacher Education
The field of in-service teacher education can be understood
in terms of various components of instruction which in-seivice
teacher educators attempt to affect as well as the various means
These components would include the follow-by which they do so.
2
ing: instructional goals, teaching styles, specific methods,
classroom instructional behavior, classroom environment, tea-
cher personal characteristics. The "means" include group sett-
>data collection and analysis tools and teacher educator
role. These components are represented in Figure 1 in grea-
ter detail.
This study dealt narrowly with the component of specific
instructional behaviors, particularly in regard to their direct
observation, analysis and evaluation. The setting was primarily
an individual, one-to-one type, although group settings were ex-
plored. Data collection and analysis tools included student
evaluations, direct classroom observation, videotape analysis,
systematic behavior coding and instructor self-evaluation. The
role of the teacher educator was that of analyst.
At the same time this study sought to contribute to the
research on "teacher effectiveness," that is, to the body of
literature which studies the specific characteristics of in-
structional situations as they relate to student outcomes. It
would seem necessary in fact for the two fields of teaching re-
search and in-service education to be closely connected if the
in-service teacher educator is to serve in any way as a media-
tor tor between educational research and educational practice.
Such a role can be vitally important for a number of rea-
sons. First, there is a growing body of knowledge on the re-
lationship between various instructional components and student
3
CD
CJ
•Hp
p CO
QJ t—l «P a cd o p d oCJ d QJ O -Pcd o p p pa) co o O cd.
E-t P cd i pQJ p qj dCP cd 00 QJ
iC P co
o cd qj
P) pCP
x E pd o dcd o a) iP
P E cd
t-t co d dO CO o X)o cd p •pid 1—l *P >
CO l o o > •pE-i co d X)2 w dW M2o
o CD cd
o P O d CD X)d •POP QJ
2 a) <P -P o E Go d •p p -p E OM o o o > cd *p
Ep a. aj d cd p pC e ft p d 00 oO o CO P QJ oo o d2 o CD « d p Po d •H CP Pw 1—4 M p 1 CD
cd p >p dPi d QJ rP t—
t
w o CO QJ
2 , *H CO i
C_) p 00 CO
< o d xjw d •H 0H p id id
p O Pw CO cd QJ CO
u d QJ S CPM M Ep o> idP4 CD
wCO
1
PO
2 00M d co
•P QJ
id t—<
o >.cd P0) co X)
H d p CO
cd p CPo do- o
co D-. pcd d o
00 H co
d
cd dd do cd
•p i—ico
P CP QJ
o co
d x) Pp d dp cd oCD od co
M r—
•
cd
oo
OOd•pXI0CJ CD
„ EE QJ
O PO CO
P >>CO COCO
cd
r—
<
o POco
•HCO >d p
CD P o 0) CO
00 d -p a. Pd QJ P d d•rl xi cd CO QJ
X) X) d d dd p P rP ocd o co cd Cft
o > EO 0)
0) Piw
p0o
X) co
•P o p d> -p d T—< o
X) QJ cd •HCO d E QJ d pd < 0) i-4 < cd
cd > O o0) 0) Pi d2 •H X
d P wd CJ 0o d <3 p p p•H E o cd o QJ
p O -P P o p ido O P d d CJ oQJ p cd QJ X d cd
< 1 co > X w P QJ
1 1 CD P d p Ep0 cd qj P p co
o i-< CO CO QJ d QJ
CJ d id M CJ
cd o CJ •Hp cd >cd 0) po ' Ep QJ
X CO
d QJ 1
oo cd d dxi d oo Md -p d cd
cd d O P QJ •
o •p P H rHoo cd P o r—i
d qj cd cp o aj
•P Eh d qj cj p>N rP Pi dcd cd i oo
i—i P > p •H
CP o W i-t
1 p 1 QJ
qj cd 44 CO prP P HI oo o QJ CO
Pi ,d CO •p
cd >. P) X
<
CO
Pco
a)
HdQJ
PP•Hp
4
indicating that in given situations, for specific instructonal
goals, with certain populations of students, certain choices
among the type and sequence of curriculum materials, methods,
teaching approach and specific instructional behaviors will
effect those objectives more completely than other choices.
Secondly, there is evidence that instructors are capable
of manipulating their teaching in terms of these components
so as to "improve" their teaching effectiveness.
Thirdly, most classroom instructors, especially at the
higher education level either are not aware of instructional
alternatives or they do not have the support and encouragement
to investigate these in their classrooms.
Finally, the role of the in-service teacher educator gains
importance in reference to the fact that the majority of in-
structors who will be teaching at the higher education level
in the next ten years are teaching in higher education insti-
tutions at the present time.
The first two of the above propositions are the subject
of Chapter II, "Review of the Literature." The third proposi-
tion, concerning instructors' knowledge of teaching components
and instructional alternatives is the subject of the present
chapter. The last proposition, given the present structure of
tenure laws and the trend toward stabilization and even decline
in the growth rate of higher education institutions, is mathe-
matically correct.
5
The Training of Higher Education Faculty in Instruction
In a Report of the President's Commission on Higher Educa-
tion (1947) the following statement was made,
College teaching is the only major learned professionfor which there does not exist a well-defined programof preparation directed toward developing the skillswhich it is essential for the practitioner to possess(p. 1).
After reviewing the graduate programs of fifty institutions
of higher education, Heiss (1970) found that in 757o of the in-
stitutions the teaching assistantship was the primary means for
preparing future college teachers and that "fewer than half of
those who held such appointments were likely to receive adequate,
systematic and continuous guidance from a senior member of the
faculty (p. 39)." Few programs designed to prepare graduates
for teaching were "interdisciplinary" and few graduate students
took coursework in any aspect of teaching. Heiss concluded,
Most graduate faculties have operated on the assump-tion that the process of becoming a researcher re-
quires rigorous exposure to theory and practice,but the art and skill of teaching comes naturallyor develops gratuitously when one is educated for
research. Thus the emphasis in most Ph. D. pro-
grams has been heavily weighted in favor of pre-
paring students to discover knowledge and only in-
cidentally, if at all, on how to impart to others
the nature and value of that knowledge (p. 37).
Arrowsmith (1967), Chairman of the Department of Classics
at the University of Texas, makes the point in much stronger
terms
,
I am suggesting what will doubtless seem paradox
or treason: There is no necessary link between
6
scholarship and education, or between research andculture, and in actual practice scholarship is nolonger a significant educational force. Scholars,to be sure, are unprecedentedly powerful, but theirpower is professional and technocratic; as educa-tors they have been eagerly disqualifying themselvesfor more than a century, and their disqualificationis now nearly total. The scholar has disowned thestudent -- that is, the student who is not a poten-tial scholar -- and the student has reasonably re-taliated by abandoning the scholar. This, I be-lieve, is the only natural reading of what I taketo be a momentous event: the secession of the stu-dent from the institutions of higher learning onthe grounds that they no longer educate and aretherefore, in his word, irrelevant (p. 110).
Eble (1971) published the results of a two-year project
(The Project to Improve College Teaching sponsored by the
Association of American Colleges and the American Association
of University Professors) which addressed the question of pre-
paration for college teaching. His booklet was based largely
on a career development questionnaire sent to faculty members
of 142 different institutions.
To the first question, "My institution (does, does not) have
an effective faculty development system," the response was over-
whelmingly negative.
To question two, "Outside of the departmental program and
budget, my institution provides specific support for (research,
teaching, service)," about 60 percent of the respondents reported
specific support for research, 10 percent reported support for
teaching and even fewer for service. Eighteen respondents said
their institutions offered no support outside of department bud-
7
gets for research, teaching or service.
Question three involved the most common form of institu-
tional support for faculty development, sabbatical leave.
Sixty percent of institutions in the survey conducted some
form of sabbatical leave,but there was much discontent in
terms of the nature of the leave and the selection procedures.
Fewer than one-fourth of these schools awarded leaves for im-
proving one's teaching, a research proposal, or a proposal
for general professional development.
In two other questions concerning specific programs for
the preparation of college teachers in graduate programs, over
half of the respondents reported no specific preparation pro-
gram aside from departmental course work, and none of insti-
tutions that provided some teacher preparation procedures had
a comprehensive program (pp. 3-4).
Other Commission and private studies, particularly those
conducted within the various disciplines (Miller and Wilson,
1963; Nowlis, Clark and Rock, 1968; Many, £t. a_l., 1969; NCTE
,
1970; Mackenzie, 1970; Dean, n.d.) support the conclusion in-
dicated above that not only are college faculty members not
formally trained in instruction, but future college faculty
are not being trained.
This situation is exacerbated at the junior college level
where the role of the instructor centers even more on class-
room instruction. A report submitted to the President and
8
Congress by the National Advisory Council on Education Pro-
fessions Development (1972) stated that instructors at the
junior and community colleges "are inadequate" and "do not
know how to teach" and that teacher education programs are
doing little to help them. The Council recommended in-
service education programs to bring the instruction into
line with the colleges' objectives.
Junior college administrators recently have called for
teacher preparation programs that would aid new instructors
in gaining a broader picture of curriculum and the needs of
students than that usually afforded in a graduate school
"subject-matter-centered" sequence (American Association of
Junior Colleges, 1969). Others (Cohen, 1970) went so far as
to request separate training institutions in which junior
college instructors might be prepared outside the "contami-
nating influence" of the research-dominated universities.
Surveys of faculty members,graduate students and ad-
ministrators also support the conclusion that college in-
structors are insufficiently trained in instruction.
A survey of 1700 University of Minnesota Ph. D.'s gra-
duated in the periods 1935 to 1949 and 1954 to 1956 (Alcia-
tore and Eckart, 1968) concerning the strengths and weaknesses
of their career preparation are summarized in the following
two tables.
9
TABLE I
RESEARCH ABILITIES JUDGED "ESSENTIAL" PRESENTLY
AND "ACQUIRED" IN GRADUATE SCHOOL
Abilities Related to ResearchPer CentEssential
Per CentAcquired
1. Familiarity with research materials andmethods 76.3 88.0
2. Skill and practice in doing research 62.3 86.2
3. Ability to do research, produce artis-tic creations 59.2 81.3
4. Ability to supervise research programs 46.5 41.5
5. Ability to use a wide range of libraryresources 43.6 70.7
TABLE II
TEACHING SKILLS JUDGED "ESSENTIAL"PRESENTLY
AND "ACQUIRED" IN GRADUATE SCHOOL
Abilities Related to TeachingPer CentEssential
Per CentAcquired
1. Skill in lecturing 72.9 27.7
2. Ability to outline objectives and or-ganize courses 71.7 40.0
3. Ability to teach or train others 60.7 35.4
4. Skill in handling discussion 60.5 30.4
5. Skill in planning for effective use of
limited time 58.9 27.0
6. Ability to direct students in use of
library 29.7 42.4
7. Skill in making and using illustrativedevices 20.8 22.8
10
The discrepancy between training perceived as "essential" and
training judged to have been "acquired" in the domain of teach-
ing far outweigh that of the research domain in these results.
In the opinion of numerous college presidents, the know-
ledge of undergraduate education among students in their Ph. D.
programs was much more problematic than the students' preoccu-
pation with research or their specialization in a field (McGrath,
1961).
University teaching assistants in physics at the Univer-
sity of Maryland (Triezenberg,1969) felt that learning to
write course objectives, learning teaching techniques, lec-
turing to classes and being observed by a faculty member were
most lacking in their present graduate program. The results
of Triezenberg ' s survey are represented in Table III.
TABLE III
TEACHING ASSISTANT SURVEY - MARYLAND
General Area of ConcernWas Should be
Part of Experience
Pre-semester orientation covering:
Course objectivesTeaching techniques
Range of teaching experience including:
Discussion of course evaluation
Participation in exam design
Lecture to lab./rec.
Lecture to whole class
Supervision and evaluation by means of:
Regular meetings with faculty
Clearly specified responsibilities
Observation of T.A. by faculty
17 73
6 74
34 88
48 81
64 85
16 55
67 87
65 82
12 77
11
TABLE III (continued)
TEACHING ASSISTANT SURVEY - MARYLAND
Was Should beGeneral Area of Concern Part of Experience
Formal course or seminar:In physics department 17 70In general education 5 27
Organized Instructional Improvement Programs in Higher Education
The response to these criticisms of college instructor pre-
paration has been an increase in both pre-service and in-service
training programs. A variety of such programs are now emerging.
The results of Heiss's study for example were presented in a
keynote address to a conference of the Commission of College Phy-
sics held at the University of Washington (West, et a_l. ,
1970),
where the conference goal was the establishment of instructional
training programs for future college physicists.
Fifty-three physicists from twenty-four institutions and
a large number of graduate students in Physics agreed upon a
model teacher preparation program including: "careful attention
to selection, orientation, training, supervision and evaluation,"
"an apprenticeship program in which the variety and depth of the
research apprenticeship is modeled for teaching and instruction,"
and "an internship program where theory becomes practice in an
atmosphere where feedback is quick and useful (p. ii)." The con-
ference members also advocated a Seminar on College Physics Tea-
ching and constructed a model outline for such a seminar.
12
The Danforth Foundation since 1964 has promoted efforts to
train college instructors via systematic teaching assistantship
programs (Danforth, 1970). One such model began in 1967 at the
University of Michigan's College Teacher Training Program, co-
ordinated through the Center for Research on Learning and Tea-
ching at that University (Koen and Ericksen, 1967). This ap-
proach moves "teaching fellows" through three sequential stages:
first the "apprenticeship" where they participate in workshops,
discussion sessions and limited apprentice teaching with feed-
back; second, teaching "assistant" in their own course or sec-
tion, still with guidance; third, "instructor" assigned to work
with two to four of the stage one trainees.
The program co-ordinator concluded that the stage three in-
structors "can provide adequate supervision." Furthermore, be-
cause the Instructors were committed to teaching they were will-
ing to devote special time and effort to helping the trainees;
they also posed "less of a threat" to the trainees than would a
faculty member (Koen and Ericksen, 1967).
Other institutions have altered the type of degree that
graduate students oriented toward college teaching might re-
ceive. At Washington University in St. Louis, a parallel de-
gree program has been established (Centra, 1972b). Over 100
graduate institutions have established some form of the Doc-
tor of Arts in College Teaching degree. This usually involves
a master's degree in a subject area, three or four education
13
courses, supervised practice teaching and a special educatonal
project (National Faculty Association, 1968).
In terms of in-service teacher education a variety of
aPProaches are also in use. Golden West College has insti-
tuted a kind of "contract" system where individual instructors
specify their teaching goals and agree upon means of measuring
these in conjunction with their department chairman, academic
dean and perhaps an instructional specialist (Cohen and Brawer
,
1969; Cohen and Shawl, 1970).
Some institutions have emphasized self-initiated faculty
projects in teaching and teaching improvement by funding fa-
culty proposals (Mathis, 1972). Programs, materials and pro-
cedures have been organized for faculty self-evaluation and
self-improvement (Mescon, et al., 1969; Boulding, 1970).
Self-improvement programs are often bolstered by student
evaluations of the faculty member, sometimes conducted by in-
dividual departments or university-wide agencies. One such
program differentiates three levels of student evaluation ques-
tionnaires, each one more specific about the faculty member's
instruction. This allows the faculty member to pinpoint the
unsuccessful aspects of his instruction and to initiate the
appropriate changes (Smock and Crooks, 1973).
Several programs depend upon inter-faculty conferencing,
especially classroom visits between beginning and experienced
faculty (Diekhoff, 1960; Gustad, 1963; Hadgkinson, 1972). At
14
least one program, the Environment for Teaching program at
Stanford's Center for Research and Development in Teaching,
is investigating the link between the "political decision-
making" activities in institutions and teaching, that is,
whether there is any connection between instructors' ability
to participate in the school's overall policy-making, job
satisfaction and teaching performance (Baldridge, 1970).
The most marked trend in in-service education, however,
has been the development of resource and improvement centers
established within and co-ordinated by universities. Typi-
cally these are staffed with experts on multiple aspects of
instruction and the various means of effecting instructional
change. Thus they can instruct faculty or departments in
curriculum design, teaching methods, use of media, etc., as
well as directly observe and critique classroom teaching.
In short, they serve a combined role of resource agent, super-
visor and researcher.
It must be concluded nevertheless that most of these
programs are "shooting in the dark," because so little is
actually known about the effectiveness of instruction. As
indicated in Chapter II, numerous in-service education tech-
niques exist which appear to alter the way in which faculty
members perform, but the vast majority of these do not mea-
sure student outcomes as a criterion of instructional im-
provement . (Burkhart ,1969).
15
The Role of Teaching in the University
The increasing number of improvement centers and the
apparent interest of faculty members, students and adminis-
trators in college and university instruction is probably a
productive trend. Certain qualifications regarding this trend
must be noted however.
One is that although many faculty have felt they were
not trained sufficiently in instruction, they presently feel
that they have gained adequate mastery of instructional tech-
nique through experience. Thus a survey of 1,085 faculty
members at six institutions (Gaff and Wilson, 1971) revealed
that ninety percent considered teaching to be a "major source
of satisfaction in their lives" and all but a few said that
their students viewed them as effective teachers.
Although studies where faculty estimates of their teach-
ing ability are compared to student estimates show that most
faculty overestimate their teaching (Centra, 1972a), the lack
of sophistication, use and credibility of measures of teacher
effectiveness have not made this apparent. It is safe to
say that most university faculty members at present have a
complacent attitude toward instructional improvement programs
This attitude is further supported by the traditional
reward system at most institutions of higher education, where
research productivity is promoted over teaching effectiveness
According to Eble (1971), "Within the profession at large,
16
the forces which work against undergraduate teaching are pro-
bably as great as those which work for it (p. 2)."
The nature of college and university education is also
unique in this regard. The very structure of a residence,
’•campus" college with light class loads for faculty and myriad
educational activities suggests that "a great deal, if not
the greater part 01 the teaching-learning function is real-
ized outside the classroom (Trent and Cohen, 1973, p. 1040)."
In fact, the real impact of university instructors may
be related to such "incidental" or "hidden" factors as their
personalities, life-styles and intellectual habits which stu-
dents "model." Real learning at college may also be due to
the open, "rich," supportive environment which is usually gen-
erated at the college level by faculty individual contact with
students and the non-monitoring of student activities. Au-
thors from Dewey (1939) to Rogers (1969) have advocated such
an environment for fostering such positive outcomes as stu-
dent independence, self-initiation and creativity. Only re-
cently have researchers inquired into this overall environ-
mental impact (Trent, 1970).
The question of instructional improvement for universi-
ty faculty also involves certain philosophical issues. It
is probably true that the United States has undergone a change
in values during the last two decades. The question of whether
the university should adapt itself to this change or attempt
17
to temper the change with traditional academic values is an
important one. Nuthall and Snook (1973) make a strong ar-
gument for retention of traditional academic disciplines
because they teach a unique way of thinking and knowing. The
scholar-teacher role which is largely based on information
transferral is certainly valid if transmitting new know-
ledge to students is an important activity. The rigor of
both the academic disciplines and the knowledge generator
role can be undermined by hasty adoption of such concepts as
"student-centeredness , " "relevance , " "teaching-counseling ,
"
or "individualization."
The real facts are that little is known about the value
of traditional instructional techniques or recently-developed
instructional techniques. Sanford’s observation in 1962 was
reasserted a decade later by the authors of the most recent
review of the literature on research in higher education (Trent
and Cohen, 1973),
The colleges will change only when more knowledge
of what they do and of what they might do has been
produced and made available to educators. The
need for theory is apparent but what is more strik-
ing is the paucity of empirical studies (p. 1012).
18
CHAPTER II
REVIEW OF THE LITERATURE
Teacher Effectiveness Research
The field of research in education which might be des-
cribed as ’’teacher effectiveness" research generally follows
a paradigm expressed in the following question: "What speci-
fic characteristics of instruction result in what learning
outcomes for what kinds of students in what kinds of educa-
tional contexts (Gage, 1963; Biddle, 1964; Meux, 1967; Cen-
tra, 1972)?" As an hypothesis this question becomes: "The
quality of learning which transpires in a given instruction-
al situation is a function of particular instructional pro-
cedures employed by a particular instructor for particular
students with particular goals in mind (Popham, 1967, p. 2)."
Biddle (1964) represented the teacher effectiveness
question in a seven-variable model depicted in Figure 2. The
import of Biddle's model is that many variables operate in a
given teaching/learning situation, and that reliable state-
ments of teacher effectiveness (statements of cause-and-ef fect
relationships between teaching factors and learning factors)
must control for these variables.
Certain instructors are effective in certain situations
more than in others. To use Biddle's example, "Some teachers
may be inspirational leaders, others warm counselors, and still
BIDDLE'S
SEVEN
-VARIABLE
-CLASS
MODEL
FOR
TEACHER
EFFECTIVENESS
19
v
• c O -° w o w u
Figure
2.
Biddle's
Seven-Variable-Class
Model
for
Teacher
Effectiveness
20
others walking encyclopedias. In certain contexts, each of
these competencies may be highly effective; in others each
might have little or negative effect (p. 19)."
The failure of researchers to grasp the complexity of
teaching/ learning situations is largely at the root of reams
of contradictory research findings. A good example is re-
search in the effectiveness of various curriculum systems
as cited by Rosenshine (1970):
The lack of information on classroom interactionhinders evaluation of a single curriculum ordifferent curricula because without this infor-mation one tends to assume that all classroomsusing the same curriculum materials constitutea homogeneous 'treatment variable.' Such an
assumption is questionable because teachersmay vary widely in what activities they selectand how they implement them. In studies whereteacher behavior in special curricula was com-
pared with the behavior of teachers in 'tra-
ditional instruction. . .there was often signi-
ficant variation in the behavior of teachers
within each group. Although the number of class-
rooms observed in these studies is small, the re-
sults are consistent enough to cause serious doubts
about whether all classrooms using the same curri-
culum constitute a single treatment variable (p. 280)
.
Other studies that have suffered from a too simplified
view of instruction were those seeking the universal quali-
ties of "good teachers." For the most part these studies were
concerned with the personal and social characteristics of tea-
chers rather than their patterns of behavior in the classroom.
The results of these investigations were negligible (Biddle,
1964). In other words, no personal characteristics of teachers
21
or general abilities were identified which could predict suc-
cessful learning outcomes. Around 1960, according to Biddle,
many educational researchers abandoned the field of competence
research as "a simple-minded approach to a vastly more complex
topic: the study of classroom interaction (p. vi)."
Ryans' study (1960) was probably a major influence on the
directions that teacher effectiveness research has taken. Ryans
focused on "teacher characteristics" meaning both "teacher pro-
perties" and "teacher behavior." His sample included approxi-
mately 6000 teachers in 1700 schools across 450 school systems,
and his data was collected over a period of ten years (1950 -
1960).
Ryans rated three dimensions of teacher behavior via di-
rect observation: "warm, understanding, friendly versus aloof,
egocentric, restricted," "responsible, businesslike, systema-
tic versus evading, unplanned, slipshod," and "stimulating,
imaginative versus dull, routine (Ryans, 1964, p. 76)."
Seven teacher properties were considered: "favorable
versus unfavorable opinions of pupils," "favorable versus un-
favorable opinions of democratic classroom procedures," "fa-
vorable versus unfavorable opinions of administrative and other
school personnel," "learning-centered" versus "child-centered"
educational viewpoints, "verbal understanding," "emotional
stability" and "validity of response."
Four student dimensions were also rated: "apathetic-
alert," "obstructive-responsible," "uncertain-confident" and
22
"dependent-initiating." The seven teacher properties were
measured by a self-report inventory consisting of 300 multi-
ple-choice and check— list items relating to personal prefer-
ences,self- judgments
,activities frequently engaged in, bio-
graphical data, etc.
The study moreover did not preconceive the teacher be-
haviors which were rated, rather a year and a half was spent
directly observing classrooms, taking down observational data
and organizing these into rater dimensions. The result was a
document entitled the Classroom Observation Record,
Finally, great care was given to the training of the
raters, including retraining at various intervals to insure
consistency. Raters were advised to focus their attention
sharply "on the specified behaviors or characteristics to be
assessed (Ryans, 1964, p. 13)."
Ryans results were definitely not clear-cut and in some
instances could be considered disappointing. For example,
in the elementary school pupil behavior showTed a positive
correlation with such teacher behaviors and characteristics
as "friendly," "systematic-businesslike," "favorable atti-
tude toward pupils," "favorable attitude toward democratic
classroom procedures." Student behavior appeared to be un-
related to teacher characteristics in secondary school.
The findings that participation in schoollike activi-
ties during childhood and adolescence were significantly re-
23
lated to positive scores on a majority of the ten teacher rat-
ing scales or that teachers in large schools scored higher than
teachers from small schools or that teachers 55 years and above
scored less well than younger instructors on all dimensions ex-
cept "systematic and businesslike classroom behavior" and "learn-
ing-centered viewpoint" may be considered somewhat trivial. The
results did not have much application to actual educational pro-
blems and their solutions.
Ryans study however pointed in many directions. He did em-
ploy direct classroom observation. He arrived at his observa-
tional categories inductively. He included a large number of
variables with careful methodology; and he did produce certain
correlations. He concluded,
We are beginning to pull together factual informa-tion about teacher behavior that permits, for thefirst time, useful descriptions of the dimensionsof teacher behavior and their interrelationshipsand interactions. This should be of significantusefulness to teacher education and to supervi-sion in the schools--a taxonomy and analysis thatmake accurate description possible and that shouldlead to more definitive and operational methodsfor 'acquiring teacher behavior' both in the tea-
cher education program and on-the-job supervision(Ryans, 1964, p. 96).
Programs and research in instructional improvement at
the present time may be classified to some extent in two cate-
gories: (1) those who define, refine and apply a list of tea-
cher competencies more or less generalizeable across different
classroom contexts--a deductive approach, and (2) those who
24
attempt to gather factual information about teacher behavior and
educational process in classrooms in a very descriptive-analyti-
cal manner an inductive approach. Teacher educators who pro-
ceed deductively with a list of teacher competencies perhaps
have more practical mobility. Those who proceed inductively
are perhaps more likely to identify critical instructional var-
iables. The two approaches nevertheless are symbiotic, as Biddle
points out:
The researcher ignores the practical problems of com-petence creation and evaluation at his peril. It isunlikely, however, that significant advances will bemade in understanding teacher competence without aclearer picture of teacher behavior and its effects(p. 20).
The Technical Skills of Teaching
The "technical skills" approach may be considered a com-
promise between high inference "traits-rating" approaches and
low inference "behaviors-ef fects" approaches. The difference
is that in the case of technical skills, instead of a trait such
as "warmth," there is a behavior such as "introduces a new idea."
These behaviors are rated however in terms of an ideal model of
their performance. The discrepancy between the instructor's
actual performance of given behaviors and the ideal performance
creates the basis for instructional change.
According to Gage (1967) technical skills are "specific
instructional techniques and procedures that a teacher may use
in the classroom (p. 607)." The choice of a particular set of
technical skills to rate is somewhat arbitrary. What is im-
25
portant according to Gage is "the attempt to analyze teaching
into limited, well-defined components that can be taught,
practiced, evaluated, predicted, controlled and understood
in a way that has proven to be altogether impossible for tea-
ching viewed in the larger chunks that occur over a period of
an hour, a day, a week or a year (p. 602).
A typical list of technical skills includes the following:
set induction," "probing questioning," "higher order question-
ing," "divergent questioning," "reinforcement," "stimulus varia-
tion," "use of examples," "planned repetition," "nonverbal cues,
"completeness of communication" and "lecturing (Miltz, 1973)."
Micro-Teaching
The technical skills approach was particularly appropriate
to laboratory settings and pre-service teacher training in that
trainees could concentrate on mastering each skill separately,
could view videotapes of teaching models, practice the skill be-
fore a small group of students and compare their performance (al
so videotaped) to that of the model (Allen and Fortune, 1967).
This format was also extremely useful for research as Gage ex-
plains :
Rather than seek criteria for the overall effective-ness of teachers in the many, varied facets of theirroles, we may have better success with criteria of
effectiveness in small, specifically defined aspectsof the teacher’s role. A sufficient number of lawsapplying to relatively pure aspects of the role, if
such laws could be developed, might eventually becombined. .. to account for the actual behavior and
26
effectiveness of teachers with pupils under gen-uine classroom conditions (p. 95).
Most of the subsequent studies using the technical skills
and laboratory approach took place at Stanford University under
the direction of N. L. Gage, Dwight Allen, Robert Bush and Fre-
derick McDonald.
According to McDonald (1973) the first experiment using
micro-teaching was performed by Aubertine (1964) on the subject
of "set induction." Since that time several studies have been
reported: "instructional set" and "cognitive closure" (Fortune,
1965), "cuing" (Unruh, 1967), "reinforcement" (McDonald and
Allen, 1.967), "explaining skills" (Rosenshine 1968, Miltz 1971),
"verbal behavior" (Davis and Smoot, 1969), "modeling" (Orme,
1966; Berliner, 1967; Koran, 1970), "vocabulary, movement in
the classroom, inclusion of content, use of questions, degree
of control and encouragement of student participation" (Beard
,
1969), "listening" and "response" (McKnight, 1970), "attending
behavior" (Salzburg, et al . , 1971; Ramp, et al., 1971), "plann-
ing" (Waimon, Bell and Ramseyer, 1971), "variability in ex-
planatory mode" (Knight, 1971), "evaluating skills" (Legge
and Asper,1972)
.
Although many of these studies attempt to corroborate hy-
potheses about the relationship of these skills to teacher ef-
fectiveness (Allen and Fortune, 1967), the vast majority do
not use student gains as the effectiveness criterion (Rosen-
27
shine, 1973). Furthermore, most of these studies occur singly
rather than in the systematic research designs envisioned by
Gage. Thus, although the general effects of a skill may be
reported with significant results, the experimental manipula-
tion of various sub—aspects of the skill or other situation
variables in general has not taken place. It must be con-
cluded in such a case that a true cause-effect relationship
between the teaching skill and student learning was not deter-
mined.
Two instances of such systematic research have occurred.
The first attempted to determine correlations between dimen-
sions of "explaining skills" and student comprehension (Bel-
gard, Rosenshine and Gage, 1971; Unruh, 1971; Rosenshine,
1971; Dell and Hiller, 1971). Although correlations were
found, the need for numerous other studies manipulating the
variables in systematic ways became apparent:
The conclusions based on correlational evidence...are interesting to examine. Student ratings ofdimensions of the teacher's performance corre-lated with how much the students learned; inaddition, students' self-reported attention to
the lesson also correlated with what they learn-ed. What immediately comes to mind as a resultof these data is the possibility of artifactualor superstitious behavior on the part of the
student. What may be happening is that the stu-
dent observes certain activities in the teacherthat alert the student to paying careful atten-
tion to what is being said. It may be that such
alerting or attention-producing stimuli are sig-
nificant variables in causing the student to
learn--perhaps even more significant than the
form and content factors of the presentation
28
which are supposedly related to good explanatoryexposition (Glaser, 1971, p. 221).
Rosenshine (1973) noted that a second series conducted
at Canterbury University, New Zealand (Wright and Nuthall,
1970; Highes, 1971; and Church, 1971) although not conclusive
have at least achieved a promising research methodology:
In each study the experimenters taught almost iden-tical lessons to existing classes, modifying thelessons only to introduce the experimental varia-tions, and monitoring the tape recording of thelessons to insure high implementation of the treat-ment and fidelity to the content (p. 124).
Often "technical skills" and micro-teaching studies in-
vestigate the laboratory as a teacher training medium, as
opposed to researching the effects of particular skills. The
results of such studies are that in general teachers trained
in the laboratory have exhibited "significant gains" in tea-
ching skills acquisition during successive teaching sessions.
Moreover they have retained these behaviors in their regular
classroom teaching or internship (Allen and Fortune, 1967;
Fortune, Cooper and Allen, 1967; Davis and Smoot, 1969; Lim-
bucher,1969; Borg, Kelley, Langer and Gall, 1970).
One researcher feels that much of what has been written
about micro-teaching is misleading and that "claims for its
effectiveness have very little substance in fact (McDonald,
1973)." McDonald’s point is that although such issues as the
length of the micro-teaching session, the effects of practice
and the transfer of training to subsequent performance are
29
important, the most important research should concern the appli-
cation of behavior modification principles to shaping student
learning; "At the present time only a relatively small number
of student behaviors can be brought under behavioral control by
applying these principles (p. 2)." "Behavior modification sys-
tems have not been developed for teaching complex behaviors
such as problem solving (p. 3)."
At least two exploratory studies have been reported where
"technical skills" and "microteaching" were used effectively
with higher education faculty and graduate teaching assistants
in an in-service education program. In the department of
General Engineering at the University of Illinois (Perlberg
and O' Bryant, 1970) twelve faculty members were videotaped in
their classrooms and discussed their instruction with an in-
structional specialist. At the same time four graduate teach-
ing assistants participated as a group in a micro-teaching
laboratory. According to the authors, "Live observation in
the classroom and discussion between researcher and students
showed a visible change in style of teaching (p. 741)."
Sixteen faculty members in the Dental Division at Tel
Aviv University (Perlberg, £t _al. , 1972) participated in a
series of microteaching sessions with the result that several
participants volunteered to continue the videotaping and in-
structional analysis process in their regular classrooms.
30
Learning Principles
Lawrence Alexander and Robert Davis have investigated
an in-service education approach at the higher education level
which is similar to the technical skills approach but which has
its roots in educational psychology and systems design.
One of their initial studies (Alexander and Davis, 1970)
involved a ten-week in-service education project with thirty
graduate teaching assistants. The instructional concepts and
skills forming the core of the program fell into five cate-
gories: organizing subject matter, classroom management, es-
tablishing a facilitative class atmosphere, evaluating stu-
dent learning or one's own instruction, and providing a model
of professional behavior. Specific behaviors within these
categories included writing and sequencing learning objec-
tives, presenting advanced organizers in class, assessing
student entry skills, asking questions, avoiding threaten-
ing practices, reinforcing participative behavior, giving
recognition to original and thoughtful contributions and
using personal illustrations of problem solving.
The teaching assistants were given a list of these be-
haviors and asked to check those which they thought might be
problematic in their classes. They then videotaped a class
and selected short segments of the tape which were illustra-
tive of these problems. These were shown to their fellow
teaching assistants. The subsequent discussion and problem-
31
solving session was controlled and moderated by the person
whose tape was being shown. During the next week the teach-
ing assistants would attempt to modify their classroom behavior
according to the recommendations that had been made. These were
videotaped and the process repeated itself. Student evalua-
tions and student debriefing about the class were also used
as data collection devices in this model.
A questionnaire asking the participants to evaluate the
various aspects of the program is replicated in Table IV.
TABLE IV
GRADUATE TEACHING ASSISTANTS RATING OF TRAINING PROGRAM
Percent Responding
SA & A N D &
I learned some new teaching skills. 94 6 0
I am more favorably disposed to
teaching as a result of the program. 71 18 11
I could have put the same amount of
time to better use in preparing for
my profession. 18 6 76
Debriefings with peers should beeliminated. 0 0 100
Videotape is a valuable and usefulfeature. 100 0 0
According to the authors,
When they entered the program, most GTA’s were de-
ficient in three general areas related to teaching
competence: command of the subject matter, knowledge
of psychological principles and instructional tech-
nology, and inter-personal sensitivity and communi-
cation skills.
As they began to acquire background concepts and princi-ples... their confidence increased and they were willingto experiment further (p.25).
32
Student Evaluations
Student evaluations of instructors are commonplace.
After much research, the general indication is that such eval-
uations do rate instructors accurately on dimensions important
to student learning (Costin and Greenough, 1971; Frey, 1973).
Systematic use of student questionnaires to improve in-
struction in the manner described by Smock and Crooks (1973)
in Chapter I is rare. Centra (1972b) has demonstrated the
value of such an approach in a two-semester study at five
different types of colleges. In his study a detailed ques-
tionnaire containing items that had been ascertained to con-
tain information which faculty members desired about their
classes, was administered to three groups. The first group
administered the questionnaire and received the results at
mid-semester. The second group administered the questionnaire
at mid-semester but did .not receive the results until the end
of the semester. A third group as well as the first two groups
administered the questionnaire at the end of the semester.
Findings were that no differences occurred between the "ex-
perimental" and "control" groups. Across both groups however
teachers who rated themselves more favorably than their stu-
dents showed the greatest change. This process repeated dur-
ing the second semester resulted in the feedback group scoring
better than a second control group of teachers on several items.
33
The author concluded, "Given enough time ... student ratings did
result in some modest instructional changes for a wide range
of teachers (p. 22)."
The results of student evaluations of teacher effective-
ness however are very similar to the "general abilities" and
"personal qualities" paradigm of early teacher effectiveness
research. A review of this literature (Centra 1972a) indicates
that students learn more from instructors who, according to the
students, "are organized in their lessons, give clear explana-
tions, stimulate their intellectual curiosity, give interesting
presentations of course material, are attentive to students'
reactions, are friendly and are flexible (p. 22)." It is doubt-
ful how much real value this information has for teacher im-
provement.
It is conceivable that extensive use of student queston-
naires in the same instructional situation over a period of time
will identify critical and useful characteristics of effective
teaching in that situation. In a study by Frey (1973) using a
sample of eight instructors of Introductory Calculus and five
instructors of Multidimensional Calculus at the university level,
the profile of the most effective instructor (validated by stu-
dent achievement criteria) was the one who (a) made the stu-
dents "work hard" and "spend a lot of time" in the course, (b)
presented the material well in terms of communicating ideas in
an "unambiguous manner," using examples and illustrations, and
34
(c) was seen as slightly "inaccessible" or impersonal (busi-
nesslike) by students. Instructors who in the judgment of
students made them feel they were learning a great deal, who
o^S'^ized the course in a very logical and orderly way, or
who graded in a fair and accurate manner were not as effec-
tive.
It is questionable in another situation for example a
humanities course with ten students, that the above pro-
file would predict effectiveness. Gump's research in the
effects of classroom environments and "role expectations"
supports this conclusion. The population of students in
calculus might expect mathematics courses to entail hard
work, presentational mode and businesslike manner, without
which factors an instructor's credibility might be damaged,
student attention lag and learning diminish.
If principles of effective teaching which can be used
as a guideline by instructors are to be generated through
student evaluations,then it would appear that this must al-
so be done in specifically defined educational contexts.
The Inductive-Analytical Approach
The inductive approach to the analysis of instruction
also has a broad literature. Rosenshine's (1973) review of
direct observational instruments estimates the number of
systems to be in excess of 120.
35
The inductive approach assumes that the correlates of
teacher effectiveness are not known . The role of the in-
service teacher educator or researcher then is to gather as
much information (data) as possible on what is actually occur
ring in an instructional situation. Only after careful des-
cription does analysis of the data take place, and any judg-
ment of the effectiveness or ineffectiveness of behavior pat-
terns are made in reference to the specific context, espe-
cially to the instructors goals (Cogan, 1973). The approach
is very similar to field observation and recording in anthro-
pology.
The observational instruments or category systems men-
tioned above are used as checklists both to facilitate re-
cording and to aid the observor in focusing on specific as-
pects of the teaching/learning situation. Many early re-
searchers (Anderson 1939; Lewin, et al., 1939; Withall, 1949)
were interested in the social-emotional climate in the class-
room. Bales and Strodtbeck (1951) studied "phases in group
problem-solving." Meux and Smith (1964) focused on the logic
of teacher-student and student-student interchanges. Systems
have been developed to record and analyze data on such sub-
jects as group size, speaker and location of speaker in the
room; cognitive and/or affective levels of instruction; tea-
cher and student "roles"; content topics and sub-topics; con-
tent or process orientation; nonverbal behavior (Rosenshine 1973).
36
Such coding of classroom phenomena may take place in the
form of absolute counts of instructional behaviors, sequencies
of behavior or sequences noted in regular time intervals. The
advantage of the last process is that it provides the researcher
and the teacher with a fairly accurate estimate of the amount of
class time occupied by various activities.
The use of the inductive approach in terms of coding systems
has had promising results. Eighteen studies reported by Flanders
(1971) indicated that in all cases attention to their teaching
behavior using category systems as well as practice and feed-
back allowed teachers to incorporate desired behaviors into their
instruction.
Eleven of the studies supported the proposition that by
learning how to code instruction by means of interaction analy-
sis categories and by interpreting their own teaching as well as
the teaching of another person, instructors tend to become more
"responsive" to student ideas. Although the results were less
clear, there was a concomitant tendency for teachers to become
more "flexible," that is, to consciously alter their teaching
mode from "direct" to "indirect" depending on the context.
Three of the studies wherein the Minnesota Teacher Atti-
tude Inventory was administered, indicated that the attitudes
of college students toward teaching and programs for the pre-
paration of teachers became more positive among those who
studied interaction analysis.
37
In six other studies conducted by Flanders, all with pre-
high school populations,
it was apparent that more student-
centered modes ("when students have opportunities to express
their ideas, and when these ideas are incorporated into the
learning activities") predicted increased student learning
and more positive student attitudes toward the teacher and
the learning activities (Flanders, 1970, p. 401).
The importance of "flexibility" is indicated by Soar (1968)
who studied 54 elementary classrooms in an effort to identify
the optimum relationships between pupil initiation and teacher
initiation with regard to various cognitive tasks. The results
of Soar's study for tasks requiring creative thinking (Curve A),
very little abstract reasoning (Curve B) and pronounced abstract
reasoning (Curve C) are depicted in Figure 3.
These data indicate that the optimum point of indirectness
for the creative task was not even reached in these classrooms,
and that it was reached sooner for the less abstract task than
for the more abstract task. In other words the data suggest
that instructional tasks of a lower cognitive order are opti-
mally taught in an instructional mode characterized by more
direct teacher behaviors and initiation, and that instructonal
tasks of a "higher" cognitive order are best taught via less
teacher initiation and more teacher indirect behaviors.
Figure 4 represents these results in relationship to "tea-
cher criticism." Here the indication is that direct teacher
38
RELATIONSHIP OF TEACHER INDIRECTNESS TO PUPIL GROWTH
n
I
a3CL
Low] [ ! ;
Teacher indirectness index High
Figure 3. Relationship of Teacher Indirectness to Pupil Growth
RELATIONSHIP OF TEACHER CRITICISM TO PUPIL GROWTH
Lqvv Teach** cwticufw rndex Hrgh
Figure 4. Relationship of Teacher Criticism to Pupil Growth
39
behavior in terms of teacher criticism contributes to pupil
growth in both the creative and less abstract tasks--not how-
ever to the more abstract tasks. In regard to the former,
continued critical teacher behavior tends to be productive
for the less abstract tasks past the point where such behavior
ceased being productive for the creative tasks. The teacher
who does not adjust the extent of his criticizing to the
particular task at hand reduces his effectiveness.
The present study incorporates the methodology of many
of the studies mentioned above, specifically the "technical
skills" approach, videotape feedback, instructional specialists
and student evaluations. These were incorporated into an in-
service teacher education model at the higher education level
by Allen and Melnick (1972). This study is the third in a
potential series of studies testing the effectiveness of that
model
.
Melnick (1972) completed the pilot study for the model at
the University of Massachusetts with a sample of seven faculty
members from the department of Computer Science. His complete
model involved eight sequential steps. First was an initial
interview to bring out the faculty member's needs and expecta-
tions in reference to instructional improvement. Second was
a video recording of thirty minutes of the faculty member's
classroom teaching. Third was a fifty-four item student eval-
uation based on a list of twenty-four technical skills of teaching.
49
The faculty member also rated himself on each item in the eval-
uation and predicted for each item what he believed the student
response would be. The faculty member and students also in-
dicated what they believed were the instructor's five strongest
and five weakest skills, and the three skills that he should
most improve on. The fourth step was the computer processing
of this data. Next was the identification of a teaching pro-
blem determined by the faculty member and an instructional spe-
cialist using all the data collected. This involved (step five)
a preliminary review of the data by instructional specialist and
faculty member together, a more intense review of the data by
each one separately (step six),and a final session isolating
a particular teaching skill which was judged most crucial for
improvement (step seven). The last stage of the model was the
faculty member's participation is certain "treatment and train-
ing" activities relating to this skill. These might include
micro-teaching, further videotapes of the classroom, viewing
films of model instructors, reviewing certain literature, ana/
or continued consultation with the instructional specialist
(Melnick, p. 32).
For the most part Melnick 's study was devoted to the iden-
tification of teaching problems and not the systematic eliminat-
ing of these problems. Much time was spent in the development
of instrumentation. No formal evaluation of the teacher improve-
ment was made. There was interest on the part of these faculty
41
to continue in this improvement process at a future time.
A second study (Noam, 1973) following this eight-step
model was conducted at the University of Massachusetts with
thirteen faculty members from the Chemistry Department.
In this study greater emphasis was placed on having facul-
ty members show change in the teaching skills which were iden-
tified as problematic. The procedures followed for enacting
this change were for the most part those sketched by Melnick.
In addition, Noam taught a demonstration lesson in one of the
Chemistry classes,conducted data review sessions with faculty
and students together, and convened two meetings of the faculty
members to discuss chemistry instruction in general. Toward
the end of the semester a final videotape was taken and analyzed
together with the initial videotape of these faculty. A ques-
tionnaire was also distributed which asked the instructors to
evaluate this improvement process.
An interaction analysis of the pairs of videotapes revealed
that the faculty members had in fact changed in the desired man-
ner. In general the percentage of instructor question-asking,
student responding and teacher clarifying or expanding on stu-
dent ideas (all goals of this group) increased significantly.
However, since there was no control group in this study,
it is not possible to know if the changes which took place were
due to the treatment. A second administration of the fifty-
four item questionnaire was not made, so that changes in the
42
students' evaluations of the instructors' teaching skills could
be observed. No achievement data was collected.
The faculty response to the improvement process was posi-
tive. Seventy-six percent of the participants indicated they
would have liked to continue improvement procedures. An equal
percentage believed they had gained new insights. All stated
that they had become aware of teaching problems. Only thirty-
nine percent felt that the process had given them an opportunity
to practice new skills (Noam, p. 63).
43
CHAPTER III
PROCEDURES
A slightly altered version of the Allen and Melnick model
(1972) discussed in Chapter II was investigated in this study.
A group of graduate teaching assistants were divided into
two experimental groups and a control group. All subjects com-
pleted a pretest which consisted of a half-hour videotape of
their classroom instruction, student evaluations and a student
achievement test.
Members of the two experimental groups individually re-
viewed their videotapes and student evaluations with the help
of an instructional specialist. One experimental group con-
tinued to meet with instructional specialists who provided them
with additional procedures for enacting changes in classroom
instruction.
After approximately eight weeks all subjects completed
a posttest consisting of the same components: a half-hour video-
tape, student evaluations and a student achievement test. All
participants including the control group reviewed their stu-
dent evaluations and videotapes at the conclusion of the study.
The pretest and posttest videotapes were analyzed along
seventeen dimensions using an adaptation of Flanders Interaction
Analysis (Amidon and Flanders, 1971). The student evaluations
were reduced to numerical scores along thirty-one individual
44
items and ten factors. The student achievement data was rated
by three subject matter specialists. These ratings and scores
were used to test differences among the three groups. The re-
mainder of this chapter describes procedures for (a) treatment,
(b) additional data collection and (c) data analysis.
Treatment
Subjects . The subjects were thirteen graduate student
teaching assistants (seven female and six male) who were teach-
ing a required Freshman Rhetoric 100 course at the University
of Massachusetts during the Winter-Spring semester of 1973.
All of the subjects had taught the course the previous semes-
ter, and some had additional teaching experience at the uni-
versity or secondary level. Nine were teaching one section of
the course during that semester and four were teaching two sec-
tions. The majority of the subjects were graduate students in
the English department; two were in the Linguistics department
and one in the Botany and German departments.
The total group was divided into an experimental group of
five, a quasi-exper imental group of four and a control group of
four by means of a table of random numbers. Three members from
the experimental and quasi-experimental group were then randomly
assigned to each of three instructional specialists. Two con-
trol group members were similarly assigned to each of two addi-
tional instructional specialists.
Participation in this study was considered part of the
45
general staff development program for instructors of this course.
This program was conducted by the Rhetoric 100 course director
and consisted of several weekly staff meetings where the theor-
etical bases for such a course as well as certain recommended
instructional methods were discussed. This staff development
program was attended by all subjects and did not include in-class
observation and instructional analysis by the course director.
Setting . Rhetoric 100 was basically a writing course with
more emphasis on the students' ability to see life situations
with increased awareness and to express their ideas with in-
tellectual accuracy, than on traditional writing mechanics. This
was considered a new and experimental orientation for this course
at the University. Students customarily wrote one or more short
papers a week and class activities most often revolved around the
analysis, discussion and evaluation cf these papers. Published
literature as well as the more technical aspects of writing were
discussed later in the semester.
Initial Interview . Each participant in the two experimental
groups underwent an initial interview wherein the theory, pur-
poses and procedures of the treatment model were described. At
the same time the instructional specialist filled out a course
information form which included a statement by the teaching assia-
tant about his perceived instructional problems, if any. The in-
structional specialist also gave the teaching assistant two of
the student evaluation forms, one to be filled out as a self-
46
evaluation and the other as a prediction of the students' eval-
uation.
At the conclusion of the interview a class was chosen to
be videotaped and a pre-videotaping form was given to the in-
structor. This solicited information on the objectives and pro-
cedures for that particular lesson. Both the self-evaluaton
,
prediction of student evaluation and pre-videotaping information
were returned to the instructional specialist before the video-
taping. The control group members underwent a much shorter in-
terviewing process with the primary focus on establishing times
for the videotaping and administration of student evaluations.
No attempt was made to explain to the control group the theory
and procedures of this improvement model nor were they informed
of the specific treatment of the other participants. They were
told that at the conclusion of the second series of data collec-
tion procedures their data would be reviewed with them by an in-
structional specialist if they so desired.
Observation . To familiarize themselves with the general
setting of the class and the particular style of the teaching
assistant, the instructional specialists observed at least one
class meeting. At that time if not before, the students were
informed of the teacher's participation in an instructional
improvement program which included videotaping and student
evaluations
.
Videotape I. A thirty-minute videotape was taken in the
47
instructors’ classrooms according to the following general for—
• the first fifteen minutes of class, the middle ten minutes,
and the final ten minutes.
Student Evaluations . The students filled out a thirty-
one item evaluation of the instructor at the end of the class
that was videotaped (Figure 5). The evaluation was designed
to elicit student ratings of the instructors' performance of
twenty-one technical skills of teaching (Figure 6). (Appendix
C contains descriptions of each of these skills.)
Interpretation and Feedback . The student evaluations
were tabulated by computer and this data as well as the video-
tape were made available to the teachers in both experimental
groups. (Appendix C contains a sample of the ’'Crosstabulations,"
"Best Skills" and "Weakest Skills" sections of the computer
printout for student evaluations.) The instructors ' s self-
evaluation, prediction of student evaluations and the actual
mean scores of the student evaluations on each item were listed
on a single three-column page.
The instructors and instructional specialists then met
for an interpretation conference. The focus of the conference
was the identifying of teaching skills which were believed to
be both important to the instructors’ courses and "problematic."
Extreme differences between the students' rating of particular
skills and the instructors' ratings were one index of problem
skills. Another was instances of skills that were rated well
48
STUDENT-CENTERED ANALYSIS OF TEACHING (SCAT)
DIRECTIONS:
We would like your opinion of your instructor'sfollowing teaching behaviors or skills. Pleaseand circle one of the responses as follows:
performance of each of thelead each ite:r. carefully
E excellent to very good performance in this are
S satisfactory performance in this area
I I would like to see some improvement in this area
I! -I—s tron 2-Lv recorr.enc improvement m this area
0 does not apply - (please use sparingly)
1. When beginning a class, lesson, or new instructionalactivity, the instructor usually makes clear whathe/she hopes to accomplish.
2. The instructor usually gets me interested at thebeginning of each class session and learningactivity.
3. The instructor presents new ideas and information atan appropriate pace, i.e. neither too fast nortoo slow.
4. The instructor arranges and presents material in aclear, well-organized fashion.
5. In this instructor's class it is usually possibleto know which points are most important, andto see the relationships between topics.
The instructor knows when to elaborate on a topicand he/ she elaborates effectively by using examplespointing out relationships, 3nd/cr giving meredetailed explanations.
7. The instructor provides useful summaries at appro-priate times, (e.g. at end of class session or endof unit.)
8. When the instructor reaches the end of any segmentof a presentation, he/ she effectively "wraps it up"by summarizing, checking for comprehension, relatingthe covered material to what is to come, and notingwhether or not the material will be taken up again.
E S I I! o
E S 1 I! 0
E S I II o
E S I T! 0
E S I II 0
E S I II 0
E S I I! 0
E S I II 0
Figure 5. Student Evaluation Form of the Clinic to Improve
University Teaching - Spring 1973
49
STUDENT-CENTERED ANALYSIS OF TEACHING (SCAT)
Again, your response choices are:
E excellent to very good performance in rhis are.-.
S satisfactory performance in tin's nmI 1 would like to see some improvement in this area
I! I strongly recommend improvement in this area
0 docs not apply - (please use this sparingly)
9. The Instructor has an effective presentation style E S I I! 0(e.g. voice quality, choice of words, body movements,etc.
)
10. The instructor inspires excitement and interest E S I I! 0in the subject matter.
11. The instructor is creative and imaginative in his/her E S I 11 0teaching methods.
12. The instructor makes students feel free to disagree E S I II 0with him/her, with fellow students, or with theread? r.gs.
13. The instructor gives open-minded presentations, of a E S I I! 0
variety of points of view.
14. The instructor relates che topics to a wide range of E S • I I! 0
fields, situations, applications, and interests.
15. The instructor encourages independent thought and, E S I I! 0
when necessary, helps students find the necessary .
resources to pursue independent study.
16. The general design of this instructor's course E S X II 0
(number jr.d choice of lectures, discussions, reading,
papers, examinations) is appropriate.
17. The level of difficulty and the amount of work in this E S I 11 0
course are about right for me.
18. The instructor allows students to work ojt alternative E S I I! 0
ways of achieving course objectives and helps students
who want to do this.
19. The instructor is an effective discussion loader E S I I' 0
(encourages contribut ioas, keeps the discussions
focused, doesn't dominate, etc.)
Figure 5. Student Evaluation Form of the Clinic to Improve
University Teaching - Spring 1973 (continued)
50
STUDENT-CENTERED ANALYSIS OF TEACHING (SCAT)
Remember, your response choices are:
E excellent to very good performance in this area
S satisfactory performance in this area
I I would like to see some Improvement in this area
XI I strongly recommend improvement in this area
0 does not apply- (please use this sparingly)
20. The instructor invites students to share their
knowledge, experiences, and opinions at appro-
priate times.
21. The instructor encourages students to listen and
to respond to each other.
22. The instructor uses questions effectively (e.g.
to focus attention on important points, to check
on student understanding, to get students to
think, etc.).
23. The instructor restates students’ questions for
clarification and answers them in a way that
the whole class understands.
E S i I ! 0
E f> I 1 ! 0
E S I 1! 0
E S 1 I ! 0
24.
The instructor provides encouragement and E
opportunity for student questions, suggestions,
comments or criticisms regarding any aspect of
this course.
25. The instructor seems to know when students are E
confused, bored or frustrated.
26. The instructor answers questions, or encourages E
and helps others to answer them, with understand-
ing and clarity.
27. The instructor demonstrates his/her active inter-
est in students as individuals by being easy to
approach, patient, and willing to help.
28. The instructor makes clear to students what is
expected of them and how their performances are
to be evaluated.
Figure 5. Student Evaluation Form of the Clinic to Improve
University Teaching - Spring 1973 (continued)
51
STUDENT-CENTERED ANALYSIS OF TEACHING (SCAT)
Remember , your response < h" i ' es arc:
F. excel lent to v*.-rj jjorid performance in this area
S sat i s_f uc_t_o_r y per ! ornvmcu in this area
I 1 would likt to see sore improvement in this area
I! 1 strongly recommend improvtsr.ent in this area
0 does not apply- (please use this sparingly)
29. The instructor evaluates student work in a clear E
and consistent wav.
1110
30. The instructor keeps the students posted on their E
pro.', teas tin < quest ion-, in class, individual
conferences, and appropriate quizzes and test...
31. The Instructor openly talks about questions of E
"right and wrong" as they relate to his/her
subject matter (e.g. does not avoid discussing
controversial Issues).
I! 0
I I! 0
Figure 5. Student Evaluation Form of the Clinic to Improve
University Teaching Spring 1973 (continued)
TWENTY-ONE TECHNICAL SKILLS OF TEACHING
AND RELATED STUDENT EVALUATION ITEMS
52
1- Set Induction (items 1 and 2)
2. Logical Organization (items 4 and 5)
3. Pacing (item 3)
4. Elaboration (item 6)
5. Closure (items 7 and 8)
6. Expression (item 9)
7. Inspiration /Charisma (item 10)
8. Level of Challenge (item 17)
9. Treatment of Divergent Views (items 12 and 13)
10. Defining and Expanding Relevance (item 14)
11. Facilitating Student Participation (items 19 and 20)
12. Facilitating Student-Student Interaction (item 21)
13. Asking Questions (item 22)
14. Answering Questions (items 23 and 26)
15. Choosing Appropriate Modes and Materials (items 11 and 16)
16. Instructional Flexibility and Individualization (items 18 and 27)
17. Facilitating Independent /Creative Inquiry (item 15)
18. Monitoring Student Response (items 24 and 25)
19. Making Expectations Clear (item 28)
20. Evaluation (items 29 and 30)
21. Defining and Expanding Ethical Contexts (item 31)
Figure 6. Twenty-One Technical Skills of Teaching and RelatedStudent Evaluation Items
53
below the overall rating. Where the skills that were rated low
could be viewed in terms of a more general problem domain, such
as "discussion leading" or "clarity," then a clearer vision of
what behaviors might undergo productive change in the classroom
was obtained.
this conference, or a second follow-up conference,
the instructional specialists attempted to clarify low skill
ratings by pointing out specific classroom episodes with the use
of the videotapes. On occasion the instructional specialists
used interaction analyses .or pointing out classroom behavior
patterns that might aid the interpretation of the ratings and
indicate productive change directions.
In the conference the videotape viewing process was focused
by the instructional specialist. The tape would be stopped at
various places where data appeared that might relate to the
student evaluations. Also, when the instructors had acknowledged
the desire to alter their instruction in specific ways, the in-
structional specialists in reviewing the videotape often would
stop the tape at points where the desired behavior changes might
have been appropriate.
Follow-up . The experimental group also engaged in follow-
up activities designed by the instructors and the instructional
specialists. Such activities were specific to the particular
change goals and included the following: successive videotaping
and feedback sessions focusing on the appearance and reinforce-
54
merit of specific behaviors; micro-teaching with feedback from stu-
dents as well as instructional specialists; videotape viewing and
group discussion of instruction in the classroom; videotape view-
ing and group discussion among several of the instructors; admin-
istering and interpreting additional student evaluations; obtain-
ing additional student feedback in the form of in-class audio-
tape recordings made be selected students; obtaining lesson spe-
cific achievement data; planning lessons; reading specific mater-
ials. Each member of the experimental group spent approximately
ten additional hours in follow-up activities.
Additional Data Collection
Videotape II,Student Evaluations . Approximately eight
weeks after the inception of the treatment process a final thir-
ty-minute videotape and set of student evaluations were completed
for both experimental groups and the control group. The same
student evaluation questionnaire was used for the posttest as
was used in the pretest.
Student Achievement Tests . During the second week and dur-
ing the eighth week of the treatment process an essay-type exam
was administered to the classes of all subjects. Two forms of
this exam were constructed by the course director. Half of both
experimental groups and the control group received one form of
the exam as a pretest. This sequence was reversed for the post-
test.
55
Data Analysis
The sequence of data collection and treatment for this
study is summarized in Figure 7.
Interaction Analysis . The first fifteen minutes of all
pretest and posttest videotapes were selected for the analysis.
This segment of each tape was the most standardized across all
subjects--ref lecting the first fifteen minutes of each class.
Two of these segments were transferred to one thirty-minute
tape and coded for rating.
Four raters were then trained in a seventeen-category
system of interaction analysis (Figure 8) developed for this
study by subscripting categories from Flanders' ten-category
system (Figure 9) (Amidon and Flanders, 1971). This training
consisted of approximately six hours of supervised practice in
using this system to analyze videotapes. Three of the four rat-
ers had had several hours of previous experiences using inter-
action analysis systems. Raters did not know which tapes were
pretest and which posttest. An audio-tape counting off three-
second intervals was played together with the videotape during
the rating. One tape segment was rated by all four raters and
the inter-rater reliability computed according to the follow-
ing Scott coefficient (Flanders, 1967, pp. 161-166):
coefficient "pi" = —1 - P e
SEQUENCE
OF
DATA
COLLECTION
AND
TREATMENT
56
to
fe pD to
O cu
fe HOp CO
fe d co & cu ofe cu 6 •iHH •H <u 4J
> > cdo Ci tu 3u tu •rl »—
<
p P M cdd o >M <5 <U war—
1
p cd pcd d p d•H cu o cuP Td CU Td•H P t> pd p •H pM CO > CO
to
PCO
<u
Hto pd d0 (U•H ep tu
td >P cuM i—i •H
1—
1
td P> CJ
(Ua w <cd p pp d do tu <ucu Td Tdt> P P•H p p> CO CO
fe 1* \ I
d> toO >—
'
fe CO cu COO P o 4->CO d CO
i-J cu cu Q)c H d Ci EhH o <uP •r~l CD p CO 4-J
fe d p d d c c£ CU cd o o O dJi—
i
cu E > •H o •rl gfe •H a> Ci P P <DW > > CU cd d cd >fe Ci cu CD P 0 P CDX cu •H P 1—1 •H M • i—l •Hw p id O w cd p M cd p
1 d o > cd > oM H <C E tu W p CU w <jCO o a. <u CLC T““
1
p o cd P Ci cd p pD cd d C p d fe p d cO' •H cu CO o cu u o cu cup Td to CU Td cu cu Td Td
•H P cd Td P p Td P pd p i
—
i •H p d •H P pM CO o > CO M > :co CO
P-t
DOPiO
H2
WP-i
Xw
CO cu CO
p o pCO d co
CU cu cu
H d Ci H0 <u co
P •H co cp cu CO pd p d d Ci d d
£ cu cd o o p p cu
cu E > •H o Td •r4 E•H CU Ci p cu p CU
> > CU cd d o cd >Ci cu CD P o 0 P CU
CU •H P T“
1
•H Ci M i—l •r-i
p id O M cd P fe M cd Pd o > cd > oM C E CU w p CL CU w <1
cd
•Hp
pd0)
TdPPCO
CC)
pO0)
Td•H
pcCU
TdPPCO
0)
naCl
CU
pd o
fe-
ed
po<u
Td•H>
pdtu
TdppCO
pdcu
TdppCOo
Figure
7.
Sequence
of
Data
Collection
and
Treatment
57
SEVENTEEN -CATEGORY INTERACTION ANALYSIS SYSTEM
1. Instructor accepts or discusses student feelings
2. Instructor praises or compliments
3^ Instructor relates to student responses
a. by simple acknowledgement or encouragement
b. by restating, clarifying, incorporating or expandingthe response
c. by probing or encountering
d. by redirecting the response to other students
e. by correcting or disapproving
4. Question-asking
a. lower order recall or convergent questioning
b. higher order divergent or evaluative questioning
5. Lecturing
a. focusing, summarizing, introducing, orienting statements
b. all other lecturing
6. Giving directions
7. Criticizing or justifying authority
8. Student talk - responding
9. Student talk - initiating
10.
Silence or confusion
a. silence
b. confusion
Figure 8. Seventeen-Category Interaction Analysis System
58
FLANDERS TEN-CATEGORY INTERACTION ANALYSIS SYSTEM
1 * Accepts Feeling : accepts and clarifies the feeling tone ofthe students in a nonthreatening manner. Feelings may bepositive or negative. Predicting or recalling feelingsis included.
woISwDt-J
Praj- se s or Encourages: praises or encourages student ac-
tion or behavior. Jokes that release tension, but notat the expense of another individual; nodding head, orsaying "urn hm?" or "go on" are included.
EhowPit-H
oM
3. Accept s or Uses Ideas of S tudent
s
: clarifying, building,or developing ideas suggested by a student. As teacherbrings more of his own ideas into play, shift to Cate-gory 5.
wo52
.J
$3
4.
5.
6
.
Asks Questions : asking a question about content or pro-cedure with the intent that a student answer.
Lecturing: giving facts or opinions about content or
procedures; expressing his own ideas, asking rhetoricalquestions
.
Giving Directions : directions, commands, or orders withwhich a student is expected to comply.
Ho§wQ
*5
Eh
Eh
W
EhCO
7. Criticizing or Justifying Authority : statements intendedto change student behavior from nonacceptable to accept-able pattern; bawling someone out; stating why the teach-er is doing what he is doing; extreme self-reference
.
Student Talk - Response : talk by students in response toteacher. Teacher initiates the contact or solicits stu-dent statement.
Student Talk - Initiation : talk by students, which theyinitiate. If "calling on" student is only to indicatewho may talk next, observer must decide whether studentwanted to talk. If he did, use this category.
10. Silence or Confusion
:
pauses, short periods of silence,and periods of confusion in which communication cannot
be understood by the observor.
Figure 9. Flanders Ten-Category Interaction Analysis System
59
where PQ is the proportion of agreement, and Pe is the propor-
tion of agreement expected by chance which is found by squaring
the proportion of tallies in each category and summing these
over all categories. Scott's coefficient can be expressed in
words as the amount that two observors exceeded chance agree-
ment divided by the amount that perfect agreement exceeds chance.
A computer program was developed for this study (deFreisse,
1973) which converted the interaction analysis scores to a 17 x
17 matrix and (by collapsing the subscripted categories) also
converted them to a Flanders 10 x 10 matrix. The program also
produced the following special measures: Ratio of Indirect Teach-
er Behaviors to Direct Behaviors (i/D Ratio), a Revised I/D Ratio,
Total Indirect Behaviors (Extended Indirect),Total Direct Be-
haviors (Extended Direct), Total Teacher Talk, Total Student
Talk, and the Student /Teacher Talk Ratio (Amidon and Flanders,
1971). When the f irst and second videotape scores of each in-
structor were submitted, the program computed a Difference Ma-
trix. This matrix for examining differences between the first
and second videotaped instructional episodes was completed for
all three groups in the study.
Because many of the categori esi n the 17-category system
appeared not to have been sufficiently differentiated by the
raters, the Flanders 10-category system was used for most of
the interaction analysis and interpretation. This was done in
three ways: (1) through comparisons of the Difference Matrices,
60
(2) through comparisons of selected interaction analysis cate-
gories, and (3) through flow charts generated from each group
matrix.
Student Evaluations A numerical score was assigned to
each of the values in the student evaluation instrument (E S
I I! 0). The mean score for each item in the pretest and post-
test evaluations of each group was computed and compared. Com-
parisons were also made by means of t-tests. This procedure
was repeated for groups of items within ten factors derived
by means of factor analysis (Figure 10). (The factor matrix
is contained in Appendix C).
Student Achievement . Three raters were trained by the
Rhetoric 100 course director to rate the student writing tests
along three dimensions: (1) "literacy," indicating the techni-
cal correctness of the writing, (2) "focus," indicating the
ability of the writers to organize their ideas around a cen-
tral purpose, and (3) "felicity," indicating the awareness dis-
played by the writer of his reader as evidenced in the use of
rhetorical devices.
Ten matched pairs of student papers from each subject's
class were rated by all three raters on each dimension. In the
case of subjects with two class sections, only the section that
was videotaped was used in the achievement rating.
The scores on the three dimensions were summed for each
student and a mean score determined for each subject's class.
61
STUDENT EVALUATION FACTORS
FACTOR 1: All items
FACTOR 2: "Clarity and Presentation (items 1, 5, 6, 7, 8, 3, 4)"
FACTOR 3: "Interest (items 2, 10, 25, 9, 11)"
FACTOR 4: "Clarity (items 1, 5, 6, 7, 8)"
FACTOR 5: "Evaluation and Feedback (items 28, 29, 30)"
FACTOR 6: "Relating to Student Responses (items 23 and 26)"
FACTOR 7: "Relating to Student Needs (items 25 and 27)"
FACTOR 8: "Discussion Leading (items 19, 20, 21, 22)"
FACTOR 9: "Openness (items 11, 12, 13, 14, 15, 18, 24, 27, 31)"
FACTOR 10: "Course Design (items 16 and 17)"
Figure 10. Student Evaluation Factors
62
A total achievement rear score for the five classes in the
ex!>er r. * s. _ group ana tne I cur classes in the quas i-exper : -
rental and control croups was carpeted tor pretest and tcsttest
?*^-scns cetveen trese scores were aided tv t-tests.
The results, interpretations and conclusions of these pro-
cedures are given detailed treatment in the following charters.
63
CHAPTER IV
RESULTS
Changes In Instructional Behavior: Interaction Analyses
In order to represent the characteristic patterns of in-
structional behavior for the experimental, quasi-experimental
and control groups, a seventeen-category system of classroom
behavior coding was employed in analyzing both pretest and
posttest videotapes. The scores derived from this system were
later reduced to a ten-category system (Amidon and Flanders,
1971). The results of both coding systems were displayed on
matrices for interpretation. (Matrices not directly discussed
in sections of this chapter are included in Appendix B.)
Difference Matrices
.
In order to identify changes in in-
structional behavior within each group from pretest to post-
test videotaping, three Difference Matrices were computed.
As indicated by the 10 x 10 Difference Matrix for the experimen-
tal group (Table V), substantial changes appear to have occurred
in at least five column totals (3, 4, 5, 8, 9) and three cells
(3-3, 5-5 and 9-9). The amount of student talk in the classes
of the experimental group decreased by I0.717o, that is, 10.717,
more of the first lesson was occupied by student talk than the
second lesson (columns 8 and 9). The amount of lecturing in-
creased by 12.217<> (column 5). The time occupied by teacher
questioning decreased by 4.907<> (column 4). The amount of ac-
64
TABLE V
EXPERIMENTAL GROUP DIFFERENCE MATRIX
Categories
. . . 1 ,I 2 I 3 I 4 I 5 I 6 I 7 I 8 I
i I
G I
I
0 I
I
0 I
I
0 I
1
0 1
I
0.O0!
0 I
t
0 I 0 I Tallies
•Of 00 I oiooi 0.00 1 0.00 1 o - n o i o r o o i
1
O.OCII
3*001I
0.0CI Percents
20 I -10 I
I
-1 I
1
-14 I -i I i i 1 I
I
o -n 71
-11 I *7 I -8 I
o * o 01
0 .59I 0 *00 1
I
0 « 8 4 I o-o’J oVl 0*661I
0-381!
0*471
0 I -2 I 57 1 2 I 0 I -2 I o I -3 I "7I 5 I
3I I I I 1 I J I
0*351• 0 • 0 0 I 0-121 4.261 0-221 0-051 0 ;1? I 0 • 00 I 0-171 0*38!
4
0 I 2 I -4 I
I
-39 I
I
3 I
I
1 I
I
0 I
I
-35 I
I
-14 !
I
- 3 I
I
• 0 • 0 0 1 0-131 0*251 2-231 0-231 o V* o -no 1 2 • 0 3 I 0-8410 • 0 1 i
5
0 I 3 I
I
1 I
1
-6 I
I
1^4 I
I
: 3 I
I
3 I
1
-2 I
I
4 I
I
1 I
1
o l3 I• o • o o i 0 - 2 1
1
0 • 0 7 T 0-271 11-861 0 • 1 9 I 0 • 2 o i 0 • 11 I0
’ 30 J
o I 2 l 0 I 0 I 0 I -3 i 0 I -1 I 0 1 -1 I
6
0 . 0 0 I o : 1 3i 0.001 0.001 0-0,1I
o r !?iI
o . no i
I
0.061I
3-00!I
0.05i
0 I 0 I 1 I 1 I 3 1 0 I 5 I 0 I *1I 1 I
7 I I I I 1 I 1 I
• 0 • 0 01 0-00 1 0*0 71 r • n 7 T
0 * 2 o l 0 -00 1 0-3410 • 01 !
0 ' 0 6 I 0*0 ?I
0 I -24 I < I -10 I -1 I l I 0 I -40 I 0 I -14 I
8 I I i I I I I 1 I
• 0 • 0 0 I 1.371 0.361 0-561 0 - 0 4 I 0 -0 71 o • no I 1.77! 0-0^1 0*851
0 I - i 7 I -2 I -9 I -3 I 0 1 -2 I 2 I .85 I -7 I
9 I I I I ! I I I I
• 0 • 0 0 1 0-971 0 • 0 4 1 n .49I 0-1 7 I o'oo I 0 • 1 2 l 0-131 4-491 0 • 4 o I
0 I •4 I -6 I -14 l n 3 2 I 3 I 6 I •13 1 14 I
10 I I I i I I I 1 I
• o • 0 0 1 0 • 2 4 I 0 .36I 0 • «0 1o • n 9
1
0 [1 4 1
0 - ?0 I 0-5lIa.74l 1 .
2
61
0 I -50 I 50 I -89 I 155 I T 3 I in I -84 I -153 I -12 I
I I I I I I I I I
TOTALSo • o 0
1 2 - 8 ?I 4 • 1 01 4 * 9 o
I 12- 2 1I 0;l51 0
.68I 4 - 1 4 I 6-571 0* 051
I/D RATIO 0.130
REVISED I/O RATIO — -0-0?3
EXTENDED INDIRECT -- 3.187
EXTENDED DIRECT 0.155
fEACMER TALK 10.650
STUDENT TALK ------- -10.714
STU/TEAC TALK RATIO -0.304
65
cepting and using student ideas (column 3) increased 4.10%.
Finally, though the percentage is small, the amount of teach-
er praise decreased 2.82% (column 2).
The remaining four columns changed insignificantly (col-
umns 1,6, 7 and 10). The 3-3 cell, indicating continuous
using or expanding on student ideas increased by 4.26%. Con-
tinuous lecturing (cell 5-5) decreased by 4.49%. Reinforce-
ment and praise (column 2) was particularly decreased follow-
ing student talk (cells 8-2 and 9-2).
The Difference Matrix for the control group (Table VI)
indicates notable changes in question-asking (column 4), lec-
turing (column 5), both student talk categories (columns 8
and 9) and silence (column 10). Columns 1, 2, 3, 6 and 7 did
not change noticeably.
Question-asking by the instructors in the control group
(column 4) became 5.17%, less of the total lesson. Lecturing
was 7.9 7% less. Teacher initiated student talk (student talk -
response, column 8) decreased by 6.3 1% while student initiated
talk increased 10.57%, and silence increased 5.96%,. Heavy change
cells for the control group were cell 4-4 (continuous teacher
question-asking, -3.78%,); cell 5-5 (continuous lecturing,
-8.30%,); cell 8-8 (continuous student responding, -4.72%,);
cell 9-9 (continuous student initiated talk, +7.46%,) and cell
10-10 (continuous silence, +3.43%,).
The quasi-experimental group (Table VII ) showed strong
66
TABLE VI
CONTROL GROUP DIFFERENCE MATRIX
Categories1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9 I 10 I
* ****** ****** ****** ****** ****** * * * * * * * * * * * * * • * * * • * * * *
1 I 0 I 0 I 1 I n i 0 I 0 I 0 I 0 I 0 I Tallies{ 1
0,091I I i I I I
o.odi 0.001 0.091 o.ooi oToni 0. 001 0 . ool 0- 0 0 i O.OOI Percents
0 I 4 I 3 I 1 I 2 1 l i 0 I 1 I 1 I 2 I
2 I I I I I I 1 I
0 • o Q 1 0-341 0-24 1 0 • 0 7 I 0-171 o-o^i 0 • 00 1 0 - 0 9 I0
*09 I O'l 71
l I 0 I 11 I -2 I -5 I 0 I 0 I 1 I 0 I 9 I
3 I I I I I I I
0.091 o.ool 0.891 0.191 0-441 o r o o i 0 . ool 0 • 091 O' oil 0-771
0 I 0 I 2 I -4? I -7 I i i 0 I -19 1 9 I 1 I
4 I I I I I I I I
0 • 0 Q1 0 on 1 0-171 3.781
0. ft T I 0 : o
9
1
0 00 1 1- 7 1 *0
6810 • 0 2 I
0 I 3 I 0 I -1? I -97 I 0 I 0 I 7 I 6 1 1 I
5 I I I I I I I I
O' op 1 0-251 0 • 0 0 1 i * 1
6
1 8.
3
n I 0 • 00 1 0 no 10
591O’ 5 1
I O' 061
0 I 0 I 0 I l t -1 I 1 I 0 I 0 I 1 I 2 I
6 I I I i I I I I
o • o 0
1
o * o 6
1
0 • 00 1 0 • 0 9 I 6 • 0 910V 0 no I 0 •00 1
3 •691 O’l 71
0 I 0 I 0 I n i 2 1 0 I 0 I 0 I 0 1 0 I
7 I 1 I I I I I
0 ' 0 0 1 o • c 6 1 0 • 00 1 n • no 1 0-171 o • o n i 0 •00 1 0 •00 10 • on I 0 • 00 1
0 I l I -6 1 -12 I 1 I 0 I 1 1- 52 I ’3
I 1 1
8 I I I I I I ,1
0* 00 1o o
61 0-5411 • o
7 1o * n
7 10 *. 0 o
10 •n
9 I 4 -7pi0•271 0*0 ?I
0 I 5 I 4 I 2 I 9 I 0 I 1 I 1 I 9 I 15 I
9 I I I I I I I I
0 • 0& 1 o^ii 0-341 0 . i6
1
0-771 0 • 0 0 1 0. 0 9I 0 • 0
9 1 7 • 4 ft I 1 • 2 6 1
0 I 2 I -1 I 7 I 4 I 1 I 0 I -6 I 24 I 42 I
10 I 1 I I I I I
.541l
0 . 00 I 0.171 0.091 (i .571 0 . 3 n I o : o =9
1
0 . OO I 0 2 •04 1 3.431
2 I 15 I 13 I-56 I -92 I 4 I 2 I
• 69 I l 6 I 73 I
1 I I I I I I I
TOTALS* 0 .1 7
l
1.231 i . o i i 5.171 7.971 0:341 0 .171 6 .31110 . 571 5.961
I/O RATIO 0.017
REVISED I/D 3ATI1 -- -0-035
EXTENDED INDIRECT -•? 1,213
EXTENDED DIRECT 0.094
TEACHER TALK -10.217
- 4.261STUDENT TALK -
stu/teac tal< ratio 0.167
67
TABLE VII
QUAS I -EXPERIMENTAL GROUP DIFFERENCE MATRIX
Categories* 1 .1 2 13 14 |5 16 17 18 19 I lO I
* 01 01 01 01 01 0! 01 01 01 0 iTallies1 * I . I I I 1 I 1 I 1
1-« 0 • 0 0 I o.ool 0.00! o.odl 0>OM C 1 0 0 I Q.ncl 0*001 0*00! 0 » oo iPercents
* 01-21 51 01 01 -1 I ? I -3 l -4 I -4 I
2 * I . I I I I I 1 I 1 1
• 0 • 0 0 1 0.161 0.74 1 n.Oll 0 • 0 1 0 r 0 **1 0.161 0*241 3 * 331 0 * 32 1
+ 0 I 1 I 30 I 7 1 1 I 0 I 0 I 2 I 2 I 3 1
3 I I 1 I I I 1 I I I
.* o.oQi 0.041 2.491 0.581 0.031 o : oo i o.noi 0.16! 3*161 0.251
• 0 1 0 I 2 1 -37 1 -4 I = 2 I o I 4 I *2 I 6 I
4 *1 . I I 1 I I 1 1 I I
••
o *o0
1
o -on l 0 • 1
6
I 2.951 0*3^1 0 *1*1 0*03 * 0*351 0 * 1 5
1
0-511
* 0 I -3 1 11-11 114 l -5 I -1 1 -6 I 0 I 0 I
5 * 1 I I 1 I 1 1 1 I I
•#
o • o o i 0*241 0*0*1 0*3*1 9.751 0 * 4 1
1
o*n*l 0*471 3 * 00 I 3*3-1
• 0 1 -2 I 0 1 0 1 *3 I -59 I o I -2 I -1 I -2 1
6 1 1 1 I 1 1 I I I I
0 • 0 0 * 0*1*1 n • o5 1 o • od
I
0*241 4 : a o i 0 • 0 0 1 0* 1*1 3* 0*1 0*1-1
• 0 I a i -11 0 1 0 I 0 I -4 1 S I t I 0 1
7 #I I 1 i 1 I I 1 I I
•• coOi o • c 6
1
0*0*1 . 00 I o*ooi o*oo ! 0*331 o*ooi 3*0*1 o « o o i
* 0 I 3 I 2 i a : -8 1 -1 I -1 I 59 I -5 1 1 I
8 * I I I i 1 I 1 I I I
# 0*001 0-251 0 - 1 7 I 0.661 g.651 0*0*1 o*o*i 7.4 C !3 • < r 1 0*0*1
• 0 1 -3 I -21 -31 -3 I -1 I -1 I 0 I -115 I -3 I
9 • 1 I 1 1 I 1 I I 1 1
*0 * col 0 • 2 4 I C . 1
6
I <. ?4 1 0*241 0*0*1 0*0*1 o-oei 9* 31 I 0 • 23 I
• 0 I 3 I 11-71 7 1 0 I 1 I 4 I *3 1 -4 I
io•
I I
0*2*11 I
0*0 91 o- 561I
0 * 1R i
I
1 0 5 1
l
o-o 9 *
1
0*3411
0*?3l1
0*2*1
* 0 I -3 I 42 1 -33 I 99 1 -69 1 -4 I 96 I -1*2 I -3 I
» I 1 I 1 I I I I I I
TOTALS* 0* 2ll 3.491 ?,561 a* 54i 5 : 6 ! 1 0* 3?I 7.361 1 3' * 7 I O' 1^1
I/O RATIO -0.035
9EVISE0 I/O 7 A Tn -* 0.444
EXTENDED INDIRECT — 4.456
extended direct -5.041
TEACHER TALK - 3.359
STUDENT TALK - -3.232
STU/TEAC T*L< RATH -0.031
68
changes in accepting and using student ideas (column 3), in
question-asking (column 4), in lecturing (column 5), in giv-
ing directions (column 6) and in both student response and
student initiation (columns 8 and 9).
The amount of class time devoted to using student ideas
in the quasi-experimental group (column 3) increased 3.49%.
Lecturing (column 5) increased by 8.587>. Giving directions
(column 6) decreased by 5.617.. Student response (column 8)
increased by 7.364, and student initiation (column 9) decreased
by 10.59%. In terms of notable cell changes, continuous use of
student ideas (cell 3-3) increased by 2.49%; continuous ques-
tion-asking (cell 4-4) decreased by 2.957.; continuous lecturing
(cell 5-5) increased by 9.757.; continuous direction-giving (cell
6-6) decreased by 4.807.; continuous student response (cell 8-8)
increased by 7.407. and continuous student initiation (cell 9-9)
decreased by 9.317..
Comparison of Difference Matrices along Selected Dimensions.
The numerical scores (number of three-second tallies),the per-
centage of the total second videotape lesson that these repre-
sent, the change in the number of tallies and the change in the
lesson percentage from the pretest videotape along a number of
significant dimensions are listed in Table VIII for all three
groups in the study.
From this table it is possible to see more clearly how each
group changed from pretest to posttest lesson. Several important
69
TABLE VIII
COMPARISON OF INTERACTION ANALYSIS DIMENSIONS
EXPERIMENTAL CONTROL QUASI-EXPERIMENTAL
1. AcceptsFeeling
23
1.54
+23
+1.54
2
.17
+2
+ .17
0
0
0
0
Tallies forsecond lessonPercent of
second lessonlst/2nd lessonsdifference (tallies)lst/2nd lessonsdifference (percent)
2. Praise 54 40 293.62 3.43 2.39-50 +15 -3
-2.32 +1.23 -.21
3. Uses 195 63 71
Student 13.09 5.40 5.84Ideas -50 + 13 +42
+4.10 +1.01 +3.49
4. Asks 120 118 120Questions 8.05 10.11 9.88
-89 -56 -33-4.90 -5.17 -2.56
5. Lecturing 508 368 52934.09 31.53 43.54+155 -82 +99
+12.21 -7.97 +8.58
6. Giving 8 2 2
Directions 0.54 .16 .16-3 +4 -69
-0.15 + .34 -5.61
70
TABLE VIII
COMPARISON OF INTERACTION ANALYSIS DIMENSIONS (continued)
EXPERIMENTAL CONTROL QUASI-EXPERIMENTAL
7. Criti-cizing
12
.81
+2
+ .17
2
.17
+2
+ .17
4
.33
-4
-.32
Tallies for
second lessonPercent of
second lessonlst/2nd lessonsdifference (tallies)lst/2nd lessonsdifference (percent)
8. Student 208 119 203Talk- 13.96 10.20 16.71Response -84 -69 +88
-4.14 -6.31 + 7.63
9. Student 206 234 141Talk- 13.83 20.05 11.60Initiation -123 126 -132
-6.57 10.57 -10.59
10. Silence- 156 215 116Confusion 10.47 18.42 9.55
-12 +73 -3
-.05 +5.96 -.13
Cell 3-3 142 36 46
9.53 3.08 3.79+57 +11 +30
+4.26 + .89 +2.49
36 43 60
2.42 3.68 4.94-39 -42 -37
-2.23 -3.78 -2.95
Cell 4-4
71
TABLE VIII
COMPARISON OF INTERACTION ANALYSIS DIMENSIONS (continued)
EXPERIMENTAL CONTROL QUASI -EXPERIMENTAL
Cell 5-5 452 313 483 Tallies for
30.34 26.82 39.75second lessonPercent of
+154 -87 +114second lessonlst/2nd lessons
+11.86 -8.30 +9.75difference (tallies)lst/2nd lessonsdifference (percent)
Cell 8-8 139
9.33-40
-1.77
74
6.34-52
-4.72
166
13.66+89
+7.40
Cell 9-9 122 169 11710.20 14.48 9.63
-85 +89 -116-4.49 +7.46 -9.31
Cell 10-10 77 121 485.17 10.37 3.95+14 +42 -4
+ 1.26 +3.43 -.23
3a. Acknow- 42 34 28ledge- 2.82 2.91 2.30ment -47 +15 +12
-2.70 +1.25 +1.00
54 42 403.62 3.60 3.29-54 -23 +19
-3.07 -2.11 +1.58
3c. Probing
72
TABLE VIII
COMPARISON OF INTERACTION ANALYSIS DIMENSIONS (continued)
EXPERIMENTAL CONTROL QUASI -EXPERIMENTAL
3d. Redi- 9 8 2 Tallies forrecting
.60 .69 .16
second lessonPercent of
second lesson-11
-.64
+6 0 lst/2nd lessonsdifference (tallies)
+ .51 0 lst/2nd lessonsdifference (percent)
5A. Orient- 89 10 65ing 5.97 .86 5.35
+19 -12 +15+1.63 -1.07 +1.28
I/D Ratio .426 .372 .291 Second Lesson-.130 + .017 -.005 lst/2nd lessons
difference
Revised I/D .944 .951 .959Ratio -.023 -.035 + .444
Extended 14.966 7.798 7.819Indirect +3.187 +1.213 +4.486
Extended .403 .171 .000Direct + .155 + .084 -5.041
Teacher 61.745 51.328 62.140Talk +10.660 -10.217 +3.359
Student 27.785 30.249 28.313Talk -10.714 +4.261 -3.232
Stud. /Teach- 7450 7589 T456er Talk Ra- -.304 +.167 -.081tio
73
patterns emerge.
First, there appears to be much more parity between the
changes that took place in the experimental and quasi-experi-
mental groups compared to the control group. For example, in
column 3, using student ideas, the experimental and quasi-
exP®t tr.en i_al groups gained 4.10% and 3.49% respectively, while
the control group gained only 1.017*.
In column 2,lecturing, although the final percentages
for all three groups on the posttest are similar (34.097,,
31.537, and 43.54/,) the tendency for both experimental and
quasi-experimental groups was to increase the amount of direct
teacher influence, that is, to increase lecturing, while for
the control group the tendency was to decrease direct in-
fluence. Since the absolute number of tallies is very high,
these percentage differences indicate substantial change.
Remarkable here is the fact that as indicated in the
above analysis, using student ideas (column 3) in the experi-
mental and quasi-experimental groups also had a tendency to
increase. In other words, both the experimental and quasi-
experimental instructors tended to be both more ’’direct" and
more "indirect” in their teaching as witnessed by the second
videotape. This change pattern is described by Flanders (1970)
as "flexible." The control group showed only a slight incease
in "indirect" teaching and a decrease in "direct" teaching.
Cells 3-3 and 5-5 reflect these similarities and differ-
74
ences. The 3-3 cell for the experimental and quasi-experimen-
tal groups increased 4.26% and 2.49% respectively, and .89%
for the control group. The percentages for the 5-5 cell were
+11.86% and +9.75%, for the experimental and quasi-experimental
groups, and -8.30%, for the control group.
The special measure of "extended indirect" from Table
VIII (which totals the number of tallies and percents in the
intersections of columns 1,2,3 and rows 1, 2, 3 of the ma-
trix) and the "total teacher talk" measure further support
the "flexible" pattern of the experimental and quasi-experi-
mental groups. The total number of indirect behaviors in-
creased 3.18%, for the experimental group, 4.486%, for the
quasi-experimental, and 1.21%, for the control.
In terms of total teacher talk, which totals columns
1 through 7 of the matrix, the experimental group showed an
increase of 10.77%, from 51.08%, to 61.74%,. The control group
decreased from 61.545%, to 51.82%,. The quasi-experimental
group reflected the experimental group with an increase of
3.35%, from 58.78%, to 62.14%,.
The total student talk (columns 8 and 9) ,ratio of stu-
dent talk to teacher talk, and cells 8-8 and 9-9 also tended
to differentiate between the experimental and quasi-experimen-
tal groups on the one hand and the control group on the other.
Both the experimental and quasi-experimental groups showed
decreases in the amount of time occupied by students in class
75
(the sum of columns 8 and 9). For the experimental and quasi-
experimental groups the decrease was 10.71% and 3.23% respec-
tively. For the control group there was an increase of 4.26%.
The resulting student /teacher talk ratio was -.304 experimen-
tal, -.081 quasi -experimental and +.167 control. Cells 8-8
and 9-9 taken together indicate a total decrease for the ex-
perimental and quasi-experimental groups in continuous student
talk (-6.26 L and -1.91/c). The control group increased 10.897o
in these combined cells.
There is also an interesting difference among the groups
in terms of "silence or confusion" (column 10). This is an
important classification in that the amount of silence and
the placement of silence indicates a great deal about the
pace or rhythm of the instruction. The amount of silence for
both experimental and quasi-experimental groups remained ap-
proximately the same for both lessons. Silence or confusion
in the classes of the control group increased by 5.9670 .
For the most part this increased silence in the control
group occurred in the 10-10 cell. In this cell the total se-
cond observation number of tallies was 121, an increase of 42
tallies and 3.437>. In other words, there was a marked in-
crease in extended (six seconds or more) silence in the con-
trol group. There was also an increase of 15 tallies in the
control group's 9-10 cell indicating more silence after a stu-
dent contribution or a change in student speakers.
76
The patterns for category 4, question-asking, were large-
ly the same for all three groups. As indicated in Table VIII
the total number of tallies during the second observation of
the experimental, control and quasi-experimental groups was
120, 118 and 120. The number of tallies and percentage of
instructor questioning decreased in all three cases: -89 tal-
lies experimental, -56 tallies control, -33 tallies quasi-
experimental .
Column 2, praise (indicating both extended general praise
as well as short encouragement and prompting) decreased in the
experimental group by 2.82%,, remained approximately the same
in the quasi-experimental group and increased by 1.23% in the
control group.
Flanders' special i/D Ratio, the relationship of indirect
teaching modes (columns 1 - 4) to total teacher talk (columns
1 - 7) is intended to trace instructor tendencies toward greater
or lesser "control" over the classroom. As described in the pre-
vious discussion the experimental and quasi-experimental goups
tended to become both more indirect and more direct: the over-
all result was a decrease in their i/D Ratio (-.130 experimen-
tal, -.005 quasi-experimental). The i/D Ratio for the control
group increased by .017.
The remaining categories of Table VIII which have not been
discussed (Column 1, Column 6, Column 7) and special measures
(Revised I/D Ratio, Extended Direct) do not offer very much
77
Information. Column 1 (accepts feeling) clearly showed differ-
ences among the groups (+1.54% experimental, +.7% control) but
the number of tallies in this category was minimal, including
none at all for the quasi-experimental group.
Columns 6 and 7 also accounted for a minor percent of the
total behavior observed: 25 tallies or .32% of the total 7,854
tallies of both observations for all groups fell into category
7. One hundred tallies or 1.237. fell into category 6, which was
also a special case because one continuous segment (59 tallies)
of one lesson concerned giving directions.
Since the Revised i/D Ratio has only categories 6 and 7
as the denominator and since Extended Direct is the total of
the 6-6, 6-7, 7-6 and 7-7 cells of the matrix, these measures
yielded little useful information.
The 17 x 17 interaction analysis matrix was designed to
identify teaching patterns in a classroom more specifically.
This was accomplished by adding to the 10 x 10 matrix seven
categories. However, category 4b (higher order questioning)
was not discriminated well enough for raters during their
training. Categories 3e and 10b contained almost no tallies.
The remaining four categories are also listed in Table VIII.
Category 5a (focusing, summarizing, introducing, orient-
ing) was differentiated from the general category of lectur-
ing. According to the data, the experimental and quasi-exper-
imental groups increased markedly in this category: 19 tallies
78
1*63% for experimental, 15 tallies 1.28% for quasi-experimen-
tal. The control group decreased by 12 tallies -1.07% in this
category. It is also noteworthy that the vast majority of
these orienting statements occurred in continuous sequence.
Of .the 89 tallies in 5a for the experimental group, 78 occurred
in the 5a-5a continuous cell; for the quasi-experimental group,
50 of 65 tallies occurred in the 5a-5a cell. Only 4 of the con-
trol group’s 10 tallies in 5a occurred in the 5a-5a cell.
Category 3a, indicating the very brief almost perfunctory
’yes," "very good," "right" kinds of teacher encouragement and
prompting of student contributions was separated from category
2, praise. At the second videotape observation the experimen-
tal group exhibited much less of this kind of behavior (-47
tallies, -2.70%). Both the control group and the quasi-experi-
mental groups however increased these behaviors: +15 tallies
1.25/o for control, +12 tallies 1.00a, for quasi-experimental.
In terms of category 3c, both the experimental and con-
trol groups decreased: -54 tallies -3.07% experimental, -23
tallies -2.11% control. The quasi-experimental group increas-
ed by 19 tallies +1.58%. In this data then there does not
appear to be a strong relationship between student-centered-
ness in terms of using student ideas (category 3) where the
experimental and quasi-experimental groups increased markedly
and student-centeredness in terms of probing or encountering
student ideas (category 3c), where only the quasi-experimental
79
group increased.
Category 3d, also separated from the general category
of questioning, showed a definite decline in the experimental
group (-11 tallies, -.64%) and a definite increase in the con-
trol group (+6 tallies, +.51%). The quasi-experimental group
for the most part did not exhibit redirecting behaviors.
As illustrated in Table IX, of sixteen categories and
special measures in the interaction analysis, half involved
the experimental and quasi-experimental groups changing in
one direction while the control group changed in the oppo-
site direction. Two showed all three groups moving in the
same direction but with the experimental and quasi-experimen-
tal groups moving much more decidedly in that direction. Five
involved some other combination.
The eight measures wherein the experimental and quasi-
experimental groups move in the same direction and the con-
trol group in the opposite direction are based on observational
categories which account for 79.84% of the total classroom be-
h^ivior fn both pretest and posttest videotapes of the combined
groups (columns 2, 5, 8, 9, 10). If the instance where both
experimental and quasi-experimental groups increase decidedly
more than the control group (column 3) is added, this figure
rises to 86.91%. In other words, if this 86.91% of the two
videotaped classroom lessons of the Rhetoric sample were view-
ed through these particular interaction analysis categories,
DIRECTION
OF
CHANGE
ON
STUDENT
EVALUATIONS
FOR
EACH
SUB-GROUP
80
5-1
CD
-CPo
r—l CO
cO a4-4 PC o04 Ci 0)
e o Ci•H oP r—
1
2CD cdCL p pX c pW pq
<U •H rO5-4 Ci r—
1
04 04 crC CL cd
& X r—
1
w •HCO 1 6c *r-l •H0 co CO•H cd
CO C3 0)
C O' COCD C6 X) cd•H c -Gn cd o
r—4 co acd CL p4-4 P oc o 5-4
CD p oe o•H rH5-4 r—
1
o04 cd CiCL p pX C cw 04 o
e oCD •H5-i Ci 04
04 04 pn CL •1—
1
X CO
w oCO 1 CLC •r-l CLO CO o•iH cd
CO P 04
c oi O004 Ge o 1 cd
C Gcd
| o
apopo
oppcoocCO
-GHIn
6-2
mJ- O mLO O r-t
• I T—
I
P t-4 Oa> co j_i
Q< fl 4J
X P cw o* ou
O VO NON IT) ,-1
• • •
\t cm cnI I
• I r—
4
P -P oCL) CO f—
i
CL cd 4J
x p cW O' ou
S-2O o LOo CM• • •
CM r—
1
1 1
1 + 4"
•i 1 1
Ci •H 004 CO CiCL cd pX P cw O'. o
oCO
OO
00 r-to U-N r-l• • •
CO r-l CM1 + 1
•1 r-l
Ci •rl O04 CO ClCL cd PX P cw O' o
ot
oCO
O r-lvoom• • •
• +
• I 1—
I
L 'H o0) CO LiH c0 uX P cW O*' o
o•
X)CO
6-2
O ON i—
1
«-! Mf O• • •
<t O H+ + +
4-)
O N vo rod) CO CO HCl r-l CM•H • • •
T3 CO <f HC + +
• I r-l
P *H Oa) co LICL cO -PX P CW O' O
oCO
_ • I oC L 'H La) 0) CO 4-1
X) CL Cd GC X p O04 W O'OpXw
vO(T"-
»H 00 rH CO rv.CM rH CO CM 140 ON CM uoCO CM CM • • • • c •
• • • CM 00 O co oCNJ 1 r-l rH + 1 i—
i
1 rH1 + + 1 +
6-2
UO ro VOS-200 00 r^ o LO r^o rH ON VD CM o OO o T—
1
• • • • • • r—l o O1 1 LO rH r-l rH • • 9
4* + + 1 i 1 +
• I r—
I
L 'rl o0) CO L4CL CO 4-1
X P cW O' O
oCM
• I r—
I
P -H OdJ co LCL (0 44x p eW O' o
uliO
9i rH
| i—
i
• 91 rH Ci •r-4 o Ci H oON Ci •r4 o 04 CO r-l 04 CO p
a) CO C4 CL cd p a ctf pX) CL cd P X P c X 2 cC X P c W O o w O' ocd w O' 0 u oO • •• O cd
co rH in
L t4 OCD co LiCL (0 44x p cW O' O
o Teacher
Talk
Exper.
+10.660
Quasi-
+3.359
Control
-10.217
DIRECTION
OF
CHANGE
ON
STUDENT
EVALUATIONS
FOR
EACH
SUB-GROUP
(continued)
81
3a)
rC+J
o
r—
1
CO
3 a4-) 3 a3 o 3(D 3 04 oE o 44 44•H O o3 r-4 X3 3 r—
1
p 4-* 4-1 0X c 3 44
w (14 P 4-1
E 3a> •pH O3 3 i—
1
Oa) 04 44
-3 a CO 3.X 1 1 3w •rH -3
3 1 E Eh3 •pH •Ho (44 CO•r4 (0 T—
f
CO 3 04 X)3 oi b0 04
3 3 PE X) cO 44
•r-4 C -3 3P to o p
r—H 3 a3 p 3+-> 3 oC O 304 3 oE O•H r—l
3 t-4 o0) 3 3a 4-» +J
X 3 3w 3 O
E a0) •pH
44 3 304 3 4-1
-3 a •r4
X 3w o
3 1 a3 •3 pO 3 o•H 3CO 3 33 oi oo04 3E X) 3•H 3 Pp 3 O
o•H4-4
3PPr-1
3 1—
1
pEh O CO cO
ro o t—
l
3 • • •
3 1 1 +PO • 1 i
—
i
3 3 •H o3 3 3 3JH P 3 4->
X 3 3• w O' O
X) O34-1
CO
82
m every category it would be observed that the experimental
and quasi -experimental groups changed, changed in the same
direction, and changed in contrast to the control group.
Flow Charts . A useful way of displaying interaction
analysis data is through flow charts. These are constructed
from interaction analysis matrices in the following way: (1)
a certain minimum number of tallies in each cell to be repre-
sented on the flow chart is chosen, (2) all "steady state"
cells (5-5, 4-4, 3-3, etc.) with this number of tallies or
more are represented on a chart in proportionately sized box-
es and all transition cells (4-8, 5-4, 3-4, etc.) with this
number of tallies or more are represented by connecting ar-
rows, and (3) all other "steady state" or transition cells
are added as necessary such that there is an arrow leading
to and exiting from each box. (For purposes of interpreta-
tion certain transition cells or steady state cells are en-
tered which have a total number of tallies below the es-
tablished minimum number.)
The minimum number of tallies chosen for this interpre-
tation was 20. The number of tallies in each flow chart en-
tered through these procedures is approximately 80% of the
total tallies recorded for the same lesson on the 10 x 10
matrix. Thus what these flow charts represent are the most
typical patterns of instructional behavior that one would ob-
serve in these classrooms eighty percent of the time.
83
The pretest flow chart for the experimental group (Fi-
gure 11) depicts large amounts of lecturing (5-5), student
initiated talk (9-9) and student responding (8-8). The amount
of teacher question-asking (4-4) and the amount of using stu-
dent ideas (3-3) are approximately equal.
The general "flow" of the instruction is to and from the
teacher’s question-asking (4-4). Thus the transitions from
lecturing to question-asking (5-4),silence to question-asking
(10-4), student initiated talk to question-asking (9-4), stu-
dent i esponding to question-asking (8-4) and teacher praise
(2-2) to question-asking (2-4) are all solid lines on the flow
chart (twenty or more tallies). This indicates a more or less
teacher-centered mode, perhaps recitational . There is also a
strong student-centered tendency, as evidenced in the relative-
ly large number of using student ideas (box 3-3).
The most typical instructional sequence is from question-
asking to student responding to short praise to question-asking.
After treatment the experimental group (Figure 12) showed
a tendency to become more teacher-centered and more student-
centered, a result which Flanders terms "flexibility" or the
tendency of teachers to alternate their basic instructional
modes in response to different instructional situations (Flan-
ders, 1970).
Lecturing for the experimental group increased by 154 tal-
lies or approximately 507c The use of student ideas increased
EXPERIMENTAL
GROUP
PRETEST
FLOW
CHART
84
Figure
11.
Experimental
Group
Pretest
Flow
Chart
EXPERIMENTAL
GROUP
POSTTEST
FLOW
CHART
85
Figure
12.
Experimental
Group
Posttest
Flow
Chart
86
57 tallies or 70%. Whereas the instructional patterns in the
experimental group's pretest flowed toward teacher question-
asking from five different directions, in the posttest this
flow was from only two directions (lecturing to question-asking,
and using student ideas to question-asking). The instructional
patterns in the experimental group's posttest flowed mainly
toward the 3-3 box, clarifying and using student ideas.
The most typical instructional sequence in the posttest
was from question-asking (4-4) to silence (10-10) to student
responding (8-8) then equally to short praise (2-2) and to
using student ideas (3-3). The tallies in the 2-2 box more-
over tended to flow to 3-3. Student talk decreased by approx-
imately 307o .
The quasi -experimental group's pretest flow chart (Fi-
gure 13) is characterized by large amounts of lecturing (5-5)
and student initiated talk (9-9), very similar to the experi-
mental group. The amount of student response in this group
was much less than that of the experimental group (77 tallies)
and the amount of continuous teacher question-asking (4-4) was
slightly greater.
The s tudent-centeredness (3-3) of the quasi-experimental
pretest group was very little (16 tallies) compared to the pre-
test 85 tallies of the experimental group. The quasi-experimen-
tal group also contained a pretest lesson involving prolonged
direction giving .
QUASI
-EXPERIMENTAL
GROUP
PRETEST
FLOW
CHART
87
Figure
13.
Quasi-experimental
Group
Pretest
Flow
Chart
88
The primary flow of the quasi-experimental group's in-
struction in the pretest was in two directions: toward teach-
er lecturing (48 tallies from four directions) and toward si-
lence (54 tallies from four directions). The most typical
instructional sequence was from question-asking (4-4) to si-
lence (10-10) to lecturing (5-5) to silence (10-10) to ques-
tion-asking (4-4).
In the quasi-experimental group's posttest (Figure 14)
the primary instructional pattern was from question-asking
to silence to lecturing to silence--almost exactly the same
as in the pretest. The shift that occurred for the experi-
mental group toward student-centeredness (3-3) as well as
teacher-centeredness (5-5) was approximated but to a much
lesser degree by the quasi-experimental group.
The control group pretest (Figure 15) was similar to the
other two groups in that lecturing (5-5) and student respond-
ing (8-8) were relatively large categories. The control group
however was characterized in the pretest by more lecturing
(400 tallies) than the other two groups and a lesser amount
of total student talk (206 tallies). There was more silence
in the control group, roughly twice as much as the experimen-
tal group. The control group was also similar to the quasi-
experimental group in terms of the small amount of using stu-
dent ideas (3-3) displayed.
QUASI
-EXPERIMENTAL
GROUP
POSTTEST
FLOW
CHART
89
o CNJ
Figure
14.
Quasi-Experimental
Group
Posttest
Flow
Chart
CONTROL
GROUP
PRETEST
FLOW
CHART
90
I
Figure
15.
Control
Group
Pretest
Flow
Chart
91
The general flow of the instructional patterns in the
control group's pretest videotape as in the experimental
group's, was toward question-asking (76 tallies from four
directions;. There were two sequences that were dominant:
question-asking to student response to question-asking and
question-asking to silence to lecturing to question-asking.
The control group posttest showed a marked drop in
question-asking, which was also true for the experimental
and quasi-experimental groups. There was also a marked de-
crease in lecturing, which ran counter to these other two
8r<-)UPs • Also in contrast was the control group's increase
in total student talk and silence.
The dominant flow of the control group's posttest in-
struction was toward silence (10-10), 69 tallies from four
directions
.
The most typical instructional sequence was from ques-
tion-asking to silence to student initiation to silence--
and then either back to question-asking or to lecturing.
The instructional pattern shift then in the control group was
toward the strengthening of the silence-student initiation,
student initiation-silence sequences, with the intervals of
student initiated talk and silence being much longer.
Significance of Changes . The use of statistics to es-
timate the significance of the changes in instructional be-
havior discussed above was judged to be inappropriate to
IONTROL
GROUP
POSTTEST
FLOW
CHART
o
Figure
16.
Control
Group
Posttest
Flow
Chart
93
the present study primarily because the sample size, thirteen,
broken up into three groups was exceedingly small.
According to Flanders (1970),who has conducted studies
of a similar nature with larger numbers of teachers, the in-
creased or decreased behavior in any one category does not
need to be large in order to be significant:
It is important to place these changes in teach-ing behavior into a reasonable perspective. Theincreased use of Category 3, for example, is on-ly a matter of 4 or 5 percent. A control groupmight average around 8 percent Category 3, whilethose who study interaction analysis are morelikely to average from 3 to 5 points higher (p. 358).
In which case several of the instructional behavior
changes revealed in the present study might be considered
significant changes, including for example Category 3 in the
experimental and quasi-experimental groups.
Inter -Rater Reliability . Rating reliability among the
four interaction analysis raters was estimated by the Scott
coefficient reported by Flanders (1967). One fifteen minute
videotaped teaching segment was scored by all raters and the
scores compared.
The resulting reliability estimates were extremely
varied, ranging from a low of .4657 (Rater 1 and Rater 4)
to a high of .7808 (Rater 2 and Rater 4). The test case re-
ported by Flanders (1967, p. 163) indicated a reliability of
.8899.
Table X represents the four raters’ scores on the rater
reliability tape in each of the categories of the 10-category
system. It is apparent from this table that the greatest part
of the discrepancy among the raters occurred in differentiat-
ing "student talk - response" from "student talk - initiation."
The difference between a student "initiation" where the
student's ideas, views or information go beyond what was re-
quested by the instructor--and a student "response," can be
a very difficult distinction to make on a videotape recording,
especially where the microphone is not aimed at student speak-
ers .
If categories 8 and 9 were considered as one category,
the racer reliability would be much greater.
Student Evaluations
A thirty-one item questionnaire asking students to eval-
uate their instructors on a number of technical teaching skills
was administered twice to all groups.
Pretest and posttest comparisons between items were aided
by t-tests. Comparisons were also made between clusters of
items suggested by factor analysis. (Detailed representation
of this data appears in Appendix C).
Table XI contains a summary of these comparisons. As
indicated, only a few of the thirty-one questionnaire items
or ten special factors showed any real change. The disparity
between the three groups in terms of the items that did appear
95
TABLE X
RATER SCORES FOR ONE FIFTEEN -MINUTE LESSON
USING THE FLANDERS TEN-CATEGORY SYSTEM
Category Rater 1 Rater 2 Rater 3 Rater
1. Accepts Feeling 0 0 0 0
2. Praises or Encourages 7 20 12 19
3. Uses Student Ideas 20 18 15 15
4. Asks Questions 37 28 32 35
5. Lecturing 44 54 47 .44
6. Giving Directions 0 6 0 3
7. Criticizing 0 0 0 0
8. Student Talk-Response 153 105 49 72
9. Student Talk-Initiation 31 83 128 92
10. Silence or Confusion 18 11 17 18
TOTALS 310 325 300 298
96
TABLE XI
COMPARISON OF STUDENT EVALUATION PRETEST AND POSTTEST SCORES
FOR INDIVIDUAL ITEMS AND FACTORS
EXPERIMENTAL QUASI-EXPERIMENTAL CONTROL
Var
.
Diff. in
MeanT-Value Diff. in
MeanT-Value Diff. in
MeanT-Value
1 . +.2076 1.32 .00 .00 +.1118 .80
2. -.0433 -.35 -.0945 -.69 +.1212 .80
3. +.1407 1.05 +.1258 .70 -.1852 •1.19
4. -.0744 -.54 +.0322 .27 +.2058 1.29
5. +.2393 1.61 +.0570 .42 -.0564 -.36
6 . -.0038 -.03 -.2778 -1.77 +.1673 1.26
7. +.2771 1.61 +.0983 .54 -.0104 -.06
8. +.1516 .90 -.1293 -.70 +.0357 .19
9. -.1268 -1.31 -.2916 -1.97* +.1964 1.47
10. +.0388 .34 -.0763 -.47 +.0687 .60
11. -.1245 -1.05 -.0782 -.53 +.0160 .10
12. -.1189 -1.44 -.2684 -2.11* +.0771 .68
13. +.1697 1.23 -.1242 -.71 +.0291 .22
14. +.3328 1.97* +.0638 .28 -.0714 -.52
15. +.1542 .73 -.2703 -1.10 -.0808 -.35
16. +.2607 1.87 +.0960 .56 -.2707 •-1.60
17. -.0648 -.50 -.2128 -1.27 +.1964 1.47
18. +.1021 .40 -.2231 -.93 +.2030 .88
19. -.0761 -.71 +.0539 .39 +.1429 .79
significant at the .05 level
97
TABLE XI (continued)
COMPARISON OF STUDENT EVALUATION PRETEST AND POSTTEST SCORES
FOR INDIVIDUAL ITEMS AND FACTORS
EXPERIMENTAL QUAS I -EXPERIMENTAL CONTROLVar
.
Diff. in
MeanT-Value Diff. in
MeanT-Value Diff. in T
Mean-Value
20. +.0040 .05 -.2785 -2.51* +.0075 .05
21. -.0011 -.01 +.0598 .43 +.1268 .93
22. -.0763 -.78 -.0501 -.42 +.1626 1.18
23. +.4083 2.74* -.3352 -1.83 +.1701 1.07
24. +.0180 .16 -.0336 -.19 -.1240 -.93
25. +.0561 .42 -.0952 -.58 -.2589 -.93
26. -.0522 -.52 -.1389 -.86 +.2133 1.51
27. +.0353 .34 -.1326 -.94 -.0056 -.04
28. +.4055 2 . 70* -.0941 -.54 +.3891 2.19*
29. +.2341 1.59 +.1405 .96 +.0216 .13
30. +.1830 1.06 -.5084 -2.04* -.2396 1.25
31. -.0098 -.05 -.1715 -.75 +.2265 1.16
FI +.0860 1.53 -.1021 -1.16 +.0558 .63
I’2 +.1335 1.55 -.0137 -.13 +.0381 .37
F3 -.0399 -.51 -.1270 -1.17 +.0547 .47
F4 +.1743 1.91 +.0497 .48 -.0504 -.45
F5 + .2744 2.38* -.1534 -1.14 + .0571 .41
F6 +.1780 1.82 -.2370 -1.74 +.1918 1.53
F7 +.0457 .50 -.1139 -.91 -.0822 -.64
98
TABLE XI (continued)
COMPARISON OF STUDENT EVALUATION PRETEST AND POSTTEST SCORES
FOR INDIVIDUAL ITEMS AND FACTORS
Var. Diff. in
MeanT-Value Diff. in
MeanT-Value Diff. in
MeanT-Value
F8 -.0373 -.55 -.0538 -.58 +.1100 .92
F9 +.0999 1.28 -.1294 -1.05 +.0401 .43
F10 +.0980 .90 -.0584 -.41 -.0372 -.31
to change is difficult to interpret. All of the notable differ-
ences that occurred for the quasi-experimental group were nega-
tive. No two of these negative items appear in any one factor
however, so there is no evident "pattern" of change. Items 12
and 20 might be viewed in terms of the instructors' relating to
students and their openness, keying from the words "feel free,"
"invites," and "share" (see Table XII, for a list of apparent
change items in the quasi-experimental group). These data point
perhaps to increased restrictiveness in this group's classrooms.
Items 6 and 23 point to poor performance in terms of "clarity."
Item 9 concerns presentation style and item seventeen refers to
the curriculum design.
The remaining student evaluation items for this group
show no change. Twenty-one of the thirty-one student evaluation
items and all seven factors showed very slight decline in the
ratings
.
TABLE XII
QUAS I -EXPERIMENTAL GROUP STUDENT EVALUATION ITEMS AND FACTORS
INDICATING CHANGE
"The instructor knows when to elaborate on a topic and he /shee laborates effectively by using examples, pointing out rela-tionships, and/or giving more detailed explanations."Mean: -.2778 T-Value: -1.77
The instructor has an effective presentation style (e.g. voicequality, choice of words, body movements, etc.)."Mean: -.2916 T-Value: -1.97*
"The instructor makes students feel free to disagree with him/her,with fellow students or with the readings."Mean: -.2684 T-Value: -2.11*
"The level of difficulty and the amount of work in this courseare about right for me."Mean: -.2128 T-Value: -1.27
The instructor invites students to share their knowledge, experiences
,and opinions at appropriate times."
Mean: -.2785 T-Value : -2.51*
"The instructor restates students' questions for clarificationand answers them in a way that the whole class understands."Mean : -.3352 T-Value: -1.83
"The instructor keeps the students posted on their progressthrough questions in class, individual conferences, and appro-priate quizzes and tests."Mean : -.5084 T-Value : -2.04*
"Relating to Student Responses (items 23 and 26)"Mean: -.2370 T-Value: -1.74
100
The control group showed noticeable change in five items and
one factor (Table XIII).
Patterns in this data however are also difficult to de-
lineate. Item 4 might be interpreted with item 9 to suggest
a general improvement in the instructors' classroom presenta-
tions. The positive direction of item 17 is offset by the
negative direction of item 16, both items referring to course
design. Item 26 points to a more positive relating to student
responses and item 28 suggests improved instructor procedures
for evaluating students. None of these items form a clear
trend with their related items as indicated by the low factor
scores throughout the control group. To some extent Factor
6, "relating to student responses (items 23 and 26)" is an
exception, but not a definitive one.
The experimental group showed marked change in ten in-
dividual items and three factors (Table XIV). Two of these
items showed change in a negative direction (items 9 and
12 ).
The three factors ("evaluation and feedback," "clarity,"
and "relating to student responses") were especially positive,
indicating that the experimental group instructors had improved
in such specific ways as making expectations clear for the
course and for individual lessons, evaluating clearly and con-
sistently, explaining and summarizing ideas, and restating,
101
TABLE XIII
CONTROL GROUP STUDENT EVALUATION ITEMS AND FACTORS
INDICATING CHANGE
4. "The instructor arranges and presents material in a clearwell-organized fashion."Mean : +.2058 T-Value : 1.29
9. The instructor has an effective presentation style (e.g.voice quality, choice of words, body movements, etc.)."Mean : +.1964 T-Value : 1.47
16. "The general design of this instructor's course (number andchoice of lectures, discussions, reading, papers, examina-tions) is appropriate."Mean : -.2707 T-Value : -1.60
17. "The level of difficulty and the amount of work in this courseare about right for me."Mean : +.1964 T-Value : 1.51
26. "The instructor answers questions, or encourages and helpsothers to answer them, with understanding and clarity."Mean : +.2133 T-Value : 1.51
28. "The instructor makes clear to students what is expected ofthem and how their performances are to be evaluated."Mean: +.3891 T-Value: 2.19
F6 "Relating to Student Responses" (items 23 and 26)Mean: +.1918 T-Value: 1.53
102
TABLE XIV
EXPERIMENTAL GROUP STUDENT EVALUATION ITEMS AND FACTORS
INDICATING CHANGE
1. "When beginning a class, lesson, or new instructional ac-tivity, the instructor usually makes clear what he/she hopesto accomplish."Mean: +.2076 T-Value : 1.32
l-
1
"1 this instructor's class it is usually possible to know
which points are most important, and to see the relation-ships between topics."Mean : +.2393 T-Value : 1.61
7. "The instructor provides useful summaries at appropriatetimes, (e.g. at end of class session or end of unit).Mean : +.2771 T-Value : 1.61
9. "The instructor has an effective presentation style (e.g.voice quality, choice of words, body movements, etc.)Mean : -.1268 T-Value : -1.31
12. The instructor makes students feel free to disagree withhim/her, with fellow students, or with the readings."Mean : -.1189 T-Value : -1.44
14. "The instructor relates the topics to a wide range offields, situations, applications, and interests."Mean : +.3328 T-Value : 1.97*
16. "The general design of this instructor's course (number andchoice of lectures, discussions, readings, papers, examina-tions) is appropriate."Mean : +.2607 T-Value: 1.87
23. "The instructor restates students' questions for clarifi-cation and answers them in a way that the whole class under-stands."Mean : +.4083 T-Value : 2.74*
28. "The instructor makes clear to students what is expected of
them and how their performances are to be evaluated."Mean: +.4055 T-Value: 2.70
103
TABLE XIV
EXPERIMENTAL GROUP STUDENT EVALUATION ITEMS AND FACTORS
INDICATING CHANGE (continued)
29. "The instructor evaluates student work in a clear and consis tent way."Mean: +.2341 T -Value : 1.59
F4 "Clarity (items 1, 5, 6, 7, 8)"Mean : +.1743 T -Value : 1.91
F5 "Evaluation and Feedback (items 28, 29,30)"Mean : +.2744 T -Value : 2.38*
F6 "Relating to Student Responses (items 23 and 26)"Mean: +.1780 T-Value: 1.82
104
clarifying and answering student questions. The overall change
factor for the experimental group (FI) was decidedly higher than
that of the control group and quasi-experlmental group.
Student Achievement
All groups in the study completed student achievement
tests. Two forms of the test were used, each one given to half
the students as a pretest and reversed as a posttest. The in-
tervening period between pretest and posttest was approximately
eight weeks. The tests were rated by three subject matter spe-
cialists according to three dimensions: "literacy," "focus,"
and "rhetorical felicity."
The results of these ratings (Table XV) indicate that none
of the groups underwent dramatic changes in achievement. It is
notable however that the average student achievement scores of
the experimental group gained 3.33 points. The scores for the
quasi -experimental group remained roughly the same. The scores
or the control group decreased by 2 points.
The achievement scores of three of the five classes in
the experimental group advanced. Three of four classes in the
quasi -experimental group advanced. Two of four classes in the
control group advanced.
Six classes received test "x" first and seven received
test "o" first. Four of the six test x - test o sequence classes
improved their achievement ratings by the second test. Four of
the seven test o - test x sequence group improved their achieve-
105
TABLE XV
STUDENT ACHIEVEMENT SCORES
EXPERIMENTAL GROUP:
pretest:
post test:
Mean
79.933
83.266
Standard Deviation
10.363
5.861
T-Value
1.34
QUAS I -EXPERIMENTALGROUP Mean Standard Deviation T-Value
pretest
:
73.750 9.725 .06
posttest: 73.583 14.061
CONTROL GROUP Mean Standard Deviation T-Value
pretest: 82.417 7.948 .46
posttest
:
80.417 15.442
106
ment ratings. The mean score for the two tests across the
entire sample differed by 3 points.
Correspondence Among Measures
The instructional problems (defined in terms of techni-
cal teaching skills) identified by the experimental and quasi-
experimental groups after the initial data analysis and feed-
back process centered on issues of "clarity," "student parti-
cipation" and "evaluation/feedback." The specific goals of
these instructors concerned improving questioning skills,
eliminating vagueness, focusing, clarifying goals and stan-
dards, summarizing and reinforcing (Table XVI).
The factors from the student evaluations in which the
experimental group showed marked improvement were "clarity,"
"relating to student responses" and "evaluation and feedback."
These factors included specific questions concerning clarify-
ing lesson objectives to students, organizing ideas, summariz-
ing, restating and clarifying student questions, making expec-
tations clear and evaluating student work.
The instructional behaviors which showed marked changes
in the experimental group were an increase in lecturing, an
increase in orienting statements and an increase in accepting
and using student ideas.
There would appear to be a logical relationship between
the improvement goals of the experimental group, the results
of the student evaluations and the observed instructional be-
INSTRUCTIONAL
PROBLEMS
AND
IMPROVEMENT
GOALS
107
COfdZDoPio
<H2:
HPiPiP-I
XwI
t—
I
COcZDO'
3H2
PiwP-I
xwPio
c0Oo4-1
c<0
E<0
>Odae
1—
1
CO
Eco
T“—
i
-CO
odPi
00d•Hco•pH
4J
CO
a>
z>
c
r
c•r4
CO
co
a)
d<u
z>
00cO
>
<mo
4-1
edoEcO
a)
pp4-1
CD
0dX<0
Pi
co•H4-1
co
a.•HO•r4
4-1
dcO
cp
4-1
cco
X)p4-1
CO
<Mo
PiucO
hJ
00a•pH
d34-1
O3d4-1
CO
X)ccO
CO
CO
CO
cCO
>•H4-1
CJ
<0
J-l
d(0
rC0cO
<04-J
CO
CO
cO
<0
dOd
ec#s
001 4J
d 40 c•H O (L>CO
d X e<D0 d O0 d
<4-1 Oco M-lX r—
H
£40 d •pH
-XO Q)60 U
*--l
d 60 M-l4J d 04J
CCO
X)p4-1
CO
X)(0
XIc(0
4-1
XCO
CO
opX)(0
Pi
do•r4
4-1
CJ
<0
d•pH
xi
4-1
4-1
•pH
x)cO
PP
X<4-1
CO
doE
X) CO
C a)
<0 >a. -HC/0 4J
44 (0 CO•i-4 2 CO
M J4 OcO
co 00o dd *r4
CO 4J
ddPU *r4
O XO <0
d(0
jd x4J d
co
co
co 00d cCO *r-4
d PD0 od d1-1 CP
0co •pH4J 4-1
d <M d<0 O dX r—
4
d CO d4J +J >CO r—
1
co
d<4-1 CO XO CO d
d d4J
d X Jsid d CJCP d d
40CO CO XPP CO CO4-1 0 CO
CP <Md d0 d CO COa d >
0 •pH
4J
CJ
CO
d r—
1
00 <M CO0 0 d E CO CO•r4 •pH 0 dCO <M d O <4-1 dCO O d d O Xd (0 co a>0 -X O co -X CJCO O d d O 0•r4 d 0 r—
H
d dO 2 0 0 a
<4-1
o
<0
-d4J
CO
CO
dco
dCJ
d1—
1
X)d dd *r4
d<0 COCO 4-1
d *r4
P dO 0CJ
d00 Od *H•H 4-1
X d<+4 P
d d<4-4
d •pH Ed d > cr dCO
401-4
0CO O
COd
r—Hco
NE X 4-1 d 0 •pH
d CO d d 0 dE•r4
4-1
(0
dOE
X!d
CO
4-1
dco
Eco
d•r4
P<0 crQ. 0)
CO d
00d•pH
do
CO
<0
pc
r
co
co
CO
dCO
p00d>
00d
•pH
do
<J
co
1
-OO
X)dd
do00
00d
•pH
x
d<0
4J<4-1
o
co
dO
OEd
co
PP
CO
CJ
pXICO
Pi
CO
CP(0
d
CO
upX)CO
Pi
CO
doE
xi co
d co
<0 >CP «HCO 4-1
pco
X)dd
CO
d0oIP
a>4-1
d <4-i
<0 oX)d co4J 4J
CO 1—
I
d<4-4 CO
O CO
•pH 4-1
+j d Xd d dip
•pHcp d
0 CO co•pH PP co4-1 4J 0d CPd d d(P 0 d CD
CP, d4-1 >> 0d 4J CO CO(0 •H X COX d 4-1 <0d d r—H+j T—
1
COC/D 0 d E
•rH 0<H <4-4 e 0O 0 d d
CO CO
Pi p: <j COCJ 0 e dd d 0 pH
P 0 0
co
CP
d -C0 X CO•H f—
4
dCO CO 0CO 4-1 •r-4
d d 4JCJ d dCO CO i-4•H 40 (0X •r4 d
.—4
CO CO XCO X dd d
r—H <0
0 d co
0 4-1
60 E dd •rH
•pH CO 0d d ad 0X •H 4-4
4J d44 d d0 4-1 -1d d d40 (0 OX CO CP(0 <0 ECO d •pH
<M CP(0
CO a> Nd N •pH
0 •,-4 co
E d dd 2
CO 60 aCO d EZD O w
do•pH
4-1
ddr—
I
d>co
Xdd
-XCJ
d40XCO
CO
CO
>•r4
4-1
oCO
<4-4
<4-| WCO co
d<4-4 dO X
CO
PJ CJ
0 Od dhJ CP
3-0030 TVLNaHIR3TXaanoao
TVLN3RIH3JX3-
1
SV06
CORRESPONDANCE
BETWEEN
IMPROVEMENT
GOALS,
BEHAVIOR
CHANGES,
AND
STUDENT
EVALUATION
SCORES
,
(EXPERIMENTAL
.
GROUP)
.
Improvement
Goals
Behavior
Changes
Student
Evaluation
Scores
108
CO
-r-'.
J-i
O ~P MOO
fcl LO
C -•pH r—
I
a) co
oo gd qj
cd ppd 'HO WQJ
> P•H *Hp p•pH C£j
W rHo aP-4 Z
CD
ep
- pm *h
po ^p oo cd
cd ,adt) d
pC Q)
>H pQJ d00 Cd Cd
cd
-C c =o o o
>H rnQJ P> cd
•H dffiP H CM*H Cd
to > •>
O QJ COP-f C CM-
CO
QJ
co
cO
p cxo CO
P <u
o «ip p
dC QJ
•hpi VO
QJ P CM00 C/d
C dcd o cjp p cd
ooo co
QJ C cm> -ft
CO QJ
O PiCP =
pf N
o 00 ^—
*
Ip d cd CO <ro •pH LTd dd cn p o rMp d >"> dp r^l QJ P d 0)CO P •pH o 'p Od 0 p oo p '—•H oo o QJ d
0) P 0 ood p d cd o d•H cd •H o •H
a)
u' /
QJ
v—
X
inCO
CO CD co cdcd 00 cd p c i
QJ d QJ d o dp P QJ •H oo p o E P •Hd d d QJ O P•H p H P d CO
o cd d C1J
d QJ d p QJ d<3 rH <3 CO Pi cr
po oo 00p d <d d CDCJ •pH m ctfd LO p p a)p d (X adp QJ p QJ •pHco p •pH 0 od 0 p oo o 4-J
•rH 00 o QJ cd cQJ P a)
d P d cd d Td•H cd •p o •p p
o +JQJ M-/ p a) CO OOco co CO CDcd 00 cd p cd 00 >.QJ d a) d 0) c PP •H p QJ p •H 0O p o E C.J CO 00d p d QJ d d QJp •H P •H po cd d cdd QJ d p d d u<! r—
1
<3 CO < cd
OOd•HCO
doopPOPodPpCO
C•H
QJ
CD
cd
QJ
pod
cd a;
i•pH
i1
B 00 p 00 1
PP «4_j
re
P d p d p P oco •p 00 •pH p •pH T)
cd •pH p 0) cp cp > cp P o o cdo 'p •p cr dP co MP p p d a ocP cd qj o oo cd p 0 u0) rP > d i“—
1
•pH •pH,n O *P p •pH o d p 3B P d d o cd o o
00 P CJ O P 0 p CO p od d B QJ 0 •pH 6 CO rP o a,•rH •p •<—
>
B p •rH p cdp aj P £> cd CO p 1—
1
> od O p P 4-Jp p P QJ p p d +J cu P d -d cr p d d Q) 00d QJ 9 C P o cd d (U gp co E cd d B cd w a) •pHp cd CO QJ •p p cd o 4JCO QJ QJ d co O d CO co 0) h oP •pH d *P P CO d p p 0d o p QJ cd d CO p p d o M-Hd d cd CX 0 p p cx o p c c •pHcd M B CO 00 Pi d CO o E M •p^ d
(items
109
havior changes of these instructors. This relationship is illus-
trated in Table XVII.
In other words, the instructors in the experimental group
appeared to change the instructional behavior which they in-
tended to change as judged by an analysis of pretest and post-
test videotaped lessons, and these changes appear to have been
lecognized and reported by the students through the student
evaluation items.
no
chapter v
DISCUSSION AND CONCLUSIONS
Purpose and Procedures
The purpose of this study was to test a particular model
for the improvement of instruction at the higher education level.
This model was characterized both by its focus on evaluating and
changing a discrete number of technical skills of teaching (ques-
tion-asking, elaboration, pacing, etc.), and its data collection,
analysis and treatment procedures. These included an initial
interview with the instructor, classroom videotaping, student
evaluations, an instructional specialist working individually with
the instructor, and various follow-up procedures.
A gi oup of thirteen graduate student teaching assistants,
all of whom were teaching a required freshman Rhetoric course
were randomly divided into three groups: experimental, quasi-
experimentai and control. The first group completed each phase
oi the instructional improvement model including follow-up pro-
cedures occupying approximately ten hours time. The second group
completed onJy the first phase of this model, ending with the
identification of teaching skills which could be improved. The
third or "control" group completed only the data collection steps
in the model.
All three groups also administered a specially devised stu-
dent achievement test to their students during the fourth week of
Ill
the semester, while the initial data collection was taking place.
A final student achievement test, videotape and student evalua-
tion were completed approximately eight weeks later, at the con-
clusion of the semester. The class size, course goals and stu-
dent assignments were similar across all sections of the course.
The pretest and posttest videotapes were analyzed and com-
pared by independent raters through a seventeen-category inter-
action analysis instrument adapted from Flanders (1971). The
achievement tests were analyzed and compared by three subject
matter specialists using a rating system constructed by the course
director. Student evaluations were computed in terms of fre-
quency counts and mean scores for each item and for ten general
factors. Changes in achievement test scores and student evalua-
tion items and factors were computed statistically by means of
t-tests. Changes in classroom instructional behavior as identi-
fied through interaction analysis were interpreted from the in-
teraction analysis matrices and flow charts representing these
matrices.
The major assumptions of the study were that the analysis
of instruction by higher education instructors using the focus
and procedures of this improvement model would result in: (1) the
identification and alteration of "problematic" teaching skills,
(2) positive change in related student evaluation scores and
(3) greater achievement as measured on pre- and post- achievement
tests
.
112
It was also assumed that instructors not exposed to the
analysis and feedback aspects of this model would not manifest
these results (or would not manifest them to the same extent).
Finally, it was assumed that instructors who were exposed only
to initial analysis and feedback without follow-up procedures
either would not manifest these results or would manifest them
to a lesser extent.
Summary of Results
Change in Instructional Behavior . The results of analyz-
ing the pretest and posttest videotapes of all three groups re-
vealed that the experimental group's general instructional pat-
tern changed from a heavy emphasis on instructor question-asking
with extended student responding and initiating to one with a
heavy emphasis on the instructor's expanding and clarifying stu-
dent ideas with extended instructor lecturing. This might be
described as an increase in "flexibility" (Flanders, 1970).
The control group's general instructional pattern shifted
from a heavy emphasis on question-asking with extended instructor
lecturing to heavy emphasis on silence with extended student talk
evolving out of that silence. This might be described not as a
trend toward student-centeredness on the part of these instruc-
tors but rather a general lessening of their overt involvement in
the class.
The quasi -experimental group shifted slightly from a gen-
eral pattern emphasizing lecturing and question-asking and si-
113
lence to one emphasizing lecturing, question-asking, student re-
sponding and silence.
Flow charts of the overall instructional patterns in the
three groups illustrated that the quasi-experimental group shifted
much less than either the control or experimental group, and that
the control group appeared to shift less than the experimental
group.
In general the two experimental groups changed in the same
direction and changed in contrast to the control group. For ex-
ample they did more clarifying and expanding of student ideas.
They lectured more and made more "orienting statements." They
also showed less silence in their classrooms, in which the ratio
of student talk to teacher talk decreased.
The extended treatment group changed more than the limited
treatment group in each of these change dimensions.
Changes in o tudent. Evaluations . Of thirty-one items on the
student evaluations, three showed positive significant change in
the experimental group; three showed significant negative change
in the quasi-experimental group and one showed significant posi-
tive change in the control group.
The experimental group improved in "defining and expanding
relevance," in "relating to student ideas" and in "making expec-
tations clear."
The quasi-experimental group regressed in "expression,"
"facilitating student participation" and in "evaluation."
114
The control group improved in "making expectations clear."
Only one general factor for any of the groups changed signifi-
cantly. This was the "evaluation and feedback" factor in the ex-
perimental group. Two other factors in that group ("clarity" and
"relating to student responses") were very close to significant
change
.
Changes in Achievement . Although none of the differences
between pretest and posttest achievement scores were significant,
it is noteworthy that the mean achievement score of the experimen-
tal group students gained 3.33 points while that of the quasi-ex-
perimental group students did not change and that of the control
group students decreased by 2 points. The scores were a sum of
three independent scores representing a rating of the mechanics
of writing, clarity of purpose and rhetorical ability.
Conclusions
Despite numerous methodological problems and limitations
discussed below, it may be said that the results of this study
tentatively support the assumptions. The classroom instruction-
al behavior,student evaluat ions and student achievement of the
instructors in the experimental group did appear to change in
desirable ways, whereas the classroom behavior of the instructors
in the quasi-experimental and control groups changed less, and
their student evaluations and student achievement for the most
part did not change in desirable ways.
The fact that the observed instructional behavior changes
115
and the student evaluation changes of the instructors in the ex-
perimental group tend to correspond with their stated problems
and improvement goals suggests that the behavior changes in this
group were intentional and were related to the treatment process.
The follow-up treatment moreover appears to have been im-
portant in that the quasi-experimental group altered its behavior
in the same directions as the experimental group (having expressed
similar change goals) but did not change to the same extent.
Finally,the achievement data tentatively support the con-
clusion that clarifying and expanding student ideas, making orient
ing statements and lecturing are critical variables in the in-
struction of this particular kind of course. Instructors who
manifested more of these behaviors from their first to their se-
cond videotape tended to be rated higher by students and to gen-
erate higher student achievement.
One interpretation of the control group's videotape post-
test results is that the instructors in the control group shift-
ed their teaching style to one that relied much more on student
initiation. Given the extremely long periods of silence that
characterized these posttest videotapes however it seems probable
that the students found this role difficult. A content analysis
of these lessons might have indicated a lack of purposefulness
and direction in the student talk.
The increased active participation of the instructors in the
experimental group both through lecturing and through clarifying
116
and/or expanding on student contributions suggests that this
group identified instructional goals and learned certain instruc-
tional behaviors through their involvement in the treatment pro-
cess which related to improving their presentations and increas-
ing their "student-centeredness." The sharper dichotomy between'
teacher-centered and student-centered instructional modes for the
experimental group indicates both greater perceptiveness and grea-
ter flexibility on the part of these instructors in choosing the
appropriate instructional mode to fit different contexts.
Limitations
Although the results suggest that this instructional improve-
ment model is effective in promoting valuable instructional change,
this conclusion must be qualified in numerous ways.
In the first place, since this mode of treatment was the on-
ly mode tested in this design, then it is possible to conclude
not this particular treatment but any systematic in—servic.e
educational treatment of higher education instructors will result
in productive behavior change.
More specifically, Flanders (1970) and others (Amidon and
Hough,1967) have achieved similar results with primary through
secondary school instructors without the use of videotape feed-
back and student evaluations. Their instructors, trained in
interaction analysis, tended to become both more distinctly tea-
cher-centered and student-centered ("flexible") in the classroom,
and their students showed corresponding gains in achievement.
117
This suggests two hypotheses. First, since interaction
analysis was used in this study as a feedback procedure by at
least two of the three instructional specialists, then exposure
to interaction analysis may have been the critical variable in
the change process — videotaping, student evaluations, etc.,
being superfluous. Second, since two different instructional
analysis procedures tended to produce similar results, then
the critical variable for change may lie in the very act of in-
structors observing, analyzing and attempting to improve their
teaching -- a question of "mind set" -- and not in a particular
improvement model.
In addition, because few situational variables were mea-
sured and the range of the measuring instruments was limited,
it is not possible to know in this study whether other instruc-
tional variables (such as the instructor's overt level of enthu-
siasm or his belief in the value of the subject matter) or in-
structor personality variables or student personality variables
or slight variations in curriculum or method were related to
the results.
The use of random selection did not insure a normal distri-
bution of such variables among the three groups. Thus it is ob-
vious from the pretest flow charts of the groups that the experi-
mental group is much more predisposed to student-centered in-
structional behaviors than the other two groups. One plausible
conclusion of this study then is that the treatment will generate
118
"flexibility" in instructors whose initial instructional patterns
are predisposed to "flexibility."
Other variables that were not measured in this study were
the actual treatment of the instructors by the instructional
specialists. In this study the instructional specialists worked
with instructors from both the experimental and quasi-experimen-
tal groups. It is possible that the treatment given to the ex-
perimental group was qualitatively different rather than quanti-
tatively different.
Furthermore,without having some measure of this treatment,
it is impossible to know what role the instructional specialist
plays in the overall model. All instructional specialists in
this study might have been characterized by client-centeredress
which fostered student-centered instructional change in the ex-
perimental groups through modeling, not through the treatment
procedures of the model.
Also, although the course goals and procedures were to a
large extent standardized, there were numerous deviations a-
mong instructors. The writing sample and the achievement rat-
ing system were composed by the course director to fit an over-
all model; some instructors with different specific goals might
have generated strong student gains in respects not measured on
the achievement test.
Finally, the results of the data in the present study were
not conclusive. With five instructors or less in each group it
119
was not possible to ascertain the level of significance for
changes in classroom instructional behaviors as recorded in the
interaction analysis categories. Furthermore, of thirty-one
items and ten factors in the student evaluations of each group,
only three items and one factor in the experimental group ac-
tually showed significant positive change (at the .05 level),
while the only tnree items that showed significant change for
the quasi-ex.perimental group changed in a negative direction.
Moreover one of the significant change items in the experimental
group also changed significantly in the control group, indicat-
ing a variable operating in the experimental situation other
than the treatment. Although trends existed for achievement
gains by the experimental group and losses in the control group,
much of the variance can be attributed to chance.
Implications for Future Research
Certainly replication of this study with a larger sample
is highly recommended. It would also be important in future
studies of this kind to measure more of the situational varia-
bles. This includes the use of additional instruments such as
those measuring non-verbal behavior as well as expansion of the
measuring instruments used in this study. The achievement test-
ing particularly should attempt to measure outcomes that are as-
certained as congruent with the specific goals of individual in-
structors .
Closely measuring a wide range of instructional variables
120
is especially necessary if the research on cause-effect relation-
ships or "laws" in instruction is to be advanced. Such research
depends on making correlations between specific instructional
variables and specific learning outcomes across a number of well
defined instructional contexts.
In order to generalize the present model, it would also
be important to apply it in different instructional contexts.
This includes different academic departments and student grade
levels as well as different instructional styles. In terms of
the latter,
it might be recommended that videotaping and inter-
action analysis be used first to identify groups of instructors
with distinctive styles. Members from each group could then be
randomly assigned to treatment and non-treatment groups.
For the purpose of improving the treatment model itself, it
would be necessary to (1) test it in similar conditions against
other models, and (2) measure more closely each variable in the
model and vary these under experimental conditions.
The apparent regression in the quasi-exparimental group
in this study in terms of student evaluations suggests another
line of future inquiry: whether improvement programs such as the
one presented here, when enacted on a short-term basis without
follow-up, can actually do more harm than good.
Finally, it would seem important for future studies to ad-
dress the question of the videotape medium itself in terms of
its propensity to foster certain kinds of instructional change
121
vis-a-vis other data collection and feedback media. It is possi-
ble for example that instructors viewing themselves on videotape
will be more inclined to alter their apparent relationship with
students rather than relationships of a less overt and more in-
tellectual nature. The latter might be better promoted through
written transcripts of the classroom or even audiotape.
Summary
Compared to many previous studies in the field of in-ser-
vice teacher education, the present study was exemplary in its
attempt to deal with the complexities of instructional situations.
Both direct observational data and student evaluational data and
achievement data were collected, analyzed and compared.
The results of these analyses and comparisons suggest numer-
ous directions for valuable future research.
The apparent correspondance between the stated improvement
goals of the instructors, the observed changes in instructional
behavior and the changes in certain student evaluation categories
among the experimental group instructors in this study tend to
support the conclusion that instructors can decide to alter their
instruction in productive ways and that students do detect and
respond to such changes.
In light of the paucity of in-service teacher education pro-
grams and models at all levels of education7-but especially at
the higher education level, the results and implications of this
study are important and deserve additional investigation.
122
REFERENCES
Alciatore and Eckert. (1968). Minnesota Ph. P 's EvaluateTheir Training . Minneapolis: University of Minnesota.
Alexander, L.T. and Davis, R.H. (1970). Developing a Sys-tem Training Program for Graduate Teaching Assistants.Michigan State University. (Mimeo.)
Allen, D.W. and Melnick, M.A. (1972). The Clinic to ImproveUniversity Teaching . W.K Kellogg Foundation Grant. Am-herst: University of Massachusetts. (Mimeo.)
Allen, D.W. and Fortune, J.C. (1967). "An Analysis of Micro-teaching: A New Procedure in Teacher Education." InMicro-teaching : A Description . Stanford, Calif.: Schoolof Education, Stanford University.
American Association of Junior Colleges (1969). In-serviceTraining for Two -Year College Faculty and Staff: A Sur-veY OI Junior and Community College Administrators.Faculty Development Project. Washington, D.C.: AAJC.
Amidon, E.J. and Flanders, N.A. (1971). The Role of theTeacher in the Classroom . Minneapolis, Minn.: Associa-tion for Productive Teaching.
Anderson, H.H. (1939). "The Measurement of Domination andof Socially Integrative Behavior in Teachers' Contactswith Children." Child Development
, 10, pp. 73-89.
Arrowsmith, W. (1967). "The Future of Teaching." In Lee,C.B.T. (ed.) (1967). Improving College Teaching .
American Council on Education: Wa-.hington, D.C.
Aubertine, H.E. (1964). An Experiment in the Set InductionProcess and I ts Application to Teaching . UnpublishedDoctoral Dissertation. Stanford: Stanford University.
Baldridge, J.V. (1970). "Images of the Future and Organiza-tional Change." Research and Development MemorandumNo. 58. Palo Alto, Calif.: Stanford Center for Researchand Development in Teaching.
Bales, R.F. and Strodtbeck, F.L . (1951). "Phases of GroupProblem Solving." Journal of Abnormal Social Psychol-ogy, 46, pp. 458-496.
123
Beard J.W. (1969). Teacher Effectiveness and Style as Relat-P_art c 1 pat 1°n in a Science Curriculum Workshop!
Unpublished Doctoral Dissertation. StanfordTTt'anf ordUniversity.
Belgard, M., Rosenshine, B.
,and Gage, N.L. (1971). ''Effec-
tiveness in Explaining: Evidence on Its Generality andCorrleation with Pupil Ratings." In Westbury, I. andBe Hack
,A. (eds.) (1971). Research Into Classroom
Processes . New York: Teachers College~Pr^s7^
Berliner, D.C. (1967). A Comparison of Different Modelingprocedures in the Acquisition of a Teaching Skill.New York: Paper Presented at AERA Meeting.
Biddle, B.J. (1964). "The Integration of Teacher Effective-ness Research." In Biddle, B.J. and Ellena, W.J. (eds.)
.... .
Contemporary Research on Teacher Effectiveness.New York: Holt, Rinehart and Winston.
~
Borg, W.R.,Kelly, M.L., Langer
,P. and Gall, M. (1970). The
Minicourse : A Micro-teaching Approach to Teacher EducaT”tion. Beverly Hills, Calif.: Macmillan Educationai“si^-vices
.
Boulding, K.B. (1970). "The Task of the Teacher in the So-Cial Sciences." In Morris, W.H. (ed.) (1970). Effec-tive College Teaching . Washington, D.C.: AmericanAssociation for Higher Education.
Burkhart, R.C. (ed.) (1969). The Asse ssment Revoliition: NewViewpoints for Teacher Evaluation . Buffalo, New York:State University of New York, Division of Teacher Edu-cation and Certification.
Centra, J.A. (1972a). The Utility of Student Ratings forInstructional
. Improvement . PR 72-16. Princeton: Edu-cational Testing Service.
Centra, J.A. (1972b). Strategies for Improving CollegeTeaching . ERIC Clearinghouse on Higher Education.Washington, D.C.: The George Washington University.
Church, R.J. (1971). "The Effects of Systematic Changes inStandard Four Science Lessons." Educational ResearchNewsletter
, 4, pp. 15-30.
Cogan,M.L. (1973). Clinical Supervision . Boston: Houghton
Mifflin Company.
124
Cohen,A.M and Shawl, W.F. (1970). "Coordinating Instruction
Through Objectives." Junior Coll ege Journal, 41(2). Dn.1 7—19
•
Cohen, A.M. and Brawer,F.B . (1969). Measuring Faculty Per-
formance. ERIC Clearinghouse for Junior Colleges, Mono-graph No. 4. Washington, D.C.: American Association ofJunior Colleges.
Cohen, E.( 19 70). Faculty for Teaching-Learning: ProposedNew Graduate Centers for the Systematic Preparation ofCommunity Col lege Teachers . ED 038 133. "(Mimeo.
)
Cos tin, F. and Greenough, W.T. (1971). "Student Ratings ofCollege Teaching: Reliability, Validity and Usefulness."Review of Educational Research, 41, No. 5, pp. 511-535.
Danfourth Foundation (1970). "Teaching Improvement Grants."Danfourth News and Notes
, 6(2), pp. 1-7.
Davis, O.L. Jr. and Smoot, B.B. (1969). Effects on the Ver-bal Teaching Behaviors of Beginning S_ec_ondar~TeachingCandidates 1 Participation in a Program of LaboratoryTeaching . Paper Presented at AERA Meeting? Austin:University of Texas.
Dean, D. (n.d.). Preservice Preparation of Colle ge BiologyTeachers : A Search for a^ Better Way . The Commission onUndergraduate Education in the Biological Sciences.
deFreisse, R. (1973). A Compute r Program for Displaying In-terac.tion Analysis Coding Systems . Amherst: Universityof Massachusetts. (Mimeo.)
Dell, D. and Hiller, J.H. (1971). "Computer Analysis ofTeachers' Explanations." In Westbury, I. and Bellack,A. (eds.) (1971). New York: Teachers College Press.
Dewey, J. (1939). Experience and Education . London: Collier-MacMillan.
Diekhoff, J.S. (1960). Tomorrow's Professors . New York: Fundfor the Advancement of Education.
Eble, K.E. (1971). Career Development of the Effective (. legeTeacher . Washington, D.C.: American Association of CollegeProfessors
.
125
Flanders, N.A. (1970). Analyzing Teach ing Behavior. ReadingMassachusetts: Addi son-Wesley
.
Flanders, N.A. (1967). "The Problems of Observer Training and
??a* 7^lllty *" In Amidon
>E * J * and Hough, J.B. (eds.)U
_
Interaction Analysi s: Theory, Research and
Application. Reading, Massachusetts: Addison-Wes ley
.
Fortune, J.C., Cooper, J.M. and Allen, D.W. (1967). "The Stan-ford Summer^Micro-teaching Clinic, 1965." The Journalof Teacher Education
, 18, Winter, pp. 389-393.
ortune, J.C. (1965). Instructional Set, Cognitive Closure,
£.n_ l£st Anxiety in the Presentation of Social StudiesContent. Unpublished Doctoral Dissertation. Stanford:Stanford University.
Frey.^P.W. (1973). Student Instructional Ratings and FacultyPerformance. New Orleans: Paper Presented at AERA Meeting
Gaff, J.G. and Wilson, R.C. (1971). "The Teaching Environment.AAUP Bulletin
, 57(4), pp. 475-493.
Gage, N.L. (1968). "An Analytical Approach to Research onInstructional Methods." Phi Delta Kappan, 6, pp. 601-606.
Gage, N.L. (1963). "Paradigms for Research on Teaching." InGage
,N.L. (ed.) (1963). Handbook of Research on Teach-
ing . Chicago: Rand McNally.
Gump, Paul V. (1964). "Environmental Guidance of the Class-room Behavioral System." In Biddle, B.J. and Ellena,W.J. (eds.). Op. Cit.
Gustad, J.W. (1963). "Orientati on and Faculty Development."The Educational Record
, July, pp. 195-213.
Heiss,A. (1970). Challenges to Graduate School. San Francis-
co: Jossey-Bass.
Hodgkinson, H. (1972). "How to Evaluate Faculty When You Don’tKnow Much About Them." The Research Reporter, 7, no. 3.Berkeley: University of California.
Hughes, D.C. (1971). "The Effects of Certain Conditions ofPupil Participation and Teacher Reacting on the Achieve-ment of Form 2 Pupils." Educational Research Newsletter,4, pp. 12-14.
126
Knight, L.E. (1971). Anfor Variable Use ofUnpublished DoctoralUniversity.
Investigation of Teacher PreparationMathematical Instructional Modes.Dissertation. Stanford: Stanford
Koen,^R. and Ericksen (1967). An Analysis of the SpecificFeatures Which Characterize 'the More Successful Programsfor the Recruitment and Training of College Teachers
.
University of Michigan: Center for Research in Learningand Teaching.
Koran, J.J. (1970). The Relative Effects of Imitation VersusProblem Solving on the Acquisition of Inquiry Behaviorby Intern Teachers . Technical Report No. 11. '
Stanford:Stanford Center for Research and Development in Teaching.
Legge^W. and Asper, L. (1972). "The Effect of VideotapedMicroteaching Lessons on the Evaluative Behavior of Pre-Student-Teachers." Journal of Teacher Education 27pp. 263-266.
'
Lev/in, K., et al. (1939). "Patterns of Aggressive Behavior
in Experimentally Created 'Social Climates.'" Journalof Social Psychology
, 10, pp. 271-299.
Limbacher, P.C. (1969). "A Study of the Effects of Microteach-ing Experiences Upon Practice Teaching Classroom Behavior."In Dissertation Abstracts International, 30, p. 189.
McDonald, F.J. (1973). A Behavior Modifi cation View of VideoPlayback : Microteaching . New Orleans: Paper Presentedat AERA Meeting.
McDonald, F.J. and Allen, D.W. (1967). Training Effects ofFeedback and Modeling Procedures on Teaching Performance.Final Report. Research Project No. OE-6-lO-OTCh uTsTOffice of Education. Stanford, Calif.: Stanford University.
McGrath, E.J. (1961). The Quality and Quantity of CollegeTeachers . New York: Columbia University Teachers College.
MacKenzie, N., Eraut, M. and Hywel, C.J. (1970). Teaching andLearning : An Introduction to New Methods and Resources inHigher Education . Belgium: Unesco and the InternationalAssociation of Universities.
McKnight,P.C. (1970). Behavioral Responsiveness of Teachers
in Verbal Interactions with Students . Unpublished Doc-toral Dissertation. Stanford: Stanford University.
127
Many, W., et al.
, (1969). In-service Education in AmericanSenior Colleges and Universities : A Status Report.Northern Illinois University, DeKalb: College “oTEdu-cation.
Mathis, B.C. (1972). "Center for the Teaching Professions -Northwestern University." In Alexander, L.T. and Yelon,S.!.. (eds.) ( 1972 ). Instructional Development Agencies’i_n Higher education
,
East Lansing, Michigan: MichiganState University.
Melnick, M.A.^ (1972). A Clinic to Improve University Teaching.Unpublished Doctoral Dissertation. Amherst: Universityof Massachusetts.
Mescon, M.H.,et al. (1969). Toward a Faculty Self-Appraisal
System . Research Paper Number 49. Bureau of Businessand Economic Research. School of Business Administration.Atlanta: Georgia State College.
Meux, M.O. (1967). "Studies of Learning in the School Setting."Review of Educational Research
, 37, No. 5, pp. 539-561.
Meux, M.O. and Smith, B. (1964). "Logical Dimensions of Teach-ing Behavior." In Biddle, B.J. and Ellena, W.J. Op. Cit .
Miller, W.S. and Wilson, K.M. (1963). Faculty Development Pro-cedures in Small Colleges - a. Southern Survey. SouthernRegional Education Board.
Miltz, R.J. (1973). Micro-teaching - a Description . Amherst:School of Education, University of Massachusetts. (Kimeo.)
Miltz, R.J. (1971). Development and Evaluation of a Manual forImproving Teachers 1 Explanations . Unpublished DoctoralDissertation. Stanford: Stanford University.
National Advisory Council on Education Professions Development.(1972). In Centra, J.A. (1972b). Op. Cit.
National Faculty Association of Community and Junior Colleges(1968). Guidelines for the Preparation of Community/Junior College Teachers . Washington, D.C.: NationalFaculty Association of Community and Junior Colleges.
Noam, J. (1973). Development and Evaluation of a Model forImproving University Teaching . Unpublished DoctoralDissertation. Amherst: University of Massachusetts.
128
Nowlis, V., Clark, K.E. and Rock, M. (1968). The GraduateStudent as Teacher . American Council on Education.
Nuthall, G. and Snook, I. (1973). "Contemporary Models ofTeaching." in Travers, R.M.W. (ed.) (1973). SecondHandbook of Research on Teaching . Chicago: Rand Mc-Nally and Company.
Orme, M.E.J. (1966). The Effects of Modeling and FeedbackVariables on the Acqui sition of a Complex TeachingStrategy . Unpublished. Doctoral Dissertation. Stan-ford: Stanford University.
Perlberg, A., ej: aH. (1972). "Microteaching and VideotapeRecordings: A New Approach to Improving Teaching."Journal of Medical Education
, 47, pp. 43-50.
Perlberg, A. and O'Bryant, D.C. (1970). "Videotaping andMicroteaching Techniques to Improve Engineering In-struction." Engineering Education
, 60, pp. 741-744.
Popham, W.J. (1967). Development of a Performance Testof Teaching Proficiency . Final Report, U.S. OfTiceor Education Project Mo. 5-0566-2-12-1. Washington,D.C.: U.S. Office of Education.
Ramp, E., Ulrich, R. and Dulaney, S. (1971). "DelayedTimeout as a Procedure for Reducing Disruptive Class-room Behavior: A Case Study." Journal of AppliedBehavior Analysis
, Vol. 4., pp. 235-240.
Report of the President's Commission on Higher Education(1947). To Improve Learning
,Vol. 4, 16. Washing-
ton, D.C.
Rogers, C.R. (1969). Freedom to Learn . Columbus, Ohio:Charles E. Merrill.
Rosenshine,B. and Furst, N. (1973). "The use of Direct
Observation to Study Teaching." In Travers, R.M.W.(ed.) (1973). Second Handbook of Research on Teaching .
Chicago: Rand McNally and Company.
Rosenshine, B. (1970). "Evaluation of Classroom Instruction."Review of Educational Research
, 40, No. 2, pp. 279-300.
Rosenshine, B. (1968). Objectively Measured Behavioral Pre -
dictors of Teacher Effectiveness in Explaining . Un-published Doctoral Dissertation. Stanford: StanfordUniversity.
129
Ryans D.C. (1964). "Research on Teacher Behavior in theContext of the Teacher Characteristics Study." inBiddle, B.J. and Ellena, W.J. (eds.). Op. Git .
Ryans, D.G. (1960). Characteristic s of Teachers: TheirDescription, Comparis on, and Appraisal. Washing tonD.C.: Amer i
c
arTcouneTT-on~^u^atioru
’
Salzburg, B.H., etal. (1971). "The Effect of Intermittent
eedback and Intermittent Contingent Access to Play onPrinting of Kindergarten Children." Journal of AppliedBehavioral Analysis
, Vol. 4, pp. 163-172.
Sanford, N. (1962). "Research and Policy in Higher Education."In Sanford, N. (ed.) (1962). The American College.New York: John Wiley.
'
Smock, R. and Crooks, T. (1973). A Plan for the Comprehen-sive Evaluati on of College Teaching . New Orl^T:Paper presented at the AERA Meeting.
Soar, R.S. (1968). "Optimum Teacher-Pupil Interaction forPupil Growth." Educational Leadership
, 26, pp. 275-
Survey of the National Council of Teachers of English (1970).In Eble
, W. (1971). Op. Cit.
Trent, J.W. and Cohen, A.M. (1973). "Research on Teachingin Higher Education." In Travers, R.M.W. (ed.) (1973).Second Handbook of Res earch on Teaching, Chicago: R.andMcNally.
Tient, J.W. "Revolution, Reformation, and Re-evaluation." InSampson, E.E.
,Korn, II.A. and associates, Student Acti-
vism and Protest . (1970). San Francisco: Jossey-Bass.
Triezenberg, D. (1969). "Graduate Student Survey - Fall 1969."University of Maryland. (Mimeo.)
Unruh, W.R. (1967). The Modali ty and Validity of Cues toLecture Effectiveness Unpublished Doctoral Dissertation.Stanford: Stanford University.
Waimon, M.D.,Bell, D.D. and Ramseyer, G.C. (1971). The Effects
of Microplanning on the Performance and Attitudes of Pro-spective Teachers
.
New York: Paper Presented at AERA Meering.
130
West, R.W.,et al. (1970). Graduate Preparation for Teaching
pissing Component.. Commission on College Physics
.
College Park: University of Maryland.
Withal 1 ,J. (1949). "The Development of a Technique for the
Measurement of Social-Emotional Climate in Classrooms."Journal of Experimental Education
, 17, pp. 347-361.
Wright, C.J. and Nuthall, G.A. (1970). "Relationships Be-tween Teacher Behaviors and Pupil Achievement in ThreeElementary Science Lessons." American Educational Re-search Journal
, 7, pp. 477-491.~
131
APPENDIX A
COURSE INFORMATION AND PRE-VIDEOTAPING INFORMATION FORMS
132
Course Information Form
1.
Department
2. Name of teacher
3. Course title
4.
Catalogue description of course (type in from catalogue).
6 .
7.
Prerequisites
Other courses
For which, if
in sequence
any, programs is this course a requirement?
8. How many students are enrolled?
9. How many of the enrolled students are:
a. Freshman? c. Juniors?
b. Sophomores? d. Seniors?
10.
What is the structural format of the course (i.e., threelectures a week, one lecture group and two discussiongroups, etc.)?
11.
How are student grades determined?
12. Are grades (check appropriate responses): pass/fail?satisfactory/unsatisf actory? A through F
13. Try to get and attach: a. syllabus c. assignmentsb. reading list d. examinations
14. Generally, what are the informational, skill, and affectivecourse objectives? (Please list on back of this sheet.)
15. What is the presumed relationship between what happens inclass and the work which students are asked to do outside class?
133
Course Information Form (continued)16.
If this course is in any sense experimental (either inobjectives or teaching patterns), describe.
17.
Has the teacher had any particular highs or difficultieswith this course? If so, describe.
18.
If there are others now teaching this course, who and/orhow many?
134
Name
Date of class
Pre -videotaping Information
To enhance our understanding of the videotape of yourclass, would you please take a few moments to answer thequestions which follow? Please note that neither specificbehavioral objectives nor an elaborate lesson plan are be-ing requested. The intention is merely to provide a frameof reference for the videotaped class.
Would you please complete this and give it to me beforeclass?
1.
Generally, what are your objectives for this class (i.e. toelaborate on specified concepts or processes, to introducea new topic, to review previously considered material, togo over assigned readings, etc.)?
2.
How do you hope to accomplish those objectives (i.e. bylecturing, class discussion, asking questions, etc)?
3.
What do you expect students to have done to prepare forthis class?
4.
What do you exp-ct students to do during class?
5.
If you think it would be useful, would you describe whatwas done in the immediately preceding class?
THANK YOU
135
APPENDIX B
INTERACTION ANALYSIS MATRICES
10 x 10 Pretest and Posttest Matrices for Each Sub
17 x 17 Pretest and Posttest Matrices for Each Sub
17 x 17 Difference Matrices for Each Sub-group
group
group
136
EXPERIMENTAL GROUP 10 x 10 PRETEST MATRIX
Categories
• •
*1 .
J 2. I 3 1 4 1 5 I 6 ! 7 1 8 1 9 l 10 1
1
0 I
1
0 I
I
0 1
I
0 1 0 I
1
0 I
I
0 1 0 I
J
0 1 0 1 Tallies•
0.00 1 0,061 0 . 00 I 0.00! 0 . 0 n i c r 0 0
1
0 . 00 1 o.ool 0 - 0 0 I 0,001 Percents0 ! 15 I 14 I ?0 I 9 1 0 ! 0 1 15 I 17 1 14 1
2 I I It
1 I I J
•0 I 0 0 I 0.931 0.871 1.241 0.561 0 r 0 0
1
0 . 00 1 0.931 1-051 0,871
0 t 2 I 85 I 17 I 11 1 2 1 0 1 7 1 >8 I 3 1
3 1 I I 1 1 l 1
• o • o Q1 0*12* 5.271 1-05I 0-6*81 0-12! O'OO 1 0.631 l- 1?1 0 ’ 1 ?I
0 I 0 I 4 I 75 1 6 I 0 1 0 1 63 1 26 I 41 1
4 t 1 1 I I l 1
• • o • oa i 0 -GO I 0«25I 4.651 0-371 0 *00 1 0 *00 2 3 . 9 1
1
i-241 2,541
0 1 1 I 0 I 26 I 2’8 1 3 1 0 1 5 l 7 I 13 1
5 i I 1 I 1 1 l
• 0-051 0 • 0 0 I 1 .61 I 18.471 0 • 1 9 I 0 • 0 0 1 0 • 3 1
1
0-831 0-812
0 1 0 I 0 i 0 1 1 1 4 I 0 1 , 2 1 0 1 4 I
6 1 I I 1 1 1 I
• O-OQ! 0 • 0 6
1
0 • 0 6
1
0 • 0 d i 0.662 0-251 0 • n 0 * 0-121 0 • 0 0 l 0 • 25
1
0 1 0 1 0 1 0 I 0 I 0 1 0 1 1 1 1 I 0 I
7 I I 1 1 1 1 I
• 0 • 0 Q 10 -00 1 0 -00 1 0 *00 1 0 *00 1
0 ;00 1 0-00 1 0 -0 6!0 * d
6
1
0 ' 00 I
0 t 47 I 18 1 22 1 5 1 1 1 0 I 179 I 3 1 17 t
8 ! I I I I 1 1
. • 0 • 0 01 2.911 1 • 1 2 I 1.361 0
• 3 1I
0 ;061
0 • 00 1 u-ioi o-1 9 1 1-0 51
0 1 34 I 15 1 22 I 6 1 0 1 2 I 0 1 2'7 I 13 I
9 1 I I 1 I I I
.0.091 2.111 0.931 1.361 6.371 0 r 0 0
1
0.121 0.00114.691 0.811
0 1 5 1 9 I 27 I 17 I 1 1 0 I 20 I 26 I 63 1
io 1 I 1 I 1 1 1
• 0 • 0 01 0-3(2 0-561 1.671 I'D’ 1
0 lo6 > O^OO 5 1*281 1-6, I 3 • 9 1
X
0 1 104 I 145 I 209 1 353 1 11 1 2 I 292 l 329 I 163 1
1 1 1 1 1 l I
TOTAL 0* of)1 6,451 8,991 {2.961 2 t . e«u
0:6fl I 0-l2» 1®. io* 2 q' 4 q1 10-822
I/O RATIO 0.556
revised i/d Ratio -- 0.967
EXTENDED indirect -* 11-779
EXTEN0FD oiRECT ---- 0.248
teacher talk 51.095
student talk 38.500
stu/teac TAL< RATlD 0.754
137
EXPERIMENTAL GROUP 10 x 10 POSTTEST MATRIX
Categories
» i i 2. I 3 1 4 1 5 I 6 I 7 I 8 1 2 1 10 I
i
0 I
I
0 I
I
0 I
I
0 I
I
0 I
I
0 1
I
n I
I
0 I 0 I o i Tallies
•o.ooi 0.001 O.OOI 0,001 0.041 olooi 0.001 o.ooi 0-001
1
o.ooi Percents0 I 5 I 13 I 6 I 8 I 1 I 1 I 4 1 •0 I 6 I
2 I I I I I I I J
• 0 • 0 0 1 0-341 0-371 o-V 0-541 o i o7
1
o-o 71 O'? 7 ! 3-6710
' 4 0 I
0 I 0 1 142 1 19 I 11 I 0 I 0 I 4 I i 1 I 8 I
3 1 I I I II
• O' 0
0
0 • 0 0 1 9.531 1.281 0-741 0:00 * o- oo 1 0-271 0*741
0• 5 4 I
4
0 1
1
2 I
J
0 I 36 I 9 I l i 0 I 28 I 6 I 38 I
0 • 0 6
1
0-131 o • o 6 1
1
2. 4?I1
0 * 6 T 1 o[ o
7 1
1
o- no 1 ! . 8 6
1
0- 4o !
I
2.551
0 I 4 I l i 20 I 45? I 0 I 3 I 3 I ll I 14 I
5 I i I I i I I
•o.ooi 0.271 0.071 1.34! 50-341 olooi 0.201 0.201 9-741 0.941
0 I 2 I 0 I 0 I 1 I 1 1 0 I 1 I 0 I 3 I
6 I I I I I I 1 I
. o • o 0J 0-131 0 • 00 I 0 - 00 I 0-671 0 :0 7 1 o • no • 0-071 9 ’ 0 0 I 0 • 2 0 1
0 I 0 I 1 1 1 I 3 I 0 1 5 l 1 I 0 I 1 I
7 I I I I ! l I I
. 0 • 0 0 T o • o 6 1 0 - 07 1 n • o 7 t 6 • ?o I o-oo* 0-341 0 • 0
7 I9 ' 0 0 1 0 ' 0
7 *
0 I 23 I 22 I 12 I 4 1 2 I o I T39 I 3 I 3 I
8 I I I I I 1 I
. 0 • 0 0 1 1,541 1.49! o • 8 i i 0-271 o [ 1 3 * o- oo 1 9.3319
’ 2 o 1 0* 20 *
0 I 17 I 13 I 13 I 3 I 0 1 0 I 2 1 1*2 I 6 I
9 I I I I I i I I
. 0 • 00 1 1.141 0-371 0-37'0
• 2 n I 0 1 0 0 *o • no 1 0-131 1 9
• 20 1 0• 4 0 *
0 I 1 1 3 I 13 I 17 I 3 I 3 I 26 I 13 I 77 I
io I 1 I I I I I I
O' oo 1 0 - 07 » 0 • 2 6
1 0-871 1 .1 4 I 0 : 2 0 * 0 • 2 0
1 1-741 9-871 5 . i7 1
0 I 54 I 195 1 120 I 508 I 8 I 12 I 208 I 256 I l56 I
I I I I I I 1 I
t^tal O' 00 I 3,621 1 3 . o9
1
8• 0 5
1
34 •o^ I 0-541 0.8^13.961 13-831 1 0 • 4 7
*
I / D 9 A T I 0.4261 / D 9 A 1 1 0.426
REVISED i/D -- 0.944
EXtENDED INDIRECT -- 14.966
EXTENDED DIRECT 0.403
TEACHER TALK - 61.745
STUDENT TALK 27.795
STU/TEAC TALK RATIO 0.450
138
QUASI -EXPERIMENTAL GROUP 10 x 10 PRETEST MATRIX
Categories
1r
! 2 I 3 1 4 1 5 16 I 7 I 8 19 1 10 1
{
0 1
I
0 1
I
0 1
I
0 1
1
0 ! 0 1
| |
0 I
1
0 1 0 1
l I
o.ooi o.ooi
o 1 Tall ies
• 0 0 I o.ooi 0.001 0.001 o.onl OToot o.ooi1
o.ooi Percents
?
0 I 7 I 2 I 5 I 3 I 1 i n I 3 1 4 1 7 I
,oo i 0.571 0.161 0.4(1 0.741 0 : o B
I
I
o.ooi1 1
0.241 J.33I1
0 . 5 7 |
0 1 0 I 16 I 3 I 6 1 0 1 n I 0 I 4 1 0 I
3 I 1 I I 1 I I I I 1
.• o o ' 0 -00 1 1.361 0-241 o
.491 o [on I 0 -00 1 0*00 10
' 33 1 0 *00 I
0 I 1 I 0 I 97 1 lo I 2 1 0 I 18 I 6 ! 19 I
4 ] I I I I 1 I 1 I I
.o.ooi 0.081 o.ooi 7.89! o . a 1
1
o t 1 6 1 o.ooi 1.461 3.491 1.541
0 I 4 I 0 I 18 I 369 I 6 I 1 I 6 1 4 1 22 I
5 I I I I I I I I 1 I
0 • 0 Q1 0-331 0 • 00 1 1 . 4 6 I 3 o . o t I 0 :4?I 0 - 0*31
0< 49 I
j> 33
I
1.79J
0 I 2 1 0 I 0 1 4 1 »9 1 0 I 3 1 11 2 1
6 1 1 I 1 t I I I I I
. Q'Ofll 0 .
1
6 I 0 • 00 1 o * n o 1 6-331 4 :a0 I 0 - 0 'J
I
0-241 j - o8 1 0*1 6
1
0 I 0 I 1 I i i 2 1 0 I 4 I 0 1 0 1 C I
7 I I I i I 1 1 I 1 I
o • o 6
1
o • o 6
1
o*o1
’ 1 0-0»l o
•
1 61 o [on
I
q.331 0 • 00 1 0 •00
l
0 • 00 I
0 I 8 I 3 I 3 1 in I 2 i 1 I 77 l 7 1 4 I
B 1 I 1 1 l i I I I I
•0,001 0.651 0.241 0.24! o.hi'i o : 1 6
I
0.081 6.261 0*57! 0.331
0 I 8 I 7 I 7 ! 3 1 1 I 1 I 0 I 233 1 13 I
9 I I I I 1 I 1 I I 1
. 0 • 00 1 0.651 0.571 0.57! 0-241 o’ o3 1 o • n 9 1 o • 0 0 1 1 3 • 9 4 1 1.0*1
0 I 2 1 0 I 19 ! 23 I 0 1 l I 8 1 14 I 52 I
10 1 I 1 1 1 i 1 I I
0.651 i
•i 4
1
1
4.231. 0 ’ 0 D 1 0.1*1 0*00* 1.54! 1
• ® 71 o[oil o-
o
9 I
0 I 32 I 29 I 153 ! 430 I 71 | 8 I 115 I 273 ! 119 I
I 1 I 1 1 1 I I 1 I
9.671TtT als* 0 • 0 0 I 2.661 2. 36112. 44134. 961 5.77! 0.651 9. 35122-201
I/D RATIO 0.2 96
revised i/o ratm -- 0.515
EXTENDED INDIRECT — 3.333
EXTENDED DIRECT 5.041
50 .
7
q nTE T ALK
STUDENT talk * 31.545
STU/TEAC TAL< S*T10 0-537
139
QUASI -EXPERIMENTAL GROUP 10 x 10 POSTTEST MATRIX
Categories*1 1213 14 t 5 16 17 le I? I 10 I
* o 1 0 1 0 1 0 1 (II 0 1 0 1 0 1 0 1 0 I n'.ii.' .
i * i i i i i i i i i i- ues
* 0 . o o i P : 0 6 1 o.ooi o.ooi 6 • 6 o I olooi o.ooi o.oot 9.001 0 . 0 0 1 Percents* 01 51111 51 3! 01 2 I 01 ol 31
2 * I . I I I 1 . I l 1 I I
* 0 * 0 Q1
0 • 4 1 1 O ' 9 !1
O ' 4!
1 0'?’ 1 0 * 00 * 0*1 6J0 • 0 0
1
0 * 0 01
0* 2 51
* 0 I 1 1 46 I 10 I 7 I 0 I 0 I 2 1 2 I 3 1
3 * I . 1 I 1 I „ I I i I I
.
* 0*001 0 • 0B 1 3.791
0 . 8?1 0.581 0 ; 00
l 0 * 00 l()
* 1 6 13 ' 1 6 1 0*251
* 0 1 1 1 2 1 60 I 6 1 0 I 0 I 22 I 4 I 25 I
4 1 l I I 1 1 1 I 1 1
.
* 0*051 o* oftl 0 • 1 6 1 4,941
0• 45 i 0 • 0 0 I
0 • 0 01 1 • 8 1 1 8*331 2 • 0
6 1
* 0 1 1 I 1 1 17 I 493 1 1 I 0 1 0 1 4 l 22 l
5 I I I I I I I I 1 1
•*
0 ’ 0 Q1 0 - 0
3 1 0 • 03 1 1 . 4 Q
I
39 .
7
7 1 oio*i n • 00 1 0 • 00
1
3*331 1 • 3 1
1
0 1 0 I 0 1 0 1 1 1 0 1 0 l 11 0 I 0 1
6 * 1 I I 1 1 I 1 1 I 1
*0 • 0 (3
1 o-ooi 0*001 0.001 0 • 03 I o-ool 0 • no 1 0 • 0 91
3 * 0 0 '• 0 • 0 0 I
* 0 l 0 1 0 I 1 1 2 1 0 1 n I 0 1 1 I 0 1
7 # I I I I l I I 1 I I
«* 0 • 0 0 I 0.061 O.OOI 0* 0 8 I 0*151 o-ool o.ooi o-ool 0 • 0 B 1 O-ool
* 0 1 11 I 5 I 11 I 2 1 1 1 0 1 166 1 2 I 5 I
8 * I I I I I I I 1
0 • 0 0 1 1 3 . 6 6 I
I 1
* o.ooi 0.911 0
.
41 I.
0.911 0 • 1 6 I oTo*! 3*16! 0*911
* 0 I 5 I 5 I 4 1 0 1 0 I 0 1 0 1 117 1 10 1
9 • I I I 1 I I 1 I 1 I
0*921* 0*00' 0 • 4 1 1 0 • 4 1
1
0 - 33 ! 0 • (in I 0 * 0 0 I 0 • no I 0 • 00
*
5.631
* 0 1 5 1 1 I 1? I 25 l 0 1 21 12 1 it 148 I
to* I
0*00!
1
0 • 4 1
1
I
n * 09
1
1
p • 99 I
I
2 • 05 I
1
0*00lI I
0- 1*1 0
. 99 I
I
3• 5 1
1
1
3.951
# 0 I 29 I 71 1 120 I 529 1 2 I 4 I 203 1 14
1 I 116 1
« I I 1 I 1 I I 1 I 1
T nT ALS* 0 • 0 0 1 2-391 5.841 9 . 881 43.541 0 ;l 61 0
.
33 ! 16. 7 * I n* 60 1 9.551
i/D 0.291
REVISED 1 /D 5 a T
I
3 -- 0.959
EXTENDED INDIRECT -- 7.819
extended direct ---- 0.000
TEACHER talk - 62.140
student talk 2 B .313
STU/TEAC TALK RATIO 0.456
140
CONTROL GROUP 10 x 10 PRETEST MATRIX
Categoriesn TT 13 14 15 ! 6 17 18 l > 1 10 |
*(•********•**•***•*****•**•*•**•*•**»*••***•••*•**•••••••*•*•« oi oi oi ci oi oi oi oi oi oi Tallies
{ * I l i 1 I i I I l t
* OiOOi o I o o i o.ooi o.ooi o.ooj o.ooi o.ooi o.ool 3.001 0.001 percents
* 01 01 61 71 31 01 oi 21 31 212 * 1 . I I l 1.1 I
j
1
}
* 0 *
0
0 1 0 * 0 nl 0
*
76
1 0
•
6 1 1 6*261 o*oo J o*oo* cl 81o’? 61 o*i B1
» 01 21 25 I 91 71 01 01 3! 31 113 * 1 I 1 1 1 I 1 1 I I
* 0 • 0 0 I 0 1 18 I 2.191 0.791 0 . ft 1 l 0: 00 1 0 • 0 0 1 0 * 261 0 • 26 1 0 • 0 9 I
* 0*1 *1l" 01 85 1 13 I 01 0 l 40 1 3 1 32 I
4 * 11 I 1 1.1 1 1
* 0*00! 0 •
0
9
1
0 • 0 G 1 7 « 4 61 1 •
1
4 1 0• 0 0
1 0 • 0 0 1 3 • 5 11 0^261 2
•
e 1
1
* 01 11 0 I 32 I 400 1 It 0 1 31 2 1 11 1
5 « 1 1 1 1 1 i I 1 1 1
* 0 > 00 1 0 • 09 1 0 • 0 0 1 2 . 3t I 35 • 1 ?I 0*0 9
1 0 HO 1 0*2 61 J*! 81 0
•
97
1
6
*
7 *
«
8
*
*
*
*
9
*
\0
T?mS*
01 01 01 0 ! 11 II °I 1 1 31 01
, 1 I 1 I
.I 1 I 1 1
0*001 0 . 06 1 0 • 0 6 1 0 • 0 0 1 6 • 6 9
1
0 ; 09 *
01 01 01 01 61 01 oi oi 61 01I 1 I 1 1.1 l * *
0 • oo 1 0 -
0
6 1 0 • 0 0 1 0 • 0 6 1 o:2 1 i.2i2Si.2:22i.2l2Si-212!li.Sl22!
0 I {2 1 12 1 19 1 61 01 0 1 126 1 71 61J | I I I I • l ‘
o • o 0
1
1 :
0
5
1
1. 051 I- 67
' 0-531 o:oo*_o;ooJiro 8|j:
8 iJ_o:53 i
pi 7J 31 61 ll 01 ol ol ‘01 11 1
I I I I 1 I 1 l 1
0 , c 0 1 0
•
6 i 1 o*26i n.531 0 •
0
9
1
0 loo * o-ool_o:ool_ 7
:o 2 l_o
:
9 _i
*"
2
*
1
*2 I 1.6 1 19 I 0 1 0 1 1 4 I 1 0 I
7
9
1
J J J I I I I *
O.OOI O'.lBI 0.191 1 .401 1.671 0:001 0 . 00 1 1.23 1 0 • S ^ 6 * 9 **
Ol ?5 I 5 q I 1 74 l 4 5 o 1 2 1 0 1 l 93 l l- 9 I l* 2
j
| | | I 1 1 1 I
0 . o 0 1 2 *
i
9 3 4,391,5.28139.5,1 o ;1B|
0 ’
i/O 9 AT 10 0.355
revised i/o Ratio -* 0.996
EXTENDED indirect -* 6.595
EXTENDED DIRECT ---* 0.098
TEACHER talk - - 61.545
STUDENT TALK - 25.998
stu/tEac tal< RATIO 0.422
141
CONTROL GROUP 10 x 10 POSTTEST MATRIX
Categories1 I 2 I 3 l 4 I 5 I 6 17 I 8 I
i
0 I
I
1 1
J
0 1
1
0 1 0] 0 I 0 I 0 I Tallies0 . 0 9 I o.onl o.ool 0.091 0*001 0.001 o.ool o.ool 3-001
I
0.001 Percents
pJ
11 I 8 I 5 I 11 0 I 3 I 4 1 4 I
0,001 0 i 34 I 0.941 0,691!
0.4MI I
0 To 9 l O.ool 0.261 0.34 1
I
0.341
3 I
36 I
I
7 !
I
7 I
j
0 1 0 1 2 t 3 I 10 I
0.09| o; 1 7
1
3.091 0.601 0.1711 1
0 T 0
0
1 0.001 0.171 0-261I
0.861
4
0 1 1 1
I
2 I
I
43 I
I
6 !
I
11 0 1
? f
21 I ll I 33 1
0*03* 0 -0 91 0-17! 3.691 0 -51 I
• l
0 -o 91 0-00* 1 ,8(, I
I
2.831
5
0 I 4 I
I
0 1
I
20 I
1
313 I
j
11 0 I 10 I 9 1 12 I
0 • 0 0 I 0-341 o » o 6 1 1 • 7{ I ?6.R5I 0 - 09
l 0-00
I
0 . 86 I 3-691I
i • 0 3 I
0 I 0 1 0 1 1 I 0 I 2 I 01 0 I 1 I 2 I
6I I I I I
•0 . 0 0 I o.oni o.ooi 0.091 0.011 0 Tl 7 J 0.001 0.001 3*091 0,17]
7
o I 0 1
I
0 I
i
0 I
I
2 I
I
01 01I I
0 I 0 I 0 I
I
• 0 • 0 0 I o • o 6
1
o.ool o.ool 0-171 o-ool 0 • 0 0
I
0-00l a-oni 0 • 0 c I
0 I 13 I 6 I 7 I 7 I 01 11 74 I 4 I 7 I
8 I 1 I I i : J
• 0 • 0 pI 1 . 11
1
0 • 5 1 I 0" 6b I 0
• 6 0 I 0*00I 0 •
0
9 i 6.341 3-341 0 • 6 0 I
9
0 1 12 I
I
7 I
l
8 I lo I 0 1 1 I 1 I 169 I 26 I
0* 01)1 1 • 031 0- 6 0> Cl. 69 I 0 .8M1 l
0-00 ! 0-0 91o- o
91 14-4811
2*231
0 I 4 I 1 I 23 I 23 I 11 0 1 8 I 34 I l2l I
10 I I I 1 I 1
0 * 0 01 0-341 o- o
91 1-971 1' 9 7 I
c • c9
10 0 0
1
0.69I ?' 9
1I
1 O'
3
7j
2 I 40 1
I
63 I
1
118 I 368 I 6 1 2 I 119 I ?34 I 215 I
TOTALS 0’1 71 3.431 5.4511
0
• 1 i 1 3l-53l 0- 5 l I o-l 71 10 ‘ 2o * 2 0* o51
I
i8.4?1
i/O RATIO --
REVISED I/O ratio 0.951
extended indirect 7.798
EXTENDED DIR ;CT -- 0.171
TEACHER TALK
STUDENT TALK 30.249
STU/TEAC TAt.4 RATIO 0.599
EXPERIMENTAL
GROUP
17
x
17
PRETEST
MATRIX
142
*O
CD •
O• *
*CD
< *CD -f*
. *
*« CD*
O*
*CD
*co -*
** CD
•
*
•
* CD» *
>o **
• +M CD
CD *in *
**-• * *-« *
* CD<X *in *
*
*CD
CD *^ *
*
•— -D —. —CD
CD CDIII * •
ro -* o
CO
0)
•Huocoa>
pcd
o
* CDCTrO *
a i cdn *ro * 'cd
*
• * f CDCD CD
crro cd
% +
. CDCD CD
< •ro * cd
«
. 4i 1CCD CD
• i -CSJ * CD
+
- « XD •
CD CD
H * CD+
*•
H CD
<=> CD
CD
I czCD CD
c\
j
'O
oCD
+ *
cv
CD i
CD CD
<X
K;
ro
• r^CVJ
COK»
uro
osi
vO
CD*• •*«
Qro
UJro
inro i
O' i
ro i c\jC\J l
H •
(M N |
• I
I — — ,
I O' I
• *0 HII • I
01 CD | o° * CD CD I CD CD
‘ *“ » — I — — — |
•— I *-• .
•IT I
C\J I C\J
• r
CD i
• —• •
* (\ i .
’ ^ * — — — I
CD I CDI
I vOi in
ro i
• i
CD I
I
• ^ |
N t
^ I
• I
ro i
» i— ~ ~ • — —• ~ i
• i *-»
CD •
>0 I 'CD• I
O I
O'
—• —» — I •—CD I
CD I C\J
— I
f\J 1
I
I
— —• I •—< •
O I
CD I
I
I
I —
CDO
• ~ ~ I
I oCD
I CDCD
* * *
* — — I
CD I I CD I I
* « * I * +
m
EXPERIMENTAL
GROUP
17
x
17
PRETEST
MATRIX
(continued)
**-*
CD *O *1
¥¥ H* *-i
CD ¥* *
*in
¥O* *
*
*CM
¥CO -+
¥
CD. **
COo in
CM
O'
fO rHITCM
* incn roin ¥ cm
*
*
*in *
* M'<* * -W-l
M- *
in
>cin
NCO
* CD¥ CD CD
III *fO ¥ o
*
* CD* CD CD
CD '»* »- • '
fO CD*
¥ sT¥ yri I CDO *
to * CD¥
• IOCD CD
cc ¥ro * cd
• ¥
CDCD CD
•
ro * o*
• ¥ so¥ ri CD
* ¥ .
CM * o«
* * CD I
• CD CD+
H # CD#
* ¥ ¥• +
CQ.in
% ¥ •
#
’ C*
f CD O CD
I CD
I CDCD CD
OCM
I CDCD
I •-
CD *
CD CD
CD
* *
*7
fN
O'n. otocm
H
roo
* * *
o
143
inin
tro
inCM* * r
¥ ¥ ¥
-X
I
•
I o• inI y-i
I
O I o• l CM
cd i roi
1 • i —° --*oCD
CD .
I CMI O'» CM
• — t —• —
*
I Cl I
CD I IO• I COCD I CM
I
' —« » »-» MO I
CD I CD• IN
» I O5 I TU
v- O I CC
oCO
¥ ¥
CD
TlTALS*
0.c0
I0i93l
5.521
8.991
6.7„*1
i^I
0.
12l
3.471
t.
55
14.
3^117,
54I0-
6flI
0•
0
0
1le
•
1
0
120
•
4019»
861
O’
561
EXPERIMENTAL
GROUP
17
x
17
POSTTEST
MATRIX
144
00-0
j£I•
0Xo0-
0
fOO’P
Io0*oJu0•
0
j^.O
jOO.C
,,0.0
r
00•
0T00T0
tOcIo
,oo.o
,ooT
EXPERIMENTAL
GROUP
17
x
17
POSTTEST
MATRIX
* IV* H CD
CD *CD * CD
• *
* Tf* *-r rv
<* * tHCD * CD
* HH
O *+
i
4»
ro
<o•
* ro. **
• *•* CD
• *>o *
CDCD
I oI CD CDI •
• CDI
* CDi ro c\j
i •
• —• •
r^ i rv
145
fO I ^• » NCD I
rvo o
*r
CD I
I
I •—I
i roi
ocsi rvj
LA
cvi
:t=> t
^ i ro<3io ro• i ro •
a> t • th o (
^ i CDro I CD CD
• >c* ro
ID * Ola * ro m-
* CvJ
rvCM
<IA
•— | •—4 I—» •—
*
i <r i > fvCM & M* CV
I
l
I CD
fo*i '<**>
I
I • *—« »—
•
CD I CDcd i ro cm
» •- i •CD I CD
I
l CD I I CDo i ro cm• i
CD 0 CDI
I
K I > fvH I O sO» rv
CM
OCO
* -oCD *
4r
* CD* CD CD
CD ** •
ro * cd
CM• «
* oC ) «ro *
m-IA
. • '
HCD *ro
CD!CD
CD CD
• —I
I CMI CMI
COM-
• r^cc
I CDCM U>
o
*
CD-X *KJ *
roicCM
CM * CD+
. * CD »
« CD*
H *
. *
. *CDLA
*
O
I CMI A)
» I * *
I
I CD I ICCD I CD CDi*| I •
CD I CDI_ I
CD I I
* » n *
roCM
* *
00
TOTALS*
i»
a*
Io*
6iJ2•
®?113.o9^
3>6?I
O^D
1
(TOO
13•
29I3*
54I
5.
97J28
,
I0•
1o»
8lIi2*
96Il3
’
83I
10'00
I0*
471
QUASI
-EXPERIMENTAL
GROUP
17
x
17
PRETEST
MATRIX
146
—• * —• «
* oCDO *
• 'll
-H
* CD*
CD -tt
CD | oI
CD I >0o I c\| tH• I
CD I CD
roro
• i
* CD
CK «*
CD+
CO '
CD
**
.
* CD* I
o *
vr »
c\j i m-
i
i —
•
ro »
ro i cm
• cm• in cmI T"f •
• CO
(-4
CD | ,
I
i roi
CM
-X3
•— — e .
CD I oCD | CD CD
I • I i •
CD | CDI
• I —• —ic: i cco I H CD
cd i ir ro• i •
CD « o .
I
CD I O• I
O *. >
ro roro
i
i ^ i
i
i cc
i •
cp
•* CDCO *in *
— * ~ i — —
^
* CD I CD* CD CD I CD CD< * • •
LA * CD I CD
9 OI
fc CDCO *
*
*CD
<x
*
*O
III •*
ro *•*
*CDm mi
ro
* : C" I C# l O ' C I CD IC
ro *
.
crro
—• •— I
cr; i laiH CD i n
• « >n *
cd i in
i
o .
CVJ rH i
' cr I
a i o I CD G I CD
I CD
CM
* CD
CD f <CDCD • CD CD• I •
O I CDI
' —• I
I oI H
CD I O » »CDrl CD I CD CD I CD CD
•* • • I •ci a i a» i
»o I lO I ‘CZ
* *
CMiDro
*
cj
iia
* * * i * *
• CDCD CD
airo
<M-
* *
CC
— I
ic I
I
13 I I
* * * I
QUASI
-EXPERIMENTAL
GROUP
17
x
17
PRETEST
MATRIX
(continued)
*CD
a ¥CD ft
*
o<CD ft
• ft
* ro
o *ft
ft oCO *
ft
H• ft
ft
—• ft —« *
-It
ft IA* ft
'O ft
147
r\;
^r
C\J
K)ro
rvin
'Ocvj
coo
roroCVJ
COCD
I •cvj
I •-
CNIT
CO
*Cn ¥ y~i
in ¥ ro¥
¥¥ CVJ
IA ¥
¥-ft
vC . Mcvj
IAC\J
GOO
ft ro¥ O
III 4 .
rO ft oft—1ft «-i ^
ft CDft CD OO 'ft »> • *
fO ft CJft
•*+ft •— —• —
•
* * crft • a i o
OftfO ft i CD
ft
*-* ft —• •—• ft ICD
ft o ocr ft
fO ft CD• ft
•“* ft
roft
• ft —
.
• CD
O
» CDO
in
ft ro OJi *
ft *
ft
ft ODft IOft
ro i oro i o cd
T tH ro
CVJ
IAGO O
IA
CO
cvj i
CVJ
ITro i
CD CD I O
>Cgo ^r
• vCvJ
H I CDI
» — _ —
.
O I CDCVJ H I CD CD
COro
roro
tH i cd
o i oi
CD I
CD I I
CD • CD• I
CD I
o-
CO I
• I
CD |
ro i
+—*i
'T I
I
-•*-* I
i
\r i
• i
rvi i
i
ro i
H I
* I
CO I
ocvj
TOTALS*
0*001
l.
3nl
l.ocl
2.361
QUASI
-EXPERIMENTAL
17
x
17
POSTTEST
MATRIX
148—• « — .
* OGC *o *
*
•*
* o<* *CD *
• *•-* *
* CD*
o **
* CD
CO -*
*
*« CD0*
* +* o
• *'O <i
** —
*
I
> o•
I
CD I CD• I
CD * »-
I
*-• I
CD I
CD I CD
r
* CDCD *in *
*—• * .
*-* CD
<T *in *
• *
•CDCD *-cr *
I c: i
CD 9 Y-t
* l
CD I
I
• »— I <~i
CD •
CD I O• r
CD I
I
I —
*
—• *
fO *•*
CDGS '»*
fO *
«• o
c> *ro *
«, * •— ,
. *CD
cr *ro *
. *
• «CD
< *fO *
• • « •
CD I
i CD •
CD f O• I
CD I
I
CD I
O » CD• i
» —• —4 —. •-» I —• —
H *
t• *
•
% I
^ *
*<
ro
inro <\j
>o
.CD
oH.CD
Oo
OPfO
inro c\j
cvj
co
oro
coCD
! *—• •—* •—
i
CD » oO CD I CD CD
00
CD I
CD I CD
I
CD I I CDCD O I CD CD
• » -
CD I OI
* * I *
oro
turo
^ i
fv. i
• •
CD I
I
- — | *—* •
CO »O I rH• I
CD |
I
ro i
ro i cd• i
o j i
i
- i *—CD I
CD I CD» * I
CD I
•
- —• I «-* '
»<r i
CD | H
'O I
ri tf N• 8
CD |
1
—< —• 1 *-•
CD |
• CD I CD• i in
CD I
I
' •“» —-• I +-•
ro i
iro i cu
».
r< I
I— — I —
•
CD I
• CD 1 CD
I •
O I
O CIO• I
o I
I
I •-« '
0 t
CD CD I CD
O I
I
I —' c I
CD 1C I O• I
1 o »
I
— — — I —
'
l CD |
CD CD I CD• I
O I
I
I -o »
CD CD I O• I
CD I
I
I *—
•
— I -*CD » •
O I o• •
CD &
I
* * I *
CO I
CD I
—• I
O I
CD I
CDI •-
• •— I
ac
cc
in
o i
h i
O I
• I
CD I
I
> —* I
C> I
O I
o I
O I
i CD I
• I
O l
* CD I
O I
• I
»
I
l_> I
CD 1
• I
O I
I
I —• •
ICD I
O I
I • t
CD I
I
• — I
CD* •
CD I
• I
O •
•
4* *
CO I
lO I
I
• I
•
QUASI
-EXPERIMENTAL
17
x
17
POSTTEST
MATRIX
(continued)
149H
CD ft
CD ft
— ft —
•
ft
< *CD ft
• **-• ft —
ft
ft roft
> **
-• ft —«
** CDft
co -ft
ft
CD
ft
*-•*-*.ft
o• ft
>o *ft
—• ft —« •
* ft
CMm * c\j
in * V<1
ft 'T< ft
IA *ft
ft •
ft
*
• K
ro
roro
ccir»
ii «ro * c
*• *— ft —
ft CDft O CD
fT 4*1ro ft cd
%• •— ft
o +ro *
*• ft *—
. ** t-4
or *ro *
4' —' ft •—
-ft
* CD
, ft *—ft
ft
» —• ft .
ft
ft ft
CD > CD
CD
O CD
CDI
A
ft ft ft
•f^
vOO vOvC*-4 ro
ccH O
VO
I CDCD CD
CD CJ
CD
ft ft ft
CC
ft ft ft
t »»I
i r*
—•—• i —
«
cd » rvo n> O
I r-|
l ^I t4
I HI
I
I
I -
— • I • •—
I CD I
CD O 3 -J• I O
CD I TTI— —• I —
o r
rvi h i in• » vO
CD G IN• t
CD I
—• I —
•
fO
I O I CD• I v
I CD I
I
-— I *—
•
1 CD »
CD l r-f
- I T'.
C. I
I
-— I *-
CD I
CD • lO• i r\j
. i —
.
o r l
ft ft i ft ft
QCCD
TTTALS*
o.00I
0.09
12.30I
5.341
3’?
J1°
•
I0-25l
3-671*
3.55
!5.35
138.-91
0.«6l
0.081-6.7,1^.601
CONTROL
GROUP
17
x
17
PRETEST
MATRIX
1504 —
.
I
* O4 —• —« _* o< 4
CD 4• *
44 CD4
CK 44
4 CD4
<© 44
•— 4 —4 CD
. 44
0 1 CD I 0• 9— ^
o » coQ 1 CD I CM r-i* 1 *1° CD I CD
* I— — * — — — I •—* «—
•
4 CD
O I
I
• —• •
• cz t
CD I
• I
'0 4 oi CD 9I 1— 4 ~ ~ — ~ I <
I —
4 CD13 4in 4
4•— 4 —
.
44 CD
< 4in 4
1 —CD 9 CDCD I CD CD• t
CD 9 CD
— 4 -« —. I • *—• «
4 :cd*9
• CD I CD
I
I
9 —
.
9
I
9 —. •
I
I CD
d
r
O 9
CD I CD• I
CD 9
4 —
.
44 CD
• 9.
CD 9
I
•— I —•*O I
(\i i n <
• •
CD 9
I- 9 —• — -
O I
O I o
en90 4
O I
I— » — — •
CD I
CD I CD
O 4 .190 4 IQ I 1 CD
4 |
4 Ocr 490 4
4 CD<X 490 4
<M 44
•— 4 t
«4>
4 O4
t-I 44
4 4 4 4• 44
9
I
9
9
9
1C «
CD I
• • 9
CD •
•— 9
CD I 9
CD 9
• 9
CD 9
9
4 I
9
9
9
9
I 90I
I
I
9 —9
I CO9
I
I
I —
-
I
9 CD•
I
I
CD 9
I— I •—CD I
r*v i m• 9 (VCD I
9
—• I —
«
CD 9
CD I CM• I
i » —«I CD 9
CD I CD
• I
O 9
4 4 4 9 4 4 4I
• CO90 » K>
I
* 9
I
aooo
NOCM
CD I
CD 9 O• I
CD 9
I— 9
O I
CD I
- 9 CMCD I
9
CD I
CD I CD• I
O 9
I
—* I —
.
CD I
cd i rv• I T-.
ftDi 9
ic i , c:CD CD I CD CD
• I •
CD I CDI
ic i mo O I (M CD• • H *O I vH
I— I
CD I CDCD CD I CD CD
• I •
CD 1 OI
I
CMin
oCD
ro-
I cCD |
—• I
I C
oIs* ^r
in
CM
• — i
IO 9
4 4 4
O
CD. » CD »
0 9 0 C
4 4 4 14 4 4
r=3
90UJ90
cr <in
CONTROL
GROUP
1,7
x
17
PRETEST
MATRIX
(continued)
151+* CD
CD ¥CD ¥
¥
•— 9
O ! Co i o o
¥¥ O< *o *
• *—• 43 —
•
¥¥ C\J
¥O
¥¥ CM
<0 ¥• •*
¥¥ CD
• *r- *
¥ H» *
O **— * —
* ¥¥ CM
CD * COin * ro
0 9 CDN I O O- c
CD I CDa
—• •— i
r-l I CD
H I CD
I •—I
I —
«
I
a h
*— —< • *—• *-» •
• Ml oin C rH CD
ro » cd* ro i _H
< *in ¥
¥
* x-l
on
CD I
t
• — I
o »
CD I
* I.
¥ CD •
* I
* -• *-*^ I
CO vr I CD CD<r ¥ cm *|M- * CM i a
i
> ~ * — — —c 3
CD I CDCD CD I CD C
III * • I •
ro ¥ cd i oi
> *— — — — i —i
CD I
CDOro *
*
•*
C 3 *ro
*— ¥ —•
* OCI>
ro ** i
CD I
I —« — —CD I I CDCD • CD CD• I
CD § CD
¥¥¥¥U9¥¥¥CDin
• —« ¥o
I
i in
CD I -r-i
CD CD I sO• I
CD I CDI
I —• »-• —
•
CD I >0CD CD I vQ CD
• » CVO I H tH
• HI
COy-1
CD I CD I CD
CD I OCD CD I tH O
9 • » I *- •
CD • CDI
I —< —• —
•
t C Il C
C f CD I nO M• I H •
' CD I HI
— — I — —« —< cd i in
C O I CV o• I «H
CD CD I
• •
a •
i
i
¥
r>*
* *
if
o
* ¥
O
o
noto
roCM
CDCD
inro
i —
•
i
tH o I CD
¥ ¥
CD
i
1
TOTALS*
0.00I
0.531
i.67I
4.391
5.
7.1
0T^3I
O.OCI
9.Q4I
D•35I1
.93I37t53l
O.^Rl
0.
00Ij6.
Sj.
19
.
0I
2*391
0.
O9I
CONTROL
GROUP
17
x
17
POSTTEST
MATRIX
152
*cd
ra . *o «
* cdfti
CD•
• i
CD*o **
*CD
•ft
CO ft
ft
ft —
«
ft
ft CD• ft
. *CD
C I
CD I CD
•—— r —o i
co i o
o i
CD | CD
|
in i
- i
CD | CD I
• I
CD • »
I
I
CM
oCM
vOcm i
• I
CD I
I
• —• I —* —
•
I
T-l t oI - •
iXD I
o I tH
I
I *-•
H |
in i ro• i
«D i*
I
oCM
• I
O I
CD I
t • I
CD •
CD I CDCD •
I •—I
• •-« I
CD I
O I
• —• » —
•
I C I
CD ft CD
l I <
I
I — • —• —
—
I CD I
CD |
CD I
ft CDCDin *
*
*CD
<in ft
*
*CD
<CM tH t
cd ri
i
CD I
CD I CD• I
CD I
O I
CD »
• ft
—* —• J —
•
CD ft
I CZ
CD I CM
CDCD
CD I CD I <
• I— »- — — I — — — I — — •
—
CD I CD I <
CD • CD I CD I ' CD »
—• I
I CCD I
*CM
M- I
ftO I
CDI CD
ft C
CD C
ft
M- ft <
ft
C
ft o <
in «ro ft i
M*
K>
OCD
¥ ••
tHCD • *K) «
o>CD I
CD ft
I
—* I —
•
CD »
CD I CD• •
O I
I
CD I
CZ I CD• I
O I
I
I
O I
*-• I
i
H ft
•— I
CD I
O »
NO
CM
*-• >
CD I
CD I
O I
O ft
ro *
. * i
ft occ ft
ro. ft
I CDCD CD
. ft • *
CM ft CDft
o in
CD I
I
•— »“• I
'r- |
i
I CMI CD I
I
I —co i
o I CMN I
H I
I
1 O I
CD CIO
I —' • I
H» •
1ft
* *-<
<fO
ro i
i
o i
CD ft CD• I
CD I
I— —• •
O ft
CD I CD( • ft
CD I
I
•— —< I —o <
»
CD ft O» ft
CD I
ft
ft ft I ft
ft
CD I
ro •
i
• i
I
cz i CD I c:
• ft
•— — i —* — i — *
i — —• —
•
• IO1 CD CD
Ol ft
• ft
O I
ft
• —. ft
» CD ft
CD I
l • ft
O ft
I
• •-« ft
O* 1
CD ft
• ft
IO ft
I
+ * * i ft ft
CONTROL
GROUP
17
x
17
POSTTEST
MATRIX
(continued)
co
cdCO ¥CD +
*
*CM
CD *»
*¥
•> *
*¥ CD
tH•CO +
¥
¥¥ CD
» ¥fx *
*^ * *-* .
. *tH
» ¥'O *
% *
in33 * CDin * ro
¥¥ *~i
•< *in
• ¥
** -<
cn**
«* rv
-*¥•*-*
• •— ¥•—•——.¥ CD
CD CDC I •
ro * cd¥
'OOJ
t :
i
to . CD
• * < CDCD CD
03 ¥ •
ro ¥ o
1^CM *-4
<X ¥ m
to ¥ CD
CM •
•— *
CDin
¥
153
I
I >0• CM
-I CM
• Cv
• COI o ^I 'OI H M"• HI Qn
• CD I CD» I
» CCD CD
CD CD I
I C I *'O I CD CO
CD t CDCD I O CD
cd cd i ro
cd i o i r'-v
CD CD I CD O I CM tH4*11 > • 4 | • • 1
CD | CD I CDI I
| , I • *—
•
o »
o
I oI •-*
I
i
i roI CM CDI H> -HI
I —
• I
rvro
c i
oNO
I —
•
I M-i ro• CMI
I —
t
I
I O
ro
co
inCD ,
CM |
» —» I
I CDI
• I— —— I *—• • I •
CD I
I
O' •
cd l ro- i incd i ro
i— i — — »
ro i cd i
O I CD- I HO I
I
*-• I f-*
CT »
cd i ro• i.
CD |
CD |
• • I
CD |
I
—• I
in i
CD |
• i
<cNO• I
CDro i
o i
go
>cCM- I
ro i cd in
in
i —
•
i
I I o:g l
O' •
CD I CD• I
CD |
i ro» vO
M- I CD I •-*
m- ro i cd cd » vr o• * • i roa i cd l CM
CD l >0
• I
CD |
I
EXPERIMENTAL
GROUP
17
x
17
DIFFERENCE
MATRIX
154
CDO oCD *CD * r->
** O
< *CD
• •*
*+ CD
o **
CD•*
CD **
** CD
I •—I
• O
CD |
• *“• I •
CD |
CD I
CAI
r • ra 1
1
roin
C\J
ro
rorv
i
i
i
i roi t
inro
CDro
C\J l
• iO I
coXT
C\M
N *
* +* CD
< *vO *
*
CDQG *in *
* CD<< #in *
• *
•*
4 CD
*
**
r c~
CD • HI
• Cl ccO n y-l CD
•— — I —. —. V—CD I vOO I tH CD
• CDl.
CM1 I
•— I
• in »
o i ^r
CD I
I— —• I *—• •
CM I
CD |
I
I
I C_ I
» CM t>
• r
CD I
I
CD |
CD CD |
• I
CD •
' *— I
>o I
• 'CD I
acO
l
I CM
CD | oI
•—• I *— —• —•tH | ^CM I CD o• I •
0-4 CDI
I •
IT*
in
•— i —CM I
'O l tH
C IO I
^ •
CM |
CM ,-t I
* • I
CD |
I^ I
0 I
r> rH I
• • •
CC3 |
I— I
CD IO o I
1 • I
CD |
O •
• I
CD »
I
* —« —• I
1C I
I tH I
* I
CD |
I
• I
o IN N |
CM . i
rH I
I
I
CD I
CDi CD I
* l.
CD f
I
'O |H 0 8
CD |
CD C I
cr>
ro
i v»• -i
c >
ro
cCD
I
CD I M-
O f CM tHv • 1:| - • 1
CD I CDI
— * —• I
•— I
CD |
CD I
1 —• I
cr i
' CD |
•— 4 —+• *
CDGT» *ro *
i •—i
I CD
•— I • —» *—CD I >0
C\i
i in
inro
CD I
— * — —in
4
4 4 4 4 4
- I —V * •
iO • CO• I H
CD •
•
I 4•
I
I
I
•
I
1 —• | .
CD I I
O I CD
4 4 14 4 4
in r
CM I CD
I— »—• t •—
f'* I
CD I CD• I
O I
I
* CD I
CD CD |
I • I
CD I
I
CD I I
CD CD I
• I
CD •
rocrro
* 4
oro
* « 4 14 4 4
IXr^
•CD |
CD |
—• — I
O I
CD O I
* §
CD I
I
IN •
tH CD I
I • I
CD I
I
I
CD I
CD O I
• I
4 4 4 1
»
EXPERIMENTAL
GROUP
17
x
17
DIFFERENCE
MATRIX
(continued)
155
CD
* —* *
o• *
• ** —• .
*
*cd *
*—4 * _
** O*
o*
** T-l*
•co +
+* fO**
• —• •
'rH I
• *-• - 1 4-4 *_ ^CD I oO I CD 0• • •O I o
I
—* I —4 —
.
«n »
CD » tH CD
I O
rO 8
COCD
» CDI
I *—4 •—4 —
4
LTV
00CMHI
» ^ —CM » CD• •
CD I
'O
* ;„* to H
• * I
* O** 4-. _co
— * OJta * 4-1
*—
.
* ro**
** —4** *0
&
CO
in
o
I I'O
i t
»-4*-* i —. —
.
H I
OJ | CD• I
CD I
I
—4 I —4 —4
onv *
'T *
III *
ro *
-4 ^ _* CD
CT?o
* 4-4
*CD
* — 4-4.
•
I
I
I
I
•
CD |
CD |
* • ‘*1
CD |
I
cd i m• i
CD I
V— I r-4
cc I
CD I
I
—* » —•
CD I
—• I
CD I
CD e CD• «
CD I
I
CD I
CD .1 CD•• I.
CD •
I
—• I —
i
CD I
CD I CD* I
CD I
I
1 —4 | —4 .
CD I
CD I CD
in
CMI
ro(M
vC
CM» r-
xr cm oro
inCM
V •
H |
in i
• »
<* i
i—» «
M* |H I
• I
V (1
in i
H |
cri
in u* i
CD I
H |
' — •
ro »
o i
• i
h i
. 4-4 |
CO IH I
CMI
CDro
* ~ ^*-4 | ^ —
CDro
* T~i
*— I —O I
CD I CD
• CT I
:C l>rt• I
CD I
I
»— • •—; cd f
CD I r~• I
CD I
o
(\i n*T~i
• CD
vOro
•rn
roc\ CM
r
' o i
T—^ |
• •
4-. _
«4f *ro *
• ** CM
CM *
4— # —. —4
*-4 *
*•
CD | CD• I
CD |
I
"-4 | *-4 4-4
v •
4-1 I CM• I
CD I
•— I - •"«
CD I |
CD I CD• I
CD •
I
f
I
• O
4-4 | 4-4
ro i
H I CD
CDI t |
CD I CD* •
CD • <
I
* • +«
I
• h«
• I
r^ i
• »
CM |
I
•
CM |
H i
• • I
ro
* *
oo
H + •
ro
v » |
m a
• iH I
(
I
QUASI-EXPERIMENTAL
GROUP
17
x
17
DIFFERENCE
MATRIX
156
CD ft
CDin o
*
in
inro
o oo
n*o
ro
ccro
ft i a* o o
<
O I
• l.o •
I
I Hcn i cm
CM
CD i
•— i
c i
* — i
cr i
i o l
(X r I O CDC I
I CCD
CD I CI
O I OO I o o• I •
g • ai
i • *-•
ic I CDCD I v-i CD
O I
I
I —ro I
(V I CM- I
CD I
I
-• — t — — i
CD I
CD I CM• I I
CD |
I
-- I •— — -
CD I
CD | o• I
C I
I
- I *—• I
O I
O I CD» *M *
O I
I
- —: C* I
to I CD
O I
I
- — I - •—'CD |
O I O• I
O I
I
•G I
CD | CD
ft ft
CD
- I
CD* •
CD •
I
* t
oro
• i
o »
•
ft i
i
ft ft
i aK>
i
i _CD • CDO I CD C• I
CD I CI
ft I ft ft ft
I
I <i xr
i
- •
ic i
CD I O1*1 i
CD I
•
o : »
o I o
ic I
o |
•— I
OI I
ft ft
CD
ft l ft ft ft
QUASI
-EXPERIMENTAL
GROUP
17
x
17
DIFFERENCE
MATRIX
157
CD . *CD I CD CD » CD CD
0ft
O « CD • CD
*+ ft —
•
— — | 4-4 —4 | _ T . ,f
in 1 O 1
* CM H | CM CD< + 1 •- | •0 O | CD 1
• ft—• ft »-« —• 1 •-4 _ _ | _ _ ^* CD 8 CO » OO* O CD | H CD 1
* • | 1 • |O * O | CD • CD*
-* * — —• —• r —4 _»_ |
* O 1 O 1 Oft NO v 1 CM CD* 1 • 1 1 1 • 1
<33 ft 0 . j IO 1 .CDft— ft *—• 1 —
•
__ t
co t CD 1 M-* CD | CD CD 1 h0 CM
. f •- b ! • I rrv ft CD 1 O 1 CD
-*
— * —
*
•-*•-• r •-4 »—* | k-4
. •* r~ 11 C 1 1 CD
* in M" 1 O 00 » CD CD' ft » • | IT • 10 * CD ft 1 M* • CD
—• ft —
•
— — | — _ |
• * 1 C 1 i V 1 >Cft co CM 1 ro CM 9 CM
2D * 0 • 1 • m- f r
in * h+
O K CD 1 CD
—• * —
4
_ —« | _ . , . |-« . . , ,
* in 1 CD • CD* CO -0 1 CD CD 1 CD CD< * • • • «. |
ir> * CD I CD 1 CD• ft
•-4 ft—* —• — I —
*
—• •—4 | •-4
ft r* 1 CD 1 CD« CD CD | CD CD 1 CD i CD
CD - * k * 1 •* CD • CD • CD
—• | —
•
•~4 >—• | _4 .
ft 1 CD 1 CDft H CD 1 CD CD » CD CD* l • » • |
M- ft CD | O 1 CDft — —• | —
•
—• —« J •4 44* CD 1 CD 1 CDO CD 1 CD CD 1 C C
III ft • « # | •
K3 * CD 1 O 1 aft
•-* ft —
.
•—» —• | — —. —
1
| B-4 *-4
O 1 CD I O* CD CD 1 CD CD 1 CD O
CT ^ • • • 1 • •l| • • Hro * CD 1 CD I a
*•—« *—
• | — —• . | *—
•
* ' cr 1 • C~ 1 cft o • c » CD : cr 1 O i cO * • 1 • 1 •
ro * CD I 1 CD 9 t CD•*
—« —• | _ —, | •— »-4 •_• « cc 1 CD I .CD* r-i 0 1 CD O 1 CD CD
CD • 1 • | 4
ro ft CD I CD 1 CD. ft
H* « M. —• . | —. *— |•—
•ft cr • CD 1 • CDtH CD 1 CD CD • CD CD
< * • • 1 • I •
to * 0 1 O 1 Oft
—4 ft _ •-« 1 •-4 —• —• | •“4 »4ft 0 1 MD 1 l CD
CM T-« 1 CM •f 1 CD CD• 1 » * | • • 4 | 1 •
cm ft CD 1 CD • CDft
*- —
•
—• —• 1 — — |
. ft O 1 • CD 1 CDJft 0 CD 1 CD CD • CD CDft • I • 1 •
H ft CD 1 CD 1 CDft
ft ft ft ft ft 1 ft ft ft 1 ft ft ft
. ft
ft CD 1
• ft in • 'O • rv
ft
1 ft • 1 • • *
ft
H CD » 4-I CDft CD CD
• 1 1
CD • 0 e CD
— — — • 44 -4CD 1 in 1 o-
CD CD 1 CM tH 1 ro CM 0• » • • 1 mCD 1 0 1 CD
— •—4 •—« |
.
—
< _4 M-4, , ,
inCD 1 H | roM- I O ro 1 to CM CD• 1 tH • 1 1 •
CD 1 tH O • CD
— •—< •—
4
|*~4
| -•-4CD 1 CD | in0 M- 1 CD O f ro CM
ac 1 * 1 1 • |
r>v f ». CD >|
— — 1 —4CD 1 CD
ft CDCD CD 1 CD O f CD CD CD
1 « 1 4- | I
CD I CD I O— •—4 • | • — J 4 —4
cc 1 CC f 1 crCD 1 r-t CD | CD 0 CD
1 • 1 1 • | •
CD 1 CD | CD
— —1 •—4 | 1—4 -4 — » ^4 —4M _ .
• 1 M > • C\N in 1 ro CM f CD CD to» * 1 t * | • I
CD I CD | CD
— •—4 ——• }*—
4
-4 | -4 _ ^4 , ,
00 1 CD | in•H CD 1 CD CD | ro CM CM1 • | •
CD 1 O « CD
^4 —. , •-.^4 , .
CC 1 CD | crCD • CD CD | CD t CD H• ft • l •CD 1 O | CD
—* 1
*_4 •-4
CO 1 M3 | M-tH Cl 1 Cvl H | CO M3 O
• 1 1 • | t •
O • CD | O-4 —4 —
• I ^4 —•144 . ,
CO 1 CO 1 CDT~t O 1 r- CD | C cr1 • 1 1 • | •
C 1 CD | O— ^4 r—« , _
CD 1 O | CDCD CD 1 CD CD | CD O 0
1-91 • • 1
• 1 • • 9
CD 1 O » CD
^4 _4 _4 t •_ —• -4 , .
O | cr i • C>0 '/' 1
1 n c | a • O 0• 1 1 • 1 •
0 • 0 1 ; CD
_ _ 1 *4 _4 , .
r- 1 O 1 ccCM r- 1 CM H | 0 0
• I 1 • | •
CD 1 CD | CD
__ _ , -4 | _V I 1 4—*
J*> • CM H | in 0• 1 • • | •
a 1 O | O_ | ^4 —• ^4 , ,
00 • CO | CC
CD 1 H CD | 4H CD H• 1*11 * 1 1 t • •
CD 1 CD ft CD
_4 _ | 44 •4. • •—4 ^4czx 1 CD 1 | CD •
CD CD 1 CD CD ft CD CD CD• | «
O 1 CD ft CD
ft ft ft 1 ft ft ft I ft ft ft ft
1 <cr - 1
1
1
a
• •
CD
«
LH
o
ooo
o •
ocd
CD
o
CDCD
CD
Cl CD
CD
CD
O
CDOc
onCD
O
CDCD
C\l
Wrl
I
00cc
m-i
o'C
C-CO
cv
m-
in
i
CD
ft
TOTALS*
o•oo1
1:221
1.
00
I
3.491
£.5M
q^qO
1
O'OO
1«*2
61(j*
11I1•231
7*291
5.6^
10
732
1
?.'36i
1C
.59l
0,4
310
.56
CONTROL
GROUP
17x17
DIFFERENCE
MATRIX
158
* ’ CD* CD CD
CD. «o « c• *
* C3* CD CD
< * •° * CD• -ft
* O* O CD+O * CD*
CDCD CD
•<0 ^ CD
*
* CDCD CD
. +N * CD#* —• —. —
•
+ : c* CD CD
. •*
>0 * CD•fc
• + 1C* CD CD
CD *ICV * CD
+
* CD* CD CD< *
lf> CD• *
* CD* O 1 CD
CD ** CD*
CDCD CD
<cd
* CDO CD
Ilf 41
ro * o4r
fO
rs» <ri
vOro
•— *— i —
«
H I <7CD I ro
LA
«
roro
C)ro
crro
—« CD
<rro +
• CD •
I
CD >
• •- I
a> i
.c •
• i
o i
i
O-l CDCD I CD CD
* • I I I- • 1
Of CDI
'•-* I •-* •—I K'
ro i o co• * i
1 CD f ' CDI
o i rcc I A i
o i
HI •
O I
CD I
I *- |
CD I
I
O » I
CD I
• I
CD I
I
• + I
I
3 I
ro i
i
• •
i
1 o • O |1 U o • H CD I
1 • 1> 9 CD 1 CD |
1
• o • fO l1 CD CD • CM
1
• • « *. |» O 1 O |
• |
• CD * N |1 CD CD • CM H|• • 1 • |
• CD 1 CD f
1
• CD 1 >0 |
1 O CD • LA rr I• • >t (•!1 TD ’ 1 CJ> |
l |• 1 _ 1• CD • CD |1 CD CD 1 O CD |
1 1•- 1 *1
1 CD 1 CD |
9 |1
• i c: 1 O |
• CD o 1 rl CD |
• o » o 1
1 _ _ _ 1 11 • fv 1 K" 1
r CM rH D N Of1 • If • |1 CD 1 CD t
» CD > CD |I CD O 1 CD CD 1
• * 1 • 1
1 CD 1 o |
1
1 CD 1 O I
• CD i CD 1 • CD CD |
C • ' • »
» CD * CD |
| ^ _ 1 —, — —. ,
1 CD » in i
• CD CD 1 CM to 1
* • i ro • i
1 O • I CM 1
| f ,
1 CD 1 CD |
1 o cr 1 O CD |
i • 1 • |
1 CD 1 CD |
| _ | ^ ,
» o * CD 1
1 O CD 1 CD CD |' 1 • « 1 4 • » |
i o * CD »
1 | M «M |
1 c~ 1 cr 1
1 O 'C • CD • CD |
1 • 1 • 1
1 1 CD 1 CD |
| __ _ _ | |M •—1 CD t IC |
1 CD CD 1 o o »
1 • 1 • 1
1 C. * CD |
> |
1 •-«|
1 CD 1 'CD I
» CD CD I O CD 1
1 • • * •
1 O 1 CD I
, _ _ 1 |
• 1 o • • CD |
1 CD CD 1 CD CD I
1 1 • 1 • • |
1 CD I CD |
|
1 CD » CD 1 I
• CD CD » O CD 1
• • • |
1 CD 1 O I
t_)
K)Oro
LUro
i
i •—
CM
I
I
CCAJ w
° O I o
ic i cr
I CD I .CD • C
CD I I
CD I CD• I
O I
»
DD I o•- I
• CDCD
I
CONTROL
GROUP
17
x
17
DIFFERENCE
MATRIX
159
C I
> o I o• IO |
I
' —• —• I —• *
i
ro c\j » cmc i
o i
• i
O I
csi
ccco >o
i •
io
o I O I CM
i oi
CMit
o «
CDin
• •
O I
I
* c * o* i id i i r-s^ ^ * H O I O o | H CDN -It - I •1 rN. | O I CD I CD
I I I
I • I •
I CDI
CD » CD• ft
CD •
i
• —I
I CD
CD I CD I CD I OO C I C CIC C I t- C
CD CD I CDCDro
c D
K>
crro
I CDCl
«
— I
C~ I
C I
• CD o • OI • •
I CD I
• •
• I
C I
•CD •
CD I
CD I
ft I
nCD
I •
CDIT
* •
CD I O CD | O |o CD ft CD CD • CD Cl 1 O CD ft
• • 1 • |
CD 1
1
CD CD | CDft
•O | K> CD | >o ft
in CM | CM 1 CD cd i ro Or |vH • 1 M" • • •H | o CD | in i
>o • CD » Nfto 'T 1 T O 1 O CD | >0 in •
00 • l CM • • 1 CMN | CM CD | H CD ft
1 1 H ft
i—• •—
•
• M —• —• | 1
O i O | *—4 |H CD I O in 1 O CD | O PO 1
• 1 1 • • 1 O * 1C » 1 o CD » ft sO ft
—
i
^ •M Mi 1 ~ MM | ^ MMft
CD I CD CD ft O |
CD CD I CD CD 1 CD CD | CD CD ft
1 • • ; • •> | • |
CD •
1
o CD ft cd a
i d 1 O' ‘ d | ri |
CD O 1 tH CD 1 CD cd i ro •• • i
CD • CD CD ft O ft
_ | M _ M a — MM |
• r-v i rr 1 d i • d 1o « 'o CM • o CD | CD O ft
• • i rv • 1
CD 1
1
CD CD | ft
1
O 1
O 1 re CD ft r^ 1
CD CD 1 H o 1 CD CD | CM CDft
• • 1 H • 1
CD 1
1
CD CD ft f
1
H ft
CD 1 cr C 1 O' 1
CD CD 1 *-> CD 1 CD CD » CD |
• 1. • 1 • 1 1 * I.
CD 1 CD CD ft O 1
m. _• Mt 1 — MM |
CO 1 H CD | N iH CD 1 l/N 1 CD CD ft CO |
• • i ro • 1
CD I O O I ft ro a
m* MM | M m. ft •— MM | M MM |
O 1 CD CD I rv i
r* c » o c: 1 d cr i CM r* |
• • » \ • 1
CD 1
1
o o 1
1
C |
N 1 CD cr i |
CM H 1 CD CD 1 o CD i >c in i
i « ri » • » *n * * r i
CD 1
1
CD CD ft
1
O ft
• C 1 cr c: i ,
T- r- 1 t- CD I d d i ro ft
1 • • 1 CM1 • |
CD 1
1
O CD ft |
1
CM |
V | o O ft ^ |
\r K) 1 y* o 1 o o i ro cr i
• 1 1 • • i ^ • i
O 1 o o | *» i
, . _ _ | — __ __ i — •MM |
O' 1 IO O 1 IO 1
I^s. m • ro CM 1 CD cd i in CM |
• • 1 • 1
O 1 o c a H ft
.. « MM 1 _ M* M 1 — MM |
CO 1 O' *C 1 r- 1
CM H 1 H CD 1 CD CD ft CD CD |
1 1*11 * • l • | • • •
CD ft
1
CD CD |
1
CD ft
CD >1 CD 1 | cd : i O' 1 1
CD CD 1 CD CD 1 CD cd • CM H ft
• • 1
O 1
•
CD CD ft
1
CD ft
ft1 to t
1 < X 1 -J 1
O 1 cr O 1 • «J I
11 1- 1
• ft r* •
1 1 ft- i
160
APPENDIX C
STUDENT EVALUATIONS
Technical Skills of Teaching Descriptions
Sample Student Evaluation Computer Printout
Factor Analysis Matrix of Student Evaluation Items
Pretest and Posttest Comparisons of Student Evaluationsfor Each Sub-group
SOME TEACHING SKILLS AND BEHAVIORS161
SET INDUCTION
Set Induction is establishing in students an affective and
activity?prediGpOSlt:io ‘1 t0 in a given instructional
skilfndU£
ir rCqUlra
fknowlnS «hen and how to exercise theill. It is appropriately exercised at the introduction
a course, class meeting, or new learning unit. Prooerimmg also demands an awareness of the need for reorientationbecause students seem to need their interest and motivationrekindled. Set Induction may be achieved by referring toprevious activities to establish a useful frame of referenceby indicating the process to be used to obtain goals( scussion, lecturing, guided discovery, etc.), by relatingthe significance of particular goals to personal or coursegoals, by defining key terminology, by providing studentswith a course outline or syllabus, etc.
Set Induction is related to Making Expectations Clear in thatthe instructor s expectations for students' work, progress,standards and participation may be one function of SetInduction. However, Set Induction may aluo include teacherbehaviors which arouse interest, motivate action, focusattention, establish relationships, etc.
LOGICAL ORGANIZATION
Logical Organization refers to the sequencing of materialseconding to some internal structure in the subject matter,the principles of effective learning, and student character-istics and nedds.
Effective performance of this skill requires not only thatthe instructor be able to sequence material and activitieseffectively
, but also that students understand the logicof the organization. In some cases, the organizing principlemay be self-evident; in others, the instructor may need tomake those principles explicit. In any case, LogicalOrganization should allow students to distinguish digressionsfrom main points, to take notes and make outlines with
relative ease, to see connections between topics tiered nt-vations times, and to recognize relationships between classcontent and course objectives.class
Logical Organization is distinguished from Set Induction inthat the latter establishes an initial frame of reference
thaie
f
L° 8iCaf
°r^anizal:iou refers to the ability to maintainthat frame of reference continuously.
PACING
Pacing is the rate at which new ideas are presented and theamount of instructional time spent on each of these ideas.
Skillful pacing involves an appropriate matching of thedifficulty and significance of the material to the amountof instructional time necessary to ensure student understand-ing and interest in that material. To accomplish this, aninstructor must be able to judge when adequate definitions,explanations, and applications are completed, to avoid spend-ing more time on an idea than its significance and/ordifficulty warrant (s) , and to modify his pace according tosituational demands.
Pacing is related to Level of Challenge in that the difficultyof the material and student: ability to deal with that material
influence the rate at which new ideas are introduced andthe time spent on those ideas.
ELABORATION
Elaboration refers to the development and clarification of anidea, definition, or process.
Effective performance of this skill involves knowing when toelaborate, as well as choosing appropriate ways to elaborate.Elaboration may be necessary when an idea is especiallysignificant, when the material is difficult, or when studentsare particularly interested in a topic. Common Elaboratiticutechniques include the use of greater detail, examples, andanalogies, as well as rephrasing points, pointing out relation-ships, explaining relevance, illustrating applications, etc.
Related skills are Monitoring Student Response (to ascertainthe necessity and effectiveness of Elaboration), Definingand Expanding Relevance (as a technique of Elaboration),Asking Questions (to check comprehension or to encourage
163students to elaborate from their experience), and AnsweringQuestions (which may provide occasion for Elaboration).
CLOSURE
Closure refers, to the instructor's ability to "wrap thingsup at appropriate points in a discussion or instructionalactivity.
Effective Closure depends upon a good sense of timing andthe ability to choose an appropriate technique. Commonly,thxs skill is exercised at the end of an instructionalactivity, class, content unit, or course, but it isappropriate whenever there is a need to re-establish ormaintain a sense of direction and purpose, or when clari-fication and review of preceding instructional activity seemsnecessary. Techniques of Closure include providing a summary,asking students to summarize main points, checking forcomprehension or consensus, noting whether or not the topicwill be pursued later, etc.
Closure is complementary to Set Induction in that good SetInduction makes Closure easier, and good Closure makes thesubsequent Set Induction easier.
EXPRESSION
Expression refers to the techniques of verbal and nonverbalcommunication.
Effective Expression not only allows an instructor to conveyinformation and ideas clearly, but also to communicate feelingsand attitudes about the material. Characteristics of thisskill include variations in body movement, facial gestures,and voice quality, use. of silence and eye contact, selectionof vocabulary appropriate to students' levels of sophistication,and other rhetorical skills.
Expression is related to several ->ther teaching skills and
behaviors in that effective performance in this skill may helppromote effective performance of the other skills (Set Induct-ion, Pacing Elaboration, etc.). Expression may be an especiallyimportant factor in Inspiration, which refers to a more generalstylistic characteristic.
INSPIRATION ANT) CHAR I SMA164
Inspiration and Charisma refers to the instto bring excitement to his teaching and hisand to engage students' interest, trust and
ructor's abilitysubject matter,respect.
Inspiration and Charisma are most often characteristics ofteachers who appear to enjoy teaching, who are enthusiasticabout their chosen subject matter, who are concerned aboutwhat and how their students are learning, who demonstrate aninvolvement in the issues of their discipline or theirprofession, who have at once a commitment to the value of whatthey are doing and yet a readiness to re-examine that commit-ment, who have confidence in themselves and their studentsto engage successfully in a given task and who can exhibit anappropriate sense of humor about themselves, their studentsand the tasks at hand.
If an instructor is successful in inspiring students andcommunicating charisma, he may need to rely less on a varietvof other skills to motivate and engage students in the learn-ing process.
LEVEL OF CHALLENGE
Level of Challenge refers to the instructor’s ability toselect materials and design learning experiences which willchallenge students' interests and abilities without makingunrealistic demands upon their time, abilities, or chancesfor success.
To find the optimum level of challenge, an instructor mustconsider student characteristics (their experiential andinformational readiness, their apparent ability level, theirinterest in the material or course, their time commitments,etc.), as well as characteristics of the subject matter(conceptual difficulty, background knowledge required, etc.).If level of Challenge is set too high, students may feelfrustration and a sense of failure; if Level of Challenge is
set too low, students may become bored and disinterested-.
Level of Challenge is related to Set Induction ( as an
occasion for establishing the initial Level of Challenge),Pacing, (Level of Challenge will influence the rate at whichnew ideas are introduced and time spent on those ideas),
Monitoring Student Response (as a way of checking whether the
material is optimimally challenging), and Making Expectations
CuTl u
VhCn an lnstructor explicitly define Ae5
level atwhich he expects students to deal with the subject matter.).
TREAi.fENT OF DIVERGENT VIEWS
Treatment of Divergent Views refers to the instructor'sability to set his material in an intellectual andattitudinal context which permits and fosters open and honesttreatment of that material by both teacher and students.
Teaching behaviors which may promote honest treatment ofdivergent vievs are acknowledging the existence of a varietyof points of view, presenting facts and ideas in a repre-sentative context of divergent opinions, distinguishing betweenfact and opinion and between raw data and interpretation,making one s own biases explicit, encouraging students toexamine a variety of points of view before making judgmentsor drawing conclusions, etc.
This teaching characteristic is related to Creative Inquiryin that Treatment of Divergent Views may provide studentswith the disposition and tools for independent, criticalinquiry. Moreover. Treatment of Divergent Views may providea context or occasion for Defining and Expanding Relevance,and the examination of divergent views may promote Student-Student and Teacher-Student Interaction.
DEFINING AND EXPANDING RELEVANCE
Defining and Expanding Relevance refers to the instructor'sability to help clarify how a particular idea or topic fitsinto broader personal, social and intellectual contexts.
Effective performance of this skill requires not only that
an instructor be able to define and expand the relevance of
a topic or idea, but also that he encourage students to engage
in a constant search for larger contexts of relevance. Some
techniques for Defining and Expanding Relevance are demonstrat-
ing the application of an idea in a variety of situations,
explaining relationships between a topic and other academic
disciplines, exploring how a topic fits in with student
interests and goals, suggesting implications for various
social issues, etc.
This skill is distinguished from Elaboration in that the
latter involves filling an immediate instructional need, while
166
Defining and Expanding Relevance refers t-n .
extend beyond the course and subject mttcr « falso distinguished from Ethical Context Skills fthJ.s««n
8
:^« d
t,
r,tandi?s and ^ 0 r ve
and ofUth0Ut che partlculir emphasis on consequences forand of one’s personal values and behaviors.
FACILITATING student participation
Ikinfinffirff^PafiClpation refers to «« instructor'sHo in eliciting student participation in the learning
fruitful directions.
^
levels’ leading it in
An instructor may wish students to participate in the learningprocess in various ways, ranging from asking and answeringquestions to sharing responsibility for class activities ordiscussions. The instructor needs to communicate when and howmuc stu cnt participation is appropriate, to encourage them togive such input, and to allow enough time for them to partic-ipate m the ways desired. Sustaining student participationinvolves maintaining a non-threatening environment and respond-ing to student, contributions in a reinforcing manner. Effect-ive direction of student participation requires that theinstructor know when and how to intervene productively (byasking questions, providing focus, tying student comments to-gether, etc.).
Facilitating Student-Student Interaction is a related skillin that interaction among and between students is one, butnot the only way, in which students may participate in theinstructional process. Asking Questions is also a relatedskill when questions are used to invite student participation.
FACILITATING STUDENT-STUDENT INTERACTION
Facilitating Student-Student Interaction refers to theinstructor's ability to promote discussion among students inthe class.
Effective performance of this skill requires that an instructorestablish a physical and attitudinai climate in which studentsmay interact with one another. This nay require anythingfrom dividing the class into small groups or arranging chairsin a circle to providing a structure in which students areassigned specific discussion roles. To maintain student-
167
mai
•»«and respond to one another *
an°ther> Pr°d then t0 e*™‘ne
:rin::r“ :r;r^:rdiscussion ^rtici^l'^St:^*^*^ ^
,#CtlV“
thate
someone's ^S"J±SSti^s
d
i^“
Faen^a
t
tlngesUfnt'»tUdi!nt I"teractl°" is °ne means of
fo^uM8J dent Participatlon - However, here we arefocusing on those teaching behaviors which encourage studentsto interact with one another and to work as a group
ASKING QUESTIONS
ms skill refers to the instructor’s ability to use a rangeof questioning techniques for a variety of instructionalpurposes.
Effective performance of this skill requires that the instructorunderstand and be able to use several types of questions,ranging from simple questions requesting factual informationto higher order questions calling for integration and applicat-ion of ideas. An instructor needs to know when to ask and howto construct questions to suit his purposes (to monitor student-progress, to engage students in critical thought, to sustainstudent attention, etc.). Once he has formulated the question,he must be able to state it clearly and in a non-threateningmanner and to direct it appropriately (to a whole class, orto an individual). An instructor may facilitate students'answering of questions by allowing enough time, by not interrupt—in£> hy rephrasing questions if necessary, by not being tooinsistent, etc.
Asking Questions focuses upon the specific techniques of effect-ive questioning but is related to several other skills (Closure,Creative Inquiry, Facilitating Student Participation, MonitoringStudent Response, Evaluation) in that an instructor may usequestions in his performance of those skills.
ANSWERING QUESTIONS
This skill refers to the instructor’s ability to listen to
168
students' questionsjudge when and howconstruct effective
for a variety of cues and meanings,questions should be answered, and toanswers.
to
Students may ask questions because they are confusedeager to contribute an idea indirectlv ^urious
’
needs to be Tlr-rt- t-oRLc
‘ * anc* t ^le instructorneeds to be alert to the reasons behind their questions Hethen needs to decide whether a question is best postponed,
-elf^ort0 a
T!!er
ftudent: 0r to Lhe inquiring student him-oeU, or answered by the instructor. If the instructor choosesto answer the question, he may help the entire class benefitby rephrasing or elaborating on the question and then phrasingnis answer in clear, concise, and generally understandable terms.
CHOOSING APPROPRIATE MODES AND MATERIAL
S
Ihis skill involves the instructor's ability to maximizelearning by integrating his objectives and personal resourcesstudent characteristics and goals, situational limitations,and the availability of instructional materials.
Effective selection of modes and materials requires that aninstructor be clear about his own objectives (to give informat-ion, to encourage discovery learning, to develop criticalthinking skills, etc.) and select inodes which will be mosteffective in reaching those objectives (discussion, lecture,reading, etc.). He needs to take into account student goals(do they want a quick introduction to the field or are theywilling to engage in thorough exploration of it, etc.) andstudent characteristics (apparent ability, background know-ledge, previous experiences, etc.). He must consider thesituational limitations (class size, meeting time and place,etc.) and deal creatively with those limitations (break theclass up into smaller groups, adjust meeting times or attend-ance rules, etc.). He must know his own strengths and weak-nesses (good lecturer, but can't get discussions going; hatesreading exams, but loves reading papers, etc.). And he mustbe aware of the availability of materials and resources (rulesfor books on reserve, supply shortages in book store, etc.).Once the instructor is clear about all of these factors, hecan better design the course modes (lectured, discussions,panels, etc.) and select the course materials (reading lists,textbooks, paper assignments, etc.).
INSTRUCTIONAL FLEXIBILITY AND INDIVIDUALIZATION
Instructional Flexibility and Individualization refers to the
ns true tor s ability to assess individualtics to design and, adapt course objectiveswith students' personal objectives, and toways for students to achieve course and/or
students' characteris-which arc consistentprovide alternativepersonal objectives.
i/ UCt° oay becone aware of Individual student talentsintet ests and needs in a variety of ways—by devisim:questionnaires to elicit such information, by inviting students™ him individually, by providing tine and co^unS -
their ide!"tl,at StUdentS feel £r« to approach him with
info-noM r c0nc,;
rns > etc " 0n« an instructor has such
for ^ud ^ may SiSn C°UrSe '•hich allow room° ‘ de’ C* 5° pursue tbelr Interests and develop their
ness willbd ‘’w
0137 cn!;
OUnter situations in which his effective-
or to andeITd
UP°n 1S WilUnSness t0 chanSt those objectivesor to allow students to substitute their own objectives. Annstructor may provide alternative ways for students to meetobjectives by suggesting several options, referring studentso other resources when his own are inadequate, asking studentsto devise their own strategies, etc.. In general, effective
performance of this skill requires an awareness of studentsas individuals, a willingness to be flexible in determiningobjectives and methods, and creativity in providing alternativecourses of action.
Flexibility and Individualization is related to most of theother teaching skills in that effective performance of thoseskills often involves the behaviors and attitudes suggestedhere.
FACILITATING INDEPENDENT/CREATIVE INQUIRY
Facilitating Independent/Creative Inquiry refers to theinstructor’s ability to demonstrate, encourage and guideindependent, original, and creative inquiry.
Facilitating Inclependent/Creative Inquiry implies not only thatthe instructor demonstrate creative treatment of the material,but also that he encourage students to approach the subjectmatter in imaginative and creative ways. An instructor mightfacilitate creative inquiry by suggesting and providing resourcesand materials for students to explore independently, by makingexplicit the processes he uses for considering ideas, designingstrategies, and making decisions, by helping students to developand utilize the scholarly tools and skills required for in-dependent research, by making himself available for individualguidance and feedback sessions, etc..
170
*ro»
students, while here we stress shying studenL“how ^'‘V6tln8they are motivated, may pursue Individual, creative work!
MONITORING student response
Monitoring student Response refers to the Instructor's skillsin noticing nonverbal indicators of student response, ininviting and encouraging verbal feedback from students on allaspects of instruction, and in responding to such feedback inconstructive ways.
Monitoring Student Response may provide an instructor withuseful information with which he may make judgments anddecisions about student progress, his own performance, classactivities and assignments, and course design, An instructormay monitor student response by being alert to nonverbalindications of confusion, curiosity, frustration, etc., duringlectures, discussion or other class activities. He may getmore complete information about student reactions by invitingand encouraging students to ask questions, express theirconcerns, make suggestions, etc., about specific instructionalactivities or about the course as a whole. In order to maintainsuch a system of student feedback, the instructor also needs toattend to the ways in which he responds to student questionsand suggestions. If he reacts defensively or simply ignorestheir suggestions, students will stop giving them. However, ifhe responds in non- threatening ways and experiments withalternative methods and activities on the basis of their suggest-ions, student feedback may provide continuous data for designinginstructional activities to meet student needs.
This skill is important in the effective performance of mostother skills because continuous awareness of student responsesto teacher behaviors may inform an instructor when he makesdecisions about how and when to exercise other skills.
MAKING EXPECTATIONS CLEAR
This skill involves the effective and timely communicationof what the instructor expects students to do, to learn, toknow, and to contribute as members of his class. The skillalso includes the instructor's communication of his criteriafor evaluating student performance.
171
Effective performance of this skill allows students to know
Partial-,atio“' StT r^yin
,i"" ° f
arl fot1 Students should know what their responsibilitiesleadings, papers, etc.; their range of options In fnrmstcontent, and treatment of assignments; work deadlines; and,
* *
^sequences for unsatisfactory or late work. They shouldalsoknow when and how they will be expected to participate in class
pff°n3
’ ectures’ etc,
» ancl what preparation they need foreffective participation. Finally, they should be awaL of theinstructor s methods of evaluation (quizzes, tests, papers,observation of class participation, etc.), the nature of evaluat-ion instruments (multiple choice quizzes, essay exams, etc.)and the criteria for evaluation (the skills, knowledge, con-’ceptual sophistication they will be asked to demonstrate)
.
This skill is complementary to Evaluation in that MakingExpectations Clear refers to the communication of what isexpected from students, while Evaluation deals with the assess-ment of how well students have fulfilled those expectations.It is also related to Set Induction, which may be an appropriatetime for an instructor to make his expectations clear.
EVALUATION
This skill involves the instructor’s ability to provide feed-back to himself and to students about progress toward statedobjectives.
Effective evaluation of student progress requires that aninstructor inform students about evaluation procedures, thathe select appiopriate evaluation techniques, and that he keepstudents posted about their progress. Students should knowwhen evaluation of their work may occur (surprise quiz anytime,mid-term exam, class observation anytime, etc.), what kindsof evaluation procedures may be used (muJtiple choice quiz,essay exams, oral questioning, self-evaluations, etc.), andwhat obj ect i ves will be evaluated (knowledge of facts, perform-ance of skills, application of principles, etc.).
Secondly, effective evaluation of student work involves theinstructor's ability to select or construct evaluation procedureswhich are appropriate for assessing desired performance andwhich are consistent with stated objectives. Lastly, aninstructor needs to know when and how to give useful feedbackto students about their progress, so that they know where theystand, what areas they need to work on, and how they may goabout improving.
Evaluation also refers to the instructor 1 ^ .
w«ays of assessing his own performance and the ef fect ivenc!^of u, course. He may get such information by looking' u“student achievement, and by asking students for feedback on
directiTLS (
?PPrP1
' lat0n "S8 ’ lnterest levo1* clarity of-lections, etc.), on specific class activities (were theyinteresting, did students feel they learned anything, etc!)on teaching performance (does the instructor lecture too muchis he an effective leader in discussions, etc.)!
’
Ideally, evaluation provides useful information for theinstructor and for students which makes further progress andgrowth possible. s anQ
This skill is complementary to making Expectations Clear ( r> eediscussion of Making Expectations Clear) and related to Monitor-ing Student Response ( as a way of evaluating teachingperformance and course effectiveness.).
DEFINING AND EXPANDING ET1TT CAT. CONTEXT
Defining and Expanding Ethical Context involves both anawareness of and explicit communication of the fact that theteaching and learning of a particular subject matter takesplace in a particular social context.
The context in which a particular learning activity goes on(and of which it is a part) implies a host of values, problemsobligations, challenges, etc., and the actions or non-actionsof teachers and students have meanings and consequences withinthat context. An instructor who exercises this skill is awareof this fact, attempts to make students aware of it, andengages in an exploration of those meanings and consequenceswith students. The stucients themselves are, or course, a partof that social context and, therefore, the instructor's conductof a course implies responsibility beyond the immediate course.
The instructor is the agent, by which the value context of agiven subject area is interpreted for the students. Thestudents are conditioned by that interpretation, and by theirown perception of the course goals, to make judgements concern-ing the relevance of the subject matter. Thus the relationshipof the discipline to the instructor, and the instructor'sinterpretation of the course goals to the students, should befully explored in the teaching process.
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
173CC I
•c,
CO
2
3CD
C/i
to
oor.
o
tuto2OatoCLcr
c2
OZDor
luIXi
«o iotoI m
Sojo
COCO
o•XoUJ
Ui
ICIO
<>— I
X I
ao*cc
> i
1 « O o XX
•-1
1
r^. 2 oo 2
!
^ 2i t-»
13 — O1 ^ X • UJ2 CO CDO
3 UJ O»- X Xu x ro ro CD ^3 o HX o • b- 21— o H rv Xto < CM in o2 o u.o IU 2
*»- fr-3 coIU to UI a2 UJ X 2‘X UJ <x.X o
1o xCO X?
U> j*-•— ro to
oCO *-
2 X UI •IUo VJL
oO co X >- H 2CO v. in - CMCO UJ CO JC *~UJ X *- > UJ CO3 IU 2 •«*- C5 b- u< c
a
7»CO X > -<c > cCO j£. 3 2 o3 o UJ b-3 X •X 2 COo < (\J o 3 •— ro fO UJ x in
oolr
rvin
i
u • cv^ -J
oo2 CO— UJ2 *2 «<
X15111 >-CD 3
-J
UJ 3X toX 3
c/ 22 a
-t
cr ujo -J-o acr -<*
b-co 2*
2 Oco t
LU IOX UJk- CO
COUi3o
I
Cm
2O
COUJ3a
a ua x-a —
*
o UJ*- 2O•2 I •
cr uj•
CO —
•
2UJ 1
UJ oX*- a
I
inro
oUJIsi
C5Xc
UjX
O I
CL '
I
ICIO 11
caUJ—
I
o
X
*x
X
Icier
CV oCM
UJCOUjCr.
Xa2<t
cr>
UJo2a• a
xof"uXI-CO 22 O
X i
UJ COX *«t
N crt C2
o•X •icric
co aO
-* —• uO 0L-C CDSC P=-
O 2»- LU
03 UJ•*- m to toco xrto coox t-l
a —. cmX
> CO-J 2•-J o<3 b-co <3 3
UJto x o »o•— co
UI- x cm*- b~ VCO UJCO UJ<1 CO-Io o-CO
ax 2 in *-«
c < »is
inm o , •
3 -X 2H <CO t—2 C—. Ou
CO 2X*- I- 1
CO2 O*-• X
inro
2O
COUj3O
COu3e
2O
COUj3O
toLU
Xo03
3UJ
uXCOXUJ
EX2
XC
cUj-<Xo ocn i-< IL(C3 J
UJ -(OO UJ
rvin
uj <t
X Uj
CO C3X 2o >o >o2 cr. a:3r 3
CVX >• M-O 03
o >-
tr iut- >CO —
•
2 r-•- o
UJ 3UJ Ll.
X u
2o
3C'
CM X H VCM w4
V NiTi
•
o• 2
UI O* *"• !-
h-*<
CO b-u IC1C •2 IcricX Q Uj o• CO i •
>— u UJ uXUJ Ul*
u. emo oX »-X “ UJO 2X LU *X 2Z CJX CV<> 15 2
CM UJ m• CO NJ“ M- 50
w-t >- a CM2 <
CO *< XUJ X«•- • u •3X o to
•X * I— o >-T. 2 m3 2 LUCO 3 cr X O M3
.c U X CO3 L. X 33 O CD - CMX m - VIU D CO >-•
CO 2 Ui3 JU X cOCJ a
UI Qdl <2UJ o u X« X JC
X> o XO — IOK) o >- K)K>a 1/ V • »-
a v CJ U •
3 >X CO X a CMo — -1- to CO oCJ* CO 2 UJ•I ;<i XX 3 1ft.
LU UJCO X2 X. - Li-o X
2 CO HUi CD UJX 2 X u1- IU 'JC X
toUJ3u
2O
COu.
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continued)
174
r. or>«
X
oXUJ'X
CO2o
ttl o U •t-
u o 2 c m»- le-.^r Ll r, r> • *- !C 1Co rl > X CD •» Icic „ « ic ic> l • S3 • » o • • CD w CDr*
13
—
#
X r-«
ew a*-
CD Ui
XCD
G-Da CD
1 • UiUI X a > a
»- UJX a
UJ 2 X. o—I in a>- o a
»- X c*n v fir. < -c
• 2•*4 UJ CD 2 a> cr c cr •< f. o2 • X » •_ 2 c X 04O fv Ui Oi Of G •
*"• - UJ o a* -rt— 2 r* f- G r4-< »— » - , n
1- - t— *- 22 — c 2Hi - 2 2 ato ua *- U LU Ocr uj t- «< y -t-a 2 X a
• K C LL n to _ cc ^ inLL LT ici X Jr C- o- o u CD IO> 1- • Ui • o IL •V »»- lr—•2 CD t— r4 2 rv C a> a AK Ui m. •— CW <A 2 *<r o tCO X o •— o i
—
'• NIL IL X Ui X MU. > UJ > - b a t-L. C •— 2 X cnJi X an — a - a
Ui <* XX ar2 » X UJ c cn a< Q •— X UJ »—O a o cn t- 2m x *\ *-» in Of O CD CD UJ <s •—
»
-< r- 2 €V V C > ^ N CD CDX »- • r. •— IT a C - crm ^ o cr • a 2x c, n X X • . cr < CDO DC o o X 2 Ow XK c »- t- o UJ o «O X u u 9- •— *- V,O r> u 2X a X X - » C*- c »“ t- X a Xcn (/) cn t- c t- »-2 LL 2 2 m cn -<
— u — 2 K 2 c
kr\
cra.
uiX
cr< 13CA *-iW IALL IdleU *— ocu z •<1 JL a
LL C.x ax aQ
UJ3 f. DW C4
3 t <r*OX o
*— 7)2 USsj ac crz- 3L C C *Ja cr1L Loxa
IA <ill *Aex< ULa o
cXo
UiX
LL!
X UJ —X c- a
UL ax a
c ^O c
XO Go —-r l.
x
— UJc —
X i-t- in
muor
LOJ
o
oXcao au
*c erCL
x aui -a
Xx m3 —
in- lin ox —
•—G
-n x-s
X DCo a-o cD 2cr **
i
t-m «*
2 W— XiM
fiC Cl—x a
u. :nc c
cr. llL_? XD—I CO
X cl_ —2 inUr LTO 3C i
LL.- ir
*- o
2o 2
C
3O
inu
inIL3O 3c
cnL.3C 3
O3C
I
33,71
M,57
‘7.34
OUFSTION
j7
T>l£
LFV^l.
OF
0IF|r
ICULTT
AMU
TMp
AMoUNf
f)K
WORK
In
THIS
OCURSfc
,
A«f-
Af»0t)r
WIGHT
r°R
MF?
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continued)
175
i
z in
> i
Mtu
I
I
•ci
L> I
I
ii
U.o
Idot/J o>- :
. .
3r to
LU DC> 9~
O^ Qcr otu —
>c >c—i “ or><2
-‘T CM3 rfZDo oX
2* 3oro toI* h-zo tu
*— CL c\ O* 12 CV
to *-k- co ^LU CO<3 CLZD .JK- IUCO
lO C33 2O <t—J H V— cr.
< LL>
or —o &-*- uO UJ-r 13tt CDH otoI? Lii
*-* LOcr h
UJ ZDZL Ch- O! j
I 1
1 to lO'O 132 CD 2» O • *«
CD i
1— 1
.
' 3 '
COx; UL
• |
jO
i cr i 2P-
;
1 UJ' '2
j i-iO 1
I cr°
!
UJ
CO IcioQlX
UJ CD UJCD 1 1 •
CD %or J
I LU—> ! • 1 UO *- CDCJ > «•
ULI2 O 3UJ - X
UJ o2
rr rUJ UJID l- in a< <UJ 2 • UJ3 ^ m XX to *-2 C>
c c LUoz
to 1— <*CO * X3 2 CO<o COCO uj oc> >—
CD C k> ro'j to
LU « • K- •> CD H 2 CO— IU CM UJ LUH CO CD XO -3 •3 —. IU o 1— &
—
u_ o inUk LU uiUJ CO h-
in UJ <2 2 H •—<t CD cro >o > X i
CO 1/ ac 2 Cto • •- a3 CM CL
n: o *0 cr xo to o <>- — K-10 Q O H
• •“ -Q
cr uj rr•- X COCO >
—
CO 22 2 O•— CO •>. •—
«
a 2 -
IU LU UJ ~x u X Xr- jc h- o
Icjo
>c- oCO
f\l
CD
I oI OL I
COLUQZ
CDZ<2UJ
o»-
COI—2UJQ3
rvLTV
roroM-
wCM
m> *occ
cvj
IOIO<=>
•cjoo
I •
COUJCtt
<ccz>oO l
LL
ao cc»~ UJO X-r •
cc o*-CO X2 O
UJ >
atX o
2o
>fr-
h~-2UJ
mro
co o—* ^
—
UJ
;
•»*
CD
I v-» M* **1
»N
CO• nr> UJ
1 CO|
I
2I <<
1
|
-OC I
2o
2*
oCD *-
ICICCD
• •
*I
uClu.
oy~
i
~
QO rlV• l— v~i
UJ CO^ x rvUJ
>- CD2
UJ>—» *-*- 2O UJUJ CDU. ZDU k- 'O >otu tr. cc
co2O o c\j
vr
*- oCO UJm xc3 o(J
oto >~LU <5 Oc/l L • CC12 CO
CMor 2o —
*
- oO CL
CC *-K- 2CO <2 I—— cr
c %-»
uj aX X
CC<x3ocr -c CO (VOu Cl C42
CO 'X ~r2O CO— or•“ UJto DLU 2ID ZDC3
CO* COco -*4 lO. vH
1 *- "— * rs.
2 L>
to touj cr12 30 o1 lOl->2 C/l
UJ —
.
a X'3 s-•»- ICICto u. CD
' ou CDO I-Ik LJ
LU^ cl*- to
3 >-K- 2cr <*
oa cdCL 2C —
•
C
J
a x2 <<t CD
UJf- cr2UJ toX XUJ toCD —<J |(J
cr ~
C\J O'C\J
cd!CJ
LU m 3 J- CDQ uj ro O inZD 3 u ozb- Cl 2 o(O X UJ
X CLoo CO MUJ LU UJ>- X 13 CO< k- >-. i
—
h- 2r/j K N O O ID V NU » C • a X ioa: X a XH CD o coa. in oc O Cdo >- o>- b-o X u CO•X •x 2cr re op~ i-in 2 co 1
—
2 •— 2 toUJ
X cr ir4UJ LU ULI CDX X X ZDb~ H- H CO
ICDLLb~«€
cr»~
coZDcru.
cro
nUJ
• auCD
LUCOZDu2ooUJcr<
2UJc3I-(O
UJXXXo2X
CO>:UJLL
CO
crab~oa:f-co2
U;X
fO
OCM
LT'- t-1
ini^>
co o CD Hrl *•!
.
n,j
CM
ii
i i
^ 1 2o O| o O
!
p~ “ - -co CO to COif LU LL1 Uir> 3 3 | 3CJ o
| o o
CMCM
2O
COu>3o
•oCM
2O
COUj
3O
CM
2O
3C3
inCM
2O
COu3CC
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continued)
176
CCLU
to
cocrLUXI
cott.
—Ducat
Q2
COLLI
CD
QC3oOz
cr >-o »—
orin -a ccz — j%-o o
CVJ oOJ
t— o inIO ZLU13c cr
-ato —ir cjLU 2:i -<
(/) Ktf N2
-JD
!«.
to*«c
to*-
I
zicica
I •
0z<t1
a:UJX
fOloo
I
K~ o u o V)to • zz
li.
—
J
t-i
OJLDo
*—• LU I b-r I
X O ot~ 1 ID 1 zCO o CL -<LD t- 1< !
ori
LU ocLU CDK- Zoo if) uj2 o H |—i • • u
_J o t— •UJ < a
i<> ^
ix u •
1
3 K z*— a <
!o •Z i IO *3<t <x t— 3
i XZ <t1Ct - * UJ >Ia
UJ t- O UJ 1 3X z z>N LLhC Oif. J—X -t c\
cl <rtoid *. -*
t- xuan *€*- c:to irz Clo OLx: c co yr
*- id o oto cr t\
o o m-»— - <o
a: u-<t crUJ <5
3«D to
UJCO oLU 2!< <VS
O V* TT
0 lO| ^
1
UJ
• e i •>- to'.5
i
•! »-^ o w
•Z3 -U*- o r-2 orui x a•*- Ic ic <*- >21010w o -I t_l
' - CDi;/ u) oID 03(X rv4
O MoCt IDCL C*
cr LD•— t—UJw O' X — tC K>
OJ Ik- a: *<r
• Cl^ Z O t-f
C tt cwa.o Ol
uj <c•K-
CO 3O 2CL -ej
CO •»
IO K tO Vn z in in- LU O
in 0 2 CDfO z> U» CVJ
*— Xto at
Li.
UJ 2X> C2I- UCO _JCL <
CVJ O'CM
Q3
toOJ
V J
UJ•X
<o:<C
• XCD
O• or
xCD
oz*IL.
OCO2 CMO
I
f~co
SIO
I
I— -3 ft lf\ rJC LL* #1^cr •—
t—‘C
to ^X3 K-« ok— ai
“3
>- GO—I 132 tO
to IO
tnto
toz
t/ in LL r - X IT 1 c U. CJ CCaj• X • Ct IT
C: • c: c X a ID • V CL LL •U OD o to o > CM :l coC3 CM cr in cr U. CM or in or cr. v. cjZ o >- o or o o a o in13 t- to k- LD >- t- 2 -- o <
u.o Q. o o o X
t— cr or or E E * a: o“"• t- C5 t- *— c- CO K t-3 (O Z co LU co to to LOz z X z z •<t z LU* UJ «-• - _l K—X rf cr ^ •H u ^ <1
ID LD LU 3 LU LU IDX X > X O X X z X UJt— t- 03 1 1- X 1— —
•
>- or
!
o CO 1 o o i
riru CM CM CM to to
1
•z Z i i
-*J
i
21
2o o O o O O1- K - K(/) to </> to w toUi LL U. U u. u13 13 r> 13 13 3O O C5 C3 O o
ICS
»cIOIC
'ZD
CD C2
CD2 to
•ft I
toI^uj
!
» •—if) uct- LUz LLCC Xu LU3K— *CO - UJ
Q. CDZ 3 oto
*- UJ LUX 3
t- 31_J to o O
uj ti- zerUJ ZD
IT
r- z trz —3 Uj3 X
to IL1
3> Xi
*-* LUt- O cr
: D Z <t-X <L X
tocr* •
CO
UJ LU K- oX z t~
CD \ UJ
1CO —
•
to1
—» tv- K- . «•
X <c z to
1
CL IL. LDCO Cl XLL* . ^ 3 —X K t—
<t o COcr <r IJJ
t— o C/3 t-i U it U i*«J
! 2 Ol K •-o tx — orX 't > CLUJ Z O
' c o — ort“ CL
Q •Cl QO > CJ <I— CO 1-r.3 O 1—
j
3 IU 3 <X
or OCt- cr y~ into z to z2 fc-* 2 C—• UJ •—« *-•
CC zLL u —X > X CLy- cd K- O
OCM CM
177
IQ 10 0 <D Hf c t<L
m• CL a. «. ,r
:»h 1
a 0 CM CM CM ITS into IO O o CO coH H1
| 1
11
I j
T)a> IK
|
1
1
P 0 1
i
P 3 I
•H+J
c
:33u
1
1
ftUi3r
1o COi
.
i COo .Hi
j1
2v_^ ft-
1
! <»*- ) •|H X
<r r* UJ X O10 2 1 »-
or -
'crCO1-*
< 3totH Ui _J ft LL *- LL !
12; 3: UJ UJ 0 CL Z UJl-H 'V, -k. u UJ _I_PS 2L Uj f= > cr I-- fi=
P-i ft- Li. V) t-JX CO ft UJ CL
Ph >*4 Q -* a CO 00 1
w 3: Ui ft 2 ft
H •- 21 a <c 2 0 3ZD UJ > <t 0 UJ
1
1
p
,
f*a ft- X 1
5* 2 CD < H 0AO LU3
<* Uia ft
*< 2 CJ2
CJ a: 0 X 0 20 UJ D ft
g < 0. -L COO co 0 CO CO — UL) LUM r-« •-* 2 <C _J C3Eh P O X CJ <t
< <c h- — ft
JD O <*< C *- 0 |-«C •3»- 2 X UJ •a: X • OrH
<f*
1 0 3 ft- - 2 3 2 O^>1v. 111 -
1
2 *-• O • •-« 2> UJ UJ CO 3 UJ _> UJw ll ft- -J CO Q UI mt«_ -cr 5? ~c LL. 2? Q: ft >~
Eh a • LL > a 1 <t C O c *»-3 0 0 Cl UJ CL O 3 •—
W UJ cn — 3 CO )— « ft
Pi 'LL) < UJ ft- UJ 0 •u — to
ZD U. «• -X l/> 3 UJ •JJ X 2 2 3E-i
CO Ui CD CD <r UJ O CJ
CO to CD c/> O O 21 -• 2f! CJ2 0 UJ 1
—
ft- *-• 3 t— Ow 21 •-» ft— _i X JO r— >~ 7) 22
ID O ft ft U. k u. to CO ft <*-J • C-l <4 2 <1 cr 2 3P-I 3 LU LU < UJ < UJ ft CO O CD
ft- ft DC X _J ft CO LU 2(/) :< IJJ L> CO 0 f- O V- fO *-•
CO U. 11 -2 Cl . <t ft ClO DC CO c iO L. to ft t- ft 2IU ft-
mjt. 3 2 LU C/J 3 <t
0 •— <t I> UJ CL < to >-
«x X 2 fO VL •-« CL UJ > 2 to
X - s: 3 X a CJ a X ft <1 ft
0 • h- LUD ' X •c > ft lx 0 .3 c •— C a r.
O 0 cn O or 0 UJ 0 LL O 0 2ft- ft 1- h- LL 1- <*-*
V*-- 2: f- 1- 3O O 0 >- O * L 0 > 0 O 03 1 3 3 3 3 • 3 3 Xcc . « rr UJ tr or a: O'. UJ ft ft t—j- CO ft- > i~ •— h- O N- • f- h •—
(/; ft- to •— 10 ti- IO CO co to 12
2 2 2 b~ x ll IT. 2 2 2.•-« Li) m-+ 0 - »— •-* »-• *—• •
c: UJ UJ XIL ID UL ii. u. 3 ft •— U- CJ ft ll ft
3 ft- a: 1*. X O IT C X X X XK CO 1— UJ ft- X ft h- ft P- k~ -
<m >o «o ro to o 'O
T* f\) rt CM CJ
PS 1 •H is '•H •1^ ft.* r >r» • r* if^ •V •IT
UD in in in in co COco 0 co
j
eo fts
1 •
j
i
j
1
1
to
;
i
CO2O
1 UjX
ft c. • .<LU 3 f- 33 > M to CO COCO »- —
•
3 X ft- 1>2 O ft^~ *2 zz •2C 3 < 0 O2 0 O O 10 0 3 DC ft- ft»2 »— O O »2 COK » u» *-• JL2 UJ - O X 0 12 LU UJ O 3 3O - to UJ F X -
*"* • CJ (/> Lk. .( tora O
iu
tf • a or 3O CO 0 0 • CC n ft- CJ CJ ft3 b— CD -c ft CJ 2 2 Oft 0 LU co — 3 3 O ^C ft-O »-» LU 3 >- CL ftcc X- •• X W O »- CO to O 2< CD ft- t—. 3 ft- LU13 0 2 U? 2 cr 3CJ t— 2 3 3 >- 3 3
CD >- ft- 2 Cl 3 ft-
ft • 2 ft- or •C 2T X 3C to CJ X. tO 1 0 — cr.
ft CD •- ft— ft 3 a. CD CO2 ft? CO O 2 -« ft 2 2 to CO
to <r ft 3 O 0 3 0 ft2 Uj 1 *-. CL tr ft- a —O CO - a ft ft- 3 0 X rJJ X— ft J 2 O ft- 2 <c >- 00* 1
—
UJ •ju 3 -ft- < < 0 3 3 2CO 0 > 5- 2 2: 3 _i 0LU 2 ft- 2 UJ *• ft- OC O •< —
•
3 3 l- 2 3 to O to 2 ft- 3 ft-
CJ Cl LL C ft »ft- >ft- 3 u~ cr. . *t
CO LU C X rr tu 2 X X (O 3 3* to ft 3 < CL tu ui to ft- 3to <c a- ft— CO CJ — 2 CO ftl— 3 3 CO 3 3 10 •3 < 0 3 •-•
2 O X > ft- CL CL •— CJ 3UJ CO 2 ft- *-* 2 CL 3 ft- 3 ft- XO IU 2 O p- LL’ O »-• ft— — ft-
3 3 O Cl ca X- Cl CJ in U>lr- O ft 2 ft ft* 2 2 0 CO 3CO ft ft— O u. r> < ID LO U) UJ
3 CO LU ft 0 IL 3 << COCO IU i X 1 • UJ 2 CO tO O CD 3UJ JJ 3 O ft- UJ 3 3 <* U O> K X e 2 2 * > C£ CZ U CL •ft-
**s b- C rr <t cr 2 ft- Z) tot— fO ft- CJ ft 0 « > 2 O * 0CO P- UJ O co CD a: O 3 CJ cr 2UJ < (O ft ft < LL X >1 2 0 <1
ft X 3 to ft X <5 CL X 3 ft-
»— ft- •—
#
ft- CO 0 0 -
a a 2 c n G d n O L n 1 • c ft-
CJ >- 0 0 O or CJ CJ 0 OC a: 2h- C L_ ft- c ft- ft- »- ft »- <1
CJ 3 LJ Q. O cr O 3 0 0 JD 0 IL CO ft-
3 3 3 3 :j 2 3 r- 2 cr
CL <4 ft 3 ft- CL L. CL UL a X a — 0- v- 2 LL 33 ft- C 1- * - ft- »- aCO 2 to <t > 0 (O CO 2 to to X to 2
ft- ft rc 2 u 2 O 2 10 2 CJ *— •—
cr 3 <i —
•
0 — *-« 3 — Xr C X CJ LU ft- ft-
u. u. u a LL ll U. 0 ft IO 3 CD 3 cr
X X X X X a X X X < T 3 X O 2 O6- 1- — <. ft- 0 ft- ft ft- tO H- ft- — X
1
•
to cm r- o vt in
CM CM i
{VJ CM
178
90iV
to
-rM
jo*
a-• t\
vro
.•»
>c\O'
' c\o
» t\a»c\
• ct
CM
ft“N,
i
1i
1
1
1
1
j
X>i
!| j
(1) !:
1 • *3 CO a: Q to
c 21
o UJ IL| l
•H• i
O|
|
*-u
i «—»f* CO
DC3 1
1
j4—
'
•*-i
.1
«
}3
1 cr 2 *~1
jc to CO o: •>“ •ft- O CD CO COl
oO
tL* 1 i2O AT ft-
f/>
UJ•X
to3 h-
UJ a3J CO
CO; ixv-/ :c • »*- 2 1
—
cr UJ a 3 cr >C/) ft- • IL >rr JL •X c ICJ . LLH X 1- <1 CD to O X »—
13 CD CO 3 » *. UJ «t cr cr 2 „ «. m t- X xO •3 LL ift- UJ X o »- IU O >- >~ UJ o •< uEh O ft- • •— >*K •2 IU • I *-« *sr ic •X »<t X 1 <1
££cc. CO 13 «* (J <JJ r— a. 3 UJ
M JL U — ** * •• 2 lO O o - CO t- CD - acvi
t— . *» UJ >- L-i LU -L — t/J C/J 2 u. u uM-i f* t/J cu i- cr CT CO u »-* 2 UJ*- o aP4 CO G >- *"• -j
:
U UJ uc UJ i-J JL > t/JCO to -J ••f :> cn CD m a CO UJ 2 CD CD m >-
P4 UJ UJ UJ Sr- Cl 2 o LU 2 Ul <1 2 3 *x r ^w cr r>i *-• S'— 2 *» a ar a CD Ul CO 3 toH CD CM u UJ o <* a 3 • a a 2 2 P- 2ID o — ft- < LU C3 oc a UJ UJ 2 UJ Ul XPL, Ct 3 u. ft H CO rj IU <r a 1 • X a IX X > h*5\' cl o o •—
•
3 • X 3 2 c_> CD o D X UJ UJ ID »—
o cr X tD a LU 1— X to »- C3 X UJ d- oQJ
a: uj UJ £X 2 to 2 UJ •3 LU L ^ 2 "V CD 2 -4: c•-t H o C3 o «*-f cr O 2 2 Q to 2UJ <c X cx •f 3 7L O m •> •w» LU -« 2 UJ tr oX — •* Cl ft— CL UJ UJ CO X CD <C IU X X »— UJ b-o »~ a: X CL u X a IU G - UJ 2 CD 2 3 J- CO •-M • n_ «*: 3 o o cr < •< to o UJ —
•
1
—
CO 2 ! UJ .j »-Eh ^ o UJ CL U « UJ 2 tt to fv; X- 1D_ 1— a •<* 2<J e a: c 9— ft- c_> C/7 -J 3 CD 3 c UJ <t
c>wHWQ£3Hco
wftJ
PL.
iCO
a -4 CO 2 to X o <>- a •
—
a a •ft- ft- 3 '
a 2 O »— 2 o o 2 2 < o > 3 2 3UJ < CO O 2 O Q X X o •-* a O O*- < LU ft— J- a • r—
«
CO X X ft- ft— • »- - »— XCO CD *—
*
i» to o to ft- ft <t a 3 ft CO Q DEC 3to 2 O rr LU to tu 3 to ft a X o to ft- (O 2 a 2 cr
a . <* )K* -«x • 2 UJ 3 ft- 3 2 o a 2 UJ *-« CL c COX* * ft— • a c to i LJ IA ) ft- ,CI> a ItJ CD a 3 • ft-
to co ~"r~ »-• rr o to a o o 2 ra o a: ft ft - 2E— 07 o 3 2 o X 1- • 2 *-» o 3 2 a 2 3 22 2 2 O a’2 UJ •—
•
C-D 3 3 a a Ci Q tr <t 7t ai O a fft- oUJ O a 3 O UJ X »— Li a a to X 3d 2 o L a 2 X Cj h- X a t • to L • X •3 LU a o a « * a w ft-
3 UJ i- 3 o O CO < - >* Q 2 to ft- Q a 2 X >- i— >— co»- 03 Lf J > ^ to \ -t ? a *-* a ft- 2 ft ft- 2CO LU tu lLA £3 CO Ul I7> X. 5 ft- to JL CO •—»' —
•
>* 0> f m as toa X CO 1 C- # 2 U.: X 5C 2 3 CO a a a a ft D> G a
L1J 1- UJ • J LU 3 3 ft— ir. a O <* X in to <t a •—
•
LU 3 3X o ir. l- <t o Liu o X a o a a to •f CD ft- ft- UJ
r- o co LU rjo a • ft- UJ o a a ft- ft <t CD to ft- t_> to lO XIL > y- i* o to x 3 ft- a a a <t a C >- <t a
(/' 3 h- 2 c:: to 3 l>- a (/ a if * a • a a a a if ca •— -4 3 3 cr 2 2 c CD 3 u. a 3 G 3 2Ul 3 3 D> O 3 a 2 to X <* O 2 rr O 2 o ft 2 a O <UJ C-) LU f7 w L3 < IU a —> o > o 3 *-• to 3
a 2 tc to LU a a IU CO to o Q j— 3 2 a to to 2 2 3 (A
> -* IX to < .J o a to •— to »-• <T' o Ui a X •— 3 a -5 a
r: a UJ o X 3 to LU 3 > j- ft :>
o Q o ^ •» tr c/ L!' ct X D Q L u a O «^- a a a a L 0 «*-
t- 2 »- C/'i o 2 to CD to O C to Q ff— ft- c G o G 3 O a ift*
o L) X •*._ CD •>- Uj ft- •—
*
h- • •— • CD o - 2 ft- ft- ft- ft GZD o f3 CO 2 o X o O Q _J to 2 a *-• o o Q o (.7 aa _ « a 3 <1 2 -4 DU 3 3 2 •— a 3 a 3 3 2 — . J j
f- to ft- »- cr 3 *-« a. o a rr a cr. O Li. CC ct: cr -t a rr GO
to to i/) < 2*~ t) CD *- «— ft- t- X a — a a ft- to b~ ft- ft* ft O7 t; LO IU >- to t/) to ft- 2 to X tn ft- cn co 2 CO to
_J a CD -j 2: a 2 2 Uj to ft-* a 2 2 2 7 G 2 LU
•D 3 o —i —
•
ft- co CD 3 X a •-4 CO
UjI 2
il a-x a .
9- <CD
UJ
2 ft
IL 33* to3 3
Li. _J3 UJft- a:
aIL LL
X UJft- *
GU. to
X ^ft- C
i
2 coLL1 V,
X UJX X
cU 3X ft-— to
UJX
IL COx a*- to
aX
UX G»- o
inCM
in-i
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continue.d)
179
|
IU
*
I
U3
i
i
i
i
i
!
i
t
i
i
a*»c\
QC-lL
<->
CD1
IDDCD
UIX
aX2
zUJrx
CJXUi
lOIDtt
CLCDz
co*-
o3cc
w2
o O1
•** a. ct ir t r*• • • •
CM CM in inIT\ M M
1
1
PO
i
PO K)
|
1
j
j
o! lO
LLI LL
1- «• cr««C CO
io
cr z y—to o i- o UCO Z • to UJ
» CO 3 fr- 73•I Q X 3IU .X -C IU >tr IU—
*
X or p~ cr £C i
1 o u UJ o b- LU1
l<i l^X X >2 IU •UJ % CJ >.< LU
i
; A* o o u o -i u. u 2 U.)
1 o »-• or CO oto Jt cj UJ tr
CJ >- O CD o c. X1 JV <* « • *4 <t X o
2 CO UJ cr -*c Xp- a 3 X
i^ UJ X UJ O X X> *- cr to CJ • LU <t
! O UJ Z5 2 o CO1 UJ >- o X u_ LU »~ 2_ to1 OD < Cj X. 2 UJj
z to O1
iu Cl o o cr kw
i
* UJ 1
—
X UJ UJ COI— LD 3 >- CL' 2
J_J I— z X <r < CO o
1
»- 1 2 U
J
2 ai
“ < -J — 3 -* h- 3 UJ CO X C <ta 3 > p— 2 o u
UJ o o z O o1
*“ X UJ . to X1 lO M 3 <c CD to »— * <t
UJ r 2T 3 to * X Xcr O CO *— 3 2 o X
iUJ 3 »l- CD CO CJ to 1
! z -ft to UJ tJ CD
i
z D UJ 2 o 3 zf- Q «-« LU Q o X <x3 X 1—
ID • 40 1- CJ 3 Uj 4. •• v> 4 ^51 >- h- CO > 3 2 CO— Z <1 3 IU •-« Xcr> — LU CO a P- CO LUfr- > c: o. UJ z CJ 3 t/; aUi J 3 ;> sc UJ CJe? h- H- UJ •— u o X X
CJ to X p- o u u. ->- <x < p- LU «
1/ C It to u (/
—r u 2 2 cr CD 2 2 c c<x c <t o X <4 o 23 •— 3 UJ *-« 2 —
•
CO z. _J CO CO UJ to CO (D CD3 x < UJ •— CO to —• <t
:> 3 o; UJa LL • Cl •Cl a Q t_ IL ao _J D I- o o o to CJp- »- u *- 1- - •»
cj O O UJ o CJ u o 3 to
i3 2 D -) 3 3 3 < 2o: <i x m X cc cr UJ X O
1i- f- o p- »- t— 3 LL' »-•
to z fO to to to - to
1^ c 2 UJ z 2 Uj If:
to *-« CO CJ 3to CX a U
LL to Uj 3 u u U LL U CO
|X L*J X o X X X LL X1- CO
1 i
h- t_> p- p- — SC o
CM CO H in o >o
i
H CM H ri
fv K> r> PO b'j. If . M i M . 1
• • • maj CO H
tM
|
t\l
i
CM CM CM
1
1
Xi i
I
j o totr-
1 21
-U3 O 1 X
trX >- •f-f- tJJ LOto •i tu CO
1 2 3 •-1
»*- O • rzCO to2 X -Ct X r- oO X CD to X»•— •»- • n 3 LL I
CJ U P- JL•M - ^ z X CJ1
—
>- <-> X »-»
2 r- (T 2 3; .UJ <L U)CO > to CJLU CD to CO _«X t- 2 a: IU Cl o fLX u <t cr r-j 2 2 O
<t CD XI O r V-X • 1— o —
«
Xo o 3 • X cc 3 to o 21^- -ft X o a O LU p- X
r- LL 2 C/D ex UI2 O X a UI X _cX Ti — CD 3 t-21 CJ 1
—
a UJ < 2 :t> XO 2 CJ X fc— -t cc11 3 o o TC toto INI a CJ a. 2 to CO
i— o to 2 o X o X> X to <1 2 O cc *- a< 2 o X UJ X<t X o Q a »-• >- u>X 1- X <c 3 .3 2X 3 3 to f- 3 oo to LU to X CO o O < »-«
2 LL 3 o P- P--
-c >- a G : X : -M CO <t2 CD cr o to n 3UJ o X H « to % p- X
•K 3 cc CO 2 to X3 X •» LU O AJ 2 JLI X «-•
X *3 2 •X X h- X O CD Xh~ O CO <t &
—
LD 3 p~ X1— CO \ <t 3 (JJ P- 1-
UI l* ui aj - m U>X X :r .V UJ IL CO X'X CO 3 3 X in to Xo X - < o X 2 X to<1 -X •» <* t- IU X CJ CJ 3X X to X 3 p- (J <t L> oa 3 cr jt f > a. a : -
-<t 3 zz U' 3 3 COX —
!
rx 2 to X «T O -> CDo >* O <4 UI UJ 3 CJ a 2*— i UJ X X X O 2 o <Co U' <t 3 o X jC — X »-
3 U X o «• a c a -v o r. 1 • 1-
i- I- 2 to o CO c CD Q cr a 2to tJ m~ LL i~ •— »— 2 p- X 5- <2 UJ 2 SC CJ ZL o »-• CJ X to P—X 2 <z 3 3 3 f- 2 CL
X w-« X cr O X . n cr. CJ OX' X cr L- »- I- tr >- XX UJ > to to Uj tn X to 2»- LL cr 3 2 X J <1 z «J •— *—
X 3 •-» —• 3 — X2 UJ 2 •M < CJ X ft- ft-
LL \ U 3 U . 3 X u if
X UJ X CO X X X 2 D c 2 Oli X X 3 cr l- P~ t— * X
CO -« J o rH inro PC CM
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continued)
180
O#c\
O* c\
c• IN
2
ZDor.
in2
a• c\
o» CN
oi«N
CO•2<X
COu:111
UJto coUJ cc
o oot-•2 toUJc x—j -f *
V) La.
UJXCO• 2
Q2
I r Ir
rv
lrlOIC
iO• c.
!
VI •1^ ri vi
j
•
i 'tocr •2
i i
UJ O toiX - a o -»
2. CZ r
O
UJ•X
>~ > ZJCC M ,y-** - —^ 11 WC/4 ULD * ^
•<
3o
to.
UJ
<*CDOCo•
a2<
)•-
3c
•Q2>-
CO*<
UJCO
—i
* - o F— — tu z d > tw 3 CL 3CO CO i 02 -d k • 2 u 3 p- CD U 'd P~2 CL O p- O O £
i.U OC o to • •O 3 U. u b~ 2 4t CD LU <t a 2 a.u UJ UJ u_ CJ UJ UJ u w P- UJ UJ to -*r 2 3X >- CL ° X c UJ <1 3 p- *- tuCD P- CO ar u. X u 1 , o o X 1- X2 a
j
< CD L c» CC UJ p- <t p-*< 2 1 2 L3 1 o 2 l « Zl UJ CL CL co o
<rJ3 >- O p- *“• cx 13 3 o X 2 2 UJ -
cr1
P~ -* CD 2 o >- X 21
W >~c O cr<r CO
j
ac <t X • 2 <c u p— CL ».* Ul CDLU UJ
1
o c CO c -«t“
-C IO CL to P- P* 23 CD i CL CD Lv 3 V— p— <0. < 3 X c 2 —O -t • CL 2 2 LL' CO X 0 2 1
^ c X 3cr o CO < X CD 3 3 O P- p- •» cc X 3-3
!CD 2 V- Ui •3 G LD • <c cr -< Cl o UaO Q X o so >- 3 O P- UJ x UU u. II m > JC
2 O 2 < *-• cr 3 2 X C X <£ CO p-1
3 p~ o 2 X *-•-* 2 <t CD 1
— LU Ul 3 • p~ »- P— • UJ <c i < •—
•
o P- CDUJ UJ cn Q D> 2 • X CO 3 r- 3 XD 2
bC • cr LU 2 P- V- V UJ tr p— 3 2 CD LO < <OC Cc > 2 3 3 P~ 2 2 u c. to CD < • in 2 «X 3 2c c * P~ UJ to Cl . CJ LL LL it
. V— U. * P- T. ip- • t- UL' > C C cr . •
X •-* X X CO LU O O a cr LU X •—« 2 Cl UJ X O UJ p-* cr Ul to * Ul U_ 3 2 X LO CL 3 2 UJ !
3 in p- X 2*- CO -< CD CO <t U. P- UU G L-> CO 3 to 1— UJ -t LU2 2 3 <C o — 3 UJ co CL CO 3 LO 3 to CD UJ Ui X CO —*
UJ o U cr 2 CJ* LU u Cl > Jr- 3 U_ CC i X Cl o k—
a •—
*
3 »— UJ (/) 2 CD a o 2 3 o a. o O %- 2 x -j:
3 p- o O »-• c UJ 2 O 2 p- P- LU U. p- P- CLp— u> 21 3 CG 3 3 a X LO t3 23. Ul ca a 1 a x; - COto LU a 2 o ir— O •— iC a u. 1- LU LU to 2 2 CL u. 2 LL LO UJ . -*
3 UJ CO X - u CO -« X LO U. ->• 3 LU «X cr < 2 < P- X.
to C CD Cr X to UJ UJ C' P-- UJ L- O UJ -< U. t.l
UJ 2 to o CO tu X CD V cr UJ X CO OC CO 3 >* » • IO cc»- CO »-• UU UU UJ 3 o <1 u iO» 3 3 O UJ U CO P- UJ »- o P- G.< cc C c to. i K- • X C! • a <
tl - 2 1 > CD c u X 2 c i/ a—
*
Uj 2 *-• p- < - c 3 is >•— 2 <1 CD m— 2 2 to CD 3 U. Cr** 2 CL
X <J > 2 w— CO b- O 2 o > a < UJ 2 O X (O O CL
< CO P- o UJ C/J p- UJ ID •>'
CO a. o —
«
cr V <t •— o UJ Ct X <> 2 CO cr X ui -t to + 2 II CD 3 <t cr to rr < X La. u. OC UJ UJ
UJ <t cr CL X cc X 3 to UJ X 2 X «* CL to •< X cr U CL X (D OLU o p— p- p-
-d CO UJ o *-• r- »“
a o C Q a G 4t c D L' c to ct G tx. r. D Cl Qo o 2 O o >- C •-« o c o _ « C cr a o O cr. CD o HI C >p- p- P~ * p- b- c p“ p— L'' P- o p- to »- P- U.* U- 2 P- to
u L» o to o :t CD CL o o 2 o X o CO o o CL o OC o—
s
u <t1113
rr
to2
21
U LL
X X
3 2o: av- —CO f—2 CO—• UJc
U CDI 3P- CO
3rr <p~
co 22 —
2:
u u.
X I
OC P-b- 2CO <c
2 P-— cro
u aX X
CO
3OC u.6- Oto
3OC
ZDcri—to
—• uc
^ - ILX b- X*- * p~
2_ _ C—. —. o — O — —
LL C LL CDXX X 2P~ O P— UJ
XCO
3rr
to LU2 X
UU XX oP- X
UJ
U'
ucc
to —
•
2
UJU CJX <t
p- a.
crp- cto 2
C '
2
CO.uo2UJ
t *—
XLU
UJCDCDUJ_JXo2X
luXlr-
XCO
soi -
2 COLU LL(X X.
CO P-u <t
k-— crCL
2 O— crCL
c. ao <t
i-
O P-
3 *<
u:p- to
to 2r. c— uj —
LL
X >-
P- COn.o
oCN,
ofVi CNJ
fOc\i
CMCVJ
COCNJ CNJ
SAMPLE
STUDENT
EVALUATION
COMPUTER
PRINTOUT
(continued)
181
iO•c
ID•-t
Cr
>
u.o%
</>
CT
2UJinuftCL
ojjC2aur2UiCLoITU’>cc rc: uj
o >a:- ecr2 cr.
2IL —T C>- a
3O1• —1
'
Uil-
I to•X u.
*-
oeoCD
»ctUI -XX UJVX il•“*
-XX CO
X UJ•— XJL o
2Ul -<IUo: uc< CLC») o-
<*>
2O
UIUJ UJrr -u. •<
rr .
O CJUJ cn r—UJ -«* |ft|
u. - -jCO Ui 1«
in CD inH 2 C Uirf? —i
u. C. CLu 2 LO Ui Ui •<
ft XCO X UJ
uV) X cr crUi >- -* 2yc O —*< X 2 COa y ^c ,rv •iO o x«- aCJ o u >-Iz X _«fc * X UJ1- CO >tr. »* v^ 2 r »
-
*-• Ui — un UiU X l uX *- X u- vO •“ u.
CM O
FACTOR
ANALYSIS
MATRIX
OF
STUDENT
EVALUATION
ITEMS
182
X —
•
<* a31 O2
cro cry- luC ) CT)
< —
•
u. <iC
oUJ x
I- 1oOr 7Q
X •—<t y-
•-•Ka o<r ct:>
criUy~iii
<
cro
o mCO COo roC_>
oir\CVJ COcd roy-i C\i
rHO vOo00 o
CD r«> CD O i
ro r-
r^ ao(M T-
^ y-i
'assess ^ ° ro o
- - -° C C C c ------
C- c c o c C
^ r-i> *T Ttro in•O r\CD CD
crou
• -*x
u.
croo<c
oroOJ
^ lr f"0 IT A. cj IA
» » ^ ^ v- I*;
,°wcio !c ,; ,;;;*l;.-
! iI i
c cc
> ac n>><o tH
OVin om n-«o «o
OQ0'O CO o*-« <> rofo vr o(V (\ H
c c:
I
in coCO 'OfO K>CO CD
lO'Cfc
i
CD CD CIO
o ro tof"- *-<r^HV No cd
cv cv. ofCfOo lO'CD
cs/o. inoo rsj
onj c\j
i
^ o TVv-fCD
CO Or» **»
rv roCD O
OO o oocjcjcdcjcjcjcjJoooo1 '
, I
ia ro cj® CO (M«Nj o cvHO vQIT IM\
1 CD 1 CD | CD
on n> fvlCO CVJ egO O' OOCVJ 7CD CD CD
iao i cd
’OCK£ S £ r ^ ^ lM v «o (\J rv
i
0 CD CD CD CD CD i
N a rHQD Ov rn K)
or co o No vO t~r fo r ^>- ho c\) r\; CDo » • • M< CD CD CD CDu.
j
ro1
y—l in tr> r\CD C\j Cvj
rr *-4 •-J cr> rrs
o CD C\J o CD- H cd n CDO • • • •<c «D CD «D CDLv- 1
0 CD CD CD CD CD
'f-'O'.-l .-fo- ^ MOO O V
^ ^ ^ ^ ^ o 'A -H >A tore
oorij^,'00' '»-rv r~- IAO IIMN
°°tt° rt° Wr,OOK! C_'Oj
oc> <=»oo oo o c* c*
c* c* r*j c*• » • I »
lP^ NNl° ro v> CO to v OV tA
»nro o® ov to to oo. sorvvo0 3 WON 0(0 o CO ,° £cJOcjoo Jo .
’c* JJCJ c*
oCOcr<c
in
or cn in *t <> or in n rs ki vr ki
^ ^ cvjo Kf o K) cocoC(V(\| o 0 vo
°ooco <r>«o«D «5 o o o J ,4
Tt'VfL^> ^ ro to A.m « n m m <1 to-.“ n Wrl OQV Hrt OH
A ^ a ' r^ to on H ook: (\i^i n, ^.'J “'I' *-0 LA to O <1 C> C\) ,A IA <r O COfo*. IA CIA -O ^ „ fo ,t riQ(Xic
I
SiSSSKSSSSS^S«ssr^s~-0 --;^ ^ T ° ” *=> ^ cr. ho to
C'coooomojj J^, i .
.*
I
OOOOOCDOOC300 CJOC3 CD O O
I I
roOrOO«»
u
J?TlrJJT r,jr^ c’' '-O' »-,o to m o- o cc
_. J-1 'A O> O' A. H“l [(1 k) H
^ J?' ' rJCt> "^or-'^'TA.HtA.tr, cv
(2 S ^ r*^ "° ^^ ° -O to to fM AJ O' C\JAJ to f\J H f\J H CN N K) o H O H OH
coc oc ao
^ ,ri tn 0,0 —• .-km a-^it\oNK)oo N HN ro AJ rr
M- o' S O^ »C JS*^ ^ Al'J toM- o. CO O' rv OO Wdoo^NHH or, tMOHMon A1 C3 Hi
OCDCDOOOCDCDOO OO O
I
a.^ ’'2 At ro *h fvi -T (\J f\J iA O O'A! VJ ST 'O rr C\Its tro^' 'tH'O<'-MA A- o <V O' H. cto O'O' -H (O ov MA JMA OJ O' vl f>tr rt C f\) o H to t\l H H OAJAI
C2 CDCOOOOOCD CDOCTOOCtO r o o oo o
yr-l CVJ m XX
ft cr• • -«/
> >
CD CDcr ct
in yf) Na o ccd cd rr
tr a a*JI< '«J < «i
> > > > >
I ! !
COO. C1HMM tt IO -ti t^ o t> O H o. r' tr in V, A (t.<-t--t . I AJ At AJ AJ AJ AJ AJ A. AJo o o o o CJ o o o o CJ CJ o r o o cr c , o o c -aoc cr a a or cr a a cr tr ao o- rr a: 'r cr ct cr cr-a I <1 . -t -4 <, H . .1 , <J -J'-J ,<I -J -J , « „ .5 „ o .,:>:> >>>>>>>>>>>>>>>> >>>
I I I •I I I I
f> rlnjroCD CD CDrr ct a«l < <>> >
I
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
EXPERIMENTAL
183
— crCh cr
A Q.
DO04O
- — CD-* Ca: - cr•— • o.
UJoo xCO o
S' UJ UJ< UJ Uj— o: crcr. c: u_
rr n:< or_ a-r a:
uj-in
in tnUJ UJto (/>
oc luuj coCD <
to Xcr uj— 2
14 1* *.# 14 l* l* tft l* It .* U l«-u 14 l* .ft l* 19 14
•> 4*4
14 19 19 14
• 9 * 9
1* 14 «4 14 14 14 14 14 14 «4
* 4 4 4 *
14 t4 14-14 14
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
EXPERIMENTAL
GROUP
(continued)
184
UL ULtr c
X
o u.
a
o £o(/) DUJ UJLU UJcr £To u.
I* It L* t* 1* v* m c* L« t* l* t* a i* t« «. i* l* t* t* ii o J *» m t* t* i* t« n ii ^i i*
cn-t oI- CE
cr a< op a.
a zcr c<t —
•
to toUJ UJ€/) tO
cr ujUJ l/l
CD *1
185
PRETEST/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
EXPERIMENTAL
GROUP
(continued)
186
O Xo
CO o
t* l# <• t*
UJ UJcr oco u
c# L* l* V t* »« C« C«-l* il t» t» L» 1* c* l» «* »• k«
« « • «
*• !• (3 a*
« « • I
cr cc< oC tt
cc
to toUJ Uito VI
V. XCC UJ— if
CL Q.3 .>o o
TOTAL
SCAT
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
EXPERIMENTAL
GROUP
(continued)
187
Ceo
Oi
3Ooo o
3 3r: oA AC O
rLrx
f«tLitv
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
EXPERIMENTAL
GROUP
(continued)
188
o t:
oto oUJ IUU. Ita. <xo u.u.
tft ift
o roto CjUl IULLi IUcc q:o u
to toUJ tuto to-X -t
L U
a a.:j r>o o
! t» Cft L# t*
* ft ft ft
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
QUASI
-EXPERIMENTAL
GROUP
189
COQ
'/P
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
QUASI
-EXPERIMENTAL
GROUP
(continued)
190
UJ _l - I
— u_ uj n
!• 4*i«- rfl l« W* l# 4* 4-* k.« «.» L* V# l» 4* l* l« l« l* l« t« 4» l* 4* *• • 4.* *t U K l* 4i i« t» U r 4* l» i« l* .ft
*- CU<* o« Q.
ct cro
ct or*1 CC
v crut iu\r i/5
< -tL U
I ri r<
Q. OL
CJ O•— I o
VtR01
6
PRETEST/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
QUASI
-EXPERIMENTAL
GROUP
(continued)
191
o Xo
tO CiUi OJCL IXOC CL12 u.
L9 %9L9- L9- 19 19 U l* It 1* »t O 19 it 19 it 19 if if !• II if if 19 19 it 19 19 .I 19 4.9 it 19 i« 19 4.9 49 19 19 19 19 19 19 19M
or or< oCi CL^ CL
to COIJLf tUto CO
CL Uiu. toCD -t
to XCL Ui— z
a ar> r>o o
D r\ ri r* A n
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
QUAS
I
-EXPERIMENTAL
GROUP
192
— • CL
i« L* it l<
•— » a
o iotn oUi LULU UJft QCO U.
«* oc. a-rr a;
t/* <rUJ UJto 00
cc ujLU COCD -*
CO >QC UJ— z
• I
H (M
a. a.U 13o oO Qis o
• »• it L« U * U ! i» in * st It i* U • %t ll it 1* * 11 1* 11 it it «t it it
o o o o o o © o o o o o o 0
GROUP
PRETEST/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
QUASI
-EXPERIMENTAL
GROUP
(continued)
193
—• • Q- I
O X »O I
CO o I
yju lu i
Uj UJ II
CC VC IO U. I
111
•b
c» l* lb lb 1 lb lb Lb lb Lb
tr oUJ iua- ujcc cco u.LU
c« lb lb lb Lb lb 4b L* 4b lb lb ib 4b ib lb >b 4b lb lb lb tb 4b Ib Lb lb lb lb lb lb ib lb- lb lb lb ib 4
-J • b•— cn v-t o ih ac i
cr. cr< oo a
o ^ t
cr o i
< — io b- i
ir < i— I
b- > I
(ML I
(V
in f/>
LU UJcr. co
UJ I b- o
• UJ• CO• IU
cr i nrcr iu » a.uj co »
cr -t $ ci coi c i / o
CO O f->
u CL CVIt fN CO
^ CM
HCL
_1 riU.cr q.
CPE'IN^SS
fufXISlLlTV
pretest/posttest
comparisons
of
student
evaluations
for
quasi
-experimental
group
(continued)
194
—• 13.
o yo
tO Q111 UJu. lu-ll: nco u.Ui
** v# IH k* k* i«- 4+ i«
u.' :_j • i*“ — CO I 5T.< < C • <r
»- tr• Q- - •
ir oUJ UJU. UJcr cro u.
: — i fx
to crUJ iutr to
i tr. ot Ui
« 13• O• O (U
Z3 _~>
o oft cr
O 13
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
195
a,
opso
oPSH£5OOPSoin
U1 J*.CO
It V# Lt-Lt-
LU LJ •" CD-« -«* Cx. r-a
t CL- CM
to aLIJ OJUJ tucr QCo u.LLI
a
cr cr< CJ
Cl^ cr
2 << •—I- ><o in
CO COUJ Ulco cr
it *.* it Lt i* vt a It Lt l ft *.« 1* l* L* it- it it- it it t it It It Lt Lt t It Lt it it Lt it Lt it it it it it it it it
n f~s A ^ ^ £ e
V*R
006
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
CONTROL
GROUP
(continued)
196
uu u.CC (Xo u.
UJ LLl
tlj UJCC CC
tS (.ft i«
• • « *
i* l* l*- C* *.»• I
* * * *
L* l« l« t» I* Ci» I l« l« L« k.* 1* • v,* t# c* • w* l ft
* # # *
* Ifto-lftlft
»«»«•)
Ui UJto to*4 *«
U U
W XCC UJ— z
H CM
a clD 3o oOr Ct
O 13
tr cr< ao ct
a za c< —
•
CC UJUJ toCC -X
-V o
r\
PRETEST/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
CONTROL
GROUP
(continued)
197
V V V SJ w u ^ v U u O
co trlu uico co-* •<
u ocr uiUJ cnCD
CL CLz>
o o
PRETEST/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
CONTROL
GROUP
(continued)
19a
Q 7.a c'< —c -2 <
cr uu
«cv to
to <oLit Ulto (/•
-t <u c;
o or* in
l_U I *-4 r>XI , • » •
I N w
torr tuUl to02 '<
3 O<3
f* IT«o Or- ir.
*cin<in in
oIT
m
m
>o
cIT«<3
CM
tro po
ooM
Ors.
roo04
/> I
IT. •
h
Or-
<3
li>
oin
V- Xor uj
i i
H CM
cv a^ IDo on o19 O
u
a
a.
caCD
inCMonr•4
>
CM
Ol
onCD
a19oaCD
-OCM(JO'
CM
a
caCD
r*CMC3nr
ono
CM
CL
ao
CM
aooc*
T-»
a.
coCD
CM
aoato
Cl
oroarr•<
>
CM
aanCD
CM
cu?
PRElESi
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
CONTROL
GROUP
(continued)
199
© O O o o
-J •— crl c
IL Li.
oc tro u.
14 14 14 14
o ro
CO oUJ UJUJ u.cr oco u.
I* *.4 14 *.»> 14 il' w4 14-14 *.4 I* 14 14 1.4 14 14 14 L* V4 L4 14 14 14 t» 14 (4 14 14 14 14 i4 14 14 *4 1#
or cr< Oc. a
(X< U-
to
O h-z <c
< •—4- >V. 1C
? K>o *-— rv
el *ruj inu in-c ai
< r-i rvj •( r»
<_> a.
*< H>uj a. c_i a. ul
J l) —
PRETEST
/POSTTEST
COMPARISONS
OF
STUDENT
EVALUATIONS
FOR
CONTROL
GROUP
(continued)
200
« CL
- ft Cl
O T.Oto CLU UJIL ILcc rrC3 u.Uia
o a.tr< U,
«=* —ft- >CO u
CO COUJ UJto lO-t nu
CO XCC UJ— 2
O. UJUJ tocr «#
Ift VJ U Cft l ft
APPENDIX D
STUDENT ACHIEVEMENT TESTS
202
SCORES OF THREE raters on STUDENT ACHIEVEMENT TESTS FOR EACH SUB-CROUP
Group~-sl Pretest Scores Test Posttest Scores
Experimental X 81 88 94 - 263 0 78 82 930 78 80 85 - 243 X 79 82 90X 50 69 73 - 192 0 72 85 87X 80 88 88 - 256 0 82 86 900 80 81 84 - 245 X 74 83 86
Control 0 80 93 97 -2 70 X 88 92 96X 71 80 84 - 235 O 63 67 670 79 80 86 - 245 X 58 71 72X 69 84 86 - 239 0 94 95 102
Quasi-Experimental X 72 84 86 - 242 0 76 85 93
0 59 66 81 - 206 X 49 56 640 74 83 83 •- 240 X 78 88 91
59 66 72 197 64 69 70 -
2 76
197
201
291
254
169
257
2030
X