observing foreign language teaching: a new method for teachers, researchers, and supervisors
TRANSCRIPT
Observing Foreign Language Teaching: A New Method for Teachers,
Researchers, and Supervisors
Leona G . Bailey
~~
ABSTRACT Numerous methods of evaluatinglan- guage teaching and learning are in common use. One accurate method is direct classroom observa- tion, t h e most widely used form of which is inter- action analysis. Since interaction analysis and FLint have been found wanting (lack of reliability, observer bias, multiplicity of categories, cumber- some recording system), t he time-interval record- ing system (a method employed in social science studies) is offered as a practical and reliable
alternative. Features of this system include: describing behaviors to be observed, recording occurrences of behavior in a t ime interval (usually ten seconds), drawing up an observational code, assessing accuracy by deriving inter-observer reliabilities, plotting data collected on a progress chart , and interpreting findings. Teachers, super- visors, and researchers may then effect changes in teaching procedures and programs based on their analysis of classroom observations.
Introduction
Once the workshops, inservice training ses- sions, and summer institutes with their intensive
presentations of new techniques have terminated,
foreign language teachers return to their class- rooms to put into practice what they have learned
about teaching. But how do they detect the changes and improvements they hope t o have made
in their instructional repertoires? How do their principals or chairpersons discover if they a r e
indeed teaching more efficiently, more effective-
ly , and with better results than before? In fact, how does a supervisor, for example, evaluate the
teachers in his or her charge on a day-to-day basis?
Lcoru. G. W l q (Ph.D., University of Kuuu) is Asmiitant Profcuor In tne tment of Modcrn L M y y t at Florid. State univeraity, Tallahassee, Fla.
All of these professionals use a variety of
methods for evaluating learning and teaching. Appraisal of learning, which is typically of a cog- nitive or affect ive nature, includes many kinds of
measurement. Exam grades, scores on standard-
ized achievement tests, student progress in suc-
ceeding classes, and results of a t t i tude question- naires a r e among t h e most widely used means of judging students' at tainments. Inspection of lesson plans, observation of the use of various media,
review of student evaluations of the class, consult-
ation of colleagues to elicit opinions--these and
many other forms of measurement a r e commonly
employed in determining teaching effectiveness. As accurate and necessary as these methods a r e in their own domains, they tend to evaluate primarily
the ar t i facts or products of teaching. In order t o
obtain a total picture of teaching, another form of
64 1
642
assessment--one which goes t o t h e h e a r t of t h e
instruct ional mode--must a lso be car r ied out . One must observe t h e vital and ongoing process of
foreign language teaching in t h e classroom i tself .
The d e g r e e t o which teachers , supervisors, and
researchers can rely on classroom observat ions as an a c c u r a t e and rel iable source of information
depends a lmost en t i re ly on t h e method of observa-
How d o t e a c h e r s d e t e c t
t h e changes and improvemen t s
t h e y hope t o have m a d e
in the i r t each ing s t r a t eg ie s?
t ion chosen. I t i s cer ta in ly t r u e t h a t w h a t is observed i s just as impor tan t as *it isobserved. Nevertheless , i t is also t r u e t h a t two o r m o r e to ta l ly d i f fe ren t evaluat ions of a t e a c h e r can be formula ted as a resul t of how t h a t teacher is ob- served .
In t h e field of foreign language teaching, a
number of observat ional methods have been
employed in r e c e n t years. The most widely ac- c e p t e d sys tem is probably Ned Flanders' in te rac-
tion analysis.' This system has been adapted for
t h e use of foreign language educa tors by G e r t r u d e
Moskowitz, whose FLint program has gained wide a c c e p t a n c e by t e a c h e r t ra iners and supervisors.
These observat ional sys tems have been t h e subject
of numerous research s tudies3 and have seen
ex tens ive pract ical application.
2
Despi te the i r popularity, in te rac t ion analysis and FLint have been found want ing in many
I . Interaction Analysis: A Manual for Observers (Ann Arbor: University of Michigan, 1960).
2. The Foreign Language Teacher Interacts (Minneapolis: Association for Productive Teaching, 1970).
3. Edmund 1. Amidon and John 8. Hough, eds., Interaction Analysis: Theory, Research, and Application (Reading, Mars.: Addison-Wesley, 1967).
respects . In a n ear l ie r numerous weak-
nesses of t h e Flanders' sys tem w e r e outlined.
Among t h e problems noted was t h e f a c t t h a t dis t inct ions between t h e severa l observat ional
ca tegor ies a r e l e f t unclear. For example, a class-
room observer would have diff icul ty ident i fying
such notions a s 'understanding' or ' acceptance of feeling' reliably. Another ca tegory e m b r a c e s t h e to ta l ly opposi te behaviors of 'silence' and 'conf u-
sion,' rendering a lmost impossible a meaningful
in te rpre ta t ion of th i s observat ion. Since observat ions a r e m a d e o n c e every t h r e e
seconds, t h e observer's t a s k is f u r t h e r impaired by
t h e necessi ty of making split-second decisions, recording such decisions on paper , and ver i fying
t h e passage of t ime. If t h e observer is unsure of which behavioral ca tegory to note , h e is ins t ruc ted
t o record t h e c lass leas t favorable to t h e instruc- tor . Inaccuracies a r e sure to ex is t in such a system. The FLint method IS even m o r e complex and,
therefore , less likely to be rel iable and useful. This sys tem increases t h e number of observat ional
Time-interval record ing has been
found to be e f f e c t i v e f o r
observing classroom behaviors .
ca tegor ies to twenty- two ( interact ion analysis
contains ten) , requiring even g r e a t e r powers of discr iminat ion of t h e observer . The avai labi l i ty of
a bilingual observer and t h e possibility t h a t such a
language-expert observer might be highly biased
a r e additional weaknesses of FLint. The problems of discr iminat ing ca tegor ies
a c c u r a t e l y and t h e obs tac les to be overcome in making observat ions a r e intensif ied by t h e lack of
4. LeonaG. Bailey,"An Observational Methodin t h e Foreign Language Classroom: A Closer Look a t Interaction Analysis," Foreign Language Annals, 8 (1975), 335-44.
643
re l iable d a t a to support t h e conclusions of in te r - a c t i o n analysis researcher^.^ Inter-observer
re l iabi l i ty (by t h e use of Scott's coef f ic ien t ) t ends
to give a higher s c o r e than would o therwise be
obtained. The poten t ia l user of in te rac t ion analysis and
FLin t m u s t also quest ion t h e prac t ica l i ty of t h e s e
systems. Will a teacher , principal, or supervisor ac tua l ly b e mot iva ted to use such unwieldy evalua-
tion methods? Interact ion analysis must b e
mas tered , apparent ly , in a t ra ining program of severa l months. The use of a complex m a t r i x to
which t h e numerous teacher / s tudent behavior ra t ios must be t ransfer red i s cumbersome. And i t
would be diff icul t for a teacher to have his c lass
videotaped, as t h e FLint sys tem suggests , thus f u r t h e r decreas ing t h e appl icabi l i ty of t h e system.
Fortunately, a re la t ive ly new method of behav- ioral observat ion, developed and f i r s t used in social sc ience research , i s avai lable to t h e observ-
e r of foreign language instruct ion. This sys tem, commonly r e f e r r e d to as 'time-interval recording,'
has a l ready been applied to l ive s tudent / teacher in te rac t ions in various classroom se t t ings and has
undergone rigorous analysis in l i teral ly hundreds of research s tudies . Time-interval recording has
been found to be a most e f f e c t i v e method for observing object ively def inable behaviors which c a n be seen and heard in t h e classroom. This sys tem has been used to record such observable behaviors as how well t eachers pay a t ten t ion to
the i r s tudents ,6 how t e a c h e r s can increase the i r use of praise,' and t h e length of t i m e t e a c h e r s
spend in c o n t a c t wi th individual students.' Scores
of o t h e r researchers have applied th i s new sys tem
for c lassroom observat ion to the i r work and have found it prac t ica l , reliable, and accura te .
What, then , does t h e observer of foreign lan- guage teaching requi re of a n observat ional
sys tem? If t h e observer i s a secondary school principal, d e p a r t m e n t chairperson, or foreign
language supervisor, h e o r s h e will need a method
The observer must f i rs t define t h e si tuation in which t h e
observations a r e to be made.
which t a k e s re la t ively l i t t l e e f f o r t to learn and by which to col lec t information in t h e shor tes t possi-
ble t ime. The sys tem should b e very a c c u r a t e and rel iable , s ince crucial decisions about a teacher
(re-assignment, re t ra ining, promotion, etc.) may be based wholly, or in par t , on what is observed in
t h e classroom. The method must be easy t o use, and t h e resul t ing information easy to ana lyze and in te rpre t so t h a t t h e principal or supervisor c a n
readily understand and explain t h e resul ts of t h e observat ions to col leagues and t h e t e a c h e r observ- ed.
In addi t ion, many t e a c h e r s a r e e a g e r to discov-
e r ways of evaluat ing t h e effects of new techni- ques used in the i r c lasses or re inforcing the i r con- f idence in t h e most common methods of instruc-
tion. In the i r e f f o r t s to improve the i r own teach- ing or to give feedback to a fellow t e a c h e r , they too must be a b l e t o use a prac t ica l , s t ra ightfor- -
5. Ned A. Flanders,"The Problems of Observer Training and Reliability," in Amidon and Hough, p: 162; Ernest Lohman et al., "A Study of the Effect of Pre-Service Training in I n t e r z h Analysis on the Verbal Behavior of Student Teachers," in Amidon and Hough, pp. 3kb-59; John 8. Hough and Richard Ober, "The Effects of Training in Interaction Analysis on the Verba l Behav- ior of Pie-Service Teachers," in Amidon and Hough, p. 343.
6. Margaret L. Cooper et g., "The Experimental Modifica- tion of Teacher Attending Beavior," Journal of Applied Behav- ior Analysis, 3 (1970). 153-57.
ward, and prec ise f o r m of classroom observation.
They require a method which will focus on d iscree t
7. Ace Cossairt e t al., "The Effects of Experimenters' Instructions, F e e d b a c k T a s Praise on Teacher Praise and Stu- 8 . John W . Scott and Don Bushell Jr., "The LmKth of dent Attending Behavior," Journal of Applied Behavior Analysis, 6 (1973), 89-100.
Teacher Contacts and Students' Off-Task Behavior," Jou&al of Applied Behavior Analysis, 7 (197s). 39-44.
644
s tudent and teacher behaviors and which will
quickly inform them about the i r progress.
Foreign language educat ion researchers , whose
needs include most of those just ment ioned, have addi t ional requi rements when invest igat ing t h e
instruct ional process. They must use a method of observat ion which has been shown repea ted ly to be
rel iable and in which their own d a t a can be proven a c c u r a t e . The system must provide a useful d a t a
base f rom which they can manipula te t h e var iables in the i r study. Last , they must be a b l e to observe
and judge t rends in t h e d a t a within a short period
of t i m e (beginning with t h e f i r s t session of obser- vation) in order to change t h e parameters of t h e
invest igat ion a s t h e d a t a d i c t a t e . Let us now look
a t t h e t ime-interval recording sys tem t o discover
The second s t e p is defining and
recording t h e behavioral
events occurring in t h e classroom.
how i t m e e t s t h e above requi rements and in what
ways i t would be appl icable to observat ions of t h e
foreign language classroom.
Specifying t h e Si tuat ion
The f i r s t procedure which t h e observer must follow is to def ine t h e s i tuat ion in which t h e obser-
vations a r e m a d e in t e r m s of i t s physical and social s e t t i n g and t h e events which occur within i t s con-
fines. The physical propert ies of t h e s tandard secondary school or col lege classroom d o not
requi re ex tens ive definition. However , changes in t h e physical environment (moving t h e class outs ide on a warm day or rearranging t h e s tudents ' desks
i n t o small conversat ion circles) must be noted by t h e observer . Likewise, t h e social s i tuat ion may be a f luc tua t ing one (fewer s tudents due to ab-
sences or t h e presence of a visitor in t he class)
which should be taken into account .
Defining and Recording Events in Observable Tcrrns
The second s t e p of t h e t ime-interval recording
systern is defining and recording t h e behavioral events which a r e occurr ing in t h e classroom. The
teacher t ra iner , supervisor , or t e a c h e r must have
defined c r i t e r i a in such a way t h a t t w o or more
observers will a g r e e on t h e occurrence of t h e behavior. Precise def ini t ions a r e crucial to relia-
bility and accuracy of observat ion. For example,
t h e nursery school teacher who wishes to assess t h e magnitude of a child's hi t t ing behavior must
def ine hi t t ing in such a way as t o d i f fe ren t ia te i t
f rom pat t ing, touching, punching, shoving, e t c . Likewise, t h e teacher t ra iner who desires t o
measure how frequent ly an in te rn gives positive
feedback to s tudents should draw up a l is t of corn- monly used praise words or expressions ("That's
great!" "You've real ly got it!" tlExcellent!'l). The next s t e p is to d e t e r m i n e t h e manner in
which a behavior is recorded. The most a c c e p t e d method i s recording t h e occurrence of a behavior in a t i m e in te rva l w h e r e t h e observer makes a m a r k in e a c h t i m e in te rva l t h e f i r s t t i m e a re- sponse occurs. The usual t i m e interval se lec ted i s
t e n to f i f t e e n seconds; in te rva ls shor te r t h a n this have been found to be diff icul t to measure reli- ably. Thus, a n observer who wished to measure t h e
amount of t i m e a foreign language t e a c h e r spent
speaking English (in ten-second intervals) would m a k e o n e mark in a box if t h e t e a c h e r u t te red a word of English within t e n seconds; if t h e teacher spoke English in t h r e e out of s ix ten-second inter-
vals, t h e observer would have marked t h r e e boxes
and would have summar ized by saying t h a t English was used 50% of t h e t i m e (see Figure 1).
In order to put confidence
in t h e observation code,
inter-obser ver agreement
would have to be above 85%.
645
FIGURE I
Foreign Language Observation Form
Teacher Questions
Student Response
Teacher Praise
Teacher English
Use of Observational Code Which categories of behavior are to be selected
for study i s usually determined by the investiga-
tor, teacher trainer, department chairperson,
supervisor, or instructor. However, the maximum
number of categories of interaction must be
restricted by practical considerations. The ob- server who attempts t o classify and record dozens
of behaviors wi l l be frustrated and, as researchers
in this area have pointed out, wi l l be more than
likely t o make poor and inaccurate judgments.
Most classroom research suggests a l im i t of f ive or
six simultaneous observations of behavior; inter-
observer agreement is reduced dramatically
beyond this number.
Assessing Observer Agreement The accuracy of the observations made i s
crucial to the degree of confidence investigators
can have in their observation system and, eventu-
ally, in the decisions which result from observing
teacthg. In almost al l research settings, accuracy
i s assessed by having two people observe the same
behaviors at exactly the same t ime and by calcu-
lat ing an inter-observer agreement score. Both
observers must know the observational code and
how t o record their observations. They must be
trained to refrain from interacting with the teach-
er or students (perhaps by viewing a f i lm or video-
tape simulation). In practical situations, i t i s
unlikely that a principal, supervisor, or teacher
would invite a second observer into the classroom
more than once, even during an extended period of
observation. However, one reliabil i ty check w i l l
give the primary observer sufficient information
about the precision of the coding.
Researchers will, of necessity, have several
observer-agreement checks made during the
The teacher trainer will probably use the system to study trends
and fluctuations. .
course of a study and then calculate the reliabil i ty
of the two observers' data. Reliabil ity i s calcu-
lated by comparing each observer's record, inter-
val by interval, scoring each comparison as 'agree'
or 'disagree' and dividing the total number of
agreements (A) by the number of agreements (A)
plus the number of disagreements (D) times one
hundred " j r r i r x 100). For example, let us say that
a behavior was observed for 180 ten-second inter-
vals (a total of thirty minutes). Observers X and Y agreed that the behavior occurred eighty-four
A
646
40
W (3 30-
z w 0 20' U w [r
3
10
intervals out of 180; they disagreed on ninety-six
intervals . Their inter-observer agreement score 84 would then be 47% (sgt- x IOO), a very poor
reliability. In order to put confidence in t h e observat ion r o d e , agreement would have to be
above 85%.
c 9 6
L
-
Analyzing and Interpret ing D a t a The classroom observat ions resul t , then, in t h e
col lect ion of d a t a which may be easi ly analyzed
and in te rpre ted a s t o s ignif icance. The f requency of t h e observat ions (every day, o n c e a week, o n c e a month, etc.) and the i r length ( ten minutes , half a n
hour, a n e n t i r e c lass period, e tc . ) a r e de te rmined by t h e behaviors t o be observed and t h e natural
considerat ions of e a c h s i tuat ion. Obviously, t h e
more f requent and t h e longer t h e observat ions, t h e
m o r e meaningful t h e d a t a will be.
Regardless of observat ion f requency and length , t h e observer should keep t r a c k of t h e
information co l lec ted by plot t ing t h e d a t a e a c h day t h a t i t is co l lec ted to c r e a t e a kind of progress
char t . A visual display of t h e behavioral changes
observed will ass is t t h e principal, supervisor, or
teacher in drawing conclusions about progress in
teaching. Figure 2 shows a graph of hypothet ical
d a t a of a col lege foreign language classroom. The number of sessions observed is marked along t h e abscissa; t h e percent of t i m e is noted on t h e
ordinate . The graph is read: "During t h e f i rs t
observat ion, 37% of Teacher A's vocal izat ions
were in English; during observat ion two, 31% w e r e
in English; observat ion t h r e e , 30% in English, etc." The sample graph also shows t h a t t h e r a t e of
Teacher B's English vocal izat ions was 34% (first
observat ion) , 35% (second observat ion) , 33% (third observat ion) , etc.
The teacher t ra iner will probably use t h e
sys tem t o s tudy t rends and f luc tua t ions on t h e
graph. A decrease in t h e f requency of a behavior
will be shown on t h e graphic c u r v e as a reduct ion in percentage; an increase in f requency will be seen
as a gain in percentage . In Figure 2, we c a n see
t h a t t h e r a t e of t h e use of English by Teacher A has
decreased f r o m 37% of all vocal izat ions to 12% by
F l C l ' R E 2
ENGLISH VOCALIZATIONS
TEACHER A
I I I I I I I I ;- 3 4 5 6 7 8 9 10 1
I
SUCCESSIVE OBSERVATIONS
647
t h e end of t h e t e n t h observat ion session; likewise,
Teacher Bls use of English did not f l u c t u a t e much
f r o m t h e 34% f i r s t observed. The in te rpre ta t ion of t h e findings is of course
t h e raison d ' i t r e of t h e observat ion process. Once
t h e d a t a are col lec ted and analyzed, t h e research-
e r o r supervisor must d e t e r m i n e the i r importance.
The f i r s t s t e p i s usually a comparison with s imilar t e a c h e r s m a d e under s imilar conditions. How much English does Teacher A use compared with
Teacher B? A second s t e p is a comparison of t h e present d a t a with t h e investigator's own expec ta-
Confusion of and ambiguity in category definitions a r e
seldom found i n time-interval recordings.
tions. Does Teacher A use much m o r e English t h a n his t ra ining and t h e instruct ional mater ia l s would d i c t a t e ? A probable third s t e p is to propose changes of t h e behavior s tudied so as to lower (or ra ise) t h e r a t e to desirable levels. A supervisor
might have recommended t h a t for t w o weeks
Teacher A visit c lasses taught by a m o r e experi- enced col league in which t h e t a r g e t language is
used a lmost exclusively; a f t e r this exposure t o a var ie ty of techniques for teaching without t h e use of English, Teacher A would be observed again, and improvements ( reduced r a t e of English vocaliza-
tions) would be noted by t h e supervisor. If no posi- t i v e effect was observed, (as in t h e case of
Teacher B) other procedures (inservice workshops, videotape recording and feedback , e tc . ) would be
ins t i tu ted , and t h e teaching observed o n c e again.
Discussion In present ing t h e pr imary f e a t u r e s o f t h e t ime-
in te rva l sys tem of observat ion, we hope to have shown t h a t th i s experimentally-developed method
f o r recording behavior c a n become a useful and pract ical tool for t h e foreign language t e a c h e r
t ra iner or supervisor a s well as for teachers them-
selves. It should r e p l a c e t h e most popular means
of classroom observat ion-- interact ion analysis-- because i t answers numerous questions raised
about t h e l a t t e r system.
Confusion of and ambigui ty in ca tegory defini- t ions a r e seldom found in t ime-interval recordings,
s ince t h e ca tegor ies a r e def ined in t e r m s of t h e occurrence or non-occurrence of observable
events . The t i m e cons t ra in ts placed on t h e inter- ac t ion analysis observer (recordings m a d e every
t h r e e seconds) and t h e resu l tan t inaccuracies of
t h e records a r e e l imina ted from t h e present
method s ince t h e normal recording interval is t e n seconds. Because t h e observer using t h e tiine- interval sys tew has a res t r ic ted number of c a t e - gories within which to work, his observations will
be m o r e a c c u r a t e . Researchers can have much
g r e a t e r confidence in the i r d a t a as a resul t of t h e
instance-by-instance rel iabi l i t ies car r ied out by t w o or m o r e observers .
The g r e a t e s t benefi ts to a c c r u e from this system a r e t h e many pract ical advantages for foreign language teachers and their supervisors.
The basics of t ime-interval recordlng can be
Teaching is a host
of observable social and
environmental interactions.
learned in a f e w hours; no special equipment i s required to m a k e classroom observations. Within
o n e recording session, t h e observer can de termine t rends in behavior and make per t inent decisions.
The sys tem is a c c u r a t e and rel iable , as numerous research s tudies in t h e classroom have shown. The
d a t a s h e e t on which t h e observat ions a r e recorded is sirnple to design and use; t h e graphic representa- tion of t h e d a t a is easy t o read and analyze.
648
Teachers--especially foreign language
teachers--carry out the i r work in a manner differ-
e n t froin a lmost any o ther profescional (librarians,
journalists, b iochemists , for exarnple). Teaching is
a host of observable social and environmental
in te rac t ions which t e a c h e r s perform in a def ined
s i tuat ion and in t h e presence of people whose
behavior t h e y wish to change. Almost all t e a c h e r s
want to be informed about t h e effects of the i r
t eaching techniques and whether or not t h e hoped-
for improvements in learning a r e ac tua l ly taking
place. Their t eaching processes can and must be
observed, no t only by t h e s e t e a c h e r s but also by
those who t ra in and supervise them and by those
who wish to inves t iga te and enhance instruct ion.
The choice of a pract ical and rel iable method of
observat ion is of fundamenta l impor tance to these
foreign language educa tors . I t is hoped t h a t t h e
t ime-interval recording system outlined here will
provide t h e necessary means for making their
observat ions and will cont r ibu te t o t h e improve- m e n t of foreign language instruct ion.
Information Needed for Summer Workshop Listing
The April 1978 issue of FLA will provide a list ing of summer workshops and ins t i tu tes for foreign language teachers. Courses in foreign language s tudy of fered during t h e s u m m e r a r e normally not included unless t h e r e a r e unique components of t h e program.
Institutions wishing to have the i r offerings includedin t h e l ist ing should provide t h e Editorial O f f i c e (ACTFL, 2 Park Ave., New York, NY 10016) with t h e following information by 10 January 1978.
(1) Dates of workshop; (2) Ti t le of workshop; (3) Name of Institution; (4) Credi t s of fe red (including graduate or under-
( 5 ) C o s t (noting what is included); ( 6 ) Features of workshop; (7) Application or registration deadline, and (8) Name and address for fur ther information.
graduate and semes ter or quar te r hours);
NDEA Products Bibliography Available Copies of List No. 8 of Forei Lan ua e Area, and
Other International Studies: A E3Eliogr:ph;df R e s e a 5 and Instructional Materials Corn le ted Under NDEA a r e now available f r e e of charge fro:: L a n g u a g e a n d r e a Research Program, International Studies Branch, Division of International Education, U.S. Off ice of Education, Washington, D C 20202.
SUMMER STUDY AT NYU IN PARIS
Graduate Program Emphasis o n language, civilization, contem- porary literature. 3-week coursc's.
M.A. may be completed in two summers: Paris and New York, or both in New York.
Candidates may enter in June or July, in New York or Paris.
Undergraduate Program 6-week intensive courses, July 3-August 11, a t various levels. 8 credits. Includes activi- ties, excursions.
Open to graduates of other disciplines seeking proficiency i n French.
Open to qualified high school seniors with parental permission.
Call (212) 598-2262 or write:
Prof. J.R. Hewitt NYU IN PARIS Faculty of Arts & Science 19 University Place New York. N.Y. 10003
LAROUSSE The Most F a m o u s Name in Dictionaries
p r e s e n t s t h e PEQUENO LAROUSSE ILUSTRADO
LAROUSSE
The outstanding reference work for all s tu dents and teachers of Spanish language and literature Revised and updated to include more that 60 000 entries and 6 000 ilustra tions and maps S19 95 7 696 pp
L a r o u s s e 8 Co., Inc. 572 Fifth Ave New York N Y 10036 Telephone (212) 575-9515 Telex 12335
Please mention Foreign Language Atlrcals when writing to advertisers