text selection

From the book (Developing translation competence- Edited by Christina Schaffner and Beverly Adab -2000, Amsterdam/Philadelphia: John Benjamins)

Text Selection for Developing Translator Competence:Why Texts From The Tourist Sector Constitute Suitable

Material

DOROTHY KELLYUniversidad de Granada

Introduction and Context

In this paper I shall explore criteria for text selection in translator training programmes at university, taking as my starting point one particular set of text types, those belonging to the tourist sector. Before beginning to comment on the reasons behind the decision to include texts from the tourist sector on the General Spanish-English Translation module at the University of Granada (Spain), however, it is appropriate to briefly situate the module in its academic context. It is a second year module on the four-year undergraduate course in Translating and Interpreting, and constitutes the students' first experience of actual translation practice after a foundation year with modules in their three working languages (mother tongue plus two foreign languages), related area studies, an introduction to linguistics for translation, and instrumental skills such as documentary research and computer skills. This module is, then, actually an introduction to practical translation skills, and is accompanied on the curriculum by a module in English-Spanish translation, together with a translation studies module. Given the university entrance system, and despite having all followed a first-year course in English language, our students, who all work with English as their first and main foreign language, have fairly diverse levels of active ability in English, although on the whole they are almost all capable of producing relatively correct and easily revisable texts in the language, which can be seen as being the minimum level necessary for undertaking a module in translation into a foreign language designed, in combination with others, to train future professionals.'

A brief parenthesis is perhaps necessary here to mention that it is my own firm belief, coinciding with authors such as McAlester (1992, 1997), Sânchez

158 TEXT SELECTION

(1994, 1997), Beeby (1996), Nobs (1996), or Way (1996), that translation into non-primary language, or non-mother tongue, is not only a' professional necessity on many national and local translation markets, but also a useful training exercise, in that carrying out the translation operation the other way round allows students to move away from their typical interpretation of the problems of translation into their mother tongue: i.e.. that the difficulty lies mainly in understanding the foreign-language original. In translating texts out of their mother tongue, they realise that understanding the original is firstly not only a question of language, but also of concepts and background knowledge, and hence of documentary research. Similarly, they realise that their translation difficulties do not end there: as they have greater difficulty reformulating in their foreign language, they pay more attention to the

reformulation process, to the search for appropriate textual and linguistic forms, learning to use sources such as parallel texts (see below) in order to do so.

This introductory module to translation into the students' first foreign language is divided basically into three parts: the first (between 10 and 15 hours) consists of short pre-translation and introductory exercises designed to help students develop approaches to different translation commission, and in particular to begin to escape literalist fidelity (see Kelly 1997a, 1997b, Sânchez 1994, 1997, Nobs 1996, Weatherby 1997) for descriptions of some of these exercises). The second pays particular attention to the translation of cultural references, specifically Spanish cultural references for English-speaking audiences (United Kingdom and United States, other English-speaking countries, English as an international language). This aspect of translation, although inevitably present in other modules on the curriculum, is not the central focus of any, hence the decision to make it one of the key elements of this one. And finally the third, which can be considered an extension of the second, to translation in the tourist sector. The basic approach adopted in the module is a functionalist approach to translation, which I have found particularly well suited to the development of translation competences, and in particular to developing skills in translation into the foreign language. The overall objective of this module could then be summed up as helping students to develop the skills necessary to undertake different kinds of translation commission, to analyse translation situations and to adopt solutions appropriate to these, with particular emphasis on awareness of the fact that there is a cultural gap between the reader of the source text and the reader of

the target text which the translator is responsible for bridging.

DOROTHY KELLY 159 Text Selection Criteria

The specific subject of this paper is the exploitation of tourist texts in this context. It is worth noting the horror in the reaction of a professor in Linguistics at the Arts Faculty of my University when he first heard that such unworthy material was used in our classes, resulting of course from the conviction which persists in some sectors that translation is only worthy of a place at university if it is literary translation! For those of us who work on university translator training courses, such reactions are fairly commonplace, and little response is necessary within our discipline to justify this type of material in our classes. I feel, however, that reflection on why we use certain text types or genres or texts from certain fields is essential if our training is to be systematic and thus ultimately efficient.

Several authors (Reiss 1971, Hatim 1984, Nord 1991, 1997, among others), have suggested that translation courses should be organised by text type although they do not coincide in their taxonomy, and in general little has been written, aside from a text's classification in one or another text type, on actual criteria for text selection. All translator trainers have surely had the experience of a text which simply doesn't work in class, and have all surely wondered why. I believe that text selection is one of the most important aspects of our teaching activity and, as such,

it is disheartening to see just how (albeit informedly!) haphazard it often is. We hear - and make ourselves - generalisations of the kind: texts must reflect the professional market; informative texts rather than expressive in the early stages, or for translation out of the mother tongue; texts must be chosen to illustrate specific translation issues.

All of these are, at least in part, valid considerations, but do they make any informative text which could constitute a professional commission and which happens to contain a metaphor or a proper name or whatever other specific point of translation, valid for a beginners' - or any other - class? Not in my opinion. Students and teachers' are immersed in a very complex learning process in which many elements come into play. The remainder of the paper will attempt to outline the different positive points the particular set of texts used by the tourist sector offers in general and in the specific context in which I use them. The set of texts, of course, is made up of a wide variety of text genres and subgenres, precisely one of its positive points. The limited scope of this paper will not allow us to look at these genres and subgenres in detail, but rather only at the set of texts as a whole.

160 TEXT SELECTION Continuity in subject

matter

Firstly, my experience in translator training would indicate that it is easier for students to develop the competences necessary to undertake different translation commissions if there is certain continuity in the subject matter or field dealt with over a period of time, that is if the teacher does not jump from one main subject area to another too frequently, particularly at the beginning of the learning process. Maintaining the subject area fairly constant allows the class to concentrate on specific aspects of the translation process, which should constitute the actual focus of the module and should be varied with particular pedagogical purposes in mind. In this respect, tourism is an ideal sector, as it not only allows that continuity, but also considerable diversity within it, thus avoiding tedious repetition, which would in turn be negative for the learning process.

Professional relevance

Secondly, tourism is one of the most important sectors of the Spanish economy, especially in Andalusia, the region our University is situated in. It is also a sector which by its very nature is multinational and multicultural, and involves a considerable amount of translation work. From this demand point of view, there is, then, an evident need for translators to be able to work in this field, and it is no doubt part of the university's social remit to meet this demand. Thirdly, and again from the market point of view, this time on the supply side, there is a tendency in Spain and, if we are to judge from some of the comments of the few authors (Duff 1981 or Newmark 1988, 1991, 1993, among others) who have made reference to translation in the tourist sector to date, also in other countries, for translations in the tourist sector to be of poor quality. This is partly because they are only rarely carried out by professionals, but rather by more or less well qualified amateurs, with varying, but usually deficient, capacity to approach the communicative situation they have undertaken to resolve. This third reason for the choice of tourism is then again a professional one: the need for more professional involvement in this activity, the development of awareness in future professionals of this need, with a view also to the education of the market.

Professional realism

Quite apart from the market's need - whether conscious or otherwise - for good professionals, there is

another reason linked to the profession for the inclusion

I

DOROTHY KELLY 161

of texts of this type on our course, and that is the belief that in order for students to develop professional competences the texts used in class should in general correspond as closely as possible to real professional situations. A fourth reason for choosing texts from this sector, then, is that they constitute authentic professionally translatable and translated material, and not artificial exercises. On this point, however, there is perhaps a need to introduce some nuance. Several authors (e.g. Gouadec 1989, 1994 or Kiraly 1995) have suggested that the closer the classroom experience is to real professional experience the better. This belief cannot perhaps be taken as an absolute at all levels of the learning process. It would seem obvious that the best way to train future translators of legal, commercial, scientific, technical or tourist texts is indeed to work with them during training, and specifically with those particular genres which are most frequent on the translation market. This does not, however, necessarily mean that other sets of texts (e.g. literature, which occupies a very small percentage of the current professional market) cannot play a role in translator training, if presented with the aim of developing specific translation awareness or competences.

Just as I believe that not all texts used in class must of necessity slavishly reflect the major areas of demand on the professional market, similarly and more importantly, I believe that the professionalisation of training must take into account that for the students this is a learning experience and, as such, care must be taken not to ask too much of them too soon. Professional demands regarding quality of the finished product, or the time in which the translation is produced, must be subordinate to the actual learning process, especially in the early stages of training. I have often found that students easily become obsessed with certain aspects of the professional world (notably doing the translation quickly, or the instrumental skills involved: word processing packages, desk-top publishing or formal aspects such as how to submit work) to the detriment of doing it well, or actual translation questions such as bridging cultural differences or respecting target text function. Quality should come before quantity in importance during the early stages of training, and indeed the former should be a pre-requisite for the latter: there is little point in producing a great amount of sub-standard text.

Grading of difficulty

In short, then, it is my opinion that texts (or translation situations, perhaps) chosen for classroom activity should in general, but not necessarily exclusively, be as real as possible regarding genre, field, translation brief, and so on, but also that demands on students should be scaled to suit their stage in the learning

162 TEXT SELECTION

process (see also Nord 1991). In the final stages of training authentic professional work for real commissioners (in the form of work placements or even with the Department or Faculty accepting professional commissions, as suggested by Gouadec 1989, 1994) have a role to play. To our fourth reason, i.e texts from the tourist sector are appropriate in this respect in that they are representative of our local, regional and national translation market, a fifth should thus be added: they are gradable in difficulty (subject matter, linguistic difficulty, degree of specialisation), in my case for the very early stages of training, due to the wide variety of texts the sector uses as mentioned above.

Textual experience

Turning now to other reasons for choosing this set of texts as a basis for class activities, and again insisting on the need for the learning process to be gradual, let us look at the importance of knowledge regarding textual conventions as part of the intercultural textual competence students should develop. The development of the competence and the acquisition of the knowledge which accompanies it depend, in the initial stages, on students' previous textual experience. Weatherby (1997) comments on the limited textual experience of her students, which also limits their awareness of textual conventions. She suggests extensive work with target language (in her case, as in ours, foreign language) texts before dealing with source language (mother

tongue) texts in order to facilitate the identification of source language/culture conventions and their differences with TL/TC conventions, introducing in this way the concept of parallel texts, how they can be useful to the translator, and what their limitations are. I have also worked extensively with parallel texts (see SnellHornby 1988) and have found them especially useful for translation out of the mother tongue, where students need to build up self-confidence in their ability to produce a valid target text. This entire process of understanding textual conventions, and the usefulness of parallel texts is easier assimilated if it begins with texts with which students are at least superficially familiar, before proceeding to text types or genres normally beyond their textual and life experience (bills of lading, contracts, scientific papers). A sixth reason for using texts from the tourist sector is, then, that they are likely already to form part of students' somewhat limited textual experience, thus facilitating the introduction of many of the concepts linked to source text analysis, analysis of the translation situation and different translation solutions.

DOROTHY KELLY 163

Textual function

Returning to the suggestions of the authors mentioned above on text type or text function taxonomies for translator training, text function is a seventh reason for selecting tourist texts. Not only do these texts usually combine to some degree the informative and operative or vocative functions3, recognised by most authors as more suitable for the early stages of training, but as these texts are used in situations with which students can identify, they find it relatively easy to assimilate the notion of text function in them; the situations they are used in are again familiar to the students and allow the student to analyze the primary and secondary functions of each individual text, thus becoming aware of the importance of text function as an overiding criterion for decision-making in the production of the target text.

Accessible content

Apart from textual conventions and functions, reason number eight is the fact that the subject matter of texts from the tourist sector is also more readily accessible for students in the early stages of their learning process, as most of the texts are addressed to a public with a fairly average knowledge of the topics covered, normally requiring little specialist knowledge to be understood.

Need for documentary research

This last point does not, of course, mean that texts from the tourist sector are devoid of comprehension, or documentary and terminological research problems: quite the contrary, and this constitutes another of the strong points of this type of text for the early stages of training, and my ninth reason for selecting them. The texts, although only very rarely intended for specialist readers, may deal with many different topics (going from sport and leisure activities to conference organisation, through art history, architecture or folklore) and occasionally from a fairly specialised perspective (the specialist explains to the layman): think, for example, of descriptions of the architectural grandeur of the Alhambra, or of the excellence of the various kinds of sherry produced in Jerez. This variety both of content and of text type within one main field allows classes to be varied, and allows the teacher to introduce some basic concepts of the application of documentary and terminological research skills to translation. In my particular case, care is taken to further facilitate the learning process by limiting the texts used in their subject matter to the city of Granada and the province of the same name, which are fortunately very rich in

164 TEXT SELECTION

kinds of tourism (coastal, mountain, rural, cultural and conference tourism are all well covered). This limit means that students are often already familiar with the monuments, villages, dishes or customs mentioned in the texts and, if not, may easily visit them or acquire information locally on them, thus allowing students to experience first hand just how important research and knowledge of the subject matter is for the translator. Once the student has mastered questions of textual conventions, and documentary and

terminological research for texts which are relatively accessible, it is easier for them to proceed to texts which are further from their own personal experience such as those we listed before: texts from the tourist sector constitute, then, sound preparation for the following stage in the training process.

Non-verbal elements

Further advantages of working with these texts is that they introduce other important aspects of the translator's professional activity which often go unnoticed on training programmes. The first of these, and my tenth reason, is the combination of written text and image (photographs, drawings, maps, and so on), the interdependence of the various elements to form a whole, which sometimes facilitates the translator's work and sometimes imposes severe constraints on it.

Quality of originals

The eleventh reason is the question of the quality of the original, one which inevitably arises at some point when dealing with Spanish originals in this sector, many of which are written without much thought for the reader or the function the text should fulfil socially, are often over-elaborate in their form of expression, following outdated conventions, and on occasion are even plagued with spelling and other errors. The functionalist approach to translation gives solid theoretical cover to the thorny issue of the translator's licence with the original, which students have considerable difficulty with; introducing the odd poor original in the module puts these theoretical and ethical issues to the students through their own translation experience.

Interpersonal Competence: Translator and Commissioner

Both of the above also bring into play the issue of the relationship between translator and commissioner, reason number twelve. There is a tendency (at least on the Spanish market) for translators not to be shown the photos,

DOROTHY KELLY 165

illustrations and so on which accompany the written text they are asked to translate. The importance of these elements underlines the need for customer education in this respect: students are thus introduced to the issues of what the translator should require of the commissioner in order to carry out the task in hand efficiently. The second question (quality of the original) also brings up the issue of customer relations for the translator. Students may be made aware (for example with a little role-play on the part of the teacher) of the need not only for customer education but also for diplomacy, as often (particularly in small towns and villages) the commissioner is also the author of the source text ; and may not take kindly to being told of its poor quality! Students thus begin '' to learn to defend their decisions on different levels: to fellow students and the teacher (translation specialists in varying degrees) and to (simulated) authors of source texts and translation commissioners (non-specialists). This creates for students an idea of their role in the complex social and communication situation which translation constitutes. In this respect, we have found that Nord's (1991) distinction between sender and author of the ST and initiator of the translation process is particularly useful.

Developing student confidence

And, last but far from least, a boon for the students' confidence as the learning process progresses and reason number thirteen for choosing tourist texts. Some authors (Kiraly 1995, Kussmaul 1995) comment on the traditional translation classroom as very negative for students' self-confidence, as most comments are error-based and thus negative. The building up of self-confidence is indeed essential in any learning process, and can only be achieved if the demands made on students are carefully graded to their level to make them feasible. In the case of translation out of their mother tongue, this need is perhaps even more acute. Here, briefly, then, an element which we use in class to promote selfconfidence among our students. Exercises in criticism of published translations from the sector, often after the same source text has been dealt with by students in class or in groups, show students that they are capable (with revision

by a native speaker, often an exchange student) of producing professional (publishable) translations which are on many occasions better than those which actually exist and are used by thousands of tourists each year in the province of Granada: clear encouragement for them to persevere!

166 TEXT SELECTION

Notes

I The criterion of revisable as the basis of translation assessment, particularly in translation out of the translator's mother tongue or primary language, has been suggested by McAlester (1992, 1997 and in this volume), and undoubtedly constitutes a valuable, if not the only, measuring stick for assessment within training programmes

2 1 am aware that some authors (see, e.g. Kiraly 1995) are not happy with the term teacher, as they believe it reflects a passive role of the student in the class: my use of it here is not to be understood in this traditional sense (I firmly believe that the learning process must be active on the part of the students), but rather for want of a better and commonly accepted term.

3 1 am using the classification derived from Buhler and adopted for translation by authors as distant in other respects as Newmark (1988) and Reiss (1971).

References

Beeby, A. 1996. Teaching Translation from Spanish to English: Worlds beyond Words. Ottawa: University of Ottawa Press. ,

Duff, A. 1981. The Third Language. Oxford: Pergamon.Gouadec, D. 1989. Le traducteur, la traduction et l'entreprise. Paris: Afnor. Gouadec, D. 1994. L'assurance qualité en traduction - perspectives professionelles,

implications pédagogiques. Plenary address at the 1st International Congresson Translation and Interpreting: Present Trends, Universidad de Las Palmas. Hatim, B. 1984. "A Text-

typological approach to syllabus design in translatortraining". The Incorporated Linguist 23: 146-149

Hatim, B. and Mason, I. 1997. The Translator as Communicator. London: Routledge.Kelly, D. 1997a. La traducciôn inversa en los planes de estudios de Traducciôn e Interpretacion. Paper

submitted to the 1st International Congress on Translation and Interpreting: Present Trends. Universidad de Las Palmas.

Kelly, D. 1997b. "La ensenanza de la traduccion inversa de text's 'generales': consideraciones metodolôgicas". In M. A. Vega and R Martin-Gaitero (eds), La palabra vertida: Investigaciones en torno a la traducciôn. Actas de los VI Encuentros Complutenses de la Traducci6n. Madrid: Universidad Complutense, 175-182.

Kiraly, D. 1995. Pathways to Translation: Pedagogy and Process. Kent: Kent State University Press.Kussmaul, P. 1995. Training the Translator. Amsterdam/Philadelphia: Benjamins.

DOROTHY KELLY 167

McAlester, G. 1992. "Teaching translation into a foreign language - status, scope and aims". In C. Dollerup and A. Loddegaard. (eds), Teaching Translation and Interpreting: Training, Talent and Experience. Amsterdam/Philadelphia: Benjamins, 291-298.

McAlester, G. in press. "The time factor: a practical evaluation criterion". In M. Grosman et al. (eds), Translation into Non-Mother Tongues. Tubingen: Stauffenburg.

Newmark, P. 1988. A Textbook of Translation. Hemel Hempstead: Prentice Hall.Newmark, P. 1991. About Translation. Clevedon: Multilingual Matters.Newmark, P. 1993. Paragraphs on Translation. Clevedon: Multilingual Matters.Nobs, M.-L. 1996. "Contra la literalidad gratuita: ejercicios preliminares a la traducciôn inversa (espanol-

alemân)". In M. Edo Julià (ed.), Actes del I Congrés Internacional sobre Traducciô, abril de 1992, (2). Bellaterra: Universitat Autbnoma de Barcelona, 409-415.

Nord, C. 199L Text Analysis in Translation. Amsterdam: Rodopi.Reiss, K. 1971. M6glichkeiten und Grenzen der Ubersetzungskritik. Munich: Hueber.Sânchez, D. 1994. Problemâtica de la traducciôn inversa: implicaciones didàcticas. Paper submitted to the 1st

International Congress on Translation and Interpreting: Present Trends, Universidad de Las Palmas.Sànchez, D 1997. Translation into non-mother tongues: How possible is it? Paper submitted to the Translation into

Non-Mother Tongues Conference, University of Ljubljana.Snell-Hornby, M. 1988. Translation Studies: An Integrated Approach. Amsterdam/Philadelphia: Benjamins.Way, C. 1996. Translating across legal systems: problems posed in translator training. Paper submitted to the

Second International TNE Conference, Budapest.Weatherby, J. 1997. Striving against literalist fidelity in Ll-L2 translation classes (Spanish-English). Paper

submitted to the Translation into NonMotherTongues Conference, University of Ljubljana.

Evaluating Translation Competence

BEVERLY ADABAston University Birmingham

Introduction

In the context of developing translation competence, one of the questions to be considered is that of how to evaluate the target text, as product of the process. This is also necessary in order to determine the level of competence achieved by the translator and to identify areas in which competence is still to be ' developed. Another consideration relates to the question of how the assessor could perform this task reliably, in the sense of a more objective and less subjectively-oriented judgement of the product. It will be argued in this paper that the identification of a set of criteria could form the basis, both for production and evaluation of the product. Improving translation competence should then be achievable, in terms of performance in transfer competence and production, through the awareness of the relative merits of different transfer strategies and careful selection from potential translation solutions.

We would also claim that setting defined criteria for this purpose can assist in raising awareness of the decision-making and revision stages of the production process. By requiring the trainee translator to focus on evaluation of different possible strategies and choices, in a more conscious selection of the most appropriate translation solution should follow. The paper will also seek to illustrate the interdependency between theory and practice which is fundamental to any translation training programme and to the development of translation competence.

What is the Purpose of Evaluation of a Target Text ?

A target text can be evaluated for different purposes: to assess the suitability of the text for its intended reader and use; to evaluate language competence

216 EVALUATING COMPETENCE

(usually L2, L3 ); to determine levels of intercultural awareness; or to identify levels and types of translation competence. Knowing the reason for evaluation and the criteria by which a text will be evaluated could help to improve the accuracy of this process, by giving a definition of the specific task in a given translation situation. In other words, in addition to understanding the purpose of the target text and the needs of the user, familiarity with the expectations of the product evaluator could be a useful factor both when selecting from possible translation alternatives and when revising choices within the text as global message. In this paper it is intended to offer an example of how evaluation of a target text can help to identify problems relating to translation competence, with reference to the sub-competences described by Neubert (in this volume) as textual, language, subject, cultural and transfer competences.

Translation in the Academic Environment .

Translation scholars who also teach translation in the academic environment are already very much aware of the different methods adopted in different universities, not only in Britain but in Europe and the rest of the world. A very simplified categorisation would probably divide these as follows. Firstly, a division into those working with literary or non-literary texts, which will usually depend on the nature of the overall programme within which translation is being taught. Hence, secondly, according to the nature of the programme. These programmes range from the traditional, with an emphasis on learning about the culture through its literature; to programmes in modem languages, with an emphasis on the socio-cultural environment and the prospective communicative demands of the working environment; to. specialist translation studies undergraduate programmes, with a focus on a combination of theory and translation practice. This latter type of programme aims to prepare students for the demands of postgraduate study and ultimately, of the professional translation environment. Of course, there is often some overlap between the different types.

In addition, the way in which the task of translation is exploited in an academic environment could be classified according to the purpose for which the exercise is being conducted. This may be for pedagogical reasons alone, to develop awareness of comparative linguistic and stylistic features, vocabulary acquisition and testing, target language fluency development, awareness of socio-political issues in the target culture. Or else the aim may be more complex, the purpose to develop a sense of how language is used to communicate experience, thoughts, feelings, aspects of social reality and

BEVERLY ADAB 217'

intellectual processes, from one culture to another. This approach draws on all of the list of competences included in the pedagogical approach; however, it also transcends this by making far more complex demands on the student, who may or may not be able to grasp all of these complexities, due in part to the form of training received in school and to the way in which translation was presented during the early stages of L2 acquisition. Finally, as already noted, the purpose may also be to train students in habits and working practices which will also lead to

competent performance in the varied and demanding conditions of the professional environment, translation or otherwise.

Traditionally, all of these different methods have often been seen as mutually exclusive and advocates of one approach have sometimes disregarded the valuable insights to be learned from other perspectives. It could be argued that whatever the final purpose of this activity in the university environment, students can benefit from an approach to translation which presents this as a real exercise in communication; in other words, an approach which adopts the fundamental -principles of the theoretical aspects of translation studies, combined with the aims of a professional translation training programme, leading to effective intercultural mediation of a message. This would entail assessment being used to evaluate target texts as practical evidence of translation competence in performance, rather than for the purpose of assessing language development.

It is clear that for the purpose of translation training programmes, evaluating language competence in L2, L3 etc is not seen as the primary objective of evaluation of the target text. The criteria of usability or suitability for intended purpose, of a target text necessarily involve consideration of the translation competence of the translator and if this is the main focus of the evaluation, then it would appear essential to reach consensus on a definition of what constitutes translation competence.

A Framework For Evaluation

This paper focuses on the need for teachers of translation in the academic (university) environment to develop a framework for the evaluation of the translated text. Such a framework could combine the need to assess the development of language skills, which may seem to require a linguistic approach to the teaching of translation, with a wider awareness of aspects of text linguistic and functional approaches, drawing also on concepts proposed by scholars such as Snell-Hornby (1988), who advocates an integrated approach to translation, and Baker (1992, 1996), who calls for an


interdisciplinary approach. Both of these scholars remind us of the need to draw on concepts from other areas of research when attempting to create an appropriate environment for the exploration, with students, of questions arising from particular problems of translation in the translation seminar. The word exploration is used rather than explanation: this is fundamental to the approach suggested here as being more productive and constructive than traditional approaches, which are often explanatory, tutor-oriented, prescriptive and wordbound. Suggestions offered are based on experience in teaching translation competence through the use of non-literary texts, but could be equally useful in the development of a framework for the translation of literary texts. The framework could be of relevance as part of a specialist translation programme, or in the context of developing communicative skills in a general undergraduate language degree programme. Development of L2 acquisition and competence should be a natural consequence of the implementation of this approach, although not its primary objective.

It is our belief that if the task of the translation teacher/ trainer is to enable students to develop greater levels of competence in the process of interlingual communication, acquisition of the different skills involved in this has to be separated from systematic and specific testing of L2 language competence at the level of syntax or lexis alone; also, indeed, from revision of L 1 competence in general. Translation Studies scholars, such as Neubert and Shreve (1992), have made major contributions to a shift from the focus on the individual sign as the unit of translation to an understanding of how these micro-signs are part of a global message, conveyed by a text, as a complete unit of meaning. Nida (1969) taught us to be aware of the importance of the socio-cultural environment and the world view of the target culture as a determining factor in the process of selecting appropriate target language (TL) units to convey a particular message. Toury (1980, 1995) stressed the need to produce a target text which will function as a fact of the target culture, with reference to norms and conventions for style and language use. Reiss and Vermeer (1984)

highlighted the importance of the prospective function of the translated text within the target culture in determining the appropriate target language signs and forms which will enable the message to fulfil its purpose for the target reader; also the need to know whether this purpose is the same as that of the source text or different, as this too will affect TL choices. The work of all of these scholars is integral to developing an awareness of how to exercise translation competence in relation to specific tasks.

BEVERLY ADAB 219

Methodology

The student of translation in the undergraduate academic environment has (usually) not yet reached a stage of sufficient L2 socio-cultural experience and language competence to be able to make decisions without some discussion of comparative and contrastive linguistic and stylistic use. Consideration of appropriacy in relation to these is very useful in the progression towards what. Robinson (1995) describes as the ability to operate on autopilot for most of the time when translating, pausing only for conscious reflection and reasoning where a previously unencountered or infrequent translation problem arises in a specific situation and context, for a given text. An appropriate analogy to give students is that of learning to drive: you have to be able to undertake various independent procedures (such as changing gear, controlling speed, looking in the mirror), without losing sight of the overall purpose towards which these processes all contribute - namely, being a safe driver. Postgraduate students. will have progressed further in this respect but will still need to reflect consciously on their language competence as part of the wider picture of translation competence. Holz-Manttari argues, in her theory of translatorial action (1984), that the expertise of the translator is imperative in determining successful TT production, within the constraints imposed by the translation situation, the translation commissioner and the intended purpose of the target text in relation to its addressees. Trainee translators would benefit if they could learn to work within this perspective.

The learner driver needs a broad vision of the overall purpose and function of the activity; also necessary is an understanding of the sub-processes or actions which will allow the learner to fulfil this purpose, individually and in combination one with the other. Perhaps one significant difference between translation and driving is that almost everyone who drives can probably describe what the controls of the car do (very specific things, like the indicator, the brake or the accelerator). Few have an extensive understanding of the how and why - of what happens under the bonnet. In translation, students not only, need to understand the broader parameters of text production; they also need to be able to lift the bonnet and explain how each individual unit interacts with , others in the text and in the context of the broader parameters of text purpose, type and structure. To continue the comparison, they need to be qualified diagnostic mechanics, not just safe or competent drivers. The method suggested for evaluation of the translated text can assist in improving understanding of the how and why, by inviting an analytical description of the sub-processes, in the context of how Nord (1997) describes translating, namely as a purposeful activity.


This is a complex and demanding goal to set for students, one which is difficult to accomplish unless students are introduced, at the start of their university degree programme, to basic theoretical concepts of translation and shown how these relate to good practice by informing decision-making, assisting in the evaluation of choices and the justification of these. It is only when students are familiar with clearly established criteria, to which they can refer when producing their own translations, that they can move towards a more constructively useful and objectifiable form of translation assessment. This is not to deny the subjective element of the perception of the assessor, based on his or her reading of the ST message, his or her TL idiolect and expectations of the level of competence of the student, all

form an integral part of the assessment equation. Students can, however, be shown that both assessor and assessee are working within the same framework, using the same criteria for evaluation and within similar broad lines of idiolect and within a similar ideolect (Robinson 1995).

Evaluation: Defining Criteria

Delisle (1998) argues that this objective, making students more aware of the how and why, cannot be achieved unless students are taught the relevant metalanguage with which to describe translation problems and potential solutions. This metalanguage should be developed on the basis of the criteria for text production which have been presented to students. It needs to be available to students so that they can account for the nature or type of translation problem, in broader terms such as Nord's pragmatic, intercultural, interlinguistic or text-specific problems. However, they should also be able, in the exploration of translation problems and possible solutions to these, to explain and describe the precise nature of the problem within a specific text and context, in relation to the translation brief for that text, so that they can justify each Target Language choice.

The considerations outlined need to form part of the teaching process, which is really a training process in how to structure and communicate messages across languages and cultures. Students also need to ensure that they are basing their selection of Target Language choices on a fairly reliable and verifiable interpretation of the Source Text message. This is why it is essential to help students to develop a critical framework for initial, pre-translation analysis of the ST; this framework for systematic ST analysis should eventually become an almost sub-conscious process. During the translation

BEVERLY ADAB 221

class it needs to be worked through, reasoned out loud, discussed, argued and defended, as a form of practice and training.

This is also why the Translation-Oriented Source Text Analysis advocated by Nord (1991) is an essential tool in the training process, as this gives students a framework for the systematic analysis of their interpretation of the ST and can guide students in TT production. Whether or not the intended TT function is similar to that of the ST, identification of the text type and awareness of how its communicative effect is achieved can alert students to the appropriate text type, stylistic conventions and linguistic devices which will produce a similar or corresponding effect on the target language addressee. This is also the point at which appropriate metalanguage can be acquired, in response to the need to describe and explain the ST prior to the next stages in the process.

At subsequent stages, the knowledge of an appropriate metalanguage will further assist the student in explaining and justifying translation strategies and decisions, by reference to the pre-determined set of evaluation criteria. These criteria should be both generalisable in their wider application and able to be applied specifically to the translation situation in hand. Criteria should relate to considerations such as intended target text purpose, text user's needs, situation of use, text typological conventions and intrasystemic language constraints. Clear parameters for these, which can be applied to all text/ message types, can be found in the work of translation scholars such as Neubert, (1992) (textual features and text production); Nord, (1997) (text analysis, translation problems and strategies); Chesterman, (1997) (norms, conventions, strategies, decision-making), whilst for more language-related considerations, Vinay and Darbelnet (1977) remain a solidly reliable source of reference. The fundamental problem, for the translation teacher, is to select as sources of reference those scholars whose work offers insights which are relevant to the overall aims of the course, so that students can apply these criteria as general principles for self-evaluation during the decision-making process, as much as post facto to the product.

If a student tackles every text for translation with a clearly defined criteria for interpretation of the message; if s/he also adopts the same framework for evaluation and selection of apparently appropriate solutions for a given situation and context; then any evaluation of the student translation

competence can adopt such a framework and can be seen to be applied as fairly and consistently as possible, within the constraints of the relative subjectivity of the assessor (cf. Adab 1998). One way of undertaking this kind of assessment is to set a task which involves not only the act of translation but also the explanation, in the form of annotations, of how and why decisions have been reached, based on which criteria. An introduction to the text, in the form of a


translation-oriented analysis of text type, function, apparent communicative intention, key themes, stylistic and linguistic devices, can explain the student's reading of the ST, its type and purpose. It may also explain, for the assessor, the source of any misunderstandings of the ST or inappropriate TL choices. Evaluation of the target text would then be conducted within the parameters set by the translator, in order to identify problems relating to translation competence. Potential translation problems should be defined as those units of meaning for which a solution, in terms of interligna transfer, required more active consideration, evaluation of choices and selection of the most appropriate TL unit. Problems can be categorised according to problem-types, based on whatever categories best fit the purpose of the assessment; they could be taken from a single approach, such as linguistic or text linguistic, or they could combine features of many different approaches, requiring a complex process of synthesis and understanding of these different approaches on the part of the student. The written discussion and explanation of individual problems invites a process of reflection on practice, which is immediately linked to decision-making and which will, it is hoped, become internalised as an integral element of the final revision process. The need to structure this reflection in a logical and reasoned manner will assist in the training process.

Evaluation: A Practical Example

Over the years this approach has been applied to the evaluation of translation choices for a Final Year course in Professional Translation, and a modified version of this approach has also worked successfully in general written language classes for Final Year students. It was only recently that more specialist translation courses have been introduced, at Aston, for First and Second Year students, within the new Modem Languages and Translation Studies undergraduate degree programme. In these courses, translation briefs and tasks are always related to real-life situations.

This approach will be illustrated by reference to an example of the kind of examination text given to students on the Final Year elective course, entitled "Professional Translation, Theory and Practice", in which students work from French (usually L2 or L3) into their mother-tongue (English is usually but not always mother-tongue - however, for this course it is assumed that this is the case). The course aims to simulate some conditions of the professional environment and to emphasise the importance of the role of Information Technology in overall translation competence (in the form of the computer as a tool and the various resources to which it offers access). For this reason, both

BEVERLY ADAB 223

the course and the final examination are taken in the computer laboratory; using a word processor, with split screens to display source and target texts; with access to all forms of electronic dictionary (mono-lingual and bi-lingual) and other internet sources, for consultation of parallel texts in each language. Additional specialist dictionaries are provided in printed form where necessary, for consultation during the examination. The examination lasts for three hours. In order to reflect the importance of translation competence (knowing how and why) as described above (ST comprehension, identification of potential problems for translation, comparative and contrastive selection of translation solutions, revision of the TT in relation to the translation brief), the weighting of the marks is 60% for the translation, i.e. for the target text and 40% for the annotations. Students receive a printed version of the text but this is also available via the departmental network for on-

screen reference and comparison. Using word-processing software allows greater flexibility of revision, speed of production and the chance to store different alternatives and review these within the overall text before making final decisions.

The text given to students is usually a complete text, or a complete section of a slightly longer text, in which case the rest of the text is given for the purpose of contextualisation. Highly domain-specific texts with specialist terminology are avoided, as are those which could be expected to require extensive lexical searches, as these could distract students from focusing on the processes of ST interpretation and of evaluation of TL choices. Students are required to read the whole text, to translate a section of about 400 words, then write a translation-oriented introduction to the text and annotations on translation problems within the section translated. Specific problems are identified for the students, to give a basis for comparison of performance. Nord's translation problem types will have formed the basis for weekly analysis of different texts, from identification of problems to discussion and justification of different possible solutions. In class discussion, as in their annotations, students are expected to draw on their understanding of criteria from the different approaches studied during the course, to describe and justify their choices of solution for the problems indicated.

However, indicating to students examples of translation problems is a somewhat artificial and subjective form of assessment, designed more to test the mechanism of annotation more than actual understanding of how to apply the criteria. To test them further and allow scope for the demonstration of an understanding of the processes involved, students are also required to select, from the section of the text which they have not translated, a fixed number of translation problems on which they have to write annotations. They are invited to select more than one example of a problem type and explain the specific


nature of each, to show an independent ability to apply this identification of problems as a prerequisite to selection of solutions.

In marking the work of each student attention is paid to different aspects of competence. The complete target text (TT) is read initially as a TL text, for coherence and overall acceptability / readability. A letter grade is allocated to the text as a measure of its usefulness as a text, for the addressees in relation to its intended function, without reference to the source text (ST). This is based on a university-wide scale of correspondence to a range of percentage points and to a pre-determined list of criteria which lists degrees of usefulness, useability and appropriateness for purpose. These criteria have been previously explained to students and correspond quite broadly to:

-

Grade A: highly appropriate for text function, accurate representation of message in acceptable language and style for the text type, very few/ almost no errors of language, style, message - reads as if written in the TL with no reference to an STGrade B: generally appropriate for text function, some minor errors of language/ style/ message but still usable with some corrections - generally reads as a TL text but with some indication of existence of ST as basis for TT productionGrade C: appropriate in parts but some significant errors - could only be used fully by someone who had access to ST and some knowledge of SLGrade D: mainly inappropriate, widely misleading, and unusable for purpose due to number of significant errorsGrade E: almost unacceptable, barely recognisable as a message, reads like an almost literal translation by a not very advanced language student, mainly inaccurateGrade F: unacceptable, unusable, not possible to make sense of any part of the message, difficult to understand TL message even with reference to ST; unacceptable level of errors

The text is read again, this time a very intensive and critical reading, with all errors being noted and underlined. Students are given a marking code at the beginning of the course; this is a list of abbreviations for different types of problem, ranging from: v- vocabulary, s- syntax, p-punctuation,

reg- register; to INT - intention, SIT- situational information, COH- cohesion and so on. Marks for any (perceived) inappropriate choices of translation solutions are deducted from a nominal starting point of one hundred; depending on the severity of the error in relation to the message or to specific criteria, deductions

BEVERLY ADAB 225

range up to three marks for a single error. In contrast, any particularly appropriate and/or pleasing choices (lexis, syntax, style, other aspects of text production) are rewarded by bonus points. The areas considered most closely, as indicative of competence in message transfer, include:

a. Language accuracy: at this stage in their university programme, students are expected to have acquired a near-native level of competence in the foreign language, so that errors of comprehension from the source language (SL) - (usually L2 but sometimes L3) - are penalised just as much as errors of production in the target language (TL) - (usually the mother-tongue). This is not to test language competence but ability to understand an SL message and write in an appropriate style in the TL. An error is anything which is inappropriate for the register and style, or simply inaccurate (syntax, spelling, punctuation, use of capital letters). Students are also expected to have a high level of comparative and contrastive awareness of differences and similarities (form/function) between the two language systems, in order to select the most appropriate TL language units

b. Accuracy of the message, in terms of subject knowlege, SL author intention and in relation to the intended target text function in culture for the specified addressees, situational and contextual knowledge in the SL culture. This is where students can demonstrate their knowledge of the source culture as much as their ability to understand a message written in a complex register and style in the SL. An error is anything which misrepresents or distorts the message in relation to the translation brief and with reference to the ST in so far as it is possible to recreate the SL message within the constraints of the translation brief

c. Assumed knowledge and needs of target reader in both SL and TL. Misrepresentations, additions or omissions are penalised according to the impact on the overall message and/or on individual details of the message.

d. Any intertextual references contained in the ST: these need to be interpreted, adapted, compensated for, made more explicit or implict, or simply omitted, depending again on the translation brief and on the TL addressees

e. Acceptability/readibility - appropriate use of language (register/ style); also included in this are textual aspects of coherence (logical information/conceptual structure) and cohesion - relevant TL mechanisms should be used wherever possible, including compensation for any text-specific effects which cannot be recreated from SL to TL


The marks thus deducted or awarded are calculated to give an overall total out of one hundred. Finally, the text is reread to see how well the initial assessment (letter grade) corresponds to the result produced by intensive marking (by matching grades to percentage scale: A= 70+; B = 60-69; C = 50-59; D = 45-49; E = 40-44; F=< 40. The plus and minus signs can be used to indicate where a grade falls within the range e.g B- = 60, B=64, B+ = 68. This is the point at which the evaluator has to exercise overall judgement, taking into account how all the criteria observed affect the usefulness of the text as produced, in order to attribute a final, overall grade and percentage within that scale. This mark is then scaled down to represent sixty percent of the total mark.

As for the annotations, a positive approach is adopted here. All and any relevant comments are ticked and given a mark, up to a nominal 100 marks. The total obtained is then scaled down to represent forty percent of the total mark. Since this part of the assessment is intended to allow students

to demonstrate knowledge it is not considered appropriate to penalise inaccurate reasoning, which is simply ignored. Conversely, sound argumentation and reasoning may be rewarded even where the actual outcome, in terms of a translation choice, is not considered to be the most appropriate choice, since this section of the assessment is concerned with ability to refer to translation theories to support decisions. Experience will eventually lead to a greater ability to make appropriate choices, provided the appropriate mechanisms for reasoned decision-making have been acquired.

If all of the stages of marking are clearly indicated on the individual student's paper, it should always be possible to point to justifiable reasons for a mark. Although of course there will always be an element of subjectivity of evaluation, this should be offset by the experience of the evaluator and by the comparability afforded by marking a set of papers at one time, which will allow for criterion-referencing within a (relatively) norm-referenced situation. Students also appreciate being rewarded for trying to apply theoretical knowledge and reasoning, even when the final choice is queried. In this way they can see the point at which their analysis may have broken down or become less reliable, as well as being given pointers as to why choices were not appropriate.

To be able to produce relevant and appropriate written annotations, students have to have understood the theoretical principles discussed during the course, also to have acquired and understood how to use the appropriate metalanguage, both of which areas of knowledge will become internalised through practice, enabling them to improve their performance in a systematic

BEVERLY ADAB 227 '

manner based on sound interpretation and judgement. The resulting discussion of problems is therefore an indication of the level and nature of the overall translation competence achieved by the individual student in the different areas of sub-competence addressed by the evaluation process, including inter alia: comprehension of a source language message; comparative and contrastive language and cultural knowledge; research competence; subject' knowledge; transfer strategies, choices, selection from alternatives; target language text production.

The writing of annotations can therefore serve as a diagnostic tool in determining those stages and skills in which a student still has problems. For annotations to serve this diagnostic purpose, the same approach needs to figure in any discussion of a text for translation, oral or written, during the translation' seminar; also to be formulated in a systematic and consistent manner, similar to the approach demanded by the actual assessment process. Regular discussion of this nature during the translation class trains students to adopt a reproducible ' approach to the performance of the translation task, including the application of theoretical principles, as well as helping to improve their awareness and overall ' competence through reflection and exposure to the insights of others. Critical evauation demands reasoned argument to justify decisions and can lead to a more self-aware and critical approach to text production.

Conclusion

Translation assessment in the university environment is a problematic issue which cannot be solved unless there are clearly defined objectives for a course. These need to be applied to evaluation of progress and competence acquired, through evaluation of individual products or translations. Students need to know what is expected of them in the translation class, which skills or subcompetences they are intended to develop through which kind of translation exercise and what knowledge is required (translation theory, subject specific, other domain-specific, language and culture) in order to develop these subcompetences. They need to understand how the various sub-competences relate to the overall objective, translation competence, through a contribution to the process and an effect on performance. Students need also to perceive the actual exercise of assessment to be not only authentic in its scope and nature, but also as objective and rationally verifiable as possible, given the nature of the process of translation and of the evaluation of the translated text as product of this process. If students are given a framework such as the one outlined above, they will see that translation can involve systematic analysis and verification,


just as much as it can be a creative process. The combination of these two skills will contribute to the development of the overall translation and communicative competence that are essential for fully competent translators.

References

Adab, B. 1998. "Evaluer les traductions en fonction de la finalite des textes". In J. Delisle and H. Lee-Jahnke (eds), L'Enseignement de la Traduction et la Traduction dans

l'Enseignement. Ottawa: Les Presses de l'Université d'Ottawa,127-133.

Baker, M. 1992. In Other Words. London: Routledge.Baker, M. 1996. "Linguistics and cultural studies". In A. Lauer et al (eds),

Ubersetzungswissenschaft in Umbruch: Festschrift für Wolfram Wilss. Tübingen: Narr, 9-19.

Chesterman, A. 1997. Mêmes of Translation. Amsterdam/Philadelphia: Benjamins. Delisle, J. 1998. "Le métalangage de l'enseignement de la traduction d'après les

manuels". In J. Delisle and H. Lee-Jahnke (eds), L'Enseignement de la

Traduction et la Traduction dans l'Enseignement. Ottawa: Les Presses del'Université d'Ottawa.

Holz-Mlinttari, J. 1984. Translatorisches Handeln. Theorie und Méthode. Helsinki:Suomalainen Tiedeakatemia.

Neubert, A. and Shreve, G. M. 1992. Translation as Text. London: Kent State University Press.

Nida, E. 1969. Towards a Science of Translating. Leiden: Brü.Nord, C. 1991. Text Analysis in Translation: Theory, Methodology and Didactic Application of a Model for

Translation. Amsterdam: Amsterdamer Publikationenzur Sprache und Literatur.

Nord, C. 1997. Translating as a Purposeful Activity: Functionalist Approaches

Explained. Manchester: St Jerome.Reiss, K. and Vermeer, H. 1984. Grundlegung einer allgemeinen Translationstheorie.

Tübingen: Niemeyer.Robinson, D. 1995. The Translator's Habit. London: Routledge.Snell-Hornby, M. 1988 Translation Studies: An Integrated Approach.

Amsterdam/Philadelphia: Benjamins.Toury, G. 1980. In Search of a Theory of Translation. Tel Aviv: The Porter Institute. Toury, G. 1995. Descriptive Translation Studies and Beyond. Amsterdam/

Philadelphia: Benjamins.Vinay, J-P. and Darbelnet, J. 1977. Stylistique Comparée de l Anglais et du Français.

Paris: Didier.

The Evaluation of Translation into a Foreign Language

GERARD MCALESTERTampere University

The Background

This article deals with the problem of translation evaluation from the point of view of the accreditation of

professional translators either within educational institutions or professional organizations. It is often claimed

that translation of a competent professional standard can only be done into the translator's language of

habitual use. However, in many countries where the first language of the population is not a major world

language, and where there is a concomitant lack of translators who have a major world language as their

language of habitual use, there exists, nevertheless, a powerful need for competent translation into such

languages (McAlester 1992), and it is desirable that such translation be done by translators who are properly

trained and qualified. The case for translation into a foreign language has been strongly presented in

Campbell (1998). However, the texts that such translators can be reasonably expected to handle

competently will normally be different in type from those that are assigned to translators working into their

mother tongue. It is unlikely, for example, that a translator would be professionally engaged in the

translation of a literary text into a language that was not his or her language of habitual use, except perhaps

as a collaborator or member of a team. Nor will the demands made with regard to the quality of the finished

product necessarily be as high as those expected of a nativespeaker translator. The requirement that the

finished product be perfect (whatever that may mean) is not always economically justified or necessary, and

even where it is, all that is required of a translator may well be a competent translation which can be passed

on to a nativespeaker editor for polishing. It would, therefore, be appropriate that both academic institutions

providing vocational courses leading to professional qualifications and accrediting bodies should reflect this

reality in their evaluation.

230 TRANSLATION INTO A FOREIGN LANGUAGE

The practice of accreditation varies according to the country. In some countries (e.g. Poland) it is by recommendation and approval by a professional body on the basis of work done, in others there is an examination (e.g. Britain, the USA), while in many countries a university training in the subject itself is considered to constitute a professional qualification. There are also various combinations of these methods. Particularly in cases where a university training is regarded as a sufficient qualification, it is of the utmost importance that it should be essentially vocational in nature and not just some offshoot of a modem languages programme with its main preoccupations in literature and/or linguistics. This fact has gradually been realised over recent decades with the setting up of dedicated translator and interpreter training courses in many countries. However, one essential requirement that such a training programme ought to fulfill is that it should be capable of evaluating the work of the students in a way that would have predictive value with regard to their potential professional competence. In other words, it would perhaps be. reasonable to regard the students who pass these courses as perhaps less than fully fledged professional translators and the degree they obtain from the university as an intermediary qualification.

The nature of university teaching permits evaluation of a student's work over a longer period, and it is not necessary to rely on a oneoff final examination, although many university systems also impose this. In contrast, the accreditation examinations set by professional bodies can reasonably test the actual professional competence of those who sit them, either by examination or by a review of professional work done, or by a combination of the two. In all cases, it is desirable that the methods used for the evaluation should be reliable, valid, objective and practical. To what extent all four, criteria can be satisfactorily met is debatable (see below). While it is people (translators) rather than products (their translations) that we essentially wish to evaluate, the condition of objectivity requires us to evaluate translators through their translations, despite Gouadec's assertion to the contrary (1989: 38): "L'évaluation de la traduction ne peut en aucun cas valoir l'évaluation du traducteur" (cf. Chesterman 1997: 137f. for a contrary view: "By their fruits, ye shall know them, surely".)

One could reasonably expect that the methods used by university departments and accrediting bodies in evaluating translation quality would show considerable agreement as to the most suitable procedures employed. One could also expect these to be defined in explicit terms; and to be based on the findings of a solid body of research on the subject. In actual fact, we find that methods vary considerably between one accrediting body and another, between one university and another, even

between different departments of the

GERARD McALESTER , 231

same university, indeed even between colleagues in the same department. The definitions of methods used are frequently inexplicit, and often the actual evaluation follows fairly rough guidelines based admittedly in the best cases on experience and common sense, but in the worst on mainly subjective impressions.

Translation Assessment Theory

Unfortunately, there is as yet no broad body of research providing a basis for suitable evaluation procedures, despite the amount of space that has been devoted to prescriptive translation theory (in which the, question of translation quality is always at least implicit and usually explicitly dealt with) as well as the plethora of criticisms of actual translations in existence. Even the concept of translation evaluation is unclear in the relevant literature, where one finds the. words (they are too inexplicitly defined to be called terms) evaluation, assessment, criticism and analysis used almost interchangeably. In this paper, I use the words in the following ways (except for quotations from other works). Translation evaluation is the placing of a value on a translation, i.e. awarding a mark, even if only a binary pass/fail one. It is this procedure in particular that should strive to fulfil the four conditions mentioned in the previous paragraph. Translation criticism consists in stating the appropriateness of a translation; this also implies a value judgement, which need not however be a quantified one, though it should perhaps be explicitly justified for it to be of any value. Translation analysis is taken to be a descriptive study of the process of creating a target text out of a source text (translation as production) or of the relationship between the target and source texts (translation as product) without ascribing a value judgement. Translation assessment I use as a cover term for the other three procedures. The verbs evaluate, criticise, etc. and the agent nouns evaluator, etc. are used analogously. Each of these procedures naturally has its place in the training of translators, and they are to some extent interdependent probably with a directionality obtaining between them: one might suppose that evaluation would be based on criticism and criticism based on analysis. A related though independent procedure is translation quality control, where the emphasis is on the assessment of the product or service per se rather than the producer.

There are two well known books devoted to the subject of translation assessment: House's Towards a Model of Translation Quality Assessment (1977), which has recently been revised and reissued as Translation Quality Assessment: A Model Revisited (1997), and Reiss's Móglichkeiten und Grenzen


der Übersetzungskritik (1971). The former is a classic in the field, but it has not led to significant developments in translation assessment, nor not to my knowledge has there been any extensive application of its suggested assessment procedures. Indeed, it is perhaps more often cited for its translation theoretical concepts of overt and covert translations and versions and types of error. There have been other notable contributions to the subject, in the German tradition from Koller (1979), Wilss (1982), Honig and Kussmaul (1984) and especially Nord (1991), and in Anglophone translation theory from Newmark (1988), Sager (1983: 121), Hewson and Martin (1991), and Williams (1989).

Following the example of House, the German tradition has tended towards an approach that seeks to be explicit and scientific, and as a result, despite some protestations to the contrary (e.g. Wilss 1982) and the overall functionalist orientation of these works, has tended to work on an analytical, atomistic or microtextual level. This, at least, is the impression gained from the actual examples of translation assessment cited in House (1977), Wilss (1982) and even Nord (1991).

Similarly, there is a strong emphasis on comparison between the source texts (ST) and target texts (TT) as the basis for translation assessment, to the extent that Koller, for example, states categorically: "... eine Ubersetzung nur im Vergleich mit dem Original analysiert und beurteilt werden kann" (Koller 1979: 206). House's model is also firmly based on STTT comparison and excludes translations between language pairs representing incompatible cultures, and those in which there is a change of function between the ST and the TT, which she dismisses as versions. Newmark (1982) similarly regards the latter type as an instance of restricted translation, and therefore outside the scope of translation theory proper. However, much of the activity that is a part of the work of many professional translators is thus excluded. I would prefer to regard as a basis for translator training and evaluation the reality of the market place rather than an a priori theory. A similar crucial assumption in most of these approaches is the fact that the ST is taken as the functional standard against which mismatches in the TT are regarded as evidence of inadequacy. Yet, as all professional translators know, many of the texts they get to translate are far from being functionally perfect, and mismatches may equally well indicate functional improvements. I naturally would not wish to argue the opposite case, either i.e. that the TT should be assessed independently of the ST for how else would it be possible to estimate the contribution of the translator in its production? And the evaluation of a text in terms of how well it fulfils its communicative purpose is notoriously difficult see Chesterman (1997: 128-133) on what he calls prospective assessment.

GERARD McALESTER ' 233

Nord's (1991) model provides a comprehensive functional account of assessment, and she makes Vermeer's concept of the skopos a central component of her method. Her approach is also less atomistic: she states "... it is the text as a whole whose functions and effects must be regarded as the crucial criteria for translation criticism" (Nord 1991: 166). She defines a translation error as "a deviation from the selected (or rather, prescribed) model of action" from the translator's standpoint, or "a frustration of expectations" concerning a certain action [...], as seen from the recipient's point of view" (Nord 1991: 170). She points out that assessment (in my terminology evaluation) is also a matter grading errors, and she suggests a hierarchy of' errors dependent on the text function, with extratextual (pragmatic and cultural) errors generally being given more weight than intratextual (linguistic) ones. Most marking schemes for translation evaluation suggest a similar weighting. The problem is how rigid may it be? Nord (1991: 172) herself says: "The overwhelming importance of the TT function sets the standard for establishing a hierarchy of errors". Does this not imply that what is a serious error in one translation may be a trivial one in another? The major defect of her approach (like that of House) is that it fails to satisfy the requirement of practicality for evaluation: in her book an assessment of English and German translations of an extract of 105 words from a Spanish novel takes up approximately eighteen pages. This kind of assessment may well be suited to translation criticism in an academic environment, but it is simply too laborious to be used by persons marking and evaluating large numbers of student translation assignments or exam papers. Another problem with Nord's approach from the point of view of practical evaluation is that it seems to equate adequacy with (near) perfection. She says of the five translations (some by established translators) that she assesses in her book: "[...] none of them meets the requirements set by text function and recipient orientation" (Nord 1991: 231). It might be more reasonable to claim that none of them fully meets these requirements. She does not indicate that there may be some level of adequacy that is acceptable without being complete.

The proposals put forward in the anglophone tradition have generally been less analytical and explicit, and some of them have hinted at the impossibility of arriving at any objective means of evaluating translations (Newmark 1982 and Sager 1983: 121). The approaches have also tended to be macrotextual, synthetic or holistic, and (with the strong exception of Newmark) less oriented towards a direct comparison of the TT with the ST. They have also underlined the position of the evaluator herself. For example, Newmark (1988: 188) stresses the importance of evaluating the translation first from the point of view of the translator's adopted strategy and only then on the suitability of this


strategy. Hewson and Martin (1991) also stress the stance of the critic and the importance of taking into account the role of the translation initiator. Although they do not offer any explicit procedures for evaluation, to judge from the examples they give, their approach is based on the functionality of the target text in relation to its purpose in the target culture and less on its correspondence with the source text. Similarly, the approach of Juan Sager (1983) is firmly rooted in the real world of business and administration translation. He states:

In many cases, a translation has to be assessed [...] in terms of the adequacy of a text for its intended purpose and the cost effectiveness of the method of production. (Sager 1983: 122, my emphasis)

Sager lists factors which constitute the parameters affecting evaluation. These include several `real life' circumstances which are often ignored elsewhere, such as the awareness of the reader that he is dealing with a translation, the use that is to be made of the translation and the revision factor (see below). He then goes on to outline the criteria for the evaluation of a translation as a product.

Most of these approaches for translation evaluation depend heavily on the concept of error. The problem is that in no case is any suggestion made concerning the amount and gravity of errors that can be tolerated for the total translation to be considered adequate. Indeed, it may well be that this is in practice and even in principle impossible. However, it is exactly what we are required to estimate in evaluating the competence of students or examination candidates for a qualification as professional translators. The problem of quantifiable evaluation is addressed by Malcolm Williams (1989), in an article dealing with translation quality control in the Canadian government's Translation Bureau. In it he emphasises the need for a system of evaluation that is reliable (consistent) and valid (predictive of the general level of quality). However, he distinguishes between, on the one hand, an evaluation system for quality control in the workplace, which he says should take into account the circumstances of production including the customer's specific requirements concerning timeliness, language quality and accuracy, etc., and, on the other, a closed evaluation system as used by universities and other accrediting bodies, which ignores such external factors and views the translation in vitro, evaluating it purely on the number and gravity of errors. This, I think, is an unnecessary distinction. There is no reason why the circumstances of production cannot be replicated in a reasonably realistic way at least in universities, and even to some extent in the examinations of other accrediting

GERARD McALESTER 233

bodies, though here logistic problems relating to the organisation of masa examinations can create severe restrictions.

Evaluation in Accreditation

Most of the national accreditation bodies that I have contacted (in Britain, USA, Sweden and Finland) use methods based on a summation of' points deducted for errors, with certain kinds of errors being more heavily penalised than others, and what appears to be an arbitrarily fixed maximum of minus points for a pass. In my own experience, the mere summation of errors in a translation has often not corresponded with my subjective evaluation of' it, Which then is right? Most evaluators, I think, would plump for their own intuitive assessment. If, on the other hand, the summation of points generally agrees with the subjective evaluation, one can ask whether the increased work it entails is justified? The evaluation of the exams conducted by the Institute of Linguists in Britain is more holistic in its approach, in that it awards points fol various aspects of the whole text, such as comprehension of the ST; accuracy and appropriateness of rendering (lexis and register) in the general language; cohesion, coherence and organisation; grammar, punctuation, spelling, etc, Candidates are also required to write a

commentary on translation problems in the general paper.In the evaluation systems of these bodies, errors which lead to misunderstanding are punished more

severely than other kinds. Whether this is a result of the fact that almost invariably the texts for translation belong to types that require a faithful translation strategy, or whether the selection of texts is made in order to enable this kind of weighting to operate effectively is unclear. However, professional translators working into a foreign language do frequently find themselves dealing with texts which are mainly or extensively expressive, operative or phatic, and in which form may play an equally or even more significant role than the transfer of information. Banal examples are the tourist brochure or much business correspondence. For example, it is at least equally important that a letter apologising for inability to meet a delivery deadline and requesting an extension be couched in the right language as it is that the actual number of man hours lost through a strike at the plant be rendered accurately as part of the excuse for delay. It is important that the texts by which we evaluate future or practising translators reflect the reality of their work. The other type of mistake that is heavily penalised is the gross misuse of the target language. The problem here is how to define gross. This kind of weighting roughly corresponds to that suggested by Nord.


These evaluation systems generally make no distinction between translation into the mother tongue and translation into a foreign language. The level required of the latter should, in my opinion, be that which is designated as revisable in the Sical III translation evaluation system described by Williams (1989). Many of the errors that are made by translators working into a foreign language are of the kind that Pym (1992) classifies as binary, i.e. gross, particularly on the linguistic level. On the other hand, they are often easy to correct by a reviser and do not necessarily involve misinformation. Another problem with categorising errors in translation into a foreign language is that what may result in an informative mistranslation is in fact a result of deficient command of the TL. For example, in Finnish the verb epailla is normally translated by the English verb doubt, often correctly. However, sometimes a better translation would be suspect. Thus a Finnish translator who renders a politician's statement: "I suspect that we shall see a considerable fall in unemployment over the coming year" as "I doubt that we shall see ..." is guilty of producing a fairly grave informative mistranslation (inversion of meaning), and yet it is the result of a simple lexical error. The question is, how is such a mistake to be categorised? By its cause or its effect? In either case it is, of course, gross.

Positive Evaluation

Nord (1991) outlines a possible method whereby the transfer competence of a student might be stated in the form of a percentage calculated by the number of adequate solutions to previously identified problems. Similar suggestions are made by Hatim and Mason (1997: 208-209) for criterion referenced evaluation. While it might be more encouraging for students to learn from their successes than their failures, in practice the system seems difficult to apply particularly at the level of linguistic problems when the translation takes place into a nonmother tongue - it is often difficult to predict in advance just what will prove to be a problem and what not. What happens where no problem has been identified, but the translator still makes a mistake? Kussmaul (1995) and Gouadec (1989) also have suggestions for positive evaluation, but neither is explicit enough to provide an applicable model. Usually, positive evaluation turns out to be merely the inverse of negative evaluation, and indeed one could paraphrase Gouadec's proposal as "What you don't get wrong you get right."

GERARD McALESTER 237 Editing Time as a

Criterion

The atomistic proposals of translation theorists outlined above are, in my, experience, generally not applicable as such, either because they basically rely on a TT-ST comparison, which is not relevant to all translation tasks, and thus °: they fail the criterion of validity, or then they are too cumbersome and time-, consuming to apply (House and Nord) and therefore do not satisfy the criterion of practicality. On the other hand, the more holistic approaches mentioned above tend to be too inexplicit or to lack objectivity. One method that falls into the holistic category and that has been suggested by Gouadec (1989) and McAlester (forthcoming) is the use of editing time as a criterion for evaluation. It is particularly suitable for the evaluation of translation carried out into a nonmother mother tongue. By placing herself in the position of a language reviser and noting the time it

takes her to correct the translation into a form suitable for its

purpose, the evaluator can arrive at a figure which, at least relatively within a, population (i.e. it is norm referenced), reflects the value of the translation as a, product (see also Gouadec 1989: 42-43). This satisfies the criterion of validity, reflecting the fact that many of the translation tasks into a non-mother tongue' pass through a native speaker language editor. And the market value of' a , translation into a foreign language in the real world is to some extent relative to the revision time. The method is also objective and quantifiable (in terms of time).

On the other hand, the criteria of reliability and practicality are less easily satisfied. For it to be reliable it should be applicable irrespective of who the evaluator is and under what conditions a text is evaluated, but the method depends on the editing speed of the individual evaluator. In this, of course, it is no different from many other marking schemes that are dependent on the performance of the marker. It may also (without adequate guidelines) be dependent on the idiosyncratic linguistic and translatorial preferences of the evaluator, but again many marking schemes are fundamentally subject to the subjective perceptions of the evaluators. Therefore, although it is not• completely satisfactory from the point of view of reliability, it is no worse in this respect than many other existing evaluation procedures.

What about practicality? In that it permits large numbers of translation texts to be evaluated in a reasonable amount of time, it is more practical than most of the approaches suggested above. There are, however, problems connected with its implementation: ensuring that the evaluator is sufficiently ,

free from distractions during her evaluating to enable her to record her editing times, or the tendency of the times taken by the evaluator editor to shorten as she goes through more translations of the same text. These problems can be


avoided if the evaluator ensures that she is isolated for the time she spends evaluating, and if she familiarises herself thoroughly with the ST and reads through a few translations before she starts correcting.

However, there are some more serious objections that can be made: the method does not distinguish between types or gravity of error. If the evaluator takes basic acceptability as her criterion (and not optimum quality), she will simply edit what needs to be edited. She will normally be familiar with the ST and the skopos of the translation (often she will have selected the text and the task herself, or at least been involved in its selection). She will make only such changes (corrections) to the TT as will enable it to function adequately for its purpose. Thus, translations that are not so much wrong as infelicitous (nonbinary in Pym's terminology) can simply be edited if necessary. This may then allow us to pass a brilliant but careless translation that would otherwise fail for a couple of errors affecting communication (the time taken to correct them may well be less than that taken on a translation that includes numerous small errors). Here, I recall the fact that, at least in an academic environment, it is the translator and his potential that we are ultimately trying to predict, and it may be easier for a budding translator to make himself more careful than to improve his basic language or transfer abilities.

The method is not analytic, and so it yields no information about the kinds of mistakes made or their causes. Its pedagogical value is, therefore low, in that it does not enable students to learn from their errors or provide material for research. But that is not its purpose. Its purpose is to place a value on a product, hopefully in a way that will be indicative of the potential of the producer. Crucially, however, the proposed method can only solve the problem of adequacy in relative terms. It offers no absolute criterion of adequacy: How much time can a translation take to correct and still be acceptable? Practical criteria such as the amount of time that a reviser would need to do the translation herself (if the revision of a translation exceeds this, it can be considered inadequate) are not reliable as they involve unstable variables such as the revision and translation working speeds of the individual reviser.

The Need for a Standard of Translation Adequacy

In this respect, then, the problem of what constitutes an adequate translation remains unsolved both in theory and in practice, where for example the directions given by various accrediting bodies to their evaluators appear arbitrary and conflicting. Some accept one gross error per translated page,

GERARD McALESTER • 239

some more, and some none. I suspect that the problem is in principle insoluble in a way that is scientifically justifiable.

However, what we might expect (and certainly sorely need) would be some more generally applicable guidelines for determining translation adequacy, i.e. a generally agreed consensus on basic standards for professional accreditation, which could also be applied in academic institutions providing translator training. After all, there are many areas of life where people are obliged to give a quantified evaluation of a product or performance. A typical example is the judge who awards points to competitors for their performances in sports like diving or figure skating, where the judge has fairly precise guidelines to use (e.g. so and so many triple jumps to be successfully completed). Nearer home, schemes have been evolved for the criterion referenced evaluation of levels of competence in foreign language learning, for example, in the Common Framework proposal of the Council of Cultural Co-operation of the Council of Europe (1996).

Evolving a similar criterion referenced evaluation framework for translation is certainly a daunting task. It should, however, be regarded as impossible to devise reliable, objective, valid and practical guidelines for the evaluation of translations. First of all a framework for assessing translation adequacy would mean setting criteria for the kinds of texts that were selected to be translated for evaluation purposes. Nord (1991: 160-161) presents a strong criticism of the kind of texts that are set for translation examinations in an academic environment. It would then be necessary to evolve a system for grading them not only on the basis of the language and special translation problems that they present but also according to the type of transfer they require (overt, covert, version, in House's terms).

The language problems could be also graded in terms of the degree of adaptation they require in transfer: grade 1 for literal translation; grade 2 for minor rephrasing; grade 3 for major rephrasing; and so on. The translation problems could be of the kind that many books on translation (e.g. Newmark1988: 193-220) draw attention to: proper names, dates, culture-bound allusions, proverbs, etc. Each

type could be awarded a weighting according to whether it required a standard solution (e.g. imperial to decimal measures) - grade 1; a solution involving research (e.g. finding the exact form of a quotation fromShakespeare or the Bible) - grade 2; or a solution requiring creativity on the part of the translator (e.g. a

suitable TT coinage for a neologism in the ST) -grade 3. The number of grading levels has been set at three merely by way ofexample, and naturally there could easily be more. Each text could then begraded according to the number and level of language and translation problemsit contains. Translations could then be evaluated on a fixed scale of marks


according to how well they perform in solving these problems, and levels of adequacy set by common (preferably international) agreement.

It would still be possible to have a condition that an adequate translation should not contain more than so and so many mistranslations or omissions or gross TL errors per a fixed number of words or pages, provided again that the ratios are generally agreed. Translation tasks could also be categorised according to whether they require faithful (overt) or communicative (covert) translation, and the solutions evaluated in this light. Here it would also be a very useful procedure to allow, or even require, those who submit translations for evaluation to append translator's commentaries. These would enable the evaluator to assess both whether the translator has achieved his aims and whether those aims were appropriate in the first place. In short, what is called for is a set of international standards of translation adequacy. This could well be done under the auspices of an international organisation such as FIT (Fédération Internationale de Traducteurs) and transmitted down through national affiliates. A profession should be able to define its own standards of competence. It would be a suitable project for the new millennium.

Notes

I In the interests of equality, I use she for the evaluator and he for the translator

References

Campbell, S. 1998. Translation into the Second Language. London: Longmont Chesterman, A. 1997. Memes of Translation. Amsterdam/Philadephia: Benjamins. Common European Framework of Reference for Modem Languages: Learning,

teaching assessment, 1996. Draft 2 of a Framework proposal. Strasbourg:Council of Europe, Council of Cultural Cooperation.

Gouadec, D. 1989. "Comprendre, évaluer, prévenir". TTR 2(2): 35-54.Hatim, B. and Mason, I. 1997. The Translator as Communicator. London and New York: Routledge.Hewson, L. and Martin, J. 1991. Redefining Translation. London and New York: Routledge.Honig, H. G. and Kussmaul, P. 1984. Strategie der Übersetzung. Tübingen: Narr House J. 1977. A Model for Translation Quality Assessment. Tübingen: Narr.House, J. 1997. Translation Quality Assessment: A Model Revisited. Tübingen: Narr.

GERARD McALESTER 241

Koller, W. 1979. Einführung in die Übersetzungswissenschafl. Heidelberg: Quelle andMeyer.

Kussmaul, P. 1995. Training the Translator. Amsterdam/Philadephia: Benjamins McAlester, G. 1992. "Teaching translation into a foreign language - status, scope and

aims". In C. Dollerup and A. Loddegaard (eds), Teaching Translation anaInterpreting. Amsterdam/Philadelphia: Benjamins, 291-297.

McAlester, G. forthcoming. "The time factor: A practical evaluation criterion". In M.Grosman, M. Kadric, I. Kovacic and M. Snell-Hornby (eds), Translation intoNon-Mother Tongues - In Professional Practice and Training. Tübingen'Stauffenburg.

Newmark, P. 1982. Approaches to Translation. Oxford: Pergamon Press. Newmark, P. 1988. A Textbook of Translation. London: Prentice Hall Nord, C. 1991. Text Analysis in Translation. Amsterdam: Rodopi.Pym, A. 1992. "Translation error analysis and the interface with language teaching"

In C. Dollerup and A. Loddegaard (eds), Teaching Translation and InterpretingAmsterdam/Philadelphia: Benjamins, 277-288.

Reiss, K. 1971. Moglichkeiten und Grenzen der Übersetzungskritik. Munich: Hueber. Sager, J. 1983. "Quality

and standards: the evaluation of translations". In C. Picker(ed.), The Translator's Handbook. London: ASLIB, 121-128.

Williams, M. 1989. "The assessment of professional translation quality: Creatingcredibility out of chaos". TTR 2(2): 13-33.

Wilss, W. 1982. The Science of Translation. Tübingen: Narr.

text selection

Documents