Third IATIS Regional Workshop – Western Balkans

Translator and Interpreter TrainingSeptember 26, 2014

Faculty of Philosophy, Novi Sad

A Corpus-based Analysis of Translator Errors Influenced by CAT Tool Design:

an experiment in back-translation

byAleksandar Kavgić

&Randall A. Major

1. INTRODUCTORY REMARKS

1. Introductory Remarks

• CAT Tools – Trados

• Parallel Corpus– LF Aligner

• CATMA– “Computer Aided Textual Markup and Analysis”

Stop!

• No one really talks like this.• Discourse should be:– a cohesive, interwoven set of connected

utterances– sentences should be an archipelago

interconnected with bridges built from anaphoric references and links.

Training College Students to Be Translators

• Undergraduate courses in translating various kinds of texts

• Master level course in Information Technology and Translating

• Question: why would students who produce (relatively) seamless translations on an undergraduate level, start producing choppy, disconnected ones in their IT&T course?

Could it be a problem of SOFTWARE?

2. BACKGROUND

2. Background

• SDL Trados - the industry standard software for producing coherent, terminologically consistent translations.

• Combines a textual framework with Translation Memory and Terminology Management

Product Development and Changes

• In 2011, Trados changed its long-running format from ST on top, TT on bottom to a side-by-side format.

Old Format (1)

Old Format (2)

New Format (1)

New Format (2)

CAT = tunnel vision?

Discussion (1)

• Some people disliked the change: It takes the focus off the Translation Unit being worked on, and it’s got a distracting amount of external information all around the work area.

• Some people liked the change: Finally you can see the larger context, both before and after, without scrolling around the document. And now you know at glance how much you still have to do to finish!

Discussion (2)

• Three of the great things about Trados:– Absolute terminological consistency– Repetitive translation is made easier (catalogues,

manuals, instruction booklets, diplomas, etc.)– Nothing gets skipped. Every TU must be translated

in some way. • That makes it good even for fiction!

Discussion (3)

• Since we were teaching them to use the old, Trados 7, is it possible that their choppy, disconnected FICTION translations were being caused by the TU focus?

• Experiment to find out!

3. METHODOLOGY

3.1. Corpus: general

Source text: Three unpublished short stories written in Serbian by our, at the time, undergraduate student (now MA track)

Thank you, Maja!

3.1. Corpus: general

Source text: Three unpublished short stories written in Serbian by our, at the time, undergraduate student (now MA track)

No copy-paste “translating” using online, previously

made translations was possible → authentic student translations.

LF Aligner -

Olifant: 3 column “TM”

3.1. Corpus: general

Number of TU (~sentences): 1924

Number of WORDS in SOURCE: 29116

Number of WORDS in TARGET: 32190

Number of WORDS in BACK TRANSLATION: 29057

3.1. Corpus: tagging scheme

Tag Set

Relevant source words/phrases

Translation Errors

Back translation Errors

3.1. Corpus: tagging scheme

TRANSLATION ERROS• Missing Element• Word Order/Focus Error:

unmarked > marked• Word Order/Focus Error:

marked > unmarked• Punctuation Error• Word Choice Error (word or

phrase)

BACK TRANSLATION ERRORS• Missing Element• Word Order/Focus Error:

unmarked > marked• Word Order/Focus Error:

marked > unmarked• Punctuation Error• Word Choice Error (word or

phrase)

+ source segment word/phrase

3.1. Corpus: tagging scheme

INTER-ANNOTATOR AGREEMENT

• Annotation was conducted by TWO ANNOTATORS:– Randy, a native speaker of (American) English– Saša (i.e. Aleksandar), a native speaker of Serbian– this enabled us to have a 360° view of the process

• Inter-annotator agreement was achieved by joint annotation of the first section of the corpus: 1 disagreement in 20 tags (~95%)

3.1. Corpus: tagging scheme

3.2. Corpus: tagged versionTAG RAW

FREQUENCY TE: Missing Element 154BTE: Missing Element 183TE: WO/FE: unmarked > marked 24BTE: WO/FE: unmarked > marked 72TE: WO/FE: marked > unmarked 38BTE: WO/FE: marked > unmarked 72TE: Punctuation Error 19BTE: Punctuation Error 91TE: Word Choice (word or phrase) 558BTE: Word Choice (word or phrase) 635

Remember, the corpus contained 1924

translation units (TUs).

3.2. Corpus: tagged versionTAG FREQUENCY

(per 100 TUs)TE: Missing Element 8BTE: Missing Element 9.51TE: WO/FE: unmarked > marked 1.25BTE: WO/FE: unmarked > marked 3.74TE: WO/FE: marked > unmarked 1.98BTE: WO/FE: marked > unmarked 3.74TE: Punctuation Error 0.99BTE: Punctuation Error 4.73TE: Word Choice (word or phrase) 29BTE: Word Choice (word or phrase) 33

4. FINDINGS

TAG FREQUENCY (per 100 TUs)

TE: Missing Element 8BTE: Missing Element 9.51TE: WO/FE: unmarked > marked 1.25BTE: WO/FE: unmarked > marked 3.74TE: WO/FE: marked > unmarked 1.98BTE: WO/FE: marked > unmarked 3.74TE: Punctuation Error 0.99BTE: Punctuation Error 4.73TE: Word Choice (word or phrase) 29BTE: Word Choice (word or phrase) 33

4.1. General trends: (un)markedness

Slight preference for unmarked, neutral word order?

Markedness in translation?

• In this research we defined markedness as:– Any deviation from the manner of presenting

information in a sentence which results in a change of the natural information focus in a language, or in amplification and/or down-toning of the naturally expected meaning

– This generally includes changes of word order, as well as omissions and additions in translation that can have this effect

4.1. General trends: unmarkedness

Konačno je ona velika količina besa prema društvu i Ijudima koju je osećala bila

dovedena na neki razuman nivo.

↓

Finally that great amount of anger that she felt towards the society and people was

brought down to a sensible level.

↓

Bes koji je osećala prema društvu i Ijudima je konačno sveden na podnošljiv nivo.

4.1. General trends: unmarkedness

Da nema poeziju, mislila je, ne bi imala smisla da živi.

↓

If she hadn’t had poetry, she thought, her life would make no sense.

↓

Mislila je da za nju život ne bi imao nikakvog smisla da joj nije bilo poezije.

Influence of CAT tools (tunnelling vision to 1 TU)?

4.1. General trends: markedness

Konobar je pogledao sa čuđenjem kada je sišla niz stepenice, prvi put primetivši koliko

je ona u stvari privlačna.

↓

When she was coming down the stairs the waiter looked at her amazed, noticing for

the first time how attractive she really was.

↓

Dok je silazila niz stepenice, konobar je pogleda u čudu, videći po prvi put koliko je

ustvari privlačna.

4.2. General trends: =equivalence

General consensus in literature on back translation:• back translation can never fully reflect the

original• 8% of back translations were exact

reproductions of the original, though• most are simple sentences, but a number of

them are complex

4.2. General trends: =equivalence

Bila je potpuno prazna.

↓

She was completely empty.

↓

Bila je potpuno prazna.

4.2. General trends: =equivalence

Irena se plašila da je drži u naručju.

↓

Irena was scared to hold her in her arms.

↓

Irena se plašila da je drži u naručju.

4.2. General trends: =equivalence

Upisao je elektrotehniku, jer je oduvek voleo prirodne pojave, fizičke procese i elektroniku.

↓

He enrolled on the Faculty of Electrical Engineering, because he always liked natural

occurrences, physical processes and electronics. ↓

Upisao je elektrotehniku, jer je oduvek voleo prirodne pojave, fizičke procese i elektroniku.

TAG FREQUENCY (per 100 TUs)

TE: Missing Element 8BTE: Missing Element 9.51TE: WO/FE: unmarked > marked 1.25BTE: WO/FE: unmarked > marked 3.74TE: WO/FE: marked > unmarked 1.98BTE: WO/FE: marked > unmarked 3.74TE: Punctuation Error 0.99BTE: Punctuation Error 4.73TE: Word Choice (word or phrase) 29BTE: Word Choice (word or phrase) 33

4.3. The old axiom is wrong?

Translators are, obviously, better attranslating into their non-mother tongue.

Carry-over effect?

or

The old axiom of translation is wrong?

Actually, our data supports claims made by Nike Pokorn…

4.3. The old axiom seems to be wrong

• English translations are, on the whole, good– not extremely good– nor excellent– but better than the back translation

• As Pokorn says: the quality of the translation, its fluency and acceptability in the target language environment depend primarily on the as yet undetermined individual abilities of the particular translator, his/her translation strategy and knowledge of the source and target cultures, and not on his/her mother tongue or the direction in which s/he is translating.

4.4. Punctuation (1)

• Run-on sentences• Sentence fragments• Interesting: Dialogue!

– Lack of knowledge of English conventions causes problems in the back-translation

“Reality is ruthless, the outside world crashes with no mercy.” I thought.

Stvarnost je nemilosrdna, spoljanji svet pada bez milosti. Mislila sam.

or“It must have been his brother’s bag.” I thought.-Mora da je to bila torba njegovog brata. Mislila sam.

4.4. Punctuation (2)

• Serbian style manuals allow two kinds of punctuation in dialogue:– „text‟– or a dash (-) at the beginning of a quotation.• [problem with the second type: where does the

quotation end?]

Here’s an example of what happens:

4.4. Punctuation (3)

-Šta bi bilo, moj Dragane. U ovom mestu se ništa ne dešava, a i ako se desi, svi odmah saznaju jer rekla-kazala funkcioniše brže od bilo koje vrste transfera informacija. (...) “What could be new, my Dragan?” Nothing happens in this place. However, if something happens, everyone gets to know it immediately. Hearsay functions faster than any other means of transferring information. (...) „Šta da ima novo, moj Dragane?“ Ništa se ovde ne dešava. A i da se desi, svi će odmah saznati. Ovde rekla-kazala radi brže od drugih sredstava obaveštavanja. (…)

4.4. Punctuation (4)

• This is a case where CONTEXT is really important for deciding when to end a quotation.

• This is a case where the old Trados SURELY contributes to errors in translation

4.5. General trends: word choice

The focus of this research was on issues in back translation caused by changes in word order, information organization, etc. possibly influenced by the use of Trados.• however some word choice errors may have

been caused by Trados, as well• focus on a single TU may cause errors in

deducing the correct meaning of polysemeous words

4.5. General trends: word choice

Smatrala je da su svi ljudi (=human beings) slepi.

↓

She considered all men (=human beings) blind. ↓

Smatrala je sve muškarce (=males) slepima.

4.5. General trends: word choice

Svako veče bi je slušali bez reči, kao omamljeni (=dazed) (by her poetry).

↓

Each night they would listen to her, not uttering a word as if they were drugged.

↓

Svake večeri bi je slušali, ne prozboreći ni reč, kao da su drogirani.

4.5. General trends: word choice

Imala je dvadeset i tri godine i imala veoma lepo lice i crnu kosu koja joj je nestašno padala u loknama.(=dark hair, cascading in unruly locks.)

↓

She was twenty-three years old and she had a very beautiful face and restless curly dark hair.

↓

Imala je 33 godine, prelepo lice i nemirnu, kovrdžavu, crnu kosu.

4.5. General trends: word choice

Upotpunjavali su se, uzdizali (=they uplifted one another).

↓

They were complementing each other – they were flying.

↓

Jedno drugo su upotpunjavali – letili su.

5. CONCLUSIONS

Conclusions - General

• The number of errors we looked at did not rise to our (wild?) expectation of 50% caused by forced TU focus. – closer to 30%

• However, our hypothesis proved to be at least partially true.

TE: 41.22 / BTE: 54.72 TAG FREQUENCY

(per 100 TUs)TE: Missing Element 8BTE: Missing Element 9.51TE: WO/FE: unmarked > marked 1.25BTE: WO/FE: unmarked > marked 3.74TE: WO/FE: marked > unmarked 1.98BTE: WO/FE: marked > unmarked 3.74TE: Punctuation Error 0.99BTE: Punctuation Error 4.73TE: Word Choice (word or phrase) 29BTE: Word Choice (word or phrase) 33

Conclusions - General

• In our further teaching of students at the department, in terms of TU focus, we need to INSIST on Newmark’s “every word in the ST must be accounted for, though not necessarily translated”.

(“no man left behind translating”)

Conclusion - Trends

• Our students tend to “neutralize” the ST in their TTs (back translation):– changing the markedness of the ST sentence– not paying careful attention to lexical choice of

polysemous words: choosing the most common meaning regardless of the context

– omit, misuse and/or misinterpret punctuation marks

– pay much more attention to translations into L2 than into L1

Conclusions• The observed trends may be:– due to Trados tunneling their vision to a single TU– due to something else

Further research is needed, witha different reasearch design.

Parallel group experimental design

Conclusions for translator training

Conclusions for translator training

Pay attention to

CONTEXT