MT and Post-Editing in Master's Level Translation Education

Constantine the Philosopher University (UKF) Nitra, SlovakiaDaša Munková, Jakub Absolon

User conference MT@Work 2015 (4 December — Brussels)

MT a PEMT in Current System of Translation Studies

Goals:• Make know principals of

MT• Basic of Post-Editing of

Machine Translation• Errors typology, most

frequent errors for EN-SK made by MT engines

• PEMT practice is a part of internship program

Courses with content connected with PEMT

• Identify differences in job description of Translator vs. Post-Editor• Trainability of PE skills and

competences• Necessary workload for PEMT

training• Recommended Methodology for

PEMT (e.g. split techniques)Post-Editor

Interpreter

Translator

Literary Translator

Challenges and Questions

Pilot Study Goals

• Find out the attitude of future translators to PEMT process (before and after pilot study)• Find out increase/decrease of productivity

comparing PEMT with HT• Find out if the productivity increase will be

the same for fast and slower translators (correlation between HT and PEMT)• Find out if there will be significant increase

of productivity after short exercises.

MT, format and text characteristics• We use Google Translate API integrated into Memsource CAT• We use exported tables for revision in word format• We use three types of texts:

HT and PEMT AMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 55.7Average Grade Level: 10.6

EVALUATIONMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 12.7Average Grade Level: 20.3

PEMT BMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 54.5Average Grade Level: 11.3

Activities in Pilot Study

PEMT basics MT Quality and usability evaluation (IT text)

HT(IT text)

MT Quality and usability evaluation (adhesives leaflet)

PEMT(IT text)

HT(IT text)

PEMT(human rights text)

PEMT(IT text)

MT Quality and usability evaluation (HR text)

PEMT(human rights text)

1.

2.3.4.5.

6.

7.8.9.

10.

Who were our respondents?

39 first year MA university students of Translation Studies in UKF (language pairs EN-SK)

Do you have any experience with commercial translation?

• No I do not translate (only at school) - 79.4% • I am translating at least 5 pages per

week - 20.6 %

Have you ever metwith demand for PEMT?

• I've never heard of PEMT 17.6%• Personally, no, but I heard about

PEMT 52.9%• Yes, but I have no practical

experience 26.5%• I have practical experience with

PEMT 2.9%

Which machine translation systems you've used?

NO

OTHER

100 %

32,4 %

8,8 %

How Do You Use MT?

• I translate unknown words 55.9%• I translate a whole sentence and then

edit them 32.4%• I only use MT for unknown languages

11.8%• I do not use it at all 0%

What Do You Think About the Quality of MT?• It is a high quality translation 0%• MT quality varies, but is a great help

55.9%• MT helps, but it must not be

automatically copied to target 35.3%• MT slows down the process of

translation 8.8%• MT is absolutely unacceptable 0%

Translation Studies students attitude toward PEMT before and after pilot study• There was significant

improvement in students attitude to PEMT although initial level was very high in comparison to average translator

before after

Switch on MT

Switch on MT

Switch-off MT Switch-off

MT

Find out increase/decrease of productivity comparing PEMT with HT

0.0% 50.0% 100.0% 150.0% 200.0% 250.0% 300.0%0

0.2

0.4

0.6

0.8

1

1.2

1.4

PEMT vs. HT acceleration

Possible reason:• Using MT is very natural for young people• Well selected text for MT• Students were not experienced translators

Find out if the productivity increase will be the same for fast and slower translators (correlation between HT and PEMT)

We found no linear dependence between • speed of HT speed and speed of PEMT• HT quality and speed of PEMT

Possible reasons:• There are other KPIs (further research needed)

higher quality lower quality

Find out if there will be significant increase of productivity after short exercises

We found no significant differencePossible reasons:• Not enough amount of

exercises (further research needed)

Added value of Study for students• Familiarizing students with the theory and current practice in the

PEMT• We gave them the opportunity of direct comparison and HT PEMT• We familiarize them with the system of human assessment PEMT• We taught them to identify major MT errors, to be able to react more

quickly on them.• We gave them possibility to measure their potential in PEMT, to be

able to decide if the PEMT will be one of their future professional activity.

Future plans• Continue in PEMT activities• Enhance research focused on identification

of KPI for PE, creating specific competency based model of PE, which can be used for PE recruitment and selection• Identification of trainable skills and

competencies• Creation of methodology for PEMT training• Creation of error typology specific for MT in

EN SK language combination

Our Goal:Carefully selected

and well trained Post-Editor

Arnold Turned His Biggest Weakness His Calves Into A

Showcase Muscle

Pilot Study Results

Based on our study we find out:• Rýchlosť prekladu (predpokladaná šikovnosť) rastie priamo úmerne

s rýchlosťou PEMT, nepotvrdil sa teda náš predpoklad, že najväčšie zrýchlenie dosiahnu tí najpomalší v preklade• Všetci testovaní dosiahli v PEMT vyššiu rýchlosť oproti HT• Priemerné zvýšenie rýchlosti oproti HT bolo 2,5 násobok čo predstavu

vyššiu hodnotu než sme predpokladali na základe výskumu (xxx)• Zvýšilo sa pozitívne vnímanie PEMT (z 91% na 99%, už na začiatku sa

prejavil mladý vek testovanej vzorky v pozitívnom postoji k MT)

Skóre:1 – 100 percente akceptovateľný preklad2 – akceptovateľný preklad (vyžaduje si menšie opravy , ale oplatí sa mať vetu predpreloženú)3 – čiastočne akceptovateľný preklad (vyžaduje si opravy, a neoplatí sa mať vetu predpreloženú)4 – neakceptovateľný preklad5 – neviem posúdiť Typológia chýb podľa DGT:slovosled - Wgramatické chyby - Gterminologické chyby- Tchyby vo význame - Mostatné – O