· omegat in the omegat user manual it is stated the following: if you do not wish to send your...
TRANSCRIPT
Agenda
Introduction Predictions MT usage & the market impact
Translators’ profession Quality Productivity Compensation
TM/MT integration Small-scale experiment at KU Leuven
Portsmouth Conference 2014
About the presenter
Postgraduate EMT in Specialised Translation Research Assistant for the SCATE project at KU Leuven Experience with training translators on CAT tools No special expertise in MT
Portsmouth Conference 2014
Smart computer-aided translation environment (SCATE) in a nutshell
Time-frame: 01/03/2014 - 28/02/2018 Consortium: KU Leuven, 6 university teams, 10 companies Funding: Flemish government, industry, IWT Primary goal:
Develop techniques to improve translation efficiency and consistency!
• How? Better exploitation of resources, e.g. translation memories, terminology…
Syntax-based fuzzy matching Automatic bilingual term extraction from comparable corpora
Stronger integration of speech recognition models with machine translation
Personalised user interfaces
Portsmouth Conference 2014
In 6 years from now… We can expect that by 2020 automated translation tools will be embedded into every device, application and website.
TAUS
13 years ago…. A decent MT machine is just around the corner…A computer will never understand, but it can translate, at least to some extent. And, since translation without understanding is meaningless, the future of the human translator is proof- sensing what a machine has pre-translated. Champollion (2001)
About Machine Translation
Portsmouth Conference 2014
Present
Is the human translator proof-reading what a machine has pre-translated?
YES
Portsmouth Conference 2014
…in the professional market, human translation still rules, though a quiet but pronounced shift to machine translation with post editing is unquestionably underway in the agencies, some of whom are also actively developing their own engines using open source software.
LT-Innovate Report 2013
What about the LSP’s?
42% of the LSPs use MT
Source: CSA
Portsmouth Conference 2014
MT for Translation
Rule-based
• Linguistic rules • Specialised dictionaries per
domain, customer, product line! • Suitable for software
documentation, technical manuals, E-learning courseware, UI
• Consistent and predictable quality
• Out-of-domain translation quality • Lack of fluency • Hard to handle exceptions to
rules • It does not cover all languages! • Expensive to develop and
customize
Statistical
• Large bilingual corpora • Terminology choices hard to
manage! • Suitable for user-generated
content • Unpredictable translation quality • Poor out-of domain quality • Does not know grammar • High CPU and disk space
required • Good fluency • Good for catching expeptions to
rules • It covers all languages! • Rapid and cost-effective
development
Hybrid
• Rules +Statistics • Better output • Solves ambiguity • More expensive to manage two
systems!
Portsmouth Conference 2014
Systran, Apertium, Babelfish, Prompt…
Google, Moses, Language Weaver, Asia Online…
Systran, Asia Online, Prompt
What is the market impact?
In a nutshell: • Impact on the translation profession
o Deprofessionalization of the translator? o New jobs, e.g. Post-Editor, MT quality evaluator… o How to maintain creativity?
• Quality o MT Process vs. quality of the MT output o What is the final expected quality?
• Productivity o Raw MT output, Light post-editing vs full post-editing
• Compensation o Per word, per hour or both?
Portsmouth Conference 2014
Benefits of MT for…
End-clients o Reduce costs o Reduce time to market o Increase productivity o Translate new content
Translators • New opportunities • Increase productivity • Higher income • More consistency
Portsmouth Conference 2014
While we see that MT technology adoption continues to grow, hopefully because of clearly demonstrated benefits and measured production efficiencies, we still see that the dialogue between the technology developers / business sponsors and translators/post- editors is often strained, and communications can often be dysfunctional and sometimes even hostile.
Kirty Vashee, eMpTy Pages
What do translators say?
Portsmouth Conference 2014
What do translators say?
…how to protect the language skills and mental function of those routinely exposed to toxic machine language. (Translator’s blog)
I fear it might be the post-MT crash (of our industry, in terms of rates and quality). (Translator, Proz )
“post-processing of machine translations”, is likely to be much more deadly to the translating community, because this tool was designed by our beloved “translation industry leaders” to simply kill off the translating profession and replace it by a new type of workers called post-processors of machine translations. (Translator’s blog)
Portsmouth Conference 2014
What do translators say?
My proposal is that you start investing time and money to “update the skills” of your human base, current freelance translators and interpreters, so that you may “flip it” around....I urge industry leaders to invite us, everyday translators and interpreters, to become an integral part (not a side story) of this equation. We have been severed from the most important conversations about our own future. Many of us are afraid of the new technologies because there is yet no clear answer to the question “what’s in it for me?”. The “industry” as a whole may well benefit from inviting us to become part of the equation going forward. If translators and interpreters do not learn –quickly and swiftly– to use 21st century technologies, we may not survive as a viable profession. Unless all the parties are on the same page, the “industry” as a whole may not evolve the way industry leaders might have anticipated. We need each other.
Claudia Brauer (2014), blog post Will you help me flip it?
Portsmouth Conference 2014
What do translators say?
…it is safe to say that the number of professional translators who are using MT has risen in the past couple of years. I suspect that you would find large differences of usage among different language groups, type of projects, experience levels, and even in the way it is used. Significantly, MT seems to have transitioned into one of many productivity tools that are useful to some but not all, and I think the stigma it used to have among translators is gradually going away.
ATA Chronicle (Feb. 2013) Conversation: Charlotte Brasler and Jost Zetzsche
Portsmouth Conference 2014
Post-editing is just a step in the MT process
Pre-production
• Consultation • Content • Customization
Production
• Piloting • Processing • Post-editing
Post-production
• Metrics • Maintenance
Based on the MT process described by Lori Thicke in “The industrial process for quality machine translation” in JoSTrans, Issue 19
Portsmouth Conference 2014
Translation vs post-editing
Translation process of a translator (TAUS 2010)
Translation process of a post-editor (TAUS 2010)
Portsmouth Conference 2014
Translators vs Post-editors Skills?
Translators • Fluent at least in two languages
o Excellent knowledge of SL o Perfect command of TL
• Excellent written and verbal communication skills
• Understand the cultural differences • Specialised domain knowledge • Word-processing skills • Knowledge of language
technologies • …
Post-editors • Excellent word-processing and editing skills
• Ability to work and make corrections directly on-screen
• Strong knowledge of general MT and its shortcomings
• Specific knowledge of the approach of the MT systems in use, especially their weaknesses and the reason for producing the encountered errors
• Knowledge of the SL, TL and the audience
• Ability to make quick decisions and reduce errors as needed
• Ability to balance post-editing speed and effort with respect to the required quality level;
• Knowledge of CAT environments and of the differences between post-editing MT output as opposed to editing the past human translations present in fuzzy matches.
Source: Doherty & Gaspari 2014, GALA blog
Portsmouth Conference 2014
Source: http://www.multilingual-tech.com/2014/05/08/language-translations-can-be-funny-right/
Quality
Portsmouth Conference 2014
Quality
Quality is relative…The evaluation of the translation quality- whether human or computerized – has to take into account the following intra-linguistic and inter-linguistic factors: morphology, syntax, content, terminology, style, conformity. (Bruderer 1978)
Typically, there are many “perfect” translations of a given source sentence. These translations may vary in word choice or in word order even when they use the same words. And yet humans can clearly distinguish a good translations from a bad one. (Papineni et al., 2002)
Portsmouth Conference 2014
Quality: MT Evaluation methods
Human • No standardised method • Word Error Rate (WER) • Subjective Sentence Error
Rate (SSER) • Metrics:
o Fluency and Adequacy (TAUS DQF, COSTA MT-Evaluation, Appraise etc.)
• Post-editing effort
Automatic • Metrics:
o BLEU o NIST o WER o METEOR o LEPOR
Portsmouth Conference 2014
Productivity
Post-editing MT output can significantly increase the productivity of human translators!
Guerberof 2009; Plitt and Masselot 2010; Federico et al. 2012; Green et al. 2013; Läubli et
al. 2013
Portsmouth Conference 2014
Productivity
TEP • Translation: 250 words per
hour • Editing: 1000 words per
hour • Proofing: 4000 words per
hour
MT Post-editing • 350-800 words per hour • …
Portsmouth Conference 2014
Productivity
Source: Asia Online, Custom MT engines with Language StudioTM
With Customised MT
Portsmouth Conference 2014
PE Compensation
• Various experiments, no standard • Hourly rates or price per word?
o 65%-85% of new word rate (Lionbridge) o 40%-85% of new word rate (CSA report) o $45/hr (Asia Online)
Portsmouth Conference 2014
PE Compensation
• It depends on certain factors: o Quality of the MT output o Translator’s productivity (speed) o Language pair o Content type
Portsmouth Conference 2014
PE Compensation
Good MT output
Less PE effort
More words per hour
More earnings?
Portsmouth Conference 2014
MT becomes even better when integrated in the translators TM environment! (O’Brien & Moorkens 2014)
A number of CAT tools connect to general and customized MT engines via API:
TM/MT integration
Portsmouth Conference 2014
OmegaT
In the OmegaT user manual it is stated the following:
If you do not wish to send your source segments to Google to get translated, untick the Google Translate menu entry in Options. Note that nothing but your source segment is sent to the MT service. The online version of Google Translate allows the user to correct the suggestion and send the corrected segment in. This feature, however, is not implemented in OmegaT.
Portsmouth Conference 2014
Post-editing analysis & Compensation
Portsmouth Conference 2014
Signs and Simptoms of Translation- Blog by Emma Goldsmith
Small-scale experiment at University of Leuven
• Goal: o To test the TM/MT integration o To investigate the impact of MT on productivity and the final translation quality
• Participants and language combinations: o 21 MA/Postgraduate students with EN > NL (Productivity test) o Translation Technology class
• Source text: o 5 short English texts, technical documentation, same level of difficulty
• Tools: o Trados Studio 2014
• Resources: o TM with output from Google
Portsmouth Conference 2014
What did we measure?
• Productivity (time): human translation vs post-editing MT
• Quality: human-translations vs post-edited translations
• The usefulness of MT
• Translators’ expectations
Portsmouth Conference 2014
How?
1. Translation from scratch, no TM/MT provided. 2. Translation with TM. 3. Translation with TM/MT. 4. Track the translation time of the whole project. 5. Answer few questions about the usefulness of MT
Portsmouth Conference 2014
Results
Productivity Total average time per text for EN-NL
Portsmouth Conference 2014
Nr of words per text varied between 140-200
Results
Quality Evaluation: Rate the translations on a scale from 1-4 in terms of:
o Clarity: How easily can you understand the translation?
o Accuracy: How much of the meaning in the source text is also expressed in the translation?
o Style: Is the language used in the translation appropriate for this content type?
Fiederer & O’Brien (2009)
Portsmouth Conference 2014
CLARITY: How easily can you understand the translation?
Portsmouth Conference 2014
Human Translation vs. Post-editing
ACCURACY: How much of the meaning in the source text is also expressed in the
translation?
Portsmouth Conference 2014
In general
ACCURACY: How much of the meaning in the source text is also expressed in the translation?
Portsmouth Conference 2014
Human Translation vs. Post-editing
STYLE: Is the language used in the translation appropriate for a software product
user manual?
Portsmouth Conference 2014
In general
STYLE: Is the language used in the translation appropriate for a software product user manual?
Portsmouth Conference 2014
Human Translation vs. Post-editing
Terminology? Collocations?
0
5
10
15
20
25
Useful for terminology
Useful for collocations
Very useless Useless Useful Very useful
Portsmouth Conference 2014
Nr o
f stu
dent
s
Do you think the machine translation output had an impact on the quality of your final translation?
Portsmouth Conference 2014
Limitations
• Students had to record the time themselves • Not all of them had a good knowledge of English • Knowledge of the technical domain • Starter-knowledge of Trados Studio 2014
Portsmouth Conference 2014
The debate that puts machine translation and professional translation on the opposite sides is misleading. The question isn’t about choosing ‘either- or’ because those two solve totally different problems. Professional translation is the best choice when your goal is to get someone to understand your message. MT is a risky choice in those kinds of situations but is a more suitable tool when you want to understand someone else’s message.
Multilizer, Translation Blog
Portsmouth Conference 2014
References • Thicke, Lori. The industrial process for quality machine translation. Retrieved from http://
www.jostrans.org/issue19/art_thicke.php on 5 November 2014. • Guerberof Arenas, Ana (2009). Productivity and Quality in the Post–editing of Outputs
from Translation Memories and Machine Translation. Localisation Focus 7(1): 11-21http://isg.urv.es/library/papers 2009_Ana_Guerberof_Vol_7-11.pdf
• Guerberof Arenas, Ana (2013) What do professional translators think about post-editing?. The Journal of Specialised Translation 19: 75-95. http://www.jostrans.org/issue19/art_guerberof.pdf
• O’Brien, Sharon (2002) Teaching Post-editing: A Proposal for Course Content. Proceedings of the 6th EAMT Workshop on “Teaching Machine Translation”. EAMT/BCS, UMIST, Manchester, UK. 99-106. http://mt-archive.info/EAMT-2002-OBrien.pdf
• Language Technology at Autodesk. Translation and Post-editing Productivity (2011).http://langtech.autodesk.com/productivity.html
• Doherty Stephen & Gaspari Federico , QT LaunchPad Project (2014) Effective Post-Editing in Human and Machine Translation Workflows. http://www.gala-global.org/blog/2014/effective-post-editing-in-human-and-machine-translation-workflows/
Portsmouth Conference 2014
References • O'Brien, Sharon and Moorkens, Joss (2014). Towards intelligent post-editing interfaces.
In: FIT XXth World Congress 2014, 4-6 Aug 2014, Berlin, Germany. Retrieved from http://doras.dcu.ie/20136/ on 5 November 2014.
• MateCat: http://www.matecat.com • SDL: http://www.sdl.com/products/sdl-trados-studio/ • Kilgray Technologies: http://kilgray.com • MemSource: http://www.memsource.com • eMpTy Pages Blog: Improving the MT Technology through Translator Dialogue. http://kv-
emptypages.blogspot.nl/2014/05/improving-mt-technology-to-translator.html • Patenttranslator blog: There Are Many Ways to Commit Suicide, But Post-Processing of
Machine Translations Would Be a Horrible Way To Go (2014) :http://patenttranslator.wordpress.com/2014/10/22/there-are-many-ways-to-commit-suicide-but-post-processing-of-machine-translations-would-be-a-horrible-way-to-go
• Translationtribulations.com
Portsmouth Conference 2014
References • Plitt.M, Masselot F. A productivity test of statistical machine translation post-editing in a typical
localisation context. Retrieved on October 21 from http://ufal.mff.cuni.cz/pbml/93/art-plitt-masselot.pdf • De Palma, Don. Post-editing in practice: http://www.tcworld.info/e-magazine/translation-and-
localization/article/post-editing-in-practice/ • Asia Online, Fair-compensation of Post-editing MT: http://www.asiaonline.net/EN/Resources/Articles/
FairCompensationForMTPostEditing.aspx. Retrieved on 27 October 2014 • Smets, Martine. Microsoft and the need for agile translations: http://cdt.europa.eu/EN/news/Pages/
default.aspx?CDTID=266 • Deliverables D5.1 Baseline Definition Results on KPI 1: Post-editing Effort: http://www.matecat.com/
deliverables/d5-1-baseline-definition/results-on-kpi-1-post-editing-effort/ • Apertium: http://www.apertium.org/index.eng.html#translation • ATA Chronicle (February 2013):http://www.atanet.org/chronicle/feature_article_february2013.php • Example of a Translation Integrated Workflow: http://www.asiaonline.net/EN/MachineTranslation/
default.aspx?QID=21
Portsmouth Conference 2014