qtlp d6.6.1.docx€¦ · web view1 july 2012, 24 months. distribution. public (internal version)...

Preparation and Launch of a Large-scale Action for Quality Translation Technology

D6.6.1: QTLaunchPad Workshop: Quality Metrics for Human and Machine Translation

Deliverable D6.6.1

QTLaunchPad Workshop:

Quality Metrics for Human and Machine Translation

Author(s): Stephen Doherty, Federico Gaspari, Josef van Genabith, Declan Groves, Ankit Srivastava (DCU)

Dissemination Level: Public (Internal Version)

Date: 24.04.2013

1



Grant agreement no. 296347

Project acronym QTLaunchPad

Project full title Preparation and Launch of a Large-scale Action for Quality

Funding scheme Coordination and Support Action

Coordinator Prof. Hans Uszkoreit (DFKI)

Start date, duration 1 July 2012, 24 months

Distribution Public (Internal Version)

Contractual date of delivery March 2013 - Extended

Actual date of delivery April 2013

Deliverable number D6.6.1

Deliverable title QTLaunchPad Workshop on Quality Metrics for Human and Machine

Type Report

Status and version Internal Draft for Review

Number of pages 28

Contributing partners DCU, DFKI, USFD

WP leader DCU

Task leader DCU

Authors Stephen Doherty, Federico Gaspari, Josef van Genabith, Declan

EC project officer Kimmo Rossi

The partners in Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI),

Dublin City University (DCU), Ireland

Institute for Language and Speech Processing, R.C. “Athena”

The University of Sheffield (USFD), United Kingdom

2



For copies of reports, updates on project activities and other QTLaunchPad-related information, contact:

DFKI GmbH

QTLaunchPad

Dr. Aljoscha Burchardt [email protected]

Alt-Moabit 91c Phone: +49 (30) 23895-1838

10559 Berlin, Germany Fax: +49 (30) 23895-1810

Copies of reports and other material can also be accessed via http://www.qt21.eu/launchpad

© 2013 - The Individual Authors

No part of this document may be reproduced or transmitted in any form, or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission from the copyright owner.

3



Contents

1 Introduction....................................................................................................................................5

1. 1 Multidimensional Quality Metrics...........................................................................................5

1.2 MQM Workshop and Related Events....................................................................................6

2 Workshop Feedback..................................................................................................................10

2.1 Metrics.....................................................................................................................................10

2.2 Applications.............................................................................................................................11

3 Conclusion...................................................................................................................................13

3.1 Summary.................................................................................................................................13

3.2 Future Directions....................................................................................................................13

Appendices........................................................................................................................................15

Appendix A - Rome Workshop Agenda.....................................................................................16

Appendix B - Workshop A5 Flyers..............................................................................................17

Appendix C - Mailing List Material..............................................................................................20

Appendix D - Rome Workshop External Participants..............................................................22

Appendix E - Miami Workshop GALA Participants...................................................................23

4



1 IntroductionThe aim of this report is to capture the discussions and feedback from two public workshops

on the QTLaunchPad quality metrics on human and machine translation. Following an

introduction to the multidimensional quality metrics (MQM) being developed in the project

and a description of the workshops’ format, this report:

1. provides specific feedback and discussion points for the MQM framework (Section 2);

2. highlights key commonalities (Section 2);

3. recommends future steps informed by the above (Section 3).

1. 1 Multidimensional Quality MetricsAs a core aspect of the QTLaunchPad project, the multidimensional quality metrics

framework (MQM) is a means of moving beyond the current shortcomings of existing

translation quality assessment. Quality assessment (QA) is an important task in the

translation workflow, especially in the context of machine translation. Traditional methods

are typically very subjective and involve assessors counting errors and their severity. This

approach has led to the formalisation of metrics for assigning errors to different types (e.g.

incorrect spelling, incorrect terminology, wrong verb agreement) and counting their

prevalence and severity in a random sample of translated content. This approach developed

into specifications such as the LISA QA Model and SAE J2450, however, these models have

not been updated consistently and have shortcomings in terms of validity and robustness.

The prevalence of the use of MT in translation and localization workshops also presents a

challenge for evaluation and QA processes due to the unique nature of MT output, and the

extensiveness of human intervention, e.g. use of raw MT output, human post-edited MT. The

“one-size-fits-all” approach of existing models simply cannot meet the needs and

expectations of a rapidly diversifying translation industry.

Following a systematic review of existing models and extensive public consultation, the

QTLaunchPad project is developing a new framework for assessing quality based on the

following principles:

Adaptability - metrics must be adaptable to specific project types as projects,

users, requirements, and scenarios are unique. This allows for metrics to be

tunable and sensitive to each application;

5



Granularity - metrics must allow for customizable degrees of granularity, from

extremely coarse to extremely fine, depending on the use case, e.g. evaluation of

gist translation vis-á-vis a detailed analysis to isolate errors;

Comparability - results must be comparable and meaningful across jobs, projects,

users, and domains of application;

Suitability - metrics must be suitable for both human and machine translation and

combinations thereof in addition to all technology and production profiles and

users;

Standardised - metrics must develop upon existing standards and established

best practice in order to optimise this progress while supporting interoperability,

proprietary methods, and customisable workflow integration;

Fairness - existing metrics conflate errors in the source, errors in the target, and

actual translation errors. This results in problems identifying the cause of the

problem, where translators are often blamed. QA should be fair and recognise

the work of translators, especially when they add value in this way.

A more extensive and practical description, and further information on implementation, is

available from the project website1, and in the publicly available deliverable D1.1.2:

Multidimensional Quality Metrics.

1.2 MQM Workshop and Related EventsAs a means of information gathering from relevant stakeholder groups, a survey was

conducted in late 2013 (whose results are documented in project deliverable D6.5.1: Report

on Requirements Gathering from Relevant Stakeholder Groups). Of 438 respondents across

approximately 40 countries, the survey examined a cross-section of the main stakeholder

groups targeted by the QTLaunchPad project: translators and LSPs, large-scale public users

of language technology (LT), providers of LT, and corporate users.

The findings of the survey highlighted current trends and best practices, and were

particularly insightful regarding the utilisation of language and translation technologies in the

context of quality assessment and MT post-editing. The survey also identified shortcomings,

for instance, in the absence of comparable and robust QA methods and in the quality

thresholds of current approaches to MT.

1 http://www.qt21.eu/launchpad/content/multidimensional-quality-metrics

6



Informed by these findings (D6.5.1), several general topics of importance were identified and

appear below as ranked by workshop participants prior to the event2:

the need to move beyond current approaches to evaluation;

the availability of high-quality corpora, datasets, systems, etc.;

funding for language and translation technologies;

the need for specialised knowledge and expertise;

the quality of current corpora, datasets, etc.;

uncertainty about the future of MT.

The project consortium identified co-location with the MultilingualWeb workshop in Rome

(March 14th, 2013) to be the ideal first venue for the QTLaunchPad MQM framework

presentation due to its location, resource cost, and audience (attendance of 150). The W3C

MultilingualWeb3 workshops are free and open to the public and concern standardisation

and best practice for multilingual and multicultural web-based information, e.g. localisation,

code standards. In sponsorship of the event, QTLaunchPad received e-mail access via the

organiser to the attendees, a full-page promotional piece for the project, and space for

exhibition in the main room of the event (pull-up poster and flyers).

The final agenda for the Rome workshop can be found in Appendix A. The MQM workshop

took place in the morning of the full-day session, where the afternoon was assigned to the

research innovation application scenarios, RIAS, as detailed in D6.7.1. Despite considerable

last-minute cancellations both the W3C and QTLP workshops have encountered, each of the

aforementioned stakeholder groups were well represented at the workshop: 7 LSPs, 2 LT

providers, 2 corporate users, and 3 large-scale public users.4 The structure of the MQM

workshop is outlined below and corresponds to the structure of feedback in Section 2 of this

report:

Overview of Metrics:

o Principles

o Issue Types

o Dimensions

Using Dimensions to Build Task-Relevant Metrics

Demonstration of MQM Tool

2 http://docs.google.com/spreadsheet/ccc?key=0AotdkIT3g3R8dEZsa2pVVElhQk5QT3FBZkRlT3Vkc1E&usp=sharing

3 http://www.multilingualweb.eu/

4 See Appendix D - names and companies will be removed from the public version of this deliverable.

7



Closing Discussion

QTLaunchPad also ran several events at the Globalization and Localization Association

(GALA) conference in Miami5 (March 17th-20th with an attendance of approximately 250

delegates with QTLaunchPad flyers in each bag) supported by the project’s sub-contractor

GALA. GALA is the world's largest association for the language industry, including

translation and localization. It is a non-profit in nature and provides resources to thousands

of its members. The QTLaunchPad presence consisted of:

1. a 45-minute talk on MQM as part of the main conference programme (43 attendees

in three-way parallel slot);

2. an interactive exhibition open through the conference;

3. a roundtable discussion on MQM (two 90-minute slots).

The exhibition space made use of the same marketing materials as the Rome event: flyers

detailing the MQM, and a pull-up banner. The combination of events allowed project

representatives many opportunities to present QTLaunchPad, the RIAS, and metrics to this

audience, which resulted in feedback and contact points for many aspects of the project

work, including feedback on RIAS as outlined in this report, the multidimensional quality

metrics (D6.6.1), and future directions (D6.8.1 and D6.8.2).

Promotional material for these events can be found on the QTLaunchPad website6, the

MultilingualWeb website7, the workshop flyer8, e-mail and social media announcements9,

and on the GALA website10. In addition to these channels, the workshop was advertised via

e-mail lists from project partners, including CNGL, MT-List, and social media (e.g. Twitter11

and LinkedIn12).

Prior to the above events, QTLaunchPad also ran a webinar13 via GALA to showcase the

MQM framework (February 21st, 2013). Following the presentation there was time for

5 http://www.gala-global.org/conference/

6 http://www.qt21.eu/launchpad/content/workshop-research-innovation-application-scenarios-%E2%80%93-rome-march-14th-2013

7 http://www.multilingualweb.eu/documents/rome-workshop/rome-program

8 See Appendix B

9 See Appendix C

10 http://www.gala-global.org/conference/qtlaunchpad-showcase

11 http://twitter.com/qtlaunchpad

12 http://www.linkedin.com/groups?gid=4807518

13 A recording of the webinar can be found here: http://www.gala-global.org/recordings-past-webinars#GSI and a PDF of the content here: http://www.gala-global.org/files/21Feb2013_QTLaunchpad_PPT.pdf

8



questions and discussion from the 75 participants (out of a total of 162 registered), and

additional follow-ups via the project’s LinkedIn page and opt-in e-mail list14.

14 http://www.dfki.de/mailman/cgi-bin/listinfo/qtlp-news%20

9



2 Workshop FeedbackThe feedback gathered at the events described has considerably helped shaping the

ongoing project developments and has directly been incorporated into the respective

Deliverables. This section summarizes main points from the feedback and discussions of the

above workshop and related events into the categories of: metrics and applications.

2.1 Metrics While it’s not always possible, it’s important to know the contexts in which metrics are

used. This includes knowledge of: users, domains, formats, tools, resources, etc.

Taking source text quality in a translation metric into account is a positive

development, but may not be immediately possible in existing workflows.

The range of issue types in the MQM needs to be practical and contain customisable

degrees of granularity. It is perhaps not possible to find a balance that suits all users,

so some level of customisation is necessary, e.g. Canadian government’s SEPT

error categories contains approximately 700 aspects, while very comprehensive it is

not applicable in industrial application.

There is also a need to look at evaluation processes, not just at error metrics. TAUS

has a Dashboard that gives options for job types, end use, and tools, etc., but it fails

to address how the evaluation data are used and if they are meaningful and effective

or not.

Further to this, the issue of comparability across jobs, projects, and evaluation

paradigms is burdensome. Metrics such as BLEU have become the de facto

standard in research, yet may not be at all meaningful for translators or buyers -

MQM must work with the existing evaluation landscape.

Further attention should be given to the nature of error categories rather than errors

themselves; this may be fruitful in terms of pre-processing, standardisation, and

overall resource saving.

There will need to be well-documented and easily accessible and understandable

content for the basis of the MQM, its usage, and its value over existing approaches.

Sensitivity to the different types of errors introduced by human translation, machine

translation, and combined approaches.

10



There should be a balance of preventative and reactive strategies for QA - human in

the loop, rather than effort wasted, e.g. post-edited content and evaluation data to be

used to improve MT processes - MQM needs to include such a balance.

Finally, the validation of the MQM is critical. To learn from existing models, there

needs to be strict reliability and validity testing, etc.

2.2 Applications It is well recognised that current models do not meet industry needs. It is apparent to

most stakeholders that there is a need for updating existing approaches to evaluation

and QA rather than following the status quo in a haphazard, internally-focused, and

reactive way.

Translators are typically not consulted in the development of metrics and their usage.

Inclusion of these groups will be valuable, especially in the proposition of the MQM

that translations are penalised for correction errors in the source.

Evaluators are busy and sometimes even ‘lazy’. Despite an extensive list of errors,

the same small number of metrics are used from a drop-down menu.

The ambiguity of certain tools or types (of errors) can be a source of the previous

point, but more generally, can lead to poor evaluation results that can vary greatly

from person to person. To combat this, ambiguity should be reduced by using clearly

defined types and procedures with cooperation between the tool and the users. (to

suit their needs).

It is also ambiguous what is meant by compatibility in MQM. Such ubiquitous

compatibility with existing models is a claim that may be difficult to accomplish in

practice.

In terms of formats: what will MQM be compatible with, what options are there with

regard to input/process/output formats and encoding?

The evaluation/QA data from MQM should also be meaningful, customisable, and

allow for different degrees of granularity, otherwise the best features of the metrics

may be lost or ignored post-evaluation.

Alongside MQM, the approach of avoiding errors in the first place should be pushed -

preventative rather than reactive steps. Thus, standardisation and pre-processing are

necessary components to high-quality translation results. Translation quality is linked

directly to pricing - higher quality equates to higher compensation. It is therefore

important to establish thresholds, just like production workflows.

11



In terms of usage, how can issues with representative sampling be addressed in

MQM? Currently, there is a need to sample and assess in QA models, while AEMs

like BLEU can assess the whole document/system.

There needs to be a clear way for users of MQM and the tool to give feedback and

be more interactive with the project - mailing lists and one-way communications are

not sufficient.

Open field testing will allow for the inclusion of as many viewpoints as possible for

different users in a variety of scenarios.

12



3 Conclusion

3.1 SummaryThe feedback QTLP has gathered was by and large very positive, supporting the work that

has been done in the first phase of the project. The project was also confirmed in choosing

an early communication strategy exposing even ongoing work to professional criticism.

In summary, the focus of stakeholders and users is apparent throughout the workshop

discussion and feedback. Stakeholder/user inclusion and buy-in are pivotal in the uptake of

new tools such as the MQM. Therefore, the entry level for its use must be low, with support

and accessible materials from the onset. The value of such adoption must be clear and

quantifiable for it to be successful. The more specific feedback relating to the metrics will be

addressed in the update to the first version of the MQM; the second, revised version will be

available in project deliverable D1.1.2.

Further to the acquisition of this valuable discussion and feedback outlined in the previous

section, the workshops resulted in quantifiable gains in terms of QTLaunchPad’s public

exposure (traffic to the project homepage, www.qt21.eu, increased from 302 unique visits in

the month of February to 524 in March, and 168 in April15), awareness raising of RIAS topics,

and membership to the project in terms of social media (e.g. increase to 71 members on

LinkedIn16), mailing list (increase to 343 new opt-in members), and individual contacts with

stakeholders.

3.2 Future DirectionsIn addition to the above, future directions for the MQM were invited where the following

points were discussed:

The project’s critical mass and development of the MQM represent sufficient

resources for large-scale field testing and further refinement with community/industry

input.

There needs to be an ongoing collaborative process where quantifiable value is

evident for all parties, especially from industry buy-in viewpoint. In the context of

MQM, this may result in collaborative efforts to test the framework in a variety of real-

world contexts where feedback from users is paramount.

15 As of April 15th - to be updated at month end.

16 For comparison, the META LinkedIn group has 356 members.

13



The roll-out of the MQM platform, translate517, needs to be coordinate effectively and

user-centric to ensure the greatest uptake and effective incorporation of feedback.

This could come in the form of usability, feedback sidebar, two-way communication

between users and QTLaunchPad, education and educational materials,

incentivisation for users to give feedback on errors, crashes, etc.

Feedback should be documented and the improvement of the MQM with this value

input should be made clear to support ownership and adoption of MQM by the user

community.

17 http://www.translate5.net/

14



Appendices

15



Appendix A - Rome Workshop Agenda

16



Appendix B - Workshop A5 Flyers

17



18



Appendix C - Mailing List Material

Message Begins

***

QTLaunchPad Workshops at MultilingualWeb, Rome

Dear Colleagues,

I wish to invite you to a workshop hosted by the EC-funded QTLaunchPad project on multidimensional quality metrics (MQM), and on use-cases for a large-scale future MT research initiative (RIAS), co-located with MultilingualWeb W3C in Rome, Italy.

Workshop Dates: March 14th, 2013

Time: MQM 09:00 - 12:00 (lunch included for both workshops); RIAS 13:00–16:00

Venue: Headquarters of the United Nations Food and Agriculture Organisation, Rome, Italy

Cost: Free

1. Workshop on Multidimensional Quality Metrics (MQM)

Translation Quality Assessment (TQA) has recently emerged as an important business topic where formal metrics such as the LISA QA Model and SAE J2450 for human translation have helped, but automatic metrics for machine translation are currently suitable only for research projects, not for production environments. QTLaunchPad has developed a unified multidimensional framework for TQA that is built around quality metrics that move beyond the limitations of existing models and focus on richness and compatibility with usability as a core feature.

This workshop focuses on the measurement of translation quality. It introduces attendees to the metrics introduced above. It will demonstrate tools for creating project type-specific metrics and ensuring their validity for actual production tasks. Participants will further be invited to provide feedback and to discuss their own quality requirements and needs to help improve and further develop the model in a discussion-oriented exploration of key issues related to TQA.

● Click here to attend and find out more about this workshop

2. Workshop on Research Innovation Application Scenarios (RIAS)

A central aim of QTLaunchPad is the preparation for a large-scale research and innovation action (QT21) in the application of research into of several core areas which have been identified in

19



close consultation with stakeholders in research and industry. These research innovation application scenarios or RIAS represent promising combinations of tasks, domains, users, industrial actors, demonstrators, innovation mechanisms, data, etc. Current suggestions under discussion include:

● Automotive: technical documentation with the end user in mind, and internal communication for multilingual environments;

● Medical: high-quality medical information for the general public, and emergency warnings (e.g. earthquakes) when multilingual data (e.g. SMS) need to be disseminated quickly and accurately;

● Public: public consultations and information;

● Media: subtitling/audio descriptors, e.g. for lectures and person-to-person communication;

● Language Learning: multi-modal communications a foreign language that is mastered only partially.

This workshop presents the progress of the exploration of these areas to participants and invites interactive discussions where attendees can add their own needs and requirements and provide welcomed feedback to the work carried out so far.

● Click here to attend and find out more about this workshop

Thanks and best wishes on behalf of the QTLaunchPad team,

Stephen

***

Message Ends

20



Appendix D - Rome Workshop External Participants

Name Organisation Stakeholder GroupStefania Cavallo University of Bologna LT Provider 1

Joanna Drugan University of East Anglia LT Provider 2

David Lewis Trinity College Dublin & CNGL Public User 1

Bettina Lieske SAP AG Corporate User 1

Christian Lieske SAP AG Corporate User 2

Isabella The Big Wave LSP 1

Radina Matic tcloc.com LSP 2

Luigi Muzii sQuid LSP 3

Alessandra AMTrad Services LSP 4

Phil Ritchie VistaTEC LSP 5

Felix Sasaki DFKI Public User 2

Marco Scarpa Milkshake Studio SRL LSP 6

Marco Trombetti Translated.net LSP 7

Feiyu Xu DFKI Public User 3

21



Appendix E - Miami Workshop GALA Participants

GALA 2013 Miami Participant List

1. Tea Dietterich Director 2M Language Services Brisbane, Australia2. Peter Redmond Senior Project Manager Ad Astra Washington DC, United States3. Lena Toolsie President and CEO Ad Astra Washington DC, United States4. Maxwell Hoffmann Global Product Evangelist, Tech Comm ADOBE SYSTEMS Beaverton, OR,

United States5. Quynn Le International Program Manager Adobe Systems, Incorporated Houston, United States6. Vitaly Pedchenko Deputy Managing Director Advanced International Translations Kiev, Ukraine7. Aleksandr Diatlov Head of Technical Support Advanced International Translations Kiev, Ukraine8. Vladimir Pedchenko Managing Director Advanced International Translations Kiev, Ukraine9. Katalin Varga managing director Afford Translations and Interpreting Ltd. Budapest, Hungary10. Christopher Carter General Manager aLanguageBank New York, United States11. Dimitra Hengen President Alpha Omega Translations Alexandria, VA 22306, United States12. Dorothee Racette President American Translators Association Saranac, United States13. George Chew Sr. Localization Specialist Amway Ada, Michigan, United States14. Catherine Guinon Manager, Digital Operations Amway Corporation Ada, United States15. Jaroslav Rázek Managing Director Aploq Ltd. Wroclaw, Poland16. Mohamed Hassan Business Development Manager Arabize Cairo, Egypt17. Roberto Ganzerli CEO Arancho Doc Bologna, Italy18. David Boe Business Development Manager, North America Atril Solutions Portland, United States19. Blandine Loze Managing Director Atril Solutions Paris, France20. Joël Sigling Director AVB Language Services Amstelveen, Netherlands21. Marie Flacassier COO BeatBabel San Diego, United States22. Burckhardt Rueffer CEO BeatBabel San Diego, United States23. Daniela Schobert Partner Blue Dimension, Lda Lisboa, Portugal24. Cristina Gil Partner Blue Dimension, Lda Lisboa, Portugal25. Tanushree Gupta General Manager, Production Braahmam Net Solutions Pvt. Ltd. Delhi-NCR,

India26. Anjula Jyala Assistant Manager, Business Development Braahmam Net Solutions Pvt. Ltd. Delhi-

NCR, India27. Anna Martirosian Resourcing and Operations Branded Translations Henderson, United States28. Alan Melby Professor; President Brigham Young University and LTAC Provo, United States29. Fabiano Cid Managing Director Ccaps Rio de Janeiro, Brazil30. David Lewis Research Assistant Professor Centre for Next Generation Localisation Dublin,

Ireland31. Dominic Jones Research Assistant Centre for Next Generation Localisation Dublin, Ireland32. Jiri Stejskal President & CEO CETRA Language Solutions Elins Park, United States33. Tommy Tomolonis Project Manager CETRA Language Solutions Philadelphia, United States34. Yu Li Senior Developer ClayTablet Technologies Toronto, Canada35. Páraic Sheridan Associate Director CNGL Dublin, Ireland36. Steve Gotz Commercial Development Manager CNGL Dublin, Ireland37. Don DePalma Chief Strategy Officer Common Sense Advisory, Inc. Cambridge, United States38. Miklos Urban General Manager Consell Translations Budapest, Hungary39. Thomas Dimmer Business Development Manager CONTRAD Olsztyn, Poland40. Marek Makosiej Managing Director CONTRAD Weranis i Makosiej Sp. J. Olsztyn, Poland41. Gary Muddyman CEO/ Managing Director Conversis Global Oxford, United Kingdom

22



42. Briana McCrory Marketing Manager Conversis Global Oxford, United Kingdom43. Tenesoya Pawlowsky Managing Director CPSL Barcelona, Spain44. Livia Florensa Managing Director CPSL Madrid, Spain45. Judy Crittenden Production Manager CTS LanguageLink Vancouver, United States46. Angelica Perez Partner Delsurtranslations Córdoba, Argentina47. Virginia Minhondo Partner Delsurtranslations Córdoba, Argentina48. Arle Lommel Senior Consultant, Language Technology Lab DFKI Berlin, Germany49. Andrew Lawless President Dig-IT / Rockant, Inc. Washington, D.C., United States50. Asta Rusakevičienė CEO Diskusija UAB Vilnius, Lithuania51. David Orban CEO Dotsub New York, United States52. Michael Smolens Founder & Chairman Dotsub New York, United States53. Marcel Bregman Sr. Localization Engineering Manager eBay Inc. San Jose, United States54. Emily Shearer Director, Client Services Echo International Pittsburgh, United States55. John Watkins President ENLASO Boise, ID, United States56. Vigdis Eriksen President & CEO Eriksen Translations/Red T Brooklyn, United States57. Jörgen Danielsen Managing Director Eule Lokalisierung GmbH Kiel, Germany58. Mark Evenepoel CEO euroscript International S.A. Bertrange, Luxembourg59. Carlos Hesano COO Executive Language Services Bloomfield Hills, United States60. Sunil Kulkarni President Fidel Localization Services Tokyo, Japan61. Karin Albiez German translator and reviewer Freelancer Barcelona, Spain62. Saoussane Mrini Business Development Future Trans Ltd Cairo, Egypt63. John Labati Chief Operating Officer G3 Translate New York, United States64. Emre Akkas Chief Localizer Globalme Language&Technology Vancouver, Canada65. Paula Ferrari Managing Director go global! Buenos Aires, Argentina66. Chenhui Chu Student Graduate School of Informatics, Kyoto U. Kyoto, Japan67. Aida Martirosyan Managing Director Haymillian London, United Kingdom68. Miguel Martinez Transcreation Director Hogarth Worldwide London, United Kingdom69. Meritxell Guitart President, Hogarth Americas Hogarth Worldwide Inc. New York, United States70. Xavier Maza Cid Language Services Manager iDISC Barcelona, Spain71. Pere Grivé-Ayguadé General Manager iDISC Barcelona, Spain72. Olga López Project Manager iDISC Barcelona, Spain73. Terena Bell CEO In Every Language Louisville, United States74. Loic Dufresne De Virel Localization Strategist Intel Corp Portland, OR, United States75. Rosana Bailone Director Interaction Translation Services Villa Mercdes, Argentina76. William Skinner Chief Translator International Monetary Fund Washington, DC, United States77. Beate Birkefeld Director of Translation Services Interpreters and Translators, Inc. (iTi)

Manchester, United States78. Ralph Strozza CEO Interpro Translation Solutions Lisle, United States79. Mitch Donaldson Project Manager Interpro Translation Solutions, Inc. Lisle, United States80. Manfred Schmitz Member, Executive Board Intertext Fremdsprachendienst e.G. Berlin, Germany81. Olga Fomenko Marketing and PR Manager InText Translation Dnipropetrovsk, Ukraine82. Céline Imbaud General Manager ITC GLOBAL TRANSLATIONS Jupiter, United States83. Yaroslav Tugarev Business Development Manager Janus Moscow, Russia84. William Rivers Executive Director JNCL / NCIS Washington, DC, United States85. Crystal Goldie Policy Analyst JNCL / NCIS Washington D.C., United States86. Miriam Valova Procurement Area Manager & Team Coord. Jonckers Translation & Engineering

Brno, Czech Republic87. Sergio Ruffolo COO JR Language Services Webster, NY, United States88. Jackie Ruffolo President JR Language Services Webster, New York, United States89. Patricia Bown Director of Sales, Americas Kilgray Translation Technologies Austin, Hungary90. Peter Yoon CEO Korean LSP Oakland, United States91. Mário Rodrigues Operations Director and Business Development L10N Studio Lisbon, Portugal

23



92. Zhang Steven Managing Director Landelion Communications Beijing, China93. Starla Keith Executive Vice President Language Services Associates, Inc. Horsham, United

States94. Scott Ludwigsen President, Translation and Localization Division LanguageLine Solutions

Portland, OR, United States95. Antonio Nebres Director of TM and Localization Engineering LanguageWorkd New York, United

States96. Fernando D'Aloia Managing Director LATIN-DATA Buenos Aires, Argentina97. Carlos Anllo Project Manager LatinLingua Buenos Aires, Argentina98. Ariel Meyer Chief Business Officer, General Manager LatinLingua Miami, United States99. Ignacio Valencia Rú Chief Orchestration Officer LatinLingua Buenos Aires, Argentina100. Steve Yoon CEO Latis Global Seoul, Korea101. Edward Kim Project Manager Latis Global Communications Seoul, South Korea102. Lori Thicke CEO LexWorks Vancouver, Canada103. Susan Amarino President Liaison Multilingual Services, Inc. Denver, United States104. Jerzy Nedoma CEO LIDO-LANG Technical Translations Kraków, Poland105. Pablo Badia Mas Production Director Linguaserve I.S. S.A. Madrid, Spain106. Pedro Diez-Orzas CEO Linguaserve I.S. S.A. Madrid, Spain107. Martin Roberts President Linguistic Systems, Inc. Cambridge, United States108. Peter Basile Director of eCommerce Lionbridge International Boston, United States109. Catalina Savill Customer Success Manager Lionbridge International Boston, United States110. Gisela Donnarumma Customer Success Manager Lionbridge International Dublin, Ireland111. Richard Sikes Principal Consultant Localization Flow Technologies Toronto, Canada112. Aki Ito President LocalizationGuy Dallas, United States113. Lilian Alves Mautone Managing Partner LocHouse Translation and DTP Rio de Janeiro, Brazil114. Serge Gladkoff President Logrus International Philadelphia/Moscow, United States115. Luciana Ramos Professional Translator and Interpreter Luciana Ramos - Translation and

Training Roldán, Argentina116. Nima Nourkami Director Sales & Marketing Lucy Software and Services Waibstadt, Germany117. Daniel Grasmick Managing Director Lucy Software and Services GmbH Waibstadt, Germany118. Jennifer Brundage Senior consultant Language Tools Lucy Software and Services GmbH

Waibstadt, Germany119. Kadri Õismaa International Marketing Manager Luisa Translation Agency Tallinn, Estonia120. Norman Newton Vice President ManpowerGroup Language Solutions Houston, United States121. Norman Newton Vice President ManpowerGroup Language Solutions Houston, United States122. Sacha O'Connell Doctoral Scholar Massey University Wellington, New Zealand123. Richard Brooks Director, Strategy & Business Development MCIS Language Services

Toronto, Canada124. Josef Kubovsky Business Development Manager MemSource Prague, Czech Republic125. David Canek CEO MemSource Prague, Czech Republic126. ADELAIDA MAIDAGAN GENERAL DIRECTOR MONDRAGON LINGUA S COOP.

MONDRAGON, Spain127. ANE RUIZ DE ZARATE Director Translation office Mondragon MONDRAGON LINGUA S

COOP. MONDRAGON, Spain128. HAMISI AMANI-DOVE Business Development Manager North America MONDRAGON

LINGUA S COOP. MONDRAGON, United States129. Arturo Quintero Member of the Board Moravia Worldwide Brno, Czech Republic130. Steven McNair Director of Sales North America MultiCorpora R&D Inc. Montreal, Canada131. Jeremy Coombs VP of Operations MultiLing Provo, United States132. Qianqian Wang Marketing Representative Multilingual Technologies Inc. Santa Clara, United

States133. Barbara Burbach Sr Localization PM NetApp Sunnyvale, United States

24



134. Aleksander Honma DIRECTOR NETWIRE CAMPINAS, Brazil135. Andrew Jones Manager - Translation Dept Nikon Precision Inc Belmont, United States136. Jill Smith-Aguirre Strategic Account Executive OmniLingua Worldwide, LLC Cedar Rapids,

United States137. Marcela Becerra Director of Localization Services Paragon Language Services, Inc. Los

Angeles, United States138. Marina Mintz VP, Business Development Paragon Language Services, Inc. Los Angeles,

United States139. Jean-Pierre Parenty CEO Parenty Reitmeier Inc. Winnipeg, Canada140. Vitor Silva Office Manager PHILOS - Comunicação Global, Lda Porto, Portugal141. Sufian Reiter Vice President of Sales Plunet Berlin, Germany142. Nancy Radloff Business Development Manager Plunet Berlin, Germany143. Tom Hoar Managing Director Precision Translation Tools Co., Ltd. Bangkok, Thailand144. Shelly Priebe CEO Priebe & Associates Austin, United States145. Engel Abalos Project Manager PTSGI Taipei, Taiwan146. James Chen Sales Manager PTSGI Taipei, Taiwan147. Square Fang President PTSGI Taipei, Taiwan148. Stephen Doherty Post-doctoral Researcher QTLP, CNGL, Dublin City University Dublin,

Ireland149. Maya Hess CEO Red T New York, United States150. Andrea Nahrwold Relationship Manager Rheinschrift Übersetzungen Cologne, Germany151. Susan Zalkow Team Lead Project Management Rheinschrift Übersetzungen Cologne,

Germany152. María Gabriela Morales Director Rosario Traducciones y Servicios S.A. Rosario, Argentina153. Rebecca Metcalf Senior Project Manager Rubric Edinburgh, Scotland154. Udi Hershkovich VP of Business Development Safaba Translation Solutions Pittsburgh,

United States155. Anu Carnegie-Brown Operations Manager Sandberg Translation Partners Whiteley, United

Kingdom156. Jesper Sandberg Managing Director Sandberg Translation Partners Ltd Whiteley, United

Kingdom157. Marcus Danei Product Manager SAP St. Leon, Germany158. Wafaa Mohiy General Manager-Egypt Saudisoft Co. Ltd Cairo, Egypt159. David Ashton VP, Consulting Services SDL Boulder, United States160. Andy Reid Product Marketing Manager SDL International Reading, United Kingdom161. Misael Diaz North America Business Manager, LSP Division SDL International Waltham,

United States162. Vicenta Ten Soriano Regional Sales Director SDL International Maidenhead, United Kingdom163. Tomas Ezpeleta Business Development Manager SDL International Waltham, United States164. Maria Azqueta Arizcun Production Manager SeproTec Multilingual Solutions Madrid, Spain165. Mieko Kuroki Graphic Designer SmartDTP Arequipa, Peru166. Evelyn Paredes Business Development Manager SmartDTP Arequipa, Peru167. Matt DeLoca VP, Sales Smartling New York, United States168. Jack Welde Founder & CEO Smartling New York, United States169. Justin Dignelli Account Executive Smartling New York, United States170. Margherita Martella Senior Lead Localization Manager Spil Games Hilversum, The

Netherlands171. Patricia Doest - Peccia Localization Project Manager Spil Games Hilversum, Netherlands172. Emilio Alesiani Operations Manager STEP.IN. S.r.l. Rome, Italy173. Grant Straker CEO Straker Translations Auckland, New Zealand174. Suzie Mays Technical Product Manager SuccessFactors/Jobs2Web Minnetonka, United

States

25



175. Cecilia Iros CEO suma Cordoba, Argentina176. Beatriz Bonnet President & CEO Syntes Language Group Centennial, United States177. Diego Bartolomé CEO tauyou <language technology> Sabadell, Spain178. Julie DiRosa International Project Manager tcworld GmbH Stuttgart, Germany179. Michael Fritz CEO tcworld GmbH Stuttgart, Germany180. Christian Hammer CEO Teknotrans AB Gothenburg, Sweden181. Mauro Bertuol CEO TERRALINGUA Porto Alegre, Brazil182. Juan Baquero CEI Texo SRL Córdoba, Argentina183. Kim Harris Managing Director text & form Berlin, Germany184. Horst Liebscher Director of Technology and Innovation text & form Berlin, Germany185. Thomas Senf Managing Director text & form Berlin, Germany186. Robert Etches CIO TextMinded Danmark Aarhus, Denmark187. Raymond Monette Vice-President, Information Technology The LanguageWorks Inc. New

York, United States188. Alicia Assini Project Management Intern The Rosetta Foundation Dublin, Ireland189. Jorge Del Rosario Project Manager - Systems and Vendor Mgmt. The World Bank

Washington, United States190. Indra Samite Dir. Business Development TILDE RIga, Latvia191. Stefan Gentz CEO TRACOM OHG Bonn, Germany192. Laura Gori Owner Trans-Edit Group Srl Milan, Italy193. Thomas Pennell Vice President TransPerfect Translations Inc New York, United States194. Richard Estevez CEO Trusted Translations, Inc. Miami, United States195. Gustavo Lucardi COO Trusted Translations, Inc. Miami, United States196. Kimberly LeFrancois Account Executive TÜV SÜD America Inc. San Diego, United States197. Kara Shannon Account Executive TÜV SÜD America Inc. San Diego, United States198. Agustina Pioli Partner Two Ways Translation Services Cordoba, Argentina199. Milena Rossi Partner Two Ways Translation Services Cordoba, Argentina200. Reinhard Schäler Dirctor, LRC University of Limerick Limerick, Ireland201. Lucia Specia Lecturer University of Sheffield Sheffield, United Kingdom202. Agustina di Pratula Production Manager Ushuaia Solutions Rosario, Argentina203. Kåre Lindahl CEO Venga Localization San Francisco, United States204. Joseph Gomes Managing Director Viva Translations Lisbon, Portugal205. Iris Capraro Galati Linguistic Manager Wells Fargo Bank Charlotte, United States206. Olga Beregovaya VP, Language Tools Welocalize San Francisco, United States207. Alex Yanishevsky senior solutions architect welocalize boston, United States208. Robert Zessar Director of Market Strategy welocalize Frederick, United States209. Gordon Husbands VP Sales & Marketing Wordbank London, United Kingdom210. Kristyna Marrero Director of Sales & Marketing Wordfast New York, United States211. Hannah Collyer-Braham Operations Director World Writers London, United Kingdom212. Brigt Skauge Transcreation Director World Writers London, United Kingdom213. Véronique Özkaya Chief Sales and Marketing Officer Xplanation Leuven, Belgium214. Bob Willans CEO XTM International Gerrards Cross, United Kingdom215. Andrzej Zydron CTO XTM International Gerrards Cross, UK, United Kingdom216. Bill Willette Sales Director XTM International Wells, United States217. Tomasz Zalewski Key Account Manager XTRF Management Systems Krakow, Poland218. Andrzej Nedoma Managing Director XTRF Management Systems Kraków, Poland

26

qtlp d6.6.1.docx€¦ · web view1 july 2012, 24 months. distribution. public (internal version)...

Documents