The TAUS Dynamic Quality Dashboard An industry collaborative platform for

translation quality measurement and trackingwww.taus.net

What About Translation Quality

Old School: “One size fits all”Since the 1980’s

LISA QA Model, SAE J2450 prescribe today’s quality processes:

1. Static:• One quality fits all purposes, all content, all audiences

2. Subjective:• Evaluations are often subjective and anecdotal

3. Costly:• QE causes friction, delays• QE can cost up to 25% of total translation costs

4. Non-transparent:• Necessity without remedy

Dynamic Quality Framework

Industry Collaborative ProgramDQF started in 2011

AdobeAppenAutodeskAVBCA TechnologiesCiscoCrestecCrosslangDellDFKIeBay EMCGoogleHewlett PackardIntelLDS ChurchLingo24

This slide may not be used or copied without permission from TAUS

Participating members

LionbridgeMedtronicMicrosoftMoraviaNikonOraclePacteraPangeanicPaypalPhilipsPTCSiemensSpil GamesSystranVMwareWelocalizeYahoo!

The TAUS Dynamic Quality Framework

Content Profiling Select the most appropriate quality evaluation

model for specific requirements Knowledge Base

Find best practices, metrics, step-by-step guides, reference templates, and use cases

DQF Tools Post-edit machine-translated segments to track

productivity Evaluate adequacy and fluency Count errors based on an error-typology Rank machine translations

From DQF Tools to Quality Dashboard

DQF ToolsSince January 2014

Tools on TAUS web site:to measure:• Productivity• Adequacy• Fluency

to review and count:• Translation errors

to get:• Stats and reports

Used by 100+ members

Quality DashboardLaunched June 2015

DQF integrated in:• CAT Tools• TMS Systems

Use of DQF plug-in provides:• Enhanced statistics• Benchmarking

Open to everyone

The Power and Value of the Quality Dashboard

• DQF collects data and generates reports on the Dashboard real-time

• Translators, managers, buyers, developers get their own stats, benchmarks and analytics

• Not only track and benchmark against your own data, but also against industry averages, between translators, customers, projects, technologies

Quality Evaluation

Business Intelligence

Quality Dashboard Demo

Using

Reports in the Quality Dashboard

TAUS DQF SERVER

CAT Tool / editor Segment/timingTranslator

TMS / GMS CAT Tool

Project manager

Project settings

TAUS DQF Infrastructure

DQF AnalysisEngine

DQF Reporting

Engine

User Managemen

t

Translator Manager Buyer

QUALITY DASHBOARD

Milliseconds per segment Source segment Target segment Edited target segments Time Language pair Project key Translator key

DQF Data Instrumentation

Open API

Test Environment https://dqf.taus.net/assets/api/v1/index.html

Open API on GitHub http://github.com/TAUSBV/dqf-api Specification Test Code Documentation Issue Tracker

Available under the MIT Open Source License

Quality Dashboard Integrators

“Microsoft Office International team is committed to the DQF model and approach and are actively partnering with TAUS to investigate how best to integrate TAUS Quality Dashboard API into our translation tool set.”

The TAUS Efficiency ScoreIntroducing a new score for measuring productivity

Core variables: Words per Hour - WPH Edits per Hour - EPH

Efficiency = WPH + EPH Normalized using Min-Max

Credit: Nikos Argyropoulos Evolving

Absolute instead of relative? Adding optional variables like quality

Harmonized error-typology

DQF & MQM HarmonizationCooperation with DFKI to harmonize DQF with MQM and standardize Error categories and metrics. A deliverable in the EU project Q21.

Harmonized error-typology

Error Review in the Quality DashboardTime spent per task

Error Review in the Quality DashboardWeighted error distribution

Error Review in the Quality DashboardDistribution of errors per task

Error Review in the Quality DashboardDistribution of errors per task

Benefits of the Quality Dashboard

The benefits of the DQF-WorldServer integration are: Get insights in translation quality, productivity and efficiency by

translator/vendor/project/process/technology/content/language/time … and being able to compare and benchmark with industry and within your

own organization Leverage investments in QE tools and processes Reduce costs of quality review Standardize QE processes across industry Less friction and disputes Increased satisfaction and flexibility

The Quality Dashboard meets a growing demand for business intelligence and elevates the discussion about quality to a level encompassing the whole language industry.

This slide may not be used or copied without permission from TAUS

THANK YOU!