mlif: the multi lingual information framework iso wd 24616 samuel cruz-lara...

MLIF: The Multi Lingual Information

FrameworkISO WD 24616

Samuel CRUZ-LARASamuel.Cruz-Lara@loria.frLORIA / INRIA, France

Pisa, 26/09/2008 ISO TC37/SC4 MLIF (S. Cruz-Lara) (c) ISO 2008

Outline

IntroductionMLIF [ISO WD 24616]Ongoing activitiesActionsDate Extension RequestConclusion

Introduction

The “Multi Lingual Information Framework” MLIFISO AWI 24616 (TC37/SC4 WG3)Nasredine Semmar (CEA, France), WG3 ConvenorSamuel Cruz-Lara (LORIA / INRIA, France), MLIF Project Leader

New Work Item Proposal

ISO TC37/SC4 MeetingChina, Beijing August 2006

ISO AWI 24616

This standard aims at proposing a specification platform for a computer-oriented representation of multilingual data within a large variety of applications such as translation memories, localization, computer-aided translation, multimedia, or electronic document management.

As with the “Terminological Markup Framework”, used in terminology [ISO 16642], the MLIF will introduce a metamodel in combination with chosen data categories that will be integrated within the TC37 Data Category Registry [ISO/DIS 12620.2] in order to allow the description of any specific domain.

The standard will thus provide a way to validate any instance of this metamodel, as well as, interoperability principles with numerous translation and localization standards.

Purpose and Justification

The extremely fast evolution of the technological development in the sector of Communication and Information Technologies, and in particular, in the field of natural language processing, makes particularly acute the question of standardization.The issues related to this standardization are of an industrial, economic and cultural nature.

The control of the interoperability between the industrial standards currently used for localization (XLIFF) , translation memory (TMX) , or any other Multi Lingual Markup Language (ML2), constitutes a major objective for a coherent and global management of these data.

The MLIF could be associated to several multimedia standards such as MPEG-4 [ISO/IEC 14496], MPEG-7 [ISO/IEC 15938], and W3C SMIL, in order to handle multilingual data within several multimedia applications such as, interactive TV, video conferencing, subtitling, karaoke and accessibility.

The MLIF may also be used in cultural heritage related activities such as, digital museums, e-learning and electronic document management.

As with the “Terminological Markup Framework” (TMF), used in terminology, the MLIF will introduce a metamodel in combination with chosen data categories.

These data categories will be derived as a subset of a Data Category Registry (DCR) [ISO/DIS 12620.2], in order to ensure interoperability between several multilingual applications and corpora.

A Data Category Specification (DCS) will define, in combination with the metamodel, the various constraints that apply to a given domain-specific information structure or interchange format.A DCS and a metamodel represent the organization of an individual application and the organization of a specific domain.

The MLIF should be considered as a unified conceptual representation of multilingual content.The MLIF is not meant to replace or to compete with any other existing standard such as TMX or XLIFF.

Rather, the MLIF is being designed with the objective of providing a common conceptual model and a platform allowing interoperability among several translation and localization standards, and by extension, their committed tools.

The asset of MLIF is the interoperability which allows experts to gather, under the same conceptual unit, various tools and representations related to multilingual data.In addition, MLIF will also make it possible to evaluate and to compare these multilingual resources and tools.

The description of all different XML elements will be done by using RelaxNG [ISO/IEC 19757-2] with the help of ODD, which is the creation and documentation language for XML schemas proposed by the TEI (Text Encoding Initiative).This follows a recent decision taken by the World Wide Web Consortium.

MLIF will also provide a variety of filters allowing transformations between several MLIF-compatible formats, as well as, several automatic validation tools.The adoption of this new work item proposal may allow to instantiate the business plan proposal of TC 37/SC4 in the perspective of creating a work specification on multilingual content (WG 3).

ISO WD 24616

Introduction

The scope of research and development in the localization and translation memory (TM) process development is very large, and numerous independent groups are working on these aspects, such as LISA, OASIS, W3C, ISO, etc.

Introduction

Under the guidance of the above-mentioned groups, many formats have been developed.Some of the major formats of specific interest for localization and TM are TMX (LISA/OSCAR) and XLIFF (OASIS).

Introduction

There are many identical requirements for all the formats irrespective of the differences in final output.For example, all the formats aim at being user-friendly, easy-to-learn, and at reusing existing databases or knowledge.

Introduction

All these formats work well in the specific field they are designed for, but they lack a synergy that would make them interoperable when using one type of information in a slightly different context, giving rise to the fear of competition between them.

As with “Terminological Markup Framework” TMF [ISO 16642] in terminology, MLIF will introduce a metamodel in combination with chosen data categories [ISO/DIS 12620.2], as a means of ensuring interoperability between several multilingual applications and corpora.

MLIF deals with multilingual corpora, multilingual fragments, and the translation relations between them.

In each domain where MLIF can be used, we may consider a specific granularity of segmentation and description, built on MAF [ISO CD 24611], Synaf [ISO CD 24615], TMF [ISO 16642] or LAF [ISO CD 24612] respectively, for morphological description, syntactical annotation, terminological description, and linguistic annotation.

MLIF will thus describe elementary linguistic segments (i.e. sentence, syntactical component, word, …).

Supporting the construction and the interoperability of localization and “Translation Memories” (TM) resources, MLIF deals with the description of a metamodel for multilingual content.

MLIF will not propose a closed list of description features.Rather, it will provide a list of Data Categories, which is much easier to update and extend.This list represents a point of reference for multilingual information in the context of various application scenarios.

Normative References

ISO 16642 Computer applications in terminology -- Terminological Markup Framework.ISO CD 24615 Syntactical Annotation Framework.ISO/DIS 12620.2 Computer applications in terminology -- Data categories.

Normative References

ISO/IEC 639-1, Information technology - ISO 639:1988, Code for the representation of names of languages.ISO 639-2 Code for the representation of names and languages-part 2:Alpha-3 code.ISO 8601 Data elements and interchange formats - Information interchange - Representation of dates and times.

Starting point of MLIF

The MLIF promotes the use of a common framework for the future development of several different formats: TMX, XLIFF, …The MLIF can be considered as a parent for all these formats, since all of them deal with multilingual data expressed in the form of segments or text units. They all can be stored, manipulated and translated in a similar manner.

MLIF Metamodel

Multi Lingual Data Collection (MLDC)Represents a collection of data containing global information and several multilingual units.

Global Information (GI)Represents technical and administrative information applying to the entire data collection. Example: title of the data collection, revision history, …

MLIF Metamodel

Multi Lingual Component (MULTI)This component represents a unique multilingual entry.

Mono Lingual Component (MONO)Part of a multilingual component containing information related to one language.

Segment Component (SEG)Textual content itself (may be “decorated” with several attributes as in SynAF [ISO CD 24615])

MLIF Metamodel

History ComponentThis generic component allows to trace modifications on the component it is anchored to (i.e. versioning).

MLIF Metamodel

In order to provide a larger description of the linguistic content, the MLIF metamodel allows anchoring of other metamodels, such as MAF (morphological description), SynAF (syntactical annotation), TMF (terminological description), LAF (linguistic annotation), or any other metamodel based on ISO 12620.2.

MLIF Metamodel

Data Categories for MLIF

All models have very similar hierarchical structure but they have different terms and methods of storing metadata relevant to them in particular.The MLIF provides a generic structure that can establish basic foundation for all other models.

Global Information (GI)/domain/

Specifies the domain on which the MLDC is dependent.

/project/Specifies a project within the domain on which the MLDC is dependent.

/source/ “A complete citation of the bibliographic information pertaining to a document or other resource. “[ISO12620] Reference to a resource from which the present resource is derived.

Global Information (GI)/sourceType/

“In multilingual and translation-oriented language resource or terminology management, the kind of text used to document the selection of lexical or terminological equivalents, collocations, and the like. “[ISO12620]“Both parallel and background texts serve as sources for information used in documenting multilingual terminology entries. “[ISO12620]

Global Information (GI)/sourceLanguage/

“In a translation-oriented language resource or terminology database, the language that is taken as the language in which the original text is written.” [ISO12620]

/note/This is an optional descriptor providing further information on any part of a content.

Multilingual Component (MULTI)/identifier/

A unique name [source:IMDI_Source_Tag]Dublin Core equivalent: DC:Identifier [source: IMDI_Source_Tag]XML equivalent “xml:id” [source: http://www.w3.org/TR/xml-id]

/class/A hierarchical high level description of the component it is anchored to.

Multilingual Component (MULTI)/subclass/

A hierarchical low level description of the component it is anchored to.

Monolingual Component (MONO)/languageIdentifier/

A unique identifier in a language resource entry that indicates the name of a language. [source:ISO12620]XML equivalent “xml:lang”

/languageLevel/ Specifies the language level of the unique language identifier associated to the monolingual component (e.g. adults, children, scientist, slang, …)

Monolingual Component (MONO)/identifier/

Monolingual Component (MONO)/class/

A hierarchical high level description of the component it is anchored to.

/subclass/A hierarchical low level description of the component it is anchored to.

Monolingual Component (MONO)/xlink/: Is a data category refinement composed by several data categories. It is used to identify a name or a resource:

/uri/: can be represented by an xlink:href attribute./type/: can be represented by an xlink:type attribute./label/: can be represented by an xlink:label attribute./title/: can be represented by an xlink:title attribute./from/: can be represented by an xlink:from attribute./to/: can be represented by an xlink:to attribute.

Data categories for MLIF

Segment Component (SEG)/identifier/

Segment Component (SEG)/class/

A hierarchical high level description of the component it is anchored to.

/subclass/A hierarchical low level description of the component it is anchored to.

Segment Component (SEG)/xlink/: Is a data category refinement composed by several data categories. It is used to identify a name or a resource:

/uri/: can be represented by an xlink:href attribute./type/: can be represented by an xlink:type attribute./label/: can be represented by an xlink:label attribute./title/: can be represented by an xlink:title attribute./from/: can be represented by an xlink:from attribute./to/: can be represented by an xlink:to attribute.

The HistoryComponent is a generic component allowing to trace modifications on the component it is anchored to (e.g., creation, modification, validation). It can be anchored onto any component of the metamodel.In MLIF metamodel, the HistoryComponent may be anchored to the GlobalInformation or to the MonoLingualComponent.

In the GlobalInformation component, the HistoryComponent keeps all information related to any modification on the context or on the domain;In the MonoLingualComponent, the HistoryComponent allows keeping all evolutions or any enhancement of the content.

HistoryComponent/transaction/

One of the steps involved in the creation, approval, and use of a specific component (approval, check, exportation, importation, input, modification, origination, standardization, userAccess, withdrawal).

HistoryComponent/date/

A date. The date is encoded according to a profile of [ISO8601] as described in [W3CDTF] and follows the YYYY-MM-DD format.

/author/The person responsible for the creation of the content.

MLIF example 1

MLIF example 2

MLIF example 3

Ongoing Activities

Ongoing activities

MLIF can be used in e-learning, interactive television programs and any other application having a user interface. It may be very helpful for future interactive television broadcasting.Within ITEA’s “Jules Verne” and “Passepartout” projects, we have identified several potential implementations of MLIF in association with interactive TV and multimedia.

Ongoing activities

The MLIF and multimedia association presents ample opportunity for giving value to different languages and cultures, as is the case in Europe and Asia.

Ongoing activities

Within the framework of ITEA “Passepartout” project, we have experimented with some basic scenarios by using:XMT (“eXtensible MPEG4 Textual format”) and,W3C SMIL (“Synchronized Multimedia Integration Language”).

Ongoing activities

We are currently collaborating with the W3C SYMM (SYnchronized MultiMedia) Working GroupSMIL (Synchronized Multimedia Integration Language)

SMILTextNew text container element

Multilinguality (yes)Linguistic granularity (no)

SMILText

This module defines new functionality for SMIL 3.0. It extends the media types available for SMIL, but does not alter any other existing functionality from SMIL 2.1 or earlier versions.

Editors: Dick Bulterman (CWI, The Netherlands), Sjoerd Mullender (CWI, The Netherlands), Samuel Cruz-Lara (LORIA / INRIA, France)TEXT IS BEAUTIFUL !

SMILText

The SMILText modules provide a text container element with an explicit content model for defining in-line text, and a set of additional elements and attributes to control explicit in-line text rendering.

SMILText

Since the SMILText elements and attributes are defined in a series of modules, designers of other markup languages can reuse these modules when they need to include a simple form of timed text functionality into their language.

Ongoing activities

We have recently joint four new projects where MLIF should be used:ITEA2 SEMbySEM (INRIA - S. Cruz-Lara)ITEA2 METAVERSE1 (INRIA - S. Cruz-Lara)FP7-ICT MEDAR (CEA - N. Semmar)ANR WEBCROSSLING (CEA - N. Semmar)

SEMbySEM

This project will provide an innovative and comprehensive Semantic Services Management System, based on a common open infrastructure, to allow management of tomorrow mixed systems (made of thousands elementary software and hardware components) with facilities to build ad-hoc dynamic visualisations of the managed systems for information, management and SLA verification purposes.

SEMbySEM

To do so, it will extensively use semantic description with the help of a « new standard » that it will provide.Keywords: Software Engineering, UML, OMG (SBVR, MOF, KDM, …Countries: Finland, France, and TurkeyINRIA’s contribution:

Multilingual Ontologies

METAVERSE1

The Metaverse1 (global standards among real and virtual worlds) project will provide a standardized global framework that enables the interoperability between virtual worlds (as for example Second Life, World of Warcraft, IMVU, Google Earth and many others) and the real world (sensors, actuators, vision and rendering, social and welfare systems, banking, insurance, travel, real estate and many others).

QuickTime™ et undécompresseur

sont requis pour visionner cette image.

METAVERSE1

The ‘Metaverse for all’ will be a special attention point aiming at the e-Inclusion of minorities in the society.Countries: Belgium, France, Germany, Greece, Israel, Luxembourg, The Netherlands, and SpainINRIA’s contribution:

Standardise the management and the representation of multilingual textual data

QuickTime™ et undécompresseur

sont requis pour visionner cette image.

FP7-ICT MEDAR

Supporting the development of tools and resources in Machine Translation and MultiLingual Information Retrieval on the basis of other partners technologies and open source codeCEA’s contribution:

Multilingual Information Retrieval

ANR WebCrossling

Developing a Machine Translation prototype based on MultiLingual Information Retrieval technologyCEA’s contribution:Multilingual Information Retrieval

Actions

MLIF is an ISO “Working Draft”[ISO WD 24616].

A Committee of Experts for MLIF has been constitutedThe proposal we have just presented needs comments, remarks, … so it will be shortly sent to the Committee of Experts

MLIF Experts Committee

Member bodiesAnnelies Kesting - NEN (The Netherlands)Bettina Seitl - ON (Austria)Young-Shik Kang - KATS (Republic of Korea)Marketa Jindrakova - CNI (Czech Republic)Roberto Ravaglia - UNI (Italy)Surayuth Boonmatat - TISI (Thailand)Toni Hittema - AFNOR (France)

ExpertsDewi Bryn Jones - Canolfan Bedwyr (UK)Elena Montiel - UPM (Spain)Elsa Sklavounou - SYSTRAN (France)Emmanuel Planas - Lingua et Machina (France)Felix Sasaki - W3CGerhard Budin - University of Vienna (Austria)Guadalupe Aguado de Cea - UPM (Spain)Harry Bunt - Tilburg University (The Netherlands)

ExpertsKey-Sun Choi - KORTERM/KAIST (Korea)Kiyong Lee - Korea University (Korea)Laurent Romary - Max Planck Digital Library (Germany), INRIA (France)Mourad Amine - Université de Montréal (Canada)Nasredine Semmar CEA (France)Nicoletta Calzolari ILC-CNR (Italy)

ExpertsSamuel Cruz-Lara - LORIA / INRIA (France)Stéphane Albucher - Business Objects (France)Wim Peters - University of Sheffield (UK)Yves Savourel ENLASO - (USA)Julien Ducret - SAFARI Consulting (France)

Actions

The basis of the discussion must be the new proposalMetamodel & Data Categories

Do we need to modify them?Do we need to take into account any new aspect?

How can we progress?Use cases!!!

Actions

Use CasesInteroperabilityPublishingLinguistic GranularitySegmentationRelated StandardsMultimediaMultilingual Ontologies…

Interoperability

TMX“the sentence contains different formatting information”

Interoperability

Interoperability TMX & MLIF

Publishing

Interoperability & Publishing

Linguistic Properties

Sentence, word, lemma, POS, …Time related issues?

Linguistic Properties

Example by: Nasredine Semmar

Segmentation

How segmentation issues will be taken into account by MLIF?

Related Standards

TEI (Text Encoding Initiative)The description of all different XML elements will be done by using RelaxNG [ISO 19757-2] with the help of ODD, which is the creation and documentation language for XML schemas proposed by the TEI.

Related Standards

W3C ITS (International Tag Set)ITS is a set of rules, expressed in elements, that provide information on how parts of a given DTD or XML Schema are related to specific internationalization & localization properties.Should ITS may be used inside MLIF (as ITS may be used in SMILText)?

Multimedia

W3C SMILSMILText

Multilinguality (yes)Linguistic Granularity (no)

Multimedia

MultimediaTimed, Multilingual Textual DescriptionsTimed, Multilingual Textual Descriptions

W3C SMIL Standardization- Development of Interactive TV Profile- Integration of Annotation Support- Definition of Temporal Text Processing

ISO MLIF Standardization- Development of MLIF format- Development of a multilingual processing pipeline- Interaction with SMIL and MPEG standards

multilingual componentmultilingual component

multilingual DBmultilingual DB

linguistic segmentlinguistic segment

l’histoire du courage d’une femme pour l’histoire du courage d’une femme pour démasquer un mystèredémasquer un mystère

Monolingual componentMonolingual component

linguistic segmentlinguistic segment

la historia da la valentía de una mujer para la historia da la valentía de una mujer para desenmascarar undesenmascarar un misteriomisterio

Monolingual componentMonolingual component

Multilingual Ontologies

In what way can MLIF be related to Multilingual Ontologies?ITEA2 SEMbySEM

Date Extension Request

MLIF [ISO CD 24616]

Current state:WarningUrgent actions to do?

Date Extension Request

Current Proposed

DIS 2008-10-13

2009-10-13

FDIS 2010-04-13

2011-04-13

IS 2010-10-13

2011-10-13

Conclusion

We suggest to develop several use cases in order to test and to validate the new metamodel and related data categoriesEach use case should be leaded by one or several members of the MLIF Experts Committee

Conclusion

A new working draft of MLIF, taken into account all proposed use cases, should be submitted to the Committee of Experts soon

Thank you!

Thank you for your attentionAny question?Mailing listmlif@loria.fr

Web sitehttp://mlif.loria.fr

mlif: the multi lingual information framework iso wd 24616 samuel cruz-lara...

cruzlara c iso

france iso tc37sc4 mlif

outlineintroductionmlif

rights reservedpisa

translation memories

numerous translation

localization standards

computeraided translation

Documents

spiral.imperial.ac.ukspiral.imperial.ac.uk/bitstream/10044/1/24616/2/1yruptur… ·...

dibujos lara

finlandia lara

dip - lara

lara (фитинги)

jimenez (lara)

carolina lara

karen lara

cristian lara

lara bohinc

j[mlif rf}dfl;saitc.gov.np/downloadfile/final agriculture...

lara 9 de juventino rosas agustin lara rosas

lara bezoz

lara waldner

earthquakes - lara

24616-j8711c 24616-j8711c 5/19/10 5:53 am page c1...

estado lara

historiaurrea (lara)

mlif: a metamodel to represent and exchange multilingual...

kívánok!! lara