mlif: the multi lingual information framework iso wd 24616 samuel cruz-lara...

MLIF: The Multi Lingual Information

FrameworkISO WD 24616

Samuel [email protected] / INRIA, France

mailto:[email protected]

Pisa, 26/09/2008 ISO TC37/SC4 MLIF (S. Cruz-Lara) (c) ISO 2008

All rights reserved

2

Outline

IntroductionMLIF [ISO WD 24616]Ongoing activitiesActionsDate Extension RequestConclusion

Introduction


All rights reserved

4

Introduction

The “Multi Lingual Information Framework” MLIFISO AWI 24616 (TC37/SC4 WG3)Nasredine Semmar (CEA, France), WG3 ConvenorSamuel Cruz-Lara (LORIA / INRIA, France), MLIF Project Leader


All rights reserved

5

New Work Item Proposal

ISO TC37/SC4 MeetingChina, Beijing August 2006

ISO AWI 24616


All rights reserved

6

Scope

This standard aims at proposing a specification platform for a computer-oriented representation of multilingual data within a large variety of applications such as translation memories, localization, computer-aided translation, multimedia, or electronic document management.


All rights reserved

7

Scope

As with the “Terminological Markup Framework”, used in terminology [ISO 16642], the MLIF will introduce a metamodel in combination with chosen data categories that will be integrated within the TC37 Data Category Registry [ISO/DIS 12620.2] in order to allow the description of any specific domain.


All rights reserved

8

Scope

The standard will thus provide a way to validate any instance of this metamodel, as well as, interoperability principles with numerous translation and localization standards.


All rights reserved

9

Purpose and Justification

The extremely fast evolution of the technological development in the sector of Communication and Information Technologies, and in particular, in the field of natural language processing, makes particularly acute the question of standardization.The issues related to this standardization are of an industrial, economic and cultural nature.


All rights reserved

10


The control of the interoperability between the industrial standards currently used for localization (XLIFF) , translation memory (TMX) , or any other Multi Lingual Markup Language (ML2), constitutes a major objective for a coherent and global management of these data.


All rights reserved

11


The MLIF could be associated to several multimedia standards such as MPEG-4 [ISO/IEC 14496], MPEG-7 [ISO/IEC 15938], and W3C SMIL, in order to handle multilingual data within several multimedia applications such as, interactive TV, video conferencing, subtitling, karaoke and accessibility.


All rights reserved

12


The MLIF may also be used in cultural heritage related activities such as, digital museums, e-learning and electronic document management.


All rights reserved

13


As with the “Terminological Markup Framework” (TMF), used in terminology, the MLIF will introduce a metamodel in combination with chosen data categories.


All rights reserved

14


These data categories will be derived as a subset of a Data Category Registry (DCR) [ISO/DIS 12620.2], in order to ensure interoperability between several multilingual applications and corpora.


All rights reserved

15


A Data Category Specification (DCS) will define, in combination with the metamodel, the various constraints that apply to a given domain-specific information structure or interchange format.A DCS and a metamodel represent the organization of an individual application and the organization of a specific domain.


All rights reserved

16


The MLIF should be considered as a unified conceptual representation of multilingual content.The MLIF is not meant to replace or to compete with any other existing standard such as TMX or XLIFF.


All rights reserved

17


Rather, the MLIF is being designed with the objective of providing a common conceptual model and a platform allowing interoperability among several translation and localization standards, and by extension, their committed tools.


All rights reserved

18


The asset of MLIF is the interoperability which allows experts to gather, under the same conceptual unit, various tools and representations related to multilingual data.In addition, MLIF will also make it possible to evaluate and to compare these multilingual resources and tools.


All rights reserved

19


The description of all different XML elements will be done by using RelaxNG [ISO/IEC 19757-2] with the help of ODD, which is the creation and documentation language for XML schemas proposed by the TEI (Text Encoding Initiative).This follows a recent decision taken by the World Wide Web Consortium.


All rights reserved

20


MLIF will also provide a variety of filters allowing transformations between several MLIF-compatible formats, as well as, several automatic validation tools.The adoption of this new work item proposal may allow to instantiate the business plan proposal of TC 37/SC4 in the perspective of creating a work specification on multilingual content (WG 3).

MLIF

ISO WD 24616


All rights reserved

22

Introduction

The scope of research and development in the localization and translation memory (TM) process development is very large, and numerous independent groups are working on these aspects, such as LISA, OASIS, W3C, ISO, etc.


All rights reserved

23

Introduction

Under the guidance of the above-mentioned groups, many formats have been developed.Some of the major formats of specific interest for localization and TM are TMX (LISA/OSCAR) and XLIFF (OASIS).


All rights reserved

24

Introduction

There are many identical requirements for all the formats irrespective of the differences in final output.For example, all the formats aim at being user-friendly, easy-to-learn, and at reusing existing databases or knowledge.


All rights reserved

25

Introduction

All these formats work well in the specific field they are designed for, but they lack a synergy that would make them interoperable when using one type of information in a slightly different context, giving rise to the fear of competition between them.


All rights reserved

26

Scope

As with “Terminological Markup Framework” TMF [ISO 16642] in terminology, MLIF will introduce a metamodel in combination with chosen data categories [ISO/DIS 12620.2], as a means of ensuring interoperability between several multilingual applications and corpora.


All rights reserved

27

Scope

MLIF deals with multilingual corpora, multilingual fragments, and the translation relations between them.


All rights reserved

28

Scope

In each domain where MLIF can be used, we may consider a specific granularity of segmentation and description, built on MAF [ISO CD 24611], Synaf [ISO CD 24615], TMF [ISO 16642] or LAF [ISO CD 24612] respectively, for morphological description, syntactical annotation, terminological description, and linguistic annotation.


All rights reserved

29

Scope

MLIF will thus describe elementary linguistic segments (i.e. sentence, syntactical component, word, …).


All rights reserved

30

Scope

Supporting the construction and the interoperability of localization and “Translation Memories” (TM) resources, MLIF deals with the description of a metamodel for multilingual content.


All rights reserved

31

Scope

MLIF will not propose a closed list of description features.Rather, it will provide a list of Data Categories, which is much easier to update and extend.This list represents a point of reference for multilingual information in the context of various application scenarios.


All rights reserved

32

Normative References

ISO 16642 Computer applications in terminology -- Terminological Markup Framework.ISO CD 24615 Syntactical Annotation Framework.ISO/DIS 12620.2 Computer applications in terminology -- Data categories.


All rights reserved

33

Normative References

ISO/IEC 639-1, Information technology - ISO 639:1988, Code for the representation of names of languages.ISO 639-2 Code for the representation of names and languages-part 2:Alpha-3 code.ISO 8601 Data elements and interchange formats - Information interchange - Representation of dates and times.


All rights reserved

34

Starting point of MLIF

The MLIF promotes the use of a common framework for the future development of several different formats: TMX, XLIFF, …The MLIF can be considered as a parent for all these formats, since all of them deal with multilingual data expressed in the form of segments or text units. They all can be stored, manipulated and translated in a similar manner.


All rights reserved

35

MLIF Metamodel


All rights reserved

36

MLIF Metamodel

Multi Lingual Data Collection (MLDC)Represents a collection of data containing global information and several multilingual units.

Global Information (GI)Represents technical and administrative information applying to the entire data collection. Example: title of the data collection, revision history, …


All rights reserved

37

MLIF Metamodel

Multi Lingual Component (MULTI)This component represents a unique multilingual entry.

Mono Lingual Component (MONO)Part of a multilingual component containing information related to one language.

Segment Component (SEG)Textual content itself (may be “decorated” with several attributes as in SynAF [ISO CD 24615])


All rights reserved

38

MLIF Metamodel

History ComponentThis generic component allows to trace modifications on the component it is anchored to (i.e. versioning).


All rights reserved

39

MLIF Metamodel

In order to provide a larger description of the linguistic content, the MLIF metamodel allows anchoring of other metamodels, such as MAF (morphological description), SynAF (syntactical annotation), TMF (terminological description), LAF (linguistic annotation), or any other metamodel based on ISO 12620.2.


All rights reserved

40

MLIF Metamodel


All rights reserved

41

Data Categories for MLIF

All models have very similar hierarchical structure but they have different terms and methods of storing metadata relevant to them in particular.The MLIF provides a generic structure that can establish basic foundation for all other models.


All rights reserved

42



All rights reserved

43


Global Information (GI)/domain/

Specifies the domain on which the MLDC is dependent.

/project/Specifies a project within the domain on which the MLDC is dependent.

/source/ “A complete citation of the bibliographic information pertaining to a document or other resource. “[ISO12620] Reference to a resource from which the present resource is derived.


All rights reserved

44


Global Information (GI)/sourceType/

“In multilingual and translation-oriented language resource or terminology management, the kind of text used to document the selection of lexical or terminological equivalents, collocations, and the like. “[ISO12620]“Both parallel and background texts serve as sources for information used in documenting multilingual terminology entries. “[ISO12620]


All rights reserved

45


Global Information (GI)/sourceLanguage/

“In a translation-oriented language resource or terminology database, the language that is taken as the language in which the original text is written.” [ISO12620]

/note/This is an optional descriptor providing further information on any part of a content.


All rights reserved

46


Multilingual Component (MULTI)/identifier/

A unique name [source:IMDI_Source_Tag]Dublin Core equivalent: DC:Identifier [source: IMDI_Source_Tag]XML equivalent “xml:id” [source: http://www.w3.org/TR/xml-id]

/class/A hierarchical high level description of the component it is anchored to.


All rights reserved

47


Multilingual Component (MULTI)/subclass/

A hierarchical low level description of the component it is anchored to.



All rights reserved

48


Monolingual Component (MONO)/languageIdentifier/

A unique identifier in a language resource entry that indicates the name of a language. [source:ISO12620]XML equivalent “xml:lang”

/languageLevel/ Specifies the language level of the unique language identifier associated to the monolingual component (e.g. adults, children, scientist, slang, …)


All rights reserved

49


Monolingual Component (MONO)/identifier/



All rights reserved

50


Monolingual Component (MONO)/class/

A hierarchical high level description of the component it is anchored to.

/subclass/A hierarchical low level description of the component it is anchored to.


All rights reserved

51


Monolingual Component (MONO)/xlink/: Is a data category refinement composed by several data categories. It is used to identify a name or a resource:

/uri/: can be represented by an xlink:href attribute./type/: can be represented by an xlink:type attribute./label/: can be represented by an xlink:label attribute./title/: can be represented by an xlink:title attribute./from/: can be represented by an xlink:from attribute./to/: can be represented by an xlink:to attribute.


All rights reserved

52

Data categories for MLIF

Segment Component (SEG)/identifier/



All rights reserved

53


Segment Component (SEG)/class/

A hierarchical high level description of the component it is anchored to.

/subclass/A hierarchical low level description of the component it is anchored to.


All rights reserved

54


Segment Component (SEG)/xlink/: Is a data category refinement composed by several data categories. It is used to identify a name or a resource:

/uri/: can be represented by an xlink:href attribute./type/: can be represented by an xlink:type attribute./label/: can be represented by an xlink:label attribute./title/: can be represented by an xlink:title attribute./from/: can be represented by an xlink:from attribute./to/: can be represented by an xlink:to attribute.


All rights reserved

55


The HistoryComponent is a generic component allowing to trace modifications on the component it is anchored to (e.g., creation, modification, validation). It can be anchored onto any component of the metamodel.In MLIF metamodel, the HistoryComponent may be anchored to the GlobalInformation or to the MonoLingualComponent.


All rights reserved

56


In the GlobalInformation component, the HistoryComponent keeps all information related to any modification on the context or on the domain;In the MonoLingualComponent, the HistoryComponent allows keeping all evolutions or any enhancement of the content.


All rights reserved

57


HistoryComponent/transaction/

One of the steps involved in the creation, approval, and use of a specific component (approval, check, exportation, importation, input, modification, origination, standardization, userAccess, withdrawal).


All rights reserved

58


HistoryComponent/date/

A date. The date is encoded according to a profile of [ISO8601] as described in [W3CDTF] and follows the YYYY-MM-DD format.

/author/The person responsible for the creation of the content.


MLIF example 1

MLIF example 2

MLIF example 3

Ongoing Activities


All rights reserved

63

Ongoing activities

MLIF can be used in e-learning, interactive television programs and any other application having a user interface. It may be very helpful for future interactive television broadcasting.Within ITEA’s “Jules Verne” and “Passepartout” projects, we have identified several potential implementations of MLIF in association with interactive TV and multimedia.


All rights reserved

64

Ongoing activities

The MLIF and multimedia association presents ample opportunity for giving value to different languages and cultures, as is the case in Europe and Asia.


All rights reserved

65

Ongoing activities

Within the framework of ITEA “Passepartout” project, we have experimented with some basic scenarios by using:XMT (“eXtensible MPEG4 Textual format”) and,W3C SMIL (“Synchronized Multimedia Integration Language”).


All rights reserved

66

Ongoing activities

We are currently collaborating with the W3C SYMM (SYnchronized MultiMedia) Working GroupSMIL (Synchronized Multimedia Integration Language)

SMILTextNew text container element

Multilinguality (yes)Linguistic granularity (no)


All rights reserved

67

SMILText

This module defines new functionality for SMIL 3.0. It extends the media types available for SMIL, but does not alter any other existing functionality from SMIL 2.1 or earlier versions.

Editors: Dick Bulterman (CWI, The Netherlands), Sjoerd Mullender (CWI, The Netherlands), Samuel Cruz-Lara (LORIA / INRIA, France)TEXT IS BEAUTIFUL !


All rights reserved

68

SMILText

The SMILText modules provide a text container element with an explicit content model for defining in-line text, and a set of additional elements and attributes to control explicit in-line text rendering.


All rights reserved

69

SMILText

Since the SMILText elements and attributes are defined in a series of modules, designers of other markup languages can reuse these modules when they need to include a simple form of timed text functionality into their language.


All rights reserved

71

Ongoing activities

We have recently joint four new projects where MLIF should be used:ITEA2 SEMbySEM (INRIA - S. Cruz-Lara)ITEA2 METAVERSE1 (INRIA - S. Cruz-Lara)FP7-ICT MEDAR (CEA - N. Semmar)ANR WEBCROSSLING (CEA - N. Semmar)


All rights reserved

72

SEMbySEM

This project will provide an innovative and comprehensive Semantic Services Management System, based on a common open infrastructure, to allow management of tomorrow mixed systems (made of thousands elementary software and hardware components) with facilities to build ad-hoc dynamic visualisations of the managed systems for information, management and SLA verification purposes.


All rights reserved

73

SEMbySEM

To do so, it will extensively use semantic description with the help of a « new standard » that it will provide.Keywords: Software Engineering, UML, OMG (SBVR, MOF, KDM, …Countries: Finland, France, and TurkeyINRIA’s contribution:

Multilingual Ontologies


All rights reserved

74

METAVERSE1

The Metaverse1 (global standards among real and virtual worlds) project will provide a standardized global framework that enables the interoperability between virtual worlds (as for example Second Life, World of Warcraft, IMVU, Google Earth and many others) and the real world (sensors, actuators, vision and rendering, social and welfare systems, banking, insurance, travel, real estate and many others).

QuickTime™ et undécompresseur

sont requis pour visionner cette image.


All rights reserved

75

METAVERSE1

The ‘Metaverse for all’ will be a special attention point aiming at the e-Inclusion of minorities in the society.Countries: Belgium, France, Germany, Greece, Israel, Luxembourg, The Netherlands, and SpainINRIA’s contribution:

Standardise the management and the representation of multilingual textual data

QuickTime™ et undécompresseur

sont requis pour visionner cette image.


All rights reserved

76

FP7-ICT MEDAR

Supporting the development of tools and resources in Machine Translation and MultiLingual Information Retrieval on the basis of other partners technologies and open source codeCEA’s contribution:

Multilingual Information Retrieval

Actions


All rights reserved

79

Actions

MLIF is an ISO “Working Draft”[ISO WD 24616].

A Committee of Experts for MLIF has been constitutedThe proposal we have just presented needs comments, remarks, … so it will be shortly sent to the Committee of Experts


All rights reserved

80

MLIF Experts Committee

Member bodiesAnnelies Kesting - NEN (The Netherlands)Bettina Seitl - ON (Austria)Young-Shik Kang - KATS (Republic of Korea)Marketa Jindrakova - CNI (Czech Republic)Roberto Ravaglia - UNI (Italy)Surayuth Boonmatat - TISI (Thailand)Toni Hittema - AFNOR (France)


All rights reserved

81


ExpertsDewi Bryn Jones - Canolfan Bedwyr (UK)Elena Montiel - UPM (Spain)Elsa Sklavounou - SYSTRAN (France)Emmanuel Planas - Lingua et Machina (France)Felix Sasaki - W3CGerhard Budin - University of Vienna (Austria)Guadalupe Aguado de Cea - UPM (Spain)Harry Bunt - Tilburg University (The Netherlands)


All rights reserved

82


ExpertsKey-Sun Choi - KORTERM/KAIST (Korea)Kiyong Lee - Korea University (Korea)Laurent Romary - Max Planck Digital Library (Germany), INRIA (France)Mourad Amine - Université de Montréal (Canada)Nasredine Semmar CEA (France)Nicoletta Calzolari ILC-CNR (Italy)


All rights reserved

83


ExpertsSamuel Cruz-Lara - LORIA / INRIA (France)Stéphane Albucher - Business Objects (France)Wim Peters - University of Sheffield (UK)Yves Savourel ENLASO - (USA)Julien Ducret - SAFARI Consulting (France)


All rights reserved

84

Actions

The basis of the discussion must be the new proposalMetamodel & Data Categories

Do we need to modify them?Do we need to take into account any new aspect?

How can we progress?Use cases!!!

Interoperability TMX & MLIF

Linguistic Properties

Example by: Nasredine Semmar


All rights reserved

95

Related Standards

TEI (Text Encoding Initiative)The description of all different XML elements will be done by using RelaxNG [ISO 19757-2] with the help of ODD, which is the creation and documentation language for XML schemas proposed by the TEI.


All rights reserved

96

Related Standards

W3C ITS (International Tag Set)ITS is a set of rules, expressed in elements, that provide information on how parts of a given DTD or XML Schema are related to specific internationalization & localization properties.Should ITS may be used inside MLIF (as ITS may be used in SMILText)?

Multimedia

MultimediaTimed, Multilingual Textual DescriptionsTimed, Multilingual Textual Descriptions

W3C SMIL Standardization- Development of Interactive TV Profile- Integration of Annotation Support- Definition of Temporal Text Processing

ISO MLIF Standardization- Development of MLIF format- Development of a multilingual processing pipeline- Interaction with SMIL and MPEG standards

multilingual componentmultilingual component

multilingual DBmultilingual DB

linguistic segmentlinguistic segment

l’histoire du courage d’une femme pour l’histoire du courage d’une femme pour démasquer un mystèredémasquer un mystère

Monolingual componentMonolingual component

linguistic segmentlinguistic segment

la historia da la valentía de una mujer para la historia da la valentía de una mujer para desenmascarar undesenmascarar un misteriomisterio

Monolingual componentMonolingual component

Date Extension Request

Conclusion


All rights reserved

106

Conclusion

We suggest to develop several use cases in order to test and to validate the new metamodel and related data categoriesEach use case should be leaded by one or several members of the MLIF Experts Committee

mlif: the multi lingual information framework iso wd 24616 samuel cruz-lara...

Documents

cruzlara c iso

france iso tc37sc4 mlif

outlineintroductionmlif

rights reservedpisa

translation memories

numerous translation

localization standards

computeraided translation