immc v3 communication protocol specifications

IMMC exchnage protocol specifications for NewCeresA.1 – Enterprise Architecture, Methods and Formats
Formats, Computer Linguistics and Metadata
IMMC v3
Communication protocol
DOCUMENT HISTORY
0.0.5 2015-11-27 Maria Kardami Release of draft version for validation
0.0.6 2016-01-08 Maria Kardami Integration of OP Units comments
0.0.7 2016-02-29 Maria Kardami Integration of A2 comments
0.0.8 2016-04-05 Maria Kardami Updated the documentation of the element cmt:environment
1.0.0 2016-04-08 Release of version 1.0.0
1.0.1 2017-02-06 Maria Kardami Updated section 3.3.1 to allow the addition of validation reports in the IMMC package, even if the result of the validation is successful Clarified the rule concerning copying the "cm:" elements from the sender's IMMC descriptor excluding the cm:extension element.
3 | P a g e
Contents
1 Introduction ..................................................................................................................................... 5
1.2 Scope ....................................................................................................................................... 5
2.1 IMMC package naming conventions ....................................................................................... 7
2.2 IMMC request types ................................................................................................................ 8
3 IMMC notifications .......................................................................................................................... 9
3.2 Technical validation ................................................................................................................. 9
3.3 Business validation ................................................................................................................ 15
3.4 Processing Status notification (response) ............................................................................. 23
4 | P a g e
Definitions, Abbreviations and Acronyms
Abbreviation Meaning
AT Authority Table produced by the Publications Office Metadata Registry team
FRBR Functional Requirements for Bibliographic Records
FTP File Transfer Protocol
PCS Production Control System
for the harmonisation and standardisation of metadata to facilitate the
information exchanges between and amongst the institutions at European level.
IMMC data channel Unique communication path between two stakeholders using the IMMC protocol.
IMMC package A ZIP format archive which must contain the IMMC descriptor file. The archive may optionally contain data files which must be referenced in the IMMC descriptor file.
IMMC transmission Exchange of IMMC packages over an IMMC data channel.
IMMC descriptor An XML file, which validates against the IMMC schema following the specific IMMC parsing rules.
Transmission identifier
(cmt:transmission @ id)
Identifier of the IMMC descriptor, which must be used by applications for a valid transmission follow up.
Applications must ensure its uniqueness within the IMMC data channel.
Technical validation Controls that concern the structure of the IMMC package (mandatory contents, IMMC package naming conventions and parsing of the IMMC descriptor).
Business validation Controls that concern the data included in the IMMC descriptor and any data files that are included in the IMMC package.
5 | P a g e
1 Introduction
1.1 Purpose of the Document
The purpose of this document is to describe the IMMC v3 communication protocol i.e. the messages
exchanged between the institutions, the Publications Office and its contractors for the transmission
of documents and their metadata.
1.2 Scope
The current specifications provide an overview of the messages exchanged in IMMC v3, but mainly
concern the technical and business validation messages as well the ones which indicate the final
status of an exchange e.g. after the publication of the resources on the internet by the Publications
Office.
2 IMMC v3 communication protocol overview
The IMMC communication protocol defines the exchanged message types between a sender and a
recipient for the completion of one-way transfers. The protocol is described in the diagram below:
RECIPIENT
SENDER
Parse
Figure 1: IMMC v3 communication protocol diagram
Senders transfer their IMMC packages over a data channel (FTP, e-TrustEx, etc.) and expect an
answer from the recipient over the same channel.
6 | P a g e
Recipients have to apply the following controls1 upon reception of an IMMC package:
Detect the IMMC descriptor. Each IMMC package shall contain only one file named after the
name of the package and bearing the extension "_immc.xml". This file is expected to be the
IMMC descriptor. In case the descriptor has not been detected the recipient has to notify the
sender that an invalid package has arrived either manually or automatically2. In case of an
automatic answer, the message will use the <cmt:receipt> element (cf. 3.2.2).
Parse the IMMC descriptor. The root element of the IMMC descriptor is expected to contain
the "schemaLocation" attribute, which shall point to the exact version of the schema to
which it conforms e.g. http://publications.europa.eu/mdr/resource/core-metadata-
schema/cm3-20160217-0/cdj/curia_transmission.xsd3. If the IMMC descriptor is not
containing the schemaLocation attribute, the parsing shall be done against a default IMMC
v3 schema, which has to be agreed in advance between the stakeholders. In case there are
parsing errors the recipient has to report back to the sender the parsing errors requesting
redelivery either manually or automatically. In case of an automatic answer, the message will
use the <cmt:receipt> element (cf. 3.2.2) and shall include the parser's report.
Send an (optional) reception notification. The reception notification will be sent using the
<cmt:receipt> element (cf. 3.2.1). It is strongly recommended to send the reception
notification only for packages that have successfully passed the parsing step in order to be
able to include also the business identifiers.
Validate the IMMC package, following the specific business automatic validation rules that
have been defined between the two stakeholders (sender and recipient). Senders and
recipients can decide on the specific automatic validation rules, which will be applied for
every particular data transfer. Successful validation notifications are optional but validation
errors shall be always reported. All automatic validation notifications will be sent using the
<cmt:receipt> element (cf. 3.3.1 and 3.3.2).
Process the IMMC package, following the specific processing rules that have been defined
between the stakeholders.
1 Failure of a step means that the subsequent steps cannot be executed.
2 Stakeholders have to agree in advance if these notifications will be done manually or automatically. They may
decide to use only manual notifications for the technical errors if they agree that these errors are rare, can
occur only during testing phases or, due to their importance and gravity, they shall be always validated
manually by a system operator (e.g. system technical problems shall always be addressed by systems or
network administrators and we cannot rely on an automatic answer in case such problems occur)
3 The IMMC XML schemas, together with the samples and the documentation are organized in different
directories, according to the domain (e.g. Case Law, General Publications etc.) and published on the MDR
website (http://publications.europa.eu/mdr/core-metadata/index.html). IMMC3 schemas are accessible online
Respond (optionally) back to the recipient providing processing status information. Process
status notifications will be sent using the specific response element of each stakeholder. The
response may consist of one or more IMMC packages (to be defined by the stakeholders).
2.1 IMMC package naming conventions
All messages exchanged within the context of the IMMC communication protocol have to be
wrapped in a ZIP archive, which filename must adhere to one of the following naming conventions:
{filename}.zip
{filename}_immc.zip
The IMMC descriptor of the IMMC package must be named {filename}_immc.xml
Every application that generates IMMC packages must ensure the uniqueness of the package
filename within the IMMC data channel.
The IMMC protocol recommends a generic naming convention for the {filename} part4 i.e.
<transmission id>_<date_time>_immc.zip
<transmission id>_<date_time>_immc.xml
YYYY Year 4 digits
MM Month 2 digits
DD Day 2 digits
hh Hour 2 digits
mm Minutes 2 digits
ss Seconds 2 digits
mmm Milliseconds 3 digits
4 Other naming convention rules for the {filename} part can be defined by the stakeholders.
8 | P a g e
2.2 IMMC request types
The IMMC request type is mainly defined with the element //cmt:workflow/cmt:phase_workflow.
The phase workflow indicates a step in the flow of the messages exchanged within a data channel. Its
use facilitates the handling and dispatching of the messages to the correct workflow.
It has been initially introduced in IMMC version 2 for indicating the different production steps of the
OJ production chain. In IMMC version 3 the list of values has been enhanced in order to cover other
production chains and workflows.
Each value can either be used as standalone and provide a specific handling instruction by itself or
require the use of additional fields to precise the request.
For example, the phase workflow "metadata" is not accompanied by additional publication request
instructions, is standalone and implies that the sender requests to update the metadata of a work in
CELLAR. On the contrary, the phase workflow "publication" is always accompanied by specific
instructions inserted in the *:t_publication_request_extension e.g. produce_formex, produce_pdf
etc.
The supported values and their usage are further explained in the table below:
phase_workflow Description Use cases
GenPub
early reading Request for the correction of a work prior to its publication.
Normally this phase is accompanied by a more precise request
in the IMMC descriptor i.e. with the element for_correction,
which can be also combined with an early diffusion on eur-lex.
caseLaw
delete Request to delete a work from CELLAR.
cancellation Request to cancel the production of a publication. caseLaw
publication reference Value used in response notifications for the final reference to
the publication in the CELLAR / EUR-Lex.
caseLaw
delivery Value used by the contractors when delivering the products
which OP has requested e.g. PDF/FORMEX, legal analysis,
summary of EU legislation.
validation status.
manual validation Value used in business notifications informing the manual
validation status.
redelivery Value used by the contractors when redelivering the products
which OP has rejected due to insufficient quality. The
stakeholders, based on the contractual agreement, will define
caseLaw
LA
LSEU
its specific semantics, the counting of attempts etc.
3 IMMC notifications
The <cmt:receipt> root element will be used for:
Reception (successful) notifications (technical ACK messages), without specific
indication on the validation or processing status of the package. It is expected,
however, that reception notifications correspond only to valid IMMC packages (i.e.
they have passed the parsing step).
Rejection notifications (technical NACK messages) due to early-processing
unrecoverable technical errors (missing IMMC descriptor, parsing errors, etc.). The
sending (either manual or automatic) of negative (NACK) replies is mandatory.
Both positive and negative business validation replies (ACK & NACK messages) that
are generated by a processing system as a reaction to an IMMC transmission:
o the sending of positive (ACK) replies is optional (unless explicitly requested for
a particular exchange from an IMMC stakeholder). Recipients may ignore
positive ACKs.
o the sending of negative (NACK) replies is mandatory, whenever an IMMC
package cannot be processed through its expected workflow. Recipients are
expected to always react on NACK messages.
The IMMC descriptors with the <cmt:receipt> root element must contain the business
identifiers when the replying system has correctly processed and identified them (i.e. for all
business validation replies).
In all cases the <cmt:response_transmission> element must indicate the transmission
identifier or the package filename to which an ACK/NACK message responds.
The validation reply shall be provided at the corresponding level
(transmission/workflow/work/expression/manifestation) to which it has been executed and
where the validation error occurred.
No data will be transmitted to the original sender. If this occurs it is implied that it
corresponds to an initiative message. The sender must keep the original packages at least
until a successful validation messages is transmitted.
3.2 Technical validation
3.2.1 Successful notifications (Technical ACK)
Scenario: The system has received an IMMC package. It was able to detect a valid IMMC
descriptor, recognize the business identifiers and wants to notify its reception
10 | P a g e
Actions
recipient responds to sender with a new IMMC package acknowledging the reception of the sender's IMMC package
sender N/A
The new IMMC package must contain only the IMMC descriptor, which consists of the
following elements:
Element Description
cmt:receipt Root element
@version_schema Optional attribute which can be used to indicate the version of the schema against which the IMMC descriptor is valid. If used, its value has to be the MDR release (specific version) identifier.
Example:
cmt:transmission
@id
The transmission identifier has to be unique within the IMMC data channel.
Each sender is responsible for the definition of the identifier values.
cmt:date_time Date and time of the transmission adhering to the format: YYYY-MM-DDThh:mm:ss.sss.
cmt:response_transmission
@type
Contains the filename of the original package (type="package") to which this new message replies to.
Example:
An IMMC stakeholder has transmitted an IMMC package with the filename {filename}.zip. This package filename will be the value of the element "cmt:response_transmission".
<cmt:response_transmission type="package">O01_61997CC0172_ELL_2015-02- 02T085546024_production_request_immc.zip</cmt:re sponse_transmission>
cmt:sender The stakeholders of the IMMC data channel have to agree on the exact values of its sub- elements.
cmt:institution Mandatory element which has to contain either:
- the code of a European institution or body (value derived from the at-corporate-body.xsd)
or
- "cmt:PRIVATE" (fixed value if the sender is a contractor)
cmt:service Mandatory element which has to contain the
name of the service sending the transmission (e.g. for Commission: SG Greffe A.1).
cmt:context Mandatory element which has to contain the
name of the application sending the transmission (e.g. for Commission: e-Greffe).
cmt:environment Optional element which has to contain one of
the values "ACCEPTANCE" or "PROD", for indicating the test or production environment respectively.
Both sender and recipient shall use the element when test data are exchanged. If the exchange concerns production data the element may not be transmitted.
cmext:name Mandatory element which has to contain a
physical person's name, ideally of the person who sent the transmission and who will be able to follow up any issues.
cmext:phone_number Optional element which has to contain a
physical person's phone number, ideally of the person who sent the transmission and who will be able to follow up any issues.
cmext:email_address Optional element which has to contain a
physical person's email address, ideally of the person who sent the transmission and who will be able to follow up any issues.
cmt:recipient Corresponds to the sender of the message to which we respond (see sub-elements below)
cmt:institution Copy value from the sender's IMMC descriptor
cmt:service Copy value from the sender's IMMC descriptor
cmt:context Copy value from the sender's IMMC descriptor
If present, copy value from the sender's IMMC descriptor
cmext:name Copy value from the sender's IMMC descriptor
cmext:phone_number If present, copy value from the sender's IMMC descriptor
cmext:email_address If present, copy value from the sender's IMMC descriptor
cmext:comment Optional element which the sender can use in
order to express any additional information concerning the transmission.
Example:
cmt:workflow
cmt:phase_workflow The element shall not be used
cm:work If present, copy from the sender's IMMC descriptor all values of its sub-elements which belong to the "cm:" namespace (except for the element cm:extension).
cm:procedure / cm:event / cm:work
If present, copy from the sender's IMMC descriptor all values of their sub-elements which belong to the "cm:" namespace (except for the element cm:extension).
3.2.2 Rejection notifications (Technical NACK)
Scenario: The system has received a digital object that was supposed to be an IMMC
package. It was not able to open the package (unzip failed), detect or parse the IMMC
descriptor, nor recognize the business identifiers
Actions
recipient responds to sender with a new IMMC package 5 indicating the problem
sender sends a corrected IMMC package
The new IMMC package must contain the IMMC descriptor and optionally a validation report.
The IMMC descriptor consists of the following elements:
Element Description
Example:
cmt:transmission
@id
5 In this scenario the IMMC descriptor is either missing or it is corrupted (not parsable IMMC descriptor file). It
is expected that, by using other means (i.e. data channel configuration) the recipient is able to identify the
original sender and that a reply via IMMC has to be provided (see also section 2).
13 | P a g e
@type
Example:
or
14 | P a g e
In this specific message, the element will not be used as there are more specific elements (validation extensions) to indicate any issues related to the transmission.
cmt:extension
@xsi:type="cmt:t_transmission_valid ation"
The validation extension must be added at the transmission level in order to notify the sender of the problem with the package. The value of the element cmext:outcome_validation must be selected from a closed list and set to "FAILED".
If it is possible to provide more information concerning the error, this can be either stated in the element cmext:validation/cmext:comment
or in an external file which filename must be referenced in the element cmext:validation/cmext:report_validation.
Example: <cmt:extension xsi:type="cmt:t_transmission_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation> <cmext:comment>No IMMC found in the package</cmext:comment>
</cmext:validation> </cmt:extension> <cmt:extension xsi:type="cmt:t_transmission_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation> <cmext:report_validation>{filename}.txt</cmext:report_validation>
cmt:workflow The element is mandatory, but it will be left
empty (<cmt:workflow/>) because no business workflow has been identified.
15 | P a g e
3.3 Business validation
3.3.1 Successful notifications (Business validation ACK)
Scenario: The system has received an IMMC package. It was able to detect an IMMC
descriptor, recognize the business identifiers and successfully validate content and metadata
adhering to the established validation rules
Actions
recipient responds to sender with a new IMMC package acknowledging the successful business level validation
sender N/A
The new IMMC package must contain the IMMC descriptor and optionally the necessary
validation report(s). The IMMC descriptor consists of the following elements:
Element Description
Example:
cmt:transmission
@id
@type
Example:
16 | P a g e
or
cmext:email_address If present, copy value from the sender's IMMC
17 | P a g e
descriptor
cmt:extension
The validation extension must be added at the
transmission level notifying that the IMMC package has passed all the business validation controls. Optionally, if the validation reports are required to be attached to the message, then these can be transmitted using the corresponding validation extension at the level to which they belong. The value of the element cmext:outcome_validation must be selected from a closed list and set to "PASSED".
Example: <cmt:extension xsi:type="cmt:t_transmission_validation"> <cmext:validation> <cmext:outcome_validation>PASSED</cmext:outcome_validation>
cmt:phase_workflow It will take the values: "automatic validation" when the message is
generated after automatic validation controls or "manual validation" when the message is
generated after manual verification controls
cmt:work_reference
@ref
If present, copy value from the sender's IMMC descriptor.
cm:work If present, copy from the sender's IMMC descriptor all values of its sub-elements which belong to the "cm:" namespace (except for the element cm:extension).
If applicable, add the validation extension at the relevant level (work/expression/manifestation).
cm:procedure / cm:event / cm:work If present, copy from the sender's IMMC descriptor all values of their sub-elements which belong to the "cm:" namespace (except for the element cm:extension). If applicable, add the validation extension at the relevant level (procedure/event/ work/expression/manifestation).
Additional information for the validation result can either be inserted in the comment field of
the validation extension (cf. case 1) or inserted in a validation report(s) file whose filename(s)
will be referenced in the element "cmext:report_validation" (cf. case 2).
18 | P a g e
Example:
OR
3.3.2 Rejection notifications (Business validation NACK)
descriptor, recognize the business identifiers and wants to notify the violation of the
established business validation rules either after automatic or manual validation
Actions
recipient responds to sender with a new IMMC package indicating the business level validation problems
sender sends a corrected IMMC package
The new IMMC package must contain the IMMC descriptor and the necessary validation
report(s). The IMMC descriptor consists of the following elements:
Element Description
Example:
xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance"
cmt:transmission
@id
@type
Example:
or
20 | P a g e
cmt:extension
@xsi:type="cmt:t_transmission_valid ation"
The validation extension notifies that the IMMC package has failed one or more of the business validation controls. Yet, it will be added at the transmission level, if and only if,
- the validation concerns the full package (e.g. naming convention of the package) or it was not possible to identify the exact level in the IMMC descriptor for adding the validation results, and - the result was negative i.e. "FAILED"
The value of the element cmext:outcome_validation must be selected from a closed list and set to "FAILED".
Example: <cmt:extension xsi:type="cmt:t_transmission_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
cmt:phase_workflow It will take the values: "automatic validation" when the message is
generated after automatic validation controls or "manual validation" when the message is
generated after manual verification controls
21 | P a g e
cmt:work_reference
@ref
cm:work If present:
1. copy from the sender's IMMC descriptor all values of its sub-elements which belong to the "cm:" namespace (except for the element cm:extension)
2. add validation extensions
a. one extension per work, if and only if, the
validation was performed at the work level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_work_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
</cmext:validation> </cm:extension>
b. one extension per expression, if and only if,
the validation was performed at the expression level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_expression_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
c. one extension per manifestation, if and only if, the validation was performed at the
manifestation level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_manifestation_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
cm:procedure / cm:event / cm:work If present:
1. copy from the sender's IMMC descriptor all values of their sub-elements which belong to the "cm:" namespace (except for the element cm:extension)
2. add validation extensions
a. one extension for the procedure, if and only if, the validation was performed at the procedure level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_procedure_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation> </cmext:validation> </cm:extension>
b. one extension for the event, if and only if,
the validation was performed at the event level and the result was negative i.e. "FAILED"
22 | P a g e
Example: <cm:extension xsi:type="cmext:t_event_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation> </cmext:validation> </cm:extension>
c. one extension per work, if and only if, the
validation was performed at the work level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_work_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
d. one extension per expression, if and only if,
the validation was performed at the expression level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_expression_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
e. one extension per manifestation, if and only if, the validation was performed at the
manifestation level and the result was negative i.e. "FAILED"
Example: <cm:extension xsi:type="cmext:t_manifestation_validation"> <cmext:validation> <cmext:outcome_validation>FAILED</cmext:outcome_validation>
Additional information for the validation result can either be inserted in the comment field of
the validation extension (cf. case 1) or inserted in a validation report(s) file whose filename(s)
will be referenced in the element "cmext:report_validation" (cf. case 2).
Example:
OR
23 | P a g e
3.4 Processing Status notification (response)
descriptor, recognize the business identifiers, successfully validate content and metadata
and complete the transaction
Actions
recipient responds to sender with a new IMMC package notifying the process status after the transaction completion
sender The transaction is considered closed. Any further action will have to trigger a new IMMC exchange (to be defined by the stakeholders)
The response IMMC package must contain the IMMC descriptor and optionally any other
data content as defined by the stakeholders. The IMMC descriptor consists of the following
elements:
The response root element will be different per stakeholder.
When OP is responding, the root element is: optrans:op_transmission_response
Example:
allowed namespaces xmlns:cmt="http://publications.europa.eu/resour ce/core-metadata-transmission/v3" xmlns:cm="http://publications.europa.eu/resourc e/core-metadata/v3" xmlns:cmext="http://publications.europa.eu/reso urce/core-metadata-extensions/v3" xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance"
When OP is responding, the additional two namespaces have to be added:
xmlns:opext="urn:eu:op:extensions" xmlns:optrans="urn:eu:op:transmission:response "
cmt:transmission
@id
cmt:response_transmission Contains the filename of the original package
24 | P a g e
@type (type="package") to which this new message replies to. The idea is that the system replies to the processed IMMC package.
Example:
or
cmt:recipient Corresponds to the sender of the processed message (see sub-elements below)
the values "ACCEPTANCE" or "PROD", for indicating the test or production environment
25 | P a g e
respectively.
cmt:workflow
cmt:phase_workflow To be defined by the stakeholders. When OP is responding the following values apply: "early reading": after proofreading "publication reference": after metadata
ingestion and/or work ingestion in CELLAR
cmt:work_reference
@ref
cm:work If present, copy from the sender's IMMC descriptor all values of its sub-elements which belong to the "cm:" namespace (except for the element cm:extension) up to the expression level.
When the response concerns the early reading, the "cmext:t_work_common_extension" has to be copied form the sender's IMMC and a new extension "opext:t_expression_early_reading" has to be added. A new DOC manifestation has to be added referencing the corrected document.
When the response concerns the publication request, the extension "opext:t_work_case_report_reference" at the expression level and the extension "opext:t_manifestation_publication_reference" at the manifestation level have to be added. New manifestations have to be added referencing the CELLAR URLs.
When the response concerns the metadata no additional information will be provided, the publication reference workflow implies the ingestion and publication of the metadata.
cm:procedure / cm:event / cm:work If present, copy from the sender's IMMC descriptor all values of their sub-elements which belong to the "cm:" namespace (except for the element cm:extension) and follow the instructions defined for the level of the cm:work.
Contents
1.2 Scope
2.1 IMMC package naming conventions
2.2 IMMC request types
3.3 Business validation
3.4 Processing Status notification (response)

immc v3 communication protocol specifications

Documents