d3.2 guidelines on interoperability...

D3.2 – Guidelines on Interoperability Assessment

of 56

Document Information

Workpackage WP.3

Leading partner ISCIII

Due Date 20

Type Report

Status Draft – v2.0

Security Public

Project URL http://stopandgoproject.eu/


of 56

This document is produced by the project: STOPandGO, 621013

CIP-ICT-PSP 7th call for proposals, 2013: Public Procurement of Innovative solutions (PPI) pilot.

ICT PSP Objective: 3.2 Supporting innovative solutions in eHealth, assisted living and for mobility, b) Active and healthy ageing and assisted living.

No. Participant organisation name Participant

short name Country Start date End date

1 Stichting Smart Homes SMH Netherlands

01-04-2014

2 Telecare Services Association TSA UK 01-04-2014

3 Instituto de Salud Carlos III ISCIII Spain 01-04-2014

4 North West Coast Academic Health Science Network

NWCAHSN UK 01-04-2014

5 Knowledge Transfer Network Limited KTN UK 01-04-2014

6** London School of Economics Enterprise LSEE UK 01-04-2014 31-05-2015

8 Federsanita Servizi FSA Italy 01-04-2014

9* Agenzia Regionale Sanitaria della Regione Campania

ARSAN Italy 01-04-2014

10** Azienda Unità Sanitaria Locale of Roma D RMD Italy 01-04-2014 01-04-2017

11* Azienda Sanitaria Provinciale of Catanzaro

CZ Italy 01-04-2014 18-04-2017

12* Eastern Cheshire Clinical Commissioning Group

EC UK 01-04-2014

13* Gemeente Helmond GMH Netherlands

01-04-2014

14 Agència de Qualitat i Avaluació Sanitàries de Catalunya

AQUAS Spain 12-10-2014

15** Junta de Comunidades de Castilla-la Mancha

CLM Spain 12-10-2014 08-02-2016

16** Servei Catala de la Salut CHS Spain 12-10-2014 30-11-2015

17* Fundacio Privada de Gestio Sanitaria de l’Hospital de la Santa Creu i Sant Pau

HSP Spain 15-10-2015

18 El Sitio de Valdelatarra S.L. VALDE Spain 01-06-2015

19 Liverpool City Council LCC UK 24-02-2016

20*** Servicio Aragones de la Salud ARAGON Spain 30-12-2016

21*** Azienda Unita Sanitaria Locale Toscana Sud-Est

ASL TSE Italy 15-12-2016

(*) Partners of the Consortium that are going to launch local tenders.

(**) Withdrawn partner

(***) In amendment

Versioning and contribution history

Version Description Comments

0.1 Structure of the document and task assignment AM, OM

0.2 Corrections and modifications: new structure AM, OM, CH

2.0 Updated IMM and questionnaire changes and other changes AM, OM


of 56

Index 1. Introduction ........................................................................................................... 4

2. Electronic Public Services in Europe ...................................................................... 4

3. Interoperability of European Public Services ........................................................ 6

4. Commission Interoperability Initiatives ................................................................ 7

5. Interoperability Maturity Models ........................................................................ 10

6 European Interoperability Maturity Model (IMM) ............................................. 13

6.1 Introduction.................................................................................................. 13

6.2 Objectives ..................................................................................................... 16

6.3 Maturity levels.............................................................................................. 17

6.4 Areas of Interoperability .............................................................................. 17

6.5 Interoperability Attributes ........................................................................... 21

6.6 Questionnaire ............................................................................................... 22

7 European Interoperability Maturity Model in STOP&GO Project ....................... 23

Annex A – Questionnaires .......................................................................................... 26

1. Service Context (A) ....................................................................................... 26

2. Service Delivery (B)....................................................................................... 29

3. Service Consumption (C) .............................................................................. 34

4. Service Management (D) .............................................................................. 42

5. Service conformance to STOP&GO Interoperability Framework (E) ........... 49

Bibliography .................................................................................................................... 54


of 56

1. Introduction This deliverable will provide the guidelines on interoperability assessment for deployments, following the recommendations of ISA (Interoperability Solutions for European Public Administrations); in particular, the Interoperability Maturity Model, and taking as a starting point the STOPandGO Interoperability Framework1 and the ISA European Interoperability Framework. Interoperability deals with the possibility to facilitate the model’s interoperations among all system types. The difficultness to develop a general method of measuring interoperability is well known. Although some models for interoperability assessment have been created in different countries during the past years, those models are for a specific field, mainly at the enterprise environment. The model proposed takes into account the Commission’s approach in these areas, with many different sectors institutions involved; it was developed during the last years, and it takes into account the common minimum recommendations which were developed at Deliverable 3.1 Interoperability requirements: STOPandGO Interoperability Framework. This document defines the Interoperability Assessment guidelines for the different stages of the procurement process, from the inception of the system to the evaluation of the implemented pilots after tendering. This task will continue with the analysis of performance of the STOPandGO Interoperability Framework in the next deliverable, based on the guidelines presented in this document. The deliverable begins defining an introduction to electronic public services in Europe

and their interoperability (sections 2 and 3 respectively); section 4 analyses briefly the

European Commission interoperability initiatives focused on ISA and ISA2; section 5 is

about the concept of Interoperability Maturity Models; section 6 reviews the European

Interoperability Maturity Model and, finally, section 7 presents the European

Interoperability Maturity Model from the STOPandGO Project perspective. The

appendix includes the questionnaires of the Interoperability Maturity Model and the

S&G Interoperability Framework at the end of the document.

2. Electronic Public Services in Europe The launch of the European strategy for the development of e-government was the “e-Europe 2002” initiative. The main objective for e-government being that Member States should ensure “generalized electronic access to main basic public services”. Concerning interactive public services the objective was that “the Member States should have ensured that basic public services are interactive, where relevant,

1 D3.1 – Interoperability requirements: STOPandGO Interoperability Framework


of 56

accessible for all, and exploit both the potential of broadband networks and of multi-platform access”. When the e-Europe programme was launched, the European Commission was aware that the regulatory framework required to undertake the implementation of some successed e-Europe initiative, was lacking. The Commission began the process of defining the indicators necessary to carry out the evaluation. A list of indicators was approved by the Council of Internal Market Ministers in November 2000. In order to specify how the indicator “Percentage of basic public services available online” had to be measured, Member States have agreed on a common list of twenty public services (12 for citizens and 8 for enterprises) for which the online sophistication is being benchmarked at national level. [1] [2] [3] Depending on the way public administrations are organised, a given Government service may imply either a single process or several business processes to be performed in a given sequence between different administrations. This is true both at the national level and the pan-European level, which is the concern of the European Interoperability Framework. eGovernment services provided in a pan-European context will rely upon the interaction between public administrations from different Member States and EU Institutions. [4] [5] The table below shows the twenty basic public services proposed by the Commission. These services already appear in the deliverable D3.1- STOP&GO Interoperability Framework. Table 1 - Twenty basic Public Services

Public Services For Citizens

Public Services For Business

1 - Income taxes: declaration, notification of assessment

1 - Social contribution for employees

2 - Job search services by labour offices 2 - Corporation tax: declaration, notification

3 - Social security contributions (3 out of the following 4) • Unemployment benefits • Child allowances • Medical costs (reimbursement or direct

settlement) • Student grants

3 - VAT: declaration, notification

4 - Personal documents (passport and driving licence) 4 - Registration of a new company

5 - Car registration (new, used and imported cars) 5 - Submission of data to statistical offices

6 - Application for building permission 6 - Customs declarations

7 - Declaration to the police (e.g. in case of theft) 7 - Environment-related permits (including reporting)

8 - Public libraries (availability of catalogues, search tools)

8 - Public procurement

9 - Certificates (birth, marriage): request and delivery

10 - Enrolment in higher education / university

11 - Announcement of moving (change of address)

12 - Health related services (e.g. interactive advice on the availability of services in different hospitals; appointments for hospitals)


of 56

At a glance it could have been thought that the concept of public services has no place in a project like STOP&GO but, some of the public services proposed by the Commission are in relation to it. On Table 1 are highlighted the defined public services which have direct relationship with the Project. This is the reason why the European Commission concept of public services and the environment relation with them are so important for the project (including all the aspects related with interoperability of public services or a method for interoperability assessment).

3. Interoperability of European Public Services On previous years, the European Commission adopted an initiative to encourage public administrations across the EU to maximize the social and economic potential of information and communication technology. More and more citizens and businesses are making use of the European single market's freedoms. However, citizens are often obliged to contact, or even to travel to, public administrations abroad to deliver or collect information or documents they need to work, study or travel within the EU. The same applies to businesses that want to establish themselves in more than one Member State. In order to overcome these constraints (so-called “e-barriers”), public administrations should be able to exchange the necessary information and cooperate to deliver public services across borders. That requires ensuring interoperability among public administrations.

Many public administrations in the Member States are already taking steps to improve interoperability for public services at national, regional and local levels, but unless Member States and the Commission act together, interoperability at EU level will lag behind. That is why the Commission has over recent years worked out a common strategy and built a common framework with Member States. European public services will often be the result of aggregating existing public services provided at various levels of government within Member States. Setting up European public services will only be feasible if those public services are designed with interoperability requirements in mind. Within the Digital Agenda for Europe, the Commission committed itself to adopt a Communication that introduced the European Interoperability Strategy (EIS)2 and the European Interoperability Framework (EIF)3, two key documents that promote interoperability among public administrations.

2 http://ec.europa.eu/isa/documents/isa_annex_i_eis_en.pdf

3 http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf


of 56

The direct beneficiaries of this action are Member States' public administrations and European Commission services that will gain in efficiency when establishing European public services and will be more aware of the risk of creating new electronic barriers if they opt for public services solutions that are not interoperable at EU level. Citizens and businesses will benefit from better European public services in their daily life when they want to extend their work or leisure activities beyond the borders of their home countries. Both the EIS and EIF recognise that interoperability has several dimensions: legal, organisational, semantic and technical. All of them are important, but thanks to the Internet and to the work of standardisation bodies and other organisations, significant progress has already been achieved in the area of technical interoperability, thereby ensuring openness, promoting reusability and fostering competition. The European Interoperability Strategy help focus EU efforts through an appropriate governance organisation and common policies and initiatives to create the environment for a trusted information exchange between public administrations. The European Interoperability Framework paves the way for public administrations in the EU to use a common approach by adopting guiding principles to allow genuine collaboration between public administrations, while modernising and rationalizing their systems to increase in a cost-efficient way their capability to provide high quality public services. The European Commission invites the Member States to continue to work together to ensure that their different efforts to achieve the interoperability of public services are aligned and take into account the European dimension at an early stage in the development of any public service that might become part of European public services in the future. In order to facilitate this collaboration, a brand new conceptual model for European public services was proposed. This model allows the identification of barriers and facilitators for the deployment of such services. [6]

4. Commission Interoperability Initiatives

Years of work around themes such as interoperability, eGovernment, open data, cloud computing and social innovation have brought within reach a suite of mature concepts and tools. The core novelty is the capacity of open data, participative tools and interoperable platforms to unleash unprecedented speed, efficiency and quality in the creation and delivery of public goods and services. The potential for interlinked and cooperative open governments joining together contributions from various sectors in an interoperable and secure manner is now high, and realising this ambition will be a crucial element for the successful building of the digital single market.


of 56

As mentioned in its 2013 annual growth survey [7], the Commission considers the cross-border interoperability of online services and the digitisation of European public administrations to be important contributors to growth and increased efficiency. Interoperability between administrations is a key enabler for more efficient and effective delivery of digital services, while sharing and re-using existing interoperability solutions could reduce the multiplication of costs. These are important productivity factors that could improve and modernise public administrations at EU, national, regional and local levels [8]. In response to the current economic crisis, the Commission has adopted another important initiative, the Europe 2020 Strategy [9], to turn the EU into a smart, sustainable and inclusive economy, delivering high levels of employment, productivity and social cohesion. Certain key challenges addressed by the Strategy relate directly to the modernisation of European public administrations. As underlined in the Digital Agenda for Europe (DAE) [10], a Europe 2020 flagship programme, interoperability is essential to maximising the social and economic potential of ICT and, consequently, DAE can take off only if interoperability is ensured. The DAE’s ‘interoperability and standards’ pillar ties in with policy priorities under other relevant initiatives such as the European Interoperability Strategy (EIS), the European Interoperability Framework (EIF) [6] and the 2012-15 e-Commission Strategy [11].

On 24 and 25 October 2013, the European Council adopted conclusions stressing that the modernisation of public administrations should continue, with the swift implementation of services, such as e-government, e-health, e-invoicing and e-procurement that rely on interoperability. This will lead to more and better digital services for citizens and enterprises across Europe, cost savings and increased efficiency, transparency and quality of service in the public sector. Interoperability programmes have come a long way since they were first introduced in 1995. The Commission has demonstrated its dedication to interoperability solutions over a remarkably long period. Today, strong commitment on all sides is essential if the Commission and the Member States are to transform and modernise services within and across Europe. In this context, interoperability, as a key enabler, has led the way to achieving ‘efficient and effective electronic cross-border and cross-sectoral interaction between […] administrations, […] enabling the delivery of electronic public services supporting the implementation of EU policies and activities’, as called for in the decision on the previous programme (ISA Decision Article 1(2) [12]).

The 2010-15 ISA Programme [12] on interoperability solutions for European public administrations had as its objective to support cooperation between European public administrations by facilitating efficient and effective electronic cross-border and cross-sector interaction enabling the delivery of electronic public services to help implement Union policies and activities. In the wider context, action under the previous ISA Programme was continuously coordinated and aligned with the ongoing work under the ICT Policy Support


of 56

Programme (ICT PSP) of the Competitiveness and Innovation Framework Programme (CIP) and/or with the Commission’s internal ICT strategy, and with action in the context of the 2011-15 European e-Government Action Plan. The ISA Programme supported these and similar initiatives whenever they contributed to interoperability between EU public administrations.

ISA2 Programme The ISA2 Programme is a programme on interoperability solutions for European public administrations, businesses and citizens for 2016-20 and succeeds the Union programme on interoperability solutions for public administrations –the ISA programme- and shall consolidate, promote and expand its activities. The ISA2 Programme shall facilitate efficient and effective electronic cross-border or cross-sector interaction between European public administrations and between them and citizens and businesses, in order to enable the delivery of electronic public services supporting the implementation of Union policies and activities. Through the ISA2 Programme, the Union shall identify, create and operate interoperability solutions implementing Union policies. These solutions shall subsequently be provided for unlimited use to other Union institutions and bodies, and to national, regional and local public administrations, thus facilitating cross-border or cross-sector interaction between them. The ISA2 Programme shall develop interoperability solutions autonomously or complement and support other Union initiatives by piloting interoperability solutions as a solution incubator or by ensuring their sustainability as a solution bridge.

The objectives of the ISA2 programme shall be to:

(a) Develop, maintain and promote a holistic approach to interoperability in the

Union in order to eliminate fragmentation in the interoperability landscape in

the Union;

(b) Facilitate efficient and effective electronic cross-border or cross-sector

interaction between European public administrations on the one hand, and

between European public administrations and businesses and citizens on the

other, and to contribute to the development of a more effective, simplified and

user-friendly e-administration at the national, regional and local levels of public

administration;

(c) Identify, create and operate interoperability solutions supporting the

implementation of Union policies and activities;

(d) Facilitate the re-use of interoperability solutions by European public

administrations.


of 56

The ISA2 programme shall take into account social, economic and other aspects of

interoperability, as well as the specific situation of SMEs and microenterprises, in order

to improve interaction between European public administrations on the one hand, and

between European public administrations and businesses and citizens on the other.

Actions under the ISA2 programme may require the procurement of external services,

which shall be subject to Union procurement rules as laid down in Regulation (EU,

Euratom) No 966/2012.

By 8 September 2016, the Commission shall develop a communication strategy, aiming

to enhance information and increase awareness with regard to the ISA2 programme

and its benefits, targeting businesses, including SMEs, and citizens, and employing

user-friendly means on the ISA2 programme's website. [13]

5. Interoperability Maturity Models

In the past, most companies and organizations created their own applications and

designed their own set of services. In the current globalized and networked society

organisations and companies need to collaborate with other organisations and

companies to exploit market opportunities and meet their own added values. As it has

said a major issue in global collaboration and cooperation is the development of

interoperability.

In order to support this collaboration and to better interoperate with the partners,

providers, subject of care, etc.; the interoperability requires being assessed and

continuously improved. One of the assessment methods is concerned with the use of

maturity models which begin to carry out in the early 80’s. A maturity model is a

framework that describes, for a specific area of interest, a number of levels of

sophistication at which activities in this area can be carried out [14] [15].

There are several interoperability maturity models developed. A number of

interoperability maturity models provide some guidance to governments interested in

developing or improving their ability to work effectively in network forms of

organization. Table 2 lists a few of these models. These models define both specific

types of capability and levels of maturity related to specific disciplines or government

policy areas. Of note, this table does not include an exhaustive list of interoperability

and capability maturity models but provides a selected list of those that capture the

complex multidimensional nature of government interoperability and areas.

Table 2 - Existing Interoperability Maturity Model Examples [16]

Policy Area or Discipline Model Year Released


of 56

Software development and systems engineering

Capability Maturity Model for Software (CMM), Carnegie Mellon

1986

Levels of Information Systems Interoperability (LISI), Carnegie Mellon

1998

Capability Maturity Model Integration (CMMI), Carnegie Mellon

2000

Defence

Organizational Interoperability Maturity Model for C2(OIMM), Australian Defence Science and Technology Organization

1999 and revised in 2003

Criminal Justice

Increasing Information Sharing Effectiveness: A Capability Assessment Model for the Justice Enterprise, Centre for Technology in Government

2005

Government Digital Information Preservation

Building State Government Digital Preservation Partnerships: A Capability Assessment and Planning Toolkit, Version 1.0, Centre for Technology in Government

2005

More Generic Government Services (often referred to as e-government)

IT Investment Management Framework (ITIM), U.S. Government Accountability Office’s (GAO)

2004

Interoperability Maturity Model (EIMM), European Union

2005

Government Interoperability Maturity Matrix (GIMM), Sarantis, Charalabidis, and Psarras

2008

Most interoperability maturity models reference the Carnegie Mellon Capability

Maturity Model (CMM) and the Carnegie Mellon Capability Maturity Model Integration

(CMMI). These models were first developed in the 1980s for software development

and systems engineering efforts and continue to be refined today. Within the last ten

years several other models have been developed. In general, these models expand

their perspectives beyond a technology development perspective (i.e., software

development or implementation) and focus on the required mix of policy,

management, as well as technology capabilities to achieve the broader goal of

improved delivery of government services and programs.

A variety of models have been developed to guide thinking across a continuum of interoperability maturity. Each adopts a unique vocabulary to express the levels and ideas, however, the models are in general consistent in terms of their characterization of interoperability capability maturity on scales ranging from low to high (see Table 3):

An organization with a low level of interoperability is characterized as working independently or in isolation from other organizations and in an ad hoc or inconsistent manner.

An organization with a high level of interoperability is characterized as being able to work with other organizations in a unified or enterprise way to


of 56

maximize the benefits of collaboration across organizations and across multiple government investments or projects (i.e., multiple networks).

Table 3 - Examples of Interoperability Maturity Levels [16]

Model

Level 1

Level 2

Level 3

Level 4

Level 5

CMMI

Initial

Managed

Defined

Quantitatively Managed

Optimizing

ITIM

Creating investment awareness

Building the investment foundation

Developing a complete investment portfolio

Improving the investment process

Leveraging IT for strategic outcomes

LISI

Isolated

Connected

Functional

Domain

Enterprise

IMM

Initial

Managed

Defined

Measured

Optimized

OIMM

Independent

Cooperative

Collaborative

Combined

Unified

EIMM

Performed

Modelled

Integrated

Interoperable

Optimizing

GIMM

Independent

Ad hoc

Collaborative

Integrated

Unified

In the middle of these maturity scales, fall those organizations that have developed some capabilities needed to collaborate, integrate, or cooperate with other organizations. However, this medium level of capability to be interoperable tends to be ad hoc, limited in scope (i.e., specific to a single network or policy or program area), and difficult to repeat or reproduce with other organizations or networks.

The existing interoperability maturity models also include a diverse mix of elements (e.g., areas of concern, goals, and interoperability attributes) considered essential to creating government interoperability. These elements cover what we refer to as dimensions of capability (or capability dimensions) needed for interoperability. [17].

The most known interoperability maturity models such as: LISI (Levels of Information

System Interoperability) [18], OIM (Organizational Interoperability Model) [19], LCIM

(Levels of Conceptual Interoperability Model) [20], and EIMM (Enterprise

Interoperability Maturity Model) [21] , deal with the a posteriori measure of

interoperability. Moreover, they focus on one single facet of interoperability (data,


of 56

technology, conceptual, Enterprise modelling, etc.). Some comparison study has been

reported in [22] and [23].

6 European Interoperability Maturity Model (IMM)

6.1 Introduction

The articles 3 and 7 of the ISA Decision specified that the ISA Programme shall support and promote the establishment of common frameworks in support cross-border and cross-sectorial interoperability by means of reports and studies. The framework is used both as a tool and a guide to identify needs in interoperability in the Member States. This involved that an action was created and delivered a first version of the Interoperability Maturity Model (IMM)4, which is continuously monitored existing practices in Member States to update de maturity model and the tool for assessment. The EC's ISA Programme developed an Interoperability Maturity Model to provide public administrations insight into two key aspects of their interoperability performance:

The current interoperability maturity level of a Public Service; Improvement priorities to reach the next level of interoperability maturity.

The Interoperability Maturity Model measures how well a public administration interacts with external entities in order to organize the efficient provisioning of its public services to other public administrations, businesses and or citizens. The IMM helps owners of a Public Service to enhance the quality of the service delivery, reduce costs and overcome integration issues by reusing available services and orchestrate services in an effective manner to maximize service outcome and benefits for citizens and public administrations. The IMM has been used by 17 Trans European Systems to assess the interoperability maturity of their public services. The Trans European Systems are the following:

ePRIOR, Support for pre and post award phase of Public Procurement DUES, Dual Use e-System EDAMIS, Electronic Data files Administration and Management Information

System eJUSTICE portal, 'Justice at a click' IMI, International Market Information system CECIS, Common Emergency Communication and Information System ECRIS, European Criminal Records Information System MH, Statistical metadata handler

4 https://joinup.ec.europa.eu/elibrary/document/interoperability-maturity-model

http://ec.europa.eu/dgs/informatics/supplier_portal/index_en.htm

http://ec.europa.eu/trade/import-and-export-rules/export-from-eu/dual-use-controls/

http://ec.europa.eu/idabc/en/document/3537/5927.html

http://ec.europa.eu/idabc/en/document/3537/5927.html

https://e-justice.europa.eu/home.do

http://ec.europa.eu/internal_market/imi-net/index_en.htm

http://www.eurocontrol.int/sites/default/files/content/documents/events/Presentations/130627-2nd-crisis-mngt-item7.1-emergency-response-centre-ec-dgecho.pdf

http://ec.europa.eu/justice/criminal/european-e-justice/ecris/index_en.htm

https://webgate.ec.europa.eu/fpfis/mwikis/sdmx/index.php/European_Statistical_System_Metadata_Handler_%28ESS_MH%29


of 56

SINAPSE, eCommunities for public policy making ESBR, European system of business registers MT@EC, Machine translation at the European Commission TACHONET, Exchange of tacho information across Member States SARI, State Aid Reporting Interactive ECN, European Competition Network INSPIRE geo-portal, Geoportal for sharing of geo-localization information EURES, European job mobility portal eTrustex, Secure exchange of natively digital or scanned documents from

system to system

Also, IMM was used by the Swedish and Greek Public Administration to evaluate a number of public services provided at national and local level.

Public Service

The IMM measures the interoperability of a Public Service. Public services are economic activities that public authorities identify as being of particular importance to different users. The following rules apply when defining a public service:

(1) The public service has a single service outcome / public decision. When multiple service outcomes are recognized, multiple public services will need to be defined and assessed separately through the IMM;

(2) The public service has a single service owner (the public administration responsible for the service). When the ownership of a service is distributed amongst multiple public administrations (e.g. multiple local administrations providing birth certificates), each service owner needs to conduct a separate assessment for his service;

(3) The public service has a single primary end user group. Service can be delivered towards three types of end users: citizens, business and other public administrations. In case the same public service is delivered to different types of end-user, these services should be assessed separately from one another through the IMM, one for each type of end user5.

(4) The public service has a visual end user interface. The IMM at the outset has been designed to evaluate services which are delivered to end users. This is a corollary to the previous design rule. The IMM shall thus not be used to only assess pure machine-to-machine services, even though this would be theoretically feasible by omitting the assessment area of Service Delivery, see 6.4.1

5 There is one exception to this, which is a service that from the organizational, legal, semantic and

technical perspective is exactly the same regardless of the end user group. Such cases are rare. Typically, services delivered to different end user groups are (slightly) different (example: the tax declaration service for citizens is different from the one for businesses).

http://europa.eu/sinapse/

http://ec.europa.eu/eurostat/documents/4031688/5931580/KS-03-13-411-EN.PDF/856ad655-c031-4c9e-a312-a9852a74c3d2?version=1.0

http://ec.europa.eu/dgs/translation/translationresources/machine_translation/index_en.htm

http://ec.europa.eu/transport/modes/road/social_provisions/tachograph/tachonet_en.htm

http://ec.europa.eu/competition/publications/cpb/2014/004_en.pdf

http://ec.europa.eu/competition/ecn/index_en.html

http://www.google.be/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=1&ved=0CCIQFjAA&url=http%3A%2F%2Finspire.ec.europa.eu%2F&ei=lHwVVdSzFoHraNLTgZgN&usg=AFQjCNHv_E0cJLhnqyZQ1CJTymM-jYzg1A&sig2=0On8iOBUUrNteb1XKTIHJQ&bvm=bv.89381419,d.d2s

https://ec.europa.eu/eures/page/index

https://joinup.ec.europa.eu/software/openetrustex/description

https://joinup.ec.europa.eu/software/openetrustex/description


of 56

So the service is delivered by different ways: a public administration to citizens (A2C),

business to organizations (A2B) and/or other public administrations (A2A). The best

way to understand the 3 cases is with the examples proposed by the model:

(A2C) Citizens (3) are offered the service to access their Electronic Health Record (1) via the eHealth portal of the Danish Sunhed (2).

(A2B) Businesses (3) are offered the service to register and pay for the filling of patents (1) via the website of the PRV (2)6.

(A2A) Administrations (3) are offered the service to obtain European vehicle information (1) via the web service of EUCARIS (2)7.

From a conceptual point of view, a public service starts with a trigger, follows a number of steps and delivers an outcome towards an end user. The outcome may, but must not necessarily, be a public decision (as IMM report example: issuing of a license involves a decision; whilst communicating the results of a job search does not). This conceptual model of public services is illustrated in Figure 1.

Figure 1 - Conceptual model for a public service

The previous model and the three cases (A2C, A2B and A2A) could adjust to the

hypothetical services procured in the STOP&GO Project environment and depending

on every procurer necessity. The A2C case could adjust to any STOPandGO Service

involving the health institution with the subject of care (citizen) directly. The A2B case

would be an example of the exchange of data between the procurers and any kind of

company which tender for the process. An example of A2A case would be between the

health institutions or procurers and another institution or Agency at the same or in

different countries, for example, in the interchange of data about the requirements of

the tendering process or about some kind of data from the subject of care.

6 PRV – The Swedish Patent and Registration Office

7 EUCARIS – European Car and Driving License Information System


of 56

Interoperability

In the context of interoperability maturity, the IMM measures how well a public service is able to interact with other organizations to realize mutually beneficial and agreed common goals through the exchange of information and reuse of services. Figure 2 displays the public service in the context of interoperability. All elements inside the internal domain that do not have an external relationship with other services or administrations, business or citizens are not relevant/ eligible for the IMM. All relationships that interconnect the public service with the outside environment are considered in the domain of interoperability and are taken into account when assessing and improving the interoperability of a public service following the IMM.

Figure 2 - Visualization of interoperability based on the service connections with the outside world (Internal domain versus external domain)

6.2 Objectives

The IMM has the objective to deliver insight into two important aspects of

interoperability maturity:

Provide insight into the current interoperability maturity of a public service

based on a set of defined interoperability attributes and maturity stages;

Provide guidelines how the public service can improve interoperability

maturity.

Although the IMM is publicly available for all interested organizations and citizens, the

main target audience are the service owners of public services that operate in an

environment in which interoperability is required to deliver a public service towards

end users.

Improving interoperability is a continuous activity. Therefore organizations are

encouraged to frequently use the model and the improvement guidelines it contains.


of 56

6.3 Maturity levels

The IMM uses a five stage model to indicate the interoperability maturity of the public service. The reason for the usage of these various maturity levels is:

To measure the interoperability maturity of the public service as a whole and of

the underlying aspects;

To indicate which capabilities and next steps are required to improve

interoperability maturity

A five stage approach is seen often in proven maturity models and is considered

ideal for assessing and improving organizational maturity. The five maturity levels

for the IMM are summarized in the Table 4 below:

Table 4 - Five maturity stages of IMM

Maturity Level Maturity Stage Interpretation

1 Ad Hoc Poor interoperability – the service has almost no interoperability in place

2 Opportunistic Fair interoperability – the service implements some elements of interoperability best practices

3 Essential Essential interoperability – the service implements the essential best practices for interoperability

4 Sustainable Good interoperability – all relevant interoperability best practices are implemented by the public services

5 Seamless Interoperability leading practice – the service is a leading example for others.

The desire interoperability level for a public service is at minimum level 4. Sustainable.

At this level the public service is considered to have implemented all relevant best

practices and should be achieved.

6.4 Areas of Interoperability

In the context of interoperability maturity, the IMM measures how well a public

service is able to interact with other organizations to realize mutually beneficial and

agreed common goals through the exchange of information and reuse of services.

Error! Reference source not found. displays all possible instances where

interoperability with the outside world may occur from the viewpoint of a public

service. Summarizing what has been said so far, the following interactions may occur.

The numbering of the areas (B, C, etc.) is based on the sections of the questionnaire.

As there is a service context section (A) in the questionnaire the numbering of the

areas starts with B.


of 56

Service Delivery (B) – Providing end-users accessibility to the public service.

Service Consumption (C) – Consumption of reusable machine-to-machine

services from other public administrations and businesses. This can include the

consumption of functionalities, base registry information and security services.

Service Management (D) – Controlling and monitoring the process flow related

to service interactions with the external domain from trigger to outcome. This

area includes Service Management aspects such as architecture, procurement

and cost-benefit analysis and the provisioning of the services towards other

administrations or business.

Figure 3 - Overview of the interoperability areas of the IMM model

The areas (hereafter referred to as Interoperability Areas) indicated in the figure above

are the object of measurement in IMM as they indicate where interoperability plays a

role from a service management, service delivery, service provisioning and service

consumption viewpoint.

6.4.1 Service Delivery (B)

The public administration delivers the public service towards end users i.e. citizens,

businesses or other administrations. It covers the interoperability aspects from an end-

user perspective only. Note that a public service only used by internal employees of

the public administration does not qualify as service delivery whilst delivery of services to

staff from other, external public administrations does. The qualifying criterion here is the

relation of the public service with the external domain as described above. The most

important interoperability aspect covered by the service delivery area is how the

service is made available to the end-users through various delivery channels (e.g.

counter, paper forms, software application, and online portal). A public service is

considered more interoperable when end-user service delivery is electronic and

supported by multiple channels and devices to enhance accessibility.


of 56

6.4.2 Service Consumption (C)

For delivering the public service towards the end user, the public service may be

required to consume services of other public administrations or businesses.

There are various types of services that can be consumed by public services:

Functional service – a common functionality (e.g. issuing a license, procurement, planning, risk assessment module) shared across organizations;

Security service – a specific type of functional service to share common security functions (e.g. identity provisioning and authentication) across organizations;

Base registry service – a specific type of functional service to share trusted, authentic and verified data (about e.g. citizens, land, vehicles) across public administrations.

Public services that consume (reuse) existing services where possible are considered more interoperable than organizations that produce (develop) their own proprietary services without reusing existing functionalities.

6.4.3 Service Management (D)

This area focuses on important Service Management aspects such as architecture,

orchestration, procurement and cost-benefit analysis that detail to what extend the

organization has mechanisms in place to facilitate for interoperability. It also addresses

the aspect that organizations that deliver (machine-to-machine) services towards other

administrations or businesses are considered more interoperable. This attribute is called

service provisioning.

Depending on the type of public service involved, the service can be either delivered

autonomously by a public administration or require service interactions with other

public administrations or businesses. These service interactions can be either based on

consumption (the public service reuses an existing services) or on provisioning (the

public service provides a service towards another organization). Service Management

encompasses the coordination of all external interactions to ensure the outcome of

the public service is achieved in the optimum manner. Organizations are considered

more interoperable when interactions with other services are managed in a central,

coordinated and consistent way.

6.4.4 Examples

The following case examples (see Table 5) illustrate the interoperability areas of

delivery, provisioning and consumption. They are taken from real-life examples based

on which the Interoperability Maturity Model has been developed and should guide

users of the model in defining and delimiting their public service’s interconnections

correctly


of 56

Table 5 - Examples of Interoperability Areas

Public Services

Service Delivery Service Consumption

Service Management (attribute service provisioning)

Electronic Health Record Access

Citizens are offered the service to access their Electronic Health Record via the eHealth portal. Case example: The service called “My Health summary” is available through the Danish eHealth portal 'Sundhed.dk' for citizens and allows authenticated users to obtain an overview of their own patient data.

Payment services Identify and access

management services

eSignature services Personal medicine

data Donor Registration Living will

registration Laboratory data

Not applicable

Online Patent Filing

Businesses are offered the service to register and pay for the filling of patents. Case example: The EPO Online Filing client application provides applicants with a standard form for filing patent applications online with the European Patent Office. Once the request is filed, the applicant receives an electronic notification of receipt. If the applicant has set up an online Mailbox , he will receive all further communication from the EPO via this Mailbox, including requests for rectifying the application and the invitation to pay claims fees

Payment services Identity and access

management services

eSignature services

Search classification service

Government E-invoicing

Business are offered the service to send online invoices towards the various government administrations. Case example: Businesses can send all their invoices in electronic format to the Dutch government. In total, more than 78 government bodies have implemented electronic invoicing. The sending and receipt of e-Invoices can take place through two channels: Digipoort (direct access or via an intermediary) or the e-Invoicing portal www.facturerenaandeoverheid.nl.

Payment services Identity and access

management services

eSignature services

Open Data provisioning

Purchasing catalogue service

Contract register Purchase order

sender Invoice receiver


of 56

6.5 Interoperability Attributes

IMM assesses each interoperability area using a set of interoperability attributes.

These interoperability attributes form the core of the IMM and are used for

measurement and improvement of interoperability maturity.

Various important sources have been utilized to build the current set of

Interoperability Attributes in this version of the model:

European Interoperability Framework - the European Interoperability

Framework (EIF) serves as an important framework for organizations to

promote and improve interoperability and therefore is considered as an

important starting point in defining the Interoperability Attributes. To make

this interrelation explicit, each interoperability attribute within IMM is linked

towards one or more EIF-layers (technical interoperability, semantic

interoperability, organizational interoperability and legal interoperability).

Alignment with various other ISA initiatives – The IMM is continuously being

aligned with and provides input into the following ISA initiatives:

o EIRA8 – European Interoperability Reference Architecture

o TES9 - Assessment of trans-European systems supporting EU policies

o NIFO10 - National Interoperability Framework Observatory

o CAMSS11 – Common Assessment Method Standards and Specifications

o SEMIC12 – Promoting semantic interoperability amongst the European

Union Member States

o Base registries13 – Access to base registries

o Cost-Benefit model14

o ICT implications15 – Assessment of ICT implications of EU legislation

o Sharing & Reuse16 – Sharing and re-use strategy

8 http://ec.europa.eu/isa/actions/02-interoperability-architecture/2-1action_en.htm


10 http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-3action_en.htm


12 http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-1action_en.htm

13 http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-2action_en.htm

14 Action tbc in next ISA work program

15 http://ec.europa.eu/isa/actions/03-ict-implications-assessment/index_en.htm

16 http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-5action_en.htm

http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-1action_en.htm

http://ec.europa.eu/isa/actions/03-ict-implications-assessment/index_en.htm


of 56

6.5.1 Interoperability Patterns

When examining the characteristics of interoperability attributes a number of patterns

emerge. The definition and combination of interoperability patterns helps in

identifying the core elements of interoperability and the way how to measure them.

Figure 4 illustrates the relationship between the interoperability maturity and the

pattern. The interoperability patterns form the basis for the interoperability scoring.

The interoperability patterns are:

1. From paper-based information exchange to digital information

exchange: a public service working with paper documents is considered

less interoperable than a public service which uses digital information;

2. From manual to automated processing: a public service manually

processing transactions is considered less interoperable than a public

service which has fully automated the process execution;

3. From ad hoc to standard: a public service developing its own (ad hoc)

protocols and formats is considered less interoperable than a public

service adopting widely used, standard-based solutions;

4. From individual to collaboration: a public service working stand-alone is

not reusing available services and therefore is considered less

interoperable than a public service which collaborates with other public

administrations and organizations where applicable.

Figure 4 - Examples of Interoperability Patterns

6.6 Questionnaire

The IMM uses a questionnaire for assessing the interoperability maturity. This section

details the question types and questionnaire structure in more detail.


of 56

6.6.1 Questionnaire Structure

This section outlines the structure of the questionnaire. The five main sections of the

questionnaire are in line with the earlier presented overview of interoperability areas

(section 4.4):

Service Context (A): This section assesses the scope of the public service (the object of measurement, i.e. the public service to examine), service landscaping and gathers important information for follow-up (contact details, etc.).

Service Delivery (B): The section assesses how the public service delivers the public service towards end-users.

Service Consumption (C): This section assesses if and how services are consumed from other administrations and businesses.

Service Management (D): This section assesses how the public service arranges the consumption and provisioning of external services and includes Service Management aspects such as architecture, procurement and cost-benefit analysis.

The questionnaire routing is sequential on the level of the main areas (A, B, C and D).

The questions within area A, B and D are also defined sequentially and do not contain

complex questionnaire routing. This is different for the Section C ‘Service

Consumption’ in which the routing is dynamically based on the given answers. [24]

In addition, some questions has been developed to test the grade of conformity of the

service with the S&G Interoperability Framework. They are in the questionnaire Service

conformance to STOP&GO Interoperability Framework (E).

7 European Interoperability Maturity Model in STOP&GO Project

The IMM was developed for some years, was published several months ago and it has

been updated on April of 2016. As it is said previously, it measures how well a public

administration interacts with external entities in order to organize the efficient

provisioning of its public services to other actors and it can be used for assessing and

improving the interoperability maturity of a public service.

The SaGIF (STOP&GO Interoperability Framework) provides a common reference

framework concerning the scope and issues addressed at the WP3 considering the four

levels of interoperability following the recommendations from the European

Interoperability Framework among others.

In this context the IMM has been created under the ISA programme (Interoperability

Solutions for European Public Administrations) the same as the European

Interoperability Framework.


of 56

The SaGIF adds to the major part of European Interoperability Framework a minimum

of other specifications based mainly on two ISO standards: the ISO 13606 and the ISO

13940, related with syntactic, semantic and organizational interoperability, which are

necessary to achieve. The European Interoperability Framework by itself describes

many interoperability actions to take into account and therefore the IMM is carried

out in the assessment of this action line.

The idea of this deliverable is to have an assessment method for evaluating the

interoperability performance in different localities and according to the next

deliverable D3.3 Analysis of overall performance of the STOPandGO Interoperability

Framework (SAGIF).

It has analysed the bibliography and the possible methods for the interoperability

assessment of the Project in the last years. The proposal in this deliverable is not to

change the work done by the organizations implied in the IMM development, because

the IMM fits with the pursued target suitably. It is mentioned that some of the

organizations or initiatives implied in the IMM development work in public

procurement, electronic data management and information systems experience, etc.

for instance. Furthermore, although the IMM is not an open source, its reproduction is

authorized. That’s why the IMM has been chosen for adapting to the STOPandGO

Interoperability Framework. Furthermore, it has been decided to add other small

questionnaire about the compliance grade with STOPandGO Interoperability

Framework to know the partners’ conformance with it.

As it was mentioned in a previous point, the model has several questionnaires

dependent on the Interoperability Areas. Therefore the Interoperability Maturity

Model (and the questionnaire related to SAGIF) are a self-assessment tool that can

provide with an analysis on the interoperability of the service to the project partners.

The idea is to spread out the questionnaires to the partners for filling them in the next

months. With the information collected from partners, interoperability rates would be

estimated and recommendations for improving the interoperability would be

performed. The questionnaires are placed in Annex A – Questionnaires

According to the STOPandGO Interoperability Framework some of the guidelines are

accomplished or achieved partially in the IMM Questionnaire and more specific in

Conformance to SAGIF Questionnaire. In Table 6 is shown the relation of concepts

between SaGIF and the Annex A Questionnaire proposed.

Table 6 - Relation of concepts between SaGIF and the questions of the Annex A Questionnaire.

SaGIF Guidelines IMM Questionnaire Conformance to SAGIF Questionnaire

Legal Interoperability C10, D11 E1, E2

Organizational D3, D6, D7, D9 E3

Semantic C6, C8, D9, D10 E3, E4, E5, E6, E7

Technical C6, D3, D9, D10 E4, E5, E6, E7, E8

General Recommendations B3, B4, B5, C7, C14, C15, D6, D8,


of 56

D9, D10, D11

The SaGIF shows some synthetic guidelines about interoperability but does not go into

the interoperability in depth. The IMM deals with more concepts about

interoperability, some of them didn’t take into account for the Deliverable 3.1

Interoperability requirements: STOPandGO Interoperability Framework, but are useful

for the interoperability environment and the S&G Project and, therefore, appear in the

questionnaire.

The Questionnaire is divided in parts and some of them according to the three

interoperability areas previously defined in this document. The section A (Service

Context) is about Context information (contact details, service description, etc.) and

does not taking into account for assessing the interoperability maturity score; the

contrary holds for sections B to D. The section B (Service Delivery) has an overall

weighting of 25% from the total maturity score. The section C (Service Consumption)

has an overall weighting of 40% and the section D (Service Management) has a 35%.

Every question within each section has a weight (if the question conditions are

achieved according to Maturity levels), so that the sum of these “weights” in the

section never overcomes the 100%.

Apart from the IMM Questionnaire, a more specific questions to SAGIF has been

planned for gathering the SAGIF introduction level to the procurers.

Finally, the section E is the questionnaire related to Service conformance to STOP&GO

Interoperability Framework. It has some and more specific questions about STOP&GO

Interoperability Framework in accordance with the four interoperability types (legal,

organisational, semantic and technical). The weight of this section is 100%.

The questions of the SAGIF compliance questionnaire have another weight and the

overall SAGIF compliance interoperability level will be processed and addressed

differently than the IMM result got previously.

The whole interoperability maturity level will be formed by two numbers, one from the

IMM Questionnaire and the other from the Conformance to SAGIF Questionnaire.


of 56

Annex A – Questionnaires

1. Service Context (A)

1. Questions

A.1

Name Contact Details

Question Type Open

Rationale Gather contact information for possible follow-up

Question Please provide your name and contact details (telephone, e-mail address).

A.2

Name Public service description

Question Type Open

Rationale Gain insight into the service the public administration provides

Question A public service is a service rendered in the public interest. What is the public service you provide to end users (either citizens, businesses or other public administrations)? Use the following criteria to define the public service:

Define the process and underlying activities to define the public service. The public service always contains three main elements (1. initiation, 2. processing and 3. delivery of an outcome). Focus on the public decision that is the outcome of the service. If there is no public decision and/or outcome, focus on the benefits the service provides to the target audience.

Define the owner of the public service (see also question A.3). A public service has typically one owner that is responsible for the outcomes of the public services. If more owners are defined – this probably will lead to the definition of multiple public services.

Define the appearance of the public service. How does the public service deliver the outcome towards the end user group? Is this a fully digital process or are manual interactions required (e.g. physical counter, etc.)? Note that IMM addresses both forms.

The public services offers benefits and an outcome towards a single end user group. If the service encompasses multiple benefits and addresses multiple end user groups, narrow down the scope of the public service to ensure the situation applies to a single, clearly delimited public service only. Please note there are situations in which the public service delivers


of 56

the outcome not directly towards an end-user group but towards other IT systems. In this scenario we assume that the public service encompasses only machine-to-machine interfacing and that the service delivery component will not be filled in during the questionnaire.

Examples Submission of yearly tax income declaration for citizens (A2C); change of residence of a citizen (A2C); online information provisioning on relevant jobs to citizens (A2C); posting of vacancies on a job portal for businesses (A2B); providing information on the whereabouts of specific cargo to businesses (A2B); providing classification services towards other related administrations for ensuring international standardisation of patent data (A2A).

A.3

Name Service owner

Question Type Open

Rationale This question determines the scope / boundaries of the public administration providing the public service.

Question Which public administration is primarily responsible for providing the public service?

Examples A tax administration; A department/unit within a tax administration, A Directorate-General (DG); A municipality.

A.4

Name End user group to which the service is delivered

Question Type Open

Rationale Determine the primary end user group to which the public service is delivered.

Question What is the primary end user group to which the public service is delivered?

Examples A specific group of businesses; A specific group of citizens; A specific group of public administrations. Note: a mix of various types of end users (administrations, businesses, citizens) indicates that the service definition of the public service is too broad. See also the explanations provided under A.2

A5

Name Administrative level

Question Type Multiple choice (>1 possible answer)

Rationale Gain insight into the reach (government tier) of the public service.


of 56

Question What is the underlying administrative level of the public service (multiple answers are possible)?

Local (e.g. city, municipality)

Regional

National

European

International


of 56

2. Service Delivery (B)

1. Questions

B.1

Name Delivery channels

EIF-Layer Technical interoperability

Weight 0%

Question type Multiple choice (>1 possible answer)

Rationale Assesses through which channels the service is delivered towards the end user. A public service is considered more interoperable when a broad set of delivery channels is used. Presence of a self-service channel is considered as a main requirement for interoperability. This includes traditional (non-digital) and digital channels.

Question Through which delivery channels is the public service made accessible towards the end user (multiple answers are possible)?

Traditional

Counter / desk

Postal

Telephone Digital

Dedicated application (functionality that needs be installed on a device by the end user before it can be used. This includes apps from an online application store)

Website and/or web portal (functionality that is directly accessible for the end user via an Internet URL)

Not applicable – the public service offers no direct delivery channel towards the end user

Examples Telephone only; Functionality that is only available via a dedicated application that needs to be installed via CD-ROM; Functionality that is made available via a portal that provides access to a set of public services (www.mijnrijksoverheid.nl). The service is made available via a dedicated website (for the public service); There is no direct delivery channel for the end user – the public service is delivered in machine-to-machine only (for example a public services that provided information towards another system).

B.2

Name Device, platform and/or browser dependency


Weight 40%

http://www.mijnrijksoverheid.nl/


of 56

Question type Multiple choice (1 answer is possible)

Rationale Assesses whether the delivery channel is device / platform / browser independent

Question Can the public service be accessed using multiple devices, platforms or browsers?

No, the public service is offered for a single device, platform and/or browser

Yes, the public service is offered for multiple but not all available devices, platform and/or browsers

Yes, the public service is offered for all common available devices, platforms and/or browsers

Examples Yes, all common browsers, platforms and devices are supported to access the public service; no, only Internet Explorer 8 is supported. - Devices: PC; Tablet; Mobile Phone - Platforms: Windows OS; Mac OS; Mobile OS - Browsers: Internet Explorer, Google Chrome; Firefox; Opera

B.3

Name Form pre-filling

EIF-Layer Semantic interoperability; Technical interoperability

Weight 40%

Question type Multiple choice (1 answer possible)

Rationale Re-use of existing trustworthy data sources in pre-filled forms should be stimulated as it minimizes end user effort and reduces the risk for erroneous data entries

Question Does the public service use pre-filling of forms?

No

Yes, pre-filling is used but only for some data fields that are electronically available

Yes, pre-filling is used for all data fields that are electronically available

Not applicable, the public service does not require user data

Examples Existing internal or external base registries (or other data sources) are used for the pre-filling of forms so name and address data are accurate. Pre-filling includes also the filling of drop-down boxes and/or auto-filling (automatic completion of key words).

B.4

Name Multilingualism

EIF-Layer Organisational interoperability; Semantic interoperability; Technical interoperability

Weight 10%



of 56

Rationale Multilingualism in the context of computing indicates that an application dynamically supports two or more languages.

Question To what extent is multilingualism supported?

Not at all

Partly, only the user interface is multilingual (two or more official EU languages supported)

Fully, the entire service (user interface, support documentation, technical specifications, etc.) as such is multilingual (two or more official EU languages supported)

Examples Multilingual support is provided for the user interface only; the entire service (user interface, functional & technical documentation, online-and offline support documentation, etc.) is made available to end users in three languages

B.5

Name Cross-referencing

EIF-Layer Organisational interoperability; Technical interoperability

Weight 5%


Rationale Promoting other related (public) services may contribute towards the overall use of (digital) public services. Public services that reference towards related (public) services therefore contribute towards overall interoperability

Question Does the public service promote the usage of its own or other (public) services through linking to/interlinking with other web sites?

No

Yes, the public service is being referenced from other sites

Yes, the public service is being referenced to other sites offering related public services

Yes, the public service is being referenced from other sites and the public service is being referenced to other sites offering related public services

Examples The service implements the organization-wide policy to link towards other public services (for example to deliver services relating to a life event). Links are typically made available via banners on the website of related public services.

B.6

Name Service Catalogue

EIF-Layer Organisational interoperability; Semantic interoperability; Technical interoperability

Weight 5%


Rationale Providing detailed information on the availability of the public service is an enabler for usage by citizens, business and administrations.


of 56

Note that what is meant here by service catalogue is a catalogue overarching various organizations (e.g. across several administrations or a national catalogue of public services).

Question Is the public service that is being delivered part of a service catalogue?

No, even though there is a Service Catalogue in place

No, because there is no Service Catalogue available

Yes, the service is included in the Service Catalogue

Examples The public service is displayed on a government portal that holds a full repository of all public services offered to citizens, to increase the awareness and usage of the public service.

2. Maturity Scoring

The overall weighting of this area towards the total maturity score is 25%

Table 7 - Scoring table: Service Delivery (B)

Question Ad hoc Opportunistic Essential Sustainable Seamless

B.1 No Score

B.2

Single Device/

platform/ browser

Multiple Devices,

platforms, browsers

All common available devices,

platforms, browsers

B.3

No pre-filling

Partial pre-filling

Full pre-filling

of Not Applicable

B.4

Not at all

Partly, only the user

interface is multilingual

Fully, the entire service

as such is multilingual

B.5

No

Yes, the public

service is referencing

to other sites

Yes, the public

service is being

referenced from other

Yes, the public service is

being referenced from other

sites and the


of 56

offering related public

services

sites)

public service is referencing to other sites

offering related)

B.6

No, even though

there is a Service

Catalogue in place

No, because

there is no Service

catalogue available

Yes, the service is

included in the Service

Catalogue


of 56

3. Service Consumption (C)

1. Questions

C.1

Name Landscaping Service Consumption

Category Multiple choice (>1 answer possible, including own-defined options)

Rationale Gain insight into the services that the public service currently consumes.

Question Please list the services which the public service has to consume in order to work:

First, indicate for the below generic services if these are required (note that this is an indication list).

Second, add specific services which are specific to the public service and required by it in order to work.

Important note: Please list both services that are consumed from within the administration (internally) and from a third party (externally). Please list both manually and digitally consumed services. Generic services (indicative list – select applicable ones):

Authentication Service

eSignature Service

e-Payment Service

Messaging Service Audio-visual Service

Data Transformation Service (yes/no)

Data Validation Service (yes/no)

Machine Translation Service (yes/no)

Data Exchange Service

Business Analytics Service

Business Reporting Service

Forms Management Service

Records Management Service

Document Management Service

Content Management Service

Access Management Service

Logging Service

Audit Service

Metadata Management Service

Networking Service

Hosting Service

Storage Service

Base registry information source Secondly: Please name any relevant specific services that are required by your public service in order to function. Again: Please include both services that are consumed from within


of 56

the administration (internally) and from a third party (externally). Please include both manually and digitally consumed services.

...

Examples See above

C.2

Name Manually or electronically consumption of services

Weight If the answer is ‘consumed manually’ the entire consumed service is seen as ‘Opportunistic’ (maturity level 2). For digital services the maturity is calculated based on questions C.3 or C.4 to C.11.

Question type Multiple choice (1 answer possible) for each selected / indicated services in question C.1

Rationale Gain insight into how the service is being consumed.

Question How do you currently consume the service (manually versus electronically)?

Consumed manually

Consumed electronically

Examples An example of electronic consumption is the tax administration digitally fetching data from the Citizen Base Register. An example for manual consumption is filling in a paper-based form at the counter of a city council officer to request a change.

C.3

Name Reusing or producing of services


Weight 0% (if the answer is ‘Reuse) or 100% (if the answer is not). If the answer is ‘Produce (develop) the service, while reuse is possible’ the entire service is seen as ‘Ad hoc’ (maturity level 1). If the answer is ‘Produce (develop) the service, because there is no fit-for-purpose service to reuse’ this service is not taken into account for the maturity scoring.

Question type Multiple choice (1 answer possible) for each selected ‘digital service’

Rationale Specify how the service is being consumed (reuse versus produce). Producing a service, while a service is available externally for reuse is considered less interoperable as it implies that the public service has “reinvented the wheel”.

Question Does the public service reuse or self-produce consumed services? (Reuse of relevant existing services vs Self Production of services)?

Self-produce the service, while relevant services are available for reuse.

Self-produce the service, because there is no fit-for-purpose service to reuse.

Reuse of an existing service

Examples The public administration uses Google Translate (external services) as


of 56

a translation service for her web portal (reuse). The identity and access management (IAM) service is developed and delivered by the administration itself while there is an institutionalized IAM-standard to use within the country of residence. This is seen as non-compliance (produce, while reuse is possible). The Tax administration holds valuable data within their own organization to perform fraud analysis. This type of data is not available externally (produce, no fit-for-purpose service to reuse).

C.4

Name Processing mode


Weight 10%


Rationale There are two types of processing modes: real-time or batch processing mode (initiated per unit of time: daily, 4 times a day, etc.).

Question What is the processing mode of the consumed service?

Batch processing only whilst real-time could be an option

Batch processing only due to legal, technical or other constraints

Both processing modes are supported

Fully real-time processing

Examples The social security office collects new data from a citizen’s base registry every week. Citizen data is update via a batch process to ensure the correct data is in place. However, if other transaction occur during the weeks the timeframe this could lead to undesirable results. Real-time processing would prevent inconsistencies and/or fraud.

C.5

Name Push-pull mechanisms


Weight 10%


Rationale The interaction mode depends on the specific context of the public service. Push consumption refers to the public service receiving automatic updates (e.g. of data) or triggers (for executing a process for example). Push consumption or having both mechanisms in place are considered more mature as these demonstrate that the public service seamlessly interconnects with the services it is consuming

Question What is the interaction mode with the service?

Pull only, whilst push could be added –Pull only due to legal or other constraints


of 56

Push only, whilst pull could be added Push only due to legal or other constraints Both mechanisms (push and pull) are being used

Examples The public service receives automatic updates from the base registry for income details (push interaction mode). Information is queried when required for pre-filling forms (pull interaction mode).

C.6

Name Common protocol usage


Weight 20%


Rationale Usage of existing protocol specifications implies a higher interoperability than developing a dedicated protocol

Question What type of protocol specification is being used for exchanging structured information?. The protocol specifies the dialog not the content of the messages.

Proprietary protocol specification

Common protocol specification

Examples A specific / unique API is considered as proprietary; the public service reuses existing SOAP (or REST) protocols (which are considered as common);

C.7

Name Reuse of network infrastructure


Weight 10%


Rationale Reuse of existing network infrastructure rather than using a private network indicates higher interoperability

Question Is the service consumed via an existing network infrastructure or a dedicated, private network?

The service is consumed via a new dedicated private network whilst it could leverage on an existing network infrastructure or the Internet

The service is consumed via a new dedicated private network due to security or other specific concerns

The service is consumed via an existing dedicated private network (e.g. sTesta)

The service is consumed using the publicly available Internet

Examples Examples comprise the reuse of existing network infrastructure within the EU such as sTesta, leverage of the Internet for accessing public services or building a new dedicated network infrastructure with the help of dedicated networking lines between administrations.


of 56

C.8

Name Semantic alignment

EIF-Layer Semantic interoperability

Weight 20%


Rationale Use of existing semantic standards (e.g. data models standards, standardised XML schemata, metadata standards, standardised reference data (e.g. code lists) is considered more interoperable than developing proprietary standards

Question To what extend are semantic standards and specifications used for data modelling of the data exchange between the public service and consumed service?

The data models have been created for the public service without using any existing semantic standards or specifications

Some proprietary semantic standards and specifications are used for creation of the data model.

The whole development of the data models is based on existing (open) semantic standards and specifications.

Examples Common XML-based standards are used widely in the service domain and are also used for provisioning the service; a unique data model is developed specifically for this service consumption.

C.9

Name Exception handling

EIF-Layer Semantic interoperability

Weight 10%


Rationale Received information may be inconsistent with internal information. Initiated transactions may lead to an unexpected response. The way how these exceptions are handled determine the level of interoperability.

Question How are exceptions resolved?

Fully Manually

Semi-automated

Fully automated

Examples The public service has no routines to handle exceptions automatically – all anomalies are processed manually by the back office; around 80% of the exceptions are resolved automatically – the remaining 20% is still processed manually by staff (semi-automatic); all exception are processed manually – no manual interference is required (fully automated).


of 56

C.10

Name Certification

EIF-Layer Organisational interoperability

Weight 10%


Rationale Certification is a success factor for ensuring working interconnections. A public service which applies for formal certification when available is considered more interoperable. Certification is a formal procedure to verify if a constituency meets the prerequisites to connect to a service. Certification may examine areas like: security, governance, technological and semantic interoperability and availability

Question Has the public service followed the certification procedure to use the service?

No, while a certification procedure is available

No, there is no certification procedure available

Yes

Examples No, although there is a separate test environment made available to test the interconnection with other systems, acceptance testing is not conducted for certification purposes; Yes, the public service has been certified conform to connection criteria.

C.11

Name Specification process


Weight 10%


Rationale An open process to establish specifications is likely to yield more interoperable results.

Question Has the public service been involved in establishing the specifications of the consumed functional service?

No, although this would have been possible

No, this was not possible

Yes

Examples There is a dedicated forum which is accessible for everybody to post ideas and participate in discussions around the public service; administrations and businesses first need to be invited to join the specification process (semi-open).

2. Maturity scoring


of 56

The overall weighting of this area towards the total maturity score is 40%.

Table 8 - Scoring Table: Service consumption (C)


C.1-C.2-C.3

Produce (develop) the service, while

reuse is possible or

Manual consumption

Scoring outcome dependent on C.4-C.11

C.4 Batch

processing while

real-time could be an option

Batch processing only due to

legal, technical or

other constraints

Both processing modes are supported

Fully real-time processing

C.5 Pull only, whilst push could be

added Pull only, due

to legal, or other

constraints

Push only whilst pull could be

added

Push only due to legal or other

constraints, both

mechanisms are used

C.6 Proprietary protocol

specification

Common protocol

specification

C.7 The service is consumed via a new dedicate

private network whilst it could leverage on an

existing network infrastructure or

the Internet

The service is consumed via a new dedicated

private network due to security or other specific

concerns

The service is consumed

via an existing private

network (e.g. sTesta)

The service is consumed using

the publicly available Internet

C.8 All data

models were created for the service without

using any existing

semantic standards

Some proprietary

semantic standards are

used

The whole development of the data models

is based on existing (open)

semantic standards and specifications.

C.9 Fully manually Semi- automated

Fully automated


of 56

C.10 No, while a certification procedure is

available

No, there is no

certification procedure available

Yes,

certification

C.11 No, although this would have been possible

No, this was not possible

Yes


of 56

4. Service Management (D)

1. Questions

These questions apply only if service consumption has been identified in section C.

D.1

Name Cost-Benefit Analysis


Weight 10%

Question type Elementary attribute

Rationale While designing the public service, a cost-benefit analysis should be made to get a deep insight in the benefits and cost reduction possibilities of a highly interoperable public service compared to proprietary development.

Question Has the public service been evaluated in terms of its cost and benefits before deciding on whether/how it should be implemented (e.g. through conducting an ex ante Business Case)?

No, cost and benefits of the public service are not identified

Yes, cost and benefits of the public service were detailed based on a common business case approach (e.g. cost-benefit analysis, total cost of ownership calculation)

Yes, cost and benefits of the public service were detailed based on a common business case approach. In addition multiple scenarios (e.g. proprietary solution versus reuse) were compared with each other to better understand the cost and benefits of increased interoperability

Examples No, the public service has not been evaluated in terms of its cost and benefits. Yes, the public service has made an inventory of all cost categories but did not detail the impact of interoperability.

D.2

Name Service Provisioning

EIF-Layer Organisational interoperability, Technical interoperability

Weight 25%


Rationale Public services that provide digital services for reuse towards other administrations and/or business contribute proactively towards a higher interoperability in the public domain.

Question Does your public service provide services towards the external environment for reuse?:

The public service makes no services available towards the external environment, while this would be possible.


of 56

The public service makes no services available towards the external environment due to constraints.

The public service makes some services available towards the external environment.

The public service makes available all services towards the external environment.

Examples The public service offers a currency conversion service to external users.

D.3

Name Procurement criteria


Weight 5%


Rationale A strong focus on certain procurement criteria can contribute to a high interoperability by avoiding common pitfalls and ensuring that services are only procured and/or developed when not available from other administrations or businesses.

Question Has standardization been a procurement criterion when procuring the service's components?:

No.

Yes, however not enforced sufficiently

Yes, and enforced to ensure compliance

Examples There is no set of specific procurement criteria. Yes, procurement criteria have been detailed but not been enforced.

D.4

Name Central point of control

EIF-Layer Organisational interoperability; Technical interoperability

Weight 10%


Rationale A central point of control facilitates the choreography of external services and provides a single source of intelligence regarding the status of individual cases

Question Does the public service feature a central point of control for choreography of externally consumed and provided services? The central point of control keeps track of all related information regarding the status of the individual cases currently active within the public service.

No

No, this is decentralized or not considered relevant

Yes

Examples All external transactions are coordinated with the help of a central


of 56

point of control – status information is always centrally available; there is no central point of control in place to monitor the status of a public transaction – this is a decentralized process and information is to be provided on request.

D.5

Name Level of automation of the choreography


Weight 10%


Rationale Automation of the choreography facilitates a rapid and seamless interaction between the public service and the consumed and provisioned services.

Question To what extent is the choreography automated?

Fully manual (all transactions are handled manually)

Semi-automated (a part of the service choreography relies on manual interference)

Fully automated (no manual interference is required)

Examples Service choreography is manual or semi-automated when the required orchestration requires (some) manual interaction. A public service is considered fully automated when all required service transactions are tracked automatically and no manual interference is required. Note that this question does not address the topic of exception handling. The service choreography can be fully automated (applying to all transactions) but still manual intervention can be required for certain exceptions or errors (this is discussed under the topic exception handling)

D.6

Name Status information

EIF-Layer Semantic interoperability; Technical interoperability

Weight 5%


Rationale Sending status information indicates that the service is seamlessly interacting with other services

Question Does the public service share status information on the cases handled with external services?

No status information shared

Yes, with some services

Yes, systematically with all services

Examples The service sends up-to-date information on the status of individual cases handled through to the service owners with which it has either a


of 56

consumption or provisioning relationship.

D.7

Name Business process definitions and rules


Weight 5%


Rationale Business process definitions and rules are the basis for day-to-day collaboration, providing actionable directives that govern the service’s interactions with the other services.

Question Does the service establish business process definitions (to describe the source and target processes of the exchange) and/or business process control rules (e.g. rules for process control, validation, quality control, tracking and tracing) jointly with the orchestrated services?

No, processes are not modelled

No, even though processes are modelled

Yes, in some cases.

Yes, systematically with all services.

Examples The collaboration business rules describe and regulate how the interoperation should take place and how the communication between service owners is established by e.g. harmonizing workflow definitions and procedures around responsibility & liability, communication and usage monitoring.

D.8

Name Business Process Management standards


Weight 5%


Rationale Business Process Management standards are (open) standards and specifications used to model and execute business processes, ideally in an interoperable manner.

Question To what extent are Business Process Management (BPM) standards applied to the orchestration of the public service?

Business processes are modelled at all

Business processes are modelled and executed on a proprietary basis

Business processes are modelled and executed using BPM standards

Examples Examples of prominent standards are Business Process Modelling Notation (BPMN) 2.0, Web Services Business Process Execution Language (WS-BPEL) 2.0 and XML Process Definition Language (XPDL)


of 56

2.1.

D.9

Name Architectural Framework


Weight 5%


Rationale Using existing common architectural frameworks ensures that the administration is leveraging best practices, avoids pitfalls and designs a public service that is interoperable with other public services and/or public service domains.

Question Has the public service considered an architecture framework in its design (EU, national level, international (open) standard)?

No, although relevant frameworks are available

No, there are no relevant frameworks available to consider

Yes, one or multiple architecture frameworks are used.

Yes, one or multiple architecture frameworks are used and the compliance is ensured by independent audits.

Examples The public services is aligned with a set of frameworks on the European-level such as EIRA (European Interoperability Reference Architecture) or at a national level (such as NORA in The Netherlands).

D.10

Name Architectural flexibility


Weight 10%


Rationale Architectural flexibility enables greater interoperability by e.g. building functionalities as software components which can be reused for different purposes and loosely coupling services with operating systems and other technologies that underlie them.

Question Has the service’s architecture been designed in a way that it is flexible for future upgrades and/or interconnections with other services?

No, the architecture cannot be considered flexible

The architecture allows for some flexibility.

Yes, the architecture is highly flexible.

Examples Highly configurable solutions typically incorporate a modular design approach (e.g. Service-Oriented-Architecture SOA) to enable flexibility and interoperability of services across multiple public administrations.

D.11


of 56

Name Specification process

EIF-Layer Legal interoperability; Organisational interoperability

Weight 10%


Rationale Providing an open process to establish specifications is likely to yield more interoperable results.

Question Has the public service established an (open) specification process in which administrations and businesses can participate?

No, the specification process is closed

Yes, participation upon invitation

Yes, open participation

Examples There is a dedicated forum which is accessible for everybody to post ideas and participate in discussions around the public service (fully open); administrations and businesses first need to be invited to join the specification process (semi-open).

2. Maturity scoring


Table 9 - Scoring table: Service Management (D)

Question Ad hoc Opportunistic Essential Sustainable Seamless D.1

No, cost and benefits of the public service are

not identified

Yes, cost and benefits of the public service were detailed

based on a common

business case approach (e.g.

cost-benefit analysis, total

cost of ownership calculation)

Yes, cost and benefits of the public service were detailed

based on a common business case approach. In addition multiple scenarios were

compared

D.2 The public service

makes no services available

towards the external

environment, while this would be possible

The public service makes

no services available

towards the external

environment due to

constraints

The public service makes some services

available towards the

external environment

The public service makes available

all services towards the

external environment

D.3 No

Yes, however not enforced

Yes, and enforced

to ensure


of 56

sufficiently compliance

D.4

No

No, this is decentralized or not considered

relevant

Yes

D.5 Fully manual

(all transactions are handled manually)

choreography

Semi-automated (a

part of the service

choreography relies on manual

interference) choreography

Fully automated (no manual

interference is required)

choreography

D.6 No status

information shared

Yes, with some

services

Yes, systematically

with all services

D.7 No, processes

are not modelled

No, even though

processes are modelled

Yes, in some cases.

Yes, systematically

with all services.

D.8 Business

processes are not

modelled at all

Business processes are modelled and executed on a

proprietary basis

Business processes are modelled and

executed using BPM standards

D.9

No, although relevant

frameworks are available

No, there are no relevant

frameworks available to

consider

Yes, one or multiple

architecture frameworks are

used.

Yes, one or multiple

architecture frameworks are

used -independent

audits compliance-.

D.10 No, the architecture cannot be considered

flexible

The architecture allows for some

flexibility.

Yes, the architecture is highly flexible.

D.11 No, the specification

process is closed

Yes,

participation upon invitation

Yes, open participation


of 56

5. Service conformance to STOP&GO Interoperability Framework (E)

1. Questions

E.1

Name Legislation Compliance about user treatment and data processing

EIF-Layer Legal Interoperability

Weight 5%


Rationale The compliance of some national or European directives is necessary to guarantee the right data treatment (as the adoption of the Directives Directive 95/46/EC and 2002/58/CE)

Question Has it taken into consideration the country legislation about the user treatment and/or data processing?.

No

Yes, partially (one or another)

Both cases

E.2

Name Security of data

EIF-Layer Legal Interoperability

Weight 5%


Rationale Preserving the security of data is essential in every service but sometimes no national legislation is accomplished.

Question Has it taken the necessary appropriate technical and organizational measures to preserve the security of data?.

No

Yes, but not taking into consideration any regional or national

directives

Yes, Directive 2002/58/CE achievement

E.3

Name Conformance to ISO EN 13940: System of concepts to support continuity of care.

EIF-Layer Organisational Interoperability, Semantic Interoperability

Weight 25%


Rationale Defining a system of organisational concepts which are common for the services is a necessity for the whole Project. Its adoption avoids possible misunderstanding with the concepts.

Question Does the service establish the use of some kind of organizational


of 56

concepts system?

No, it is not planned

Yes, a particular organisational system of concepts is used

Yes, some ISO EN 13940 concepts are used

Yes, totally conformance with ISO EN 13940

E.4

Name Reference Model

EIF-Layer Semantic Interoperability, technical interoperability

Weight 10%


Rationale A reference model is a key piece in the dual model (information/knowledge). It contents the basic entities for representing any information in EHR.

Question Is some standard reference model used?

No

Yes, a proprietary reference model is used

Yes, a standard reference model is used (OpenEHR, RIM HL7

v3, etc.) 17

Yes, the ISO EN 13606 reference model

E.5

Name Export information


Weight 20%


Rationale An interoperability requisite for the STOP&GO interoperability Framework (SAGIF) is the information reusing for upgrading, futures deployments or for transferring the information to another systems. A positive assessment of this question eases the interoperability maturity of the service.

Question Is the exportation of information taking into account for futures developments?

No information is exported

Yes, according to a proprietary reference model extract

Yes, according to other standards (OpenEHR, RIM HL7 v3,

etc.)18

Yes, according to ISO EN 13606 Reference Model Extract

17

Please, indicate what is the standard used 18

Please, indicate what is the standard used


of 56

E.6

Name Use of Archetypes (or concept models)


Weight 10%


Rationale Archetypes are one of the key piece in the dual model (information/knowledge). It contents the formal definitions of clinical concepts (knowledge) in EHR.

Question Have extracts conforming to archetypes been using?

No extracts or archetypes are used.

Yes, the extracts are according to a proprietary archetypes.

Yes, the extracts are according to other standards (OpenEHR,

RIM HL7 v3, etc.) 19

Yes, the extracts are according to ISO EN 13606 Archetypes

E.7

Name Availability of Archetypes (or concept models)


Weight 10%


Rationale The availability of an archetypes repository make easier the adaptation of one standard and the possibility of sharing knowledge. It avoids the work and task duplication.

Question Are the archetypes (or concept models) available for consulting?

No archetypes (or concept models) are used.

No, archetypes (or concept models) are not available

Yes, archetypes (or concept models) are available for local

consulting.

Yes, archetypes (or concept models) are publically available

for consulting.

E.8

Name Extracts codification


Weight 15%


Rationale

Question Are the extracts codification according to XML-Schemas?

No archetypes or extracts are used.

Yes, some of the codification is according to XML-Schemas

Yes, the whole extract codification is according to XML-

Schemas.

19

Please, indicate what is the standard used.


of 56

2. Maturity scoring


Table 10 - Scoring table: Service conformance to SaGIF (E)


E.1 No

Yes, partially (one or

another)

Both cases

E.2

No

Yes, but not taking into

consideration any regional or

national directives

Yes, Directive 2002/58/CE achievement

E.3

No, it is not planned

Yes, a particular

organisational system of

concepts is used

Yes, some ISO EN 13940

concepts are used

Yes, totally conformance with ISO EN

13940

E.4

No

Yes, a proprietary reference

model is used

Yes, a standard reference

model is used (OpenEHR, RIM HL7 v3,

etc.)

Yes, the ISO EN

13606 reference model

E.5

No information is exported

Yes, according to a

proprietary reference

model extract

Yes, according to other

standards (OpenEHR, RIM HL7 v3,

etc.)

Yes, according to ISO EN 13606

Reference Model Extract

E.6 No extracts

or archetypes are used.

Yes, the extracts are

according to a proprietary archetypes.

Yes, the extracts are according to

other standards (OpenEHR, RIM HL7 v3,

Yes, the extracts are according to

ISO EN 13606 Archetypes


of 56

etc.)

E.7 No

archetypes (or concept models) are

used

No, archetypes (or

concept models) are not available

Yes, archetypes (or

concept models) are available for

local consulting

Yes, archetypes (or concept models) are

publically available for consulting.

E.8 No

archetypes or extracts are used.

Yes, some of the

codification is according to

XML-Schemas

Yes, the whole extract

codification is according to

XML-Schemas.


of 56

Bibliography

[1] European Commission, “eEurope 2005: An information society for all.

COM(2002) 263 final.,” 2002 5 28. [Online]. Available: http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2002:0263:FIN:EN:PDF.

[Accessed January 2016].

[2] IDA -Interchange of Data between Administrations- and Capgemini, “European

Commission. Study on stakeholder requirements for pan-Euroepan eGovernment

Services. Final Report. Ranking and description of PEGS. V1.3,” [Online].

Available: http://ec.europa.eu/idabc/servlets/Docc7f6.pdf?id=19649. [Accessed

October 2015].

[3] European Commission, “European Commission -Towards interoperability for

European public services. ISA European Interoperability Framework (EIF) for

European public services. COM(2010) 744 final,” 16 December 2010. [Online].

Available: http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf.

[Accessed January 2016].

[4] European Commission and Capgemini, “Online Availability of Public Services:

How Is Europe Progressing? (2006). Web Based Survey on Electronic Public

Services,” June 2006. [Online]. Available: http://bookshop.europa.eu/es/online-

availability-of-public-services-how-is-europe-progressing--pbKK0414477.

[Accessed December 2015].

[5] European Commission, Capgemini, IDC, Rand Europe, Sogeti and DTi, “Digitizing

Public Services in Europe: Putting ambition into action. 9th Benchmark

Measurement,” December 2010. [Online]. Available:

https://ec.europa.eu/digital-agenda/sites/digital-agenda/files/egov_report.pdf.


[6] European Commission, “Towards interoperability for European public services,

Commission Communication to the European Parliament, the Council, the

European Economic and Social Committee and the Committee of the Regions,

COM(2010) 744 final, 16.12.2010.,” 16th December 2010. [Online]. Available:

http://ec.europa.eu/isa/documents/isa_iop_communication_en.pdf. [Accessed

November 2015].

[7] European Commission, “Commission 2013 annual growth survey, COM(2012)

750 final, 28.11.2012,” 2012.

[8] European Commission, “European Interoperability Strategy 2012.

Implementation Review, report on political, socio-economic and legal factors.,”


of 56

2012.

[9] European Commission, “Commission Communication EUROPE 2020 A strategy

for smart, sustainable and inclusive growth,,” 2010. [Online]. Available:

http://eur-


[Accessed Oct 2015].

[10] European Commission, “A Digital Agenda for Europe, Commission

Communication to the European Parliament, the Council, the European

Economic and Social Committee and the Committee of the Regions, COM(2010)

245 final, 28.8.2010,” 2010. [Online]. Available: http://eur-



[11] European Commission, “e-Commission 2012-15: Delivering user-centric digital

services, Communication from VP Šefčovič to the Commission, SEC(2012) 492

final, 1.8.2012,” 2012.

[12] European Commission, “(Decision 922/2009/EC of the European Parliament and

of the Council of 16 September 2009 on interoperability solutions for European

public administrations (ISA), OJ L 260, 3.10.2009, p. 20.),” 2009. [Online].

Available: http://ec.europa.eu/isa/documents/isa_lexuriserv_en.pdf. [Accessed

September 2015].

[13] European Commission, “Decision 2015/2240 of the European Parliament and ...

of 25 November 2015 establishing a programme on interoperability solutions and

common frameworks for European public administrations, businesses and

citizens (ISA2 programme). OJ L 318, 4.12.2015, p.16,” [Online]. Available:

http://ec.europa.eu/isa/documents/celex_en.pdf. [Accessed January 2016].

[14] Alonso J., De Soria I., Orue-Echevarria L., Vergara M., “Enterprise Collaboration

Maturity Model (ECMM): Preliminary Definition and Future Challenges.,” in

Enterprise Interoperability IV, Part VII, Heidelberg, Springer, 2010, p. 429–438.

[15] Guédria W., Naudet Y., Chen D. et al., “A Maturity Model Assessing

Interoperability Potential.,” in Enterprise, Business-Process and Information

Systems Modeling. 12th International Conference, BPMDS 2011, and 16th

International Conference, EMMSAD 2011, Heidelberg, Springer, 2011, pp. 276-

283.

[16] Center for Technology in Goverment, “Center for Technology in Goverment.

University at Albany. State University of New York,” [Online]. Available:

http://www.ctg.albany.edu/publications/reports/improving_government_intero

perability?chapter=5t. [Accessed January 2016].


of 56

[17] Cresswell, A.M., T.A. Pardo, D.S. Canestraro, and S.S. Dawes., “Why Assess

Information Sharing Capability?,” The Center for Technology in Government, the

Research Foundation of State University of New York, Albany, NY, 2005.

[18] C4ISR Interoperability Working Group, “Levels of information systems

interoperability (LISI),” Tech. report, US Department of Defense, Washington,

1998.

[19] Clark, T., Jones, R., “Organizational interoperability maturity model for c2,” in

Proc. of the 1999 Command and Control Research and Technology Symposium,

Washington, 1999.

[20] Tolk, A., Muguira, J.A., “The levels of conceptual interoperability model,” in Fall

Simulation Interoperability Workshop, USA, 2003.

[21] “ATHENA.: Advanced Technologies for Interoperability of Heterogeneous

Enterprise Networks and their Applications,” FP6-2002-IST1, Integrated Project

Proposal, UE, 2003.

[22] H. Panetto, “Towards a classification framework for interoperability of enterprise

applications.,” International Journal of Computer Integrated Manufacturing, vol.

8, no. 20, pp. 727-740, 2007.

[23] Guédria, W., Naudet, Y., Chen, D, “A Maturity Model for Enterprise

Interoperability,” in 4th IFAC/IFIP, OTM Workshop, Vilamoura, 2009.

[24] ISA - Intereporability solutions for European Public Administratios. European

Commission, “Interoperability Maturity Model,” March 2016. [Online]. Available:

http://ec.europa.eu/isa/ready-to-use-solutions/imm_en.htm. [Accessed May

2016].

d3.2 guidelines on interoperability...

Documents