future internet technologies for manufacturing 24/04/2014 … · 2017-04-25 · project id 604674...

Project ID 604674 FITMAN – Future Internet Technologies for MANufacturing

24/04/2014 Deliverable D4.4

D4.4 – FITMAN Technical / Business Indicators for Smart Factory

V1.5

Document Owner: Giacomo Tavola POLIMI

Contributors: Chiara Galbusera – POLIMI, Stefano Perini – POLIMI, Alessandro Canepa – Piacenza,

Pierluigi Petrali – Whirlpool, Ignacio Arconada – TRW, June Sola – INNOVALIA, Angelo

Naselli – Softeco, Marco Masetti – Softeco, Mauro Isaja – Engineering, Guy Doumeingts

– IVLAB

Dissemination: Public Contributing to: WP 4.4 - FITMAN Technical / Business Indicators for Smart Factory

Date: 24.04.2014

Revision: 1.5



2

VERSION HISTORY

VERSION DATE NOTES AND COMMENTS

0.1 14/10/2013 TOC and initial draft

0.2 4/12/2013 First Draft of Technical Indicators and Business Performance Indicators for

Whirlpool Trial (POLIMI)

0.5 03/02/2014 First Draft of the document (POLIMI)

0.8 27/03/2014 Final version of the document (POLIMI), unless of:

TRW final business indicators identification,

as-is and target values for Whirlpool, TRW and Piacenza,

filled business indicators templates of TRW and Piacenza

0.9 28/3/2014 KPI analysis and business scenarios refinement

0.95 28/3/2014 Document restructuring:

1. Missed Actual Value and Target for Piacenza

2. To finalize Technical indicators

3. To finalize Self Certification

4. To finalize the cross trial evaluation of PI

1.0 10/4/2014 Final Version for internal revision

1.1 11/4/2014 INNO Comments

IV-LAB final validation of Smart Factory PI

1.2 12/4/2014 Final version for peer review

1.3 19/04/2014 Sergio Gusmeroli Comments

1.4 21/04/2014 Addressing Sergio’s Comments (see below SG1, SG2 and SG3)

1.5 24/04/2014 Alignment with D4.1, Refinement with Smart Factory Trials

Final Version

DELIVERABLE PEER REVIEW SUMMARY

ID Comments by KJ Addressed ()

Answered (A)

1 Describe why it is not possible to have common BIs. (RR#3, cross trial

assessment)

A - To be finalized in answer to RR#3, see also

GT2

2

In section “1.2. Structure of the Document” include also reference to 4.1 Process

Assistant & 4.2.Specific Issues

3 Update Figure 1 and Table 3 (self-certification of SEs only)

4 Table Captions before (above) tables.

Applies to all tables.



3

5

BI AS-IS and Target not defined for

Piacenza in version 0.95

6

Remove section 3.4.2 Self-certification for

Smart Factory Trials GEs

Remove section 3.4.3 Self-certification

for Smart Factory Trials TSC and TICs

7

User community definition missing

Business Indicators Templates in

Annexes.

A - These values are not available at the moment and will be provided during T2.5 actual

execution of V&V process, as needed for Survey

Form delivery to stakeholders

8

Whirlpool and WHR used with the same meaning throughout the document. Recommendation; please use “Whirlpool”

in all places.

9 Additional Comments may be raised

after cross checking with D5.4 and D6.4

GT1

WHIRLPOL and Piacenza Trials not

considering: Actions to react

depending on the value of the PI in PI

indicator description

GT2

Insert in conclusions, clustering of

indicators as defined in Bruxelles 31/3

and 1/4

A - To be finalized in RR3 answer presentation

SG1 Make more evident the VOICE of

CUSTOMERS A - Refinement of Trial objectives definition

with Trial Owners (Whirlpool and Piacenza).

SG2 Avoid Duplication with WP2 and T4.1

1. Minimum description of methodology,

references to D2.2 and D2.3 (some

detail move to Annex for reference)

2. Alignment with D4.1

SG3 PI Definition process with Trial Added 8 ANNEX IV Business

Performance Indicators refinement process

to describe PI refinement process.



4

Table of Contents

EXECUTIVE SUMMARY ....................................................................................................................................... 6

1. INTRODUCTION ............................................................................................................................................ 7

1.1. Objectives of the Task 4.4 ..................................................................................................................... 7 1.2. Structure of the Document .................................................................................................................... 7

2. VERIFICATION AND VALIDATION CONCEPTS .................................................................................... 9

2.1. Methodological Approach for Indicators ............................................................................................... 9 2.2. Methodological Approach for Business Performance Indicators ...........................................................10 2.3. Methodological Approach for Technical Indicators .............................................................................11 2.4. Methodological Approach for Software Verification .............................................................................11

3. FITMAN SMART FACTORY TRIALS TECHNICAL AND BUSINESS PERFORMANCE INDICATORS

12

3.1. Data Collection Form ........................................................................................................................12 3.2. Business Performance Indicators for Smart Factory Trials...................................................................13

3.2.1. Whirlpool Trial ............................................................................................................................................. 14 3.2.2. Piacenza Trial ................................................................................................................................................ 21 3.2.3. TRW Trial ..................................................................................................................................................... 28

3.3. Technical Indicators for Smart Factory Trials ......................................................................................33 3.4. Self-certification for Smart Factory Trials SW Components ..................................................................33

4. CONCLUSIONS & NEXT STEPS .................................................................................................................35

4.1. Process Assistant .................................................................................................................................37 4.2. Specific Issues .....................................................................................................................................37 4.3. Next steps ............................................................................................................................................39

5. ANNEX I : REFERENCES ............................................................................................................................40

6. ANNEX II : GLOSSARY AND TERMINOLOGY ........................................................................................41

7. ANNEX III: BUSINESS PERFORMANCE INDICATORS TEMPLATES .................................................42

7.1. Whirlpool – Business Performance Indicators Templates .....................................................................42 7.2. Piacenza – Business Performance Indicators Templates .......................................................................44 7.3. TRW – Business Performance Indicators Templates .............................................................................45

8. ANNEX IV BUSINESS PERFORMANCE INDICATORS REFINEMENT PROCESS ..............................51

8.1. Whirlpool ............................................................................................................................................51 8.2. Piacenza .............................................................................................................................................52 8.3. TRW ....................................................................................................................................................53

9. ANNEX V: TECHNICAL INDICATORS......................................................................................................55

9.1. Functional Technical Indicators ..........................................................................................................55 9.2. Non-Functional Technical Indicators ...................................................................................................56 9.3. Software Verification (Self Certification) .............................................................................................57



5

Tables

Table 1 Data Collection Form ....................................................................................................... 12

Table 2 Whirlpool SW components ............................................................................................... 15 Table 3 Whirlpool relevant events ................................................................................................. 16

Table 4 Whirlpool – Phase 1 ECOGRAI ....................................................................................... 17 Table 5 Whirlpool – Output from ECOGRAI ................................................................................ 18

Table 6 Whirlpool Business Performance Indicators AS-IS and Target values............................... 19 Table 7 Piacenza SW components ................................................................................................. 21

Table 8 Piacenza – Phase 1 ECOGRAI ......................................................................................... 22 Table 9 Piacenza – Output from ECOGRAI .................................................................................. 23

Table 10 Piacenza Business Performance Indicators AS-IS and Target values ............................... 28 Table 11 TRW SW components .................................................................................................... 29

Table 12 TRW – Phase 1 ECOGRAI............................................................................................. 29 Table 13 TRW - Output from ECOGRAI ...................................................................................... 30

Table 14 TRW Business Performance Indicators AS-IS and Target values .................................... 31 Table 15 GEs and SEs for Smart Factory Trials............................................................................. 33

Table 16 SEs and related Software Developers .............................................................................. 33 Table 17 Summary Technical PIs .................................................................................................. 35

Table 18 Summary Business Performance Indicators .................................................................... 35 Table 19 Whirlpool Results after the First iteration ....................................................................... 51

Table 20 Whirlpool Results after the Second iteration ................................................................... 51 Table 21 Piacenza Results after the First iteration ......................................................................... 52

Table 22 Piacenza Results after the Second iteration ..................................................................... 53 Table 23 TRW Results after the First iteration ............................................................................... 53

Table 24 TRW Results after the Second iteration .......................................................................... 54 Table 25 Functional Technical Indicators ...................................................................................... 55

Table 26 Non-functional Technical Indicators ............................................................................... 56 Table 27 Self-certification Methodology ....................................................................................... 57

Figures

Figure 1 Business Performance Indicators, Technical Indicators and Verification Tests ................ 10

Figure 2 ECOGRAI methodology approach .................................................................................. 10 Figure 3 Whirlpool - WU Production line...................................................................................... 14

Figure 4 Piacenza Timing of clothing main activities .................................................................... 24 Figure 5 Piacenza exploitation per department in 2013 .................................................................. 25

Figure 6 Piacenza Production Process ........................................................................................... 25 Figure 7 Piacenza energy consumption - 1 .................................................................................... 26

Figure 8 Piacenza energy consumption - 2 .................................................................................... 26



6

Executive Summary

This document is devoted to the definition of the experimentation metrics to be employed

for FITMAN Smart Factory Trials impact assessment, both from the technical and business

perspective. The approach and methodology has been defined in WP2 FITMAN

Verification & Validation Method including the development of suitable tools for data

collection and consolidation. This task defines and coordinates the definition and

measurement of suitable metrics that will be used during experimentation and will support

the implementation of the Smart Factory trials best practices. This task is responsible of

consolidating the WP knowledge in the form of guidelines about successful FITMAN

Platform instantiation, FITMAN platform experimentation management and FITMAN

Smart Factory experimentation metric dashboard and metric information collection. It will

create a solid knowledge foundation for T7.1, synthesis of Use Case Trials Experiences and

Consolidation of results, T8.1 FITMAN Use Case Trials comparative evaluation and future

Phase III extensions, where SMEs can benefit from these practices in terms of new services

generation and platform regional instantiation.

The three SMART Factory Trials has been involved in a comprehensive exercise aiming to

identify a significant set of Business Performance Indicators (implementing WP2

methodology) able to demonstrate actual impact of adoption of FITMAN paradigm in

solution development. This has been the occasion for business stakeholders in the three

companies to approach the evaluation of the performances of their business processes from

a different perspective. In some cases (Whirlpool) some of the identified indicators are

brand new and despite AS-IS values are not available, the trial owner decides to implement

them as they perceive these indicators will greatly help to improve the effectiveness of the

monitoring of the processes.

The teams have been involved in an iterative process to come to the definition of PIs

according to the simplified ECOGRAI methodology. The final results are reported in Table

18 Summary Business Performance Indicators. It is remarkable to note that, with the join efforts

of Trail team and Project team, we came to a very compact set of Business Performance

Indicators for each of the identified Business Scenarios in trials.

Main focus of T4.4 was to come to the definition of the indicators able to monitor the

performances of the deployed system, specifying the AS-IS values and the expected results

(Target). T2.5 will take the ownership (continuing T2.4 - Instantiation of V&V Assessment

Package per Trial activities) of conducting the actual collection of Data and implementing

tools, organizing and monitoring the data gathering process.

Next release of D4.4 will address, based on experience gained in the first months of

implementation of trial, the refinement of identified Indicators, the management of data

analysis and related feedback to trials, preparing the knowledge base for T7.1 Synthesis of

Use Case Trials Experiences.

Contributing Partners: Polimi, Piacenza, Softeco, TRW, INNOVALIA, Whirlpool,

Engineering, IPK , TXT, IV-LAB.



7

1. Introduction

In the present document, on the base of WP2 V&V Methodology Technical and Business

Performance Indicators for each Smart Factory Trial are identified and explained in detail.

On one hand, the Technical Indicators have been extracted from WP2 and then properly refined or

aggregated in order to respond as better as possible to the specific FITMAN Trials environments. In

this respect, three Indicators have been designed in order to validate each of the Software

Component (i.e. GEs/SEs) implemented in the Smart Factory Trials, taking into account their

effective operation once instantiated in the specific Trial Use Case.

For this kind of Indicators an overall evaluation by the specific Trial is requested. Then, other five

Indicators have been designed in order to have feedbacks about the whole Trial Integrated Solution,

collecting in this case the different perceptions of all the stakeholders of the Trial via a community

based survey. The final result is a solid common framework able to validate at different levels of

detail the IT aspects of the Smart Factory Trials.

On the other hand, the methodology for the definition of the Business Performance Indicators (i.e.

the Simplified ECOGRAI Methodology) has been derived from WP2 and then applied to the

different Smart Factory Trials. In particular, the Business Performance Indicators represent the final

mediation between the theoretical initial list derived against the Trials Business Requirements [1]

and the specific needs and possibilities of the Trials. The final list of Business Performance

Indicators in fact has been validated by the different Trial Owners, who checked the specific data

availability and measurability with the help of their IT Support Partners. In this second case, the

final result is hence a customized solution for each of the Smart Factory Trial, according to their

specific Business Requirements and Business Scenarios.

1.1. Objectives of the Task 4.4

The objective of Deliverable 4.4, as from the Description of Work, is to define and coordinate the

suitable metrics that will be used during experimentation and coordinate the generation of the Smart

Factory best practices. This task will be responsible of consolidating the WP knowledge in the form

of guidelines about successful FITMAN Platform instantiation, FITMAN platform experimentation

management and FITMAN Smart Factory experimentation metric dashboard and metric

information collection. This task will create a solid knowledge foundation for future Phase III

extensions, where SMEs can benefit from these practices in terms of new services generation and

platform regional instantiation.

1.2. Structure of the Document

In Chapter 1 – Introduction, an overall presentation of the document structure and contribution to

other Deliverables/Tasks is provided.

In Chapter 2 – Verification and Validation Concepts, the methodological approach associated to

each Smart Factory Trial is presented, including both Business Performance and Technical

Indicators. Key elements are mentioned, while a complete reference can be found in D2.1 FITMAN

Verification & Validation Method and Criteria [1], D2.2 FITMAN Business and Technical

Indicators [2] and D2.3 FITMAN Verification & Validation generic Assessment Package [3].

In Chapter 3 - FITMAN Smart Factory Trials Technical (IT) and Business Performance Indicators,

the Technical and Business Performance Indicators of each Smart Factory Trial are presented in

detail including the criteria and approach for their identification and consolidation.

In Chapter 4 – Conclusions & Next Steps, the overall conclusions and the explanation of the

planned future actions are given.

In this section is also introduced the concept of Process Assistant, a structured and unique

repository of all collected information during the validation and verification process, as well with



8

non-structured (free format text) information describing the issues and significant event

encountered during the trial development and that will be utilized in WP7 and WP8.

The specific issues encountered in setting up the PIs system are described based mainly on

confidentiality reasons.

Contribution to other Deliverables/Tasks

D4.4 is providing contribution to:

T2.5 Continuous adaptation and support of the V&V package in the trials

T7.1 Synthesis of Use Case Trials Experiences

T8.1 FITMAN Use Case Trials comparative evaluation

T8.2 FITMAN Expanded Trials Proposition, T8.3 FITMAN SMEs Innovation Preparation,

T8.4 FITMAN Support to Phase III Expansion of Use Cases



9

2. Verification and Validation Concepts

The concepts described in this Chapter rely on the results provided by WP2. The next Paragraphs

2.1and 2.2 describe the specific instantiation of the results of WP2 for the Smart Factory Trials, and

in particular the methodological approach to define the indicators and tests used. It needs to be

remarked that as a key characteristic of the approach same concepts apply to the three domains

(SMART, Virtual and Digital) and eventually it is suitable to be adopted for other trials in Phase III.

These contents will also support the creation of the best practices for this kind of Trials.

General aspects and ideas of the concept are drafted below:

1. Meaningful values of Indicators from end user companies (Piacenza, TRW and Whirlpool)

will be identified and mapped (in case of confidentiality issues) to anonymized units. AS

IS/TO BE values of Indicators are identified and mapped to units for confidentiality

purposes e.g. map time in hours to time in units. This will still lead to show the

improvement but without giving the specific numbers of the company.

2. Procedure for the assessment of the trials:

• Represent the current process identifying key functional aspects and meaningful

parameters (time, cost, …)

• Based on the configuration of the trials, analysis of the architecture, identifying the

Trial’s components GEs/SEs/TSCs/TICs

• Implementation of the system is carried out by Trial owner and technology partners

• Collection of indicators with appropriate selection of time frames and granularity of

information

• Evaluation

3. The evaluation should include:

• Intuitively applicable use of the measurement system (end-user)

• Benefits from the GEs/SEs/TSCs/TICs (positive/negative/comments)

complexity

granularity

e.g. “The SE is too complex and we need only a part of it”

2.1. Methodological Approach for Indicators

The methodological approach for the instantiation includes three main elements: the Business

Performance Indicators, the Technical Indicators and the Verification Tests.

Technical Indicators (which cover from P5 to T1 steps of the FITMAN V&V Methodology

developed in D2.1 [1]) aims at measuring technical performances of the software components and

of the entire solution, in order to understand if the product is built and works in the right way. A

reduced number of 8 indicators has been selected among a wider list: five of them are non-

functional and more qualitative users opinions, three of them are functional and evaluated at each

software component level); these indicators are replicated for all the trials. Business Performance

Indicators (which cover the T2 step of the FITMAN Methodology [1]) have been identified at

Business Scenario level through the ECOGRAI process [3], according to the trials objectives. For

each Business Performance Indicator, the trials are required to report the current value, the target

value they want to achieve and the values after the solution implementation. In order to perform P1-

P5 steps of the FITMAN Methodology [1], the software components are evaluated through the

Verification tests.

The Business Performance Indicators and the functional Technical Indicators are addressed by the

Trial Owner; the non-functional Technical Indicators require the crowd engagement, therefore all

the trial team members; the Software Components developers are responsible to evaluate their

components with recommended or alternative techniques, and report results through a self-

certification.

The methodological approach for the instantiation is represented in the following scheme:



10

Figure 1 Business Performance Indicators, Technical Indicators and Verification Tests

In the next paragraph, each of the three adopted techniques represented in the picture is described.

For a detailed description of the methodology, please refer [1] and [2]

2.2. Methodological Approach for Business Performance Indicators

The objective of a Performance Indicators system is to see what’s happen in the controlled system

in order to make the right decisions at the right time.

The figure below shows these principles and related results of various WP’s:

Figure 2 ECOGRAI methodology approach

For FITMAN Smart Factory Trials Business Performance Indicators collection, a simplified version

of ECOGRAI [3] has been used. It includes only three phases in order to facilitate the application

and to be in line with the size of the use cases and the duration of the project. The phases are:

First Phase: Description of the system in which the Performance Indicators (PIs) will be defined,

including Functions, Processes, Boundaries and Business Objectives.

Second Phase: According to the Objectives of the system the owner of the system determines the

potential actions to reach them (called Decision Variables (DV) or Action Variables (AV)).

Third Phase: the Performance Indicators indicate or characterize the reaching of the Objectives by

using the DV/AV.

For details on ECOGRAI methodology and FITMAN implementation, please refer [2].

Business process 1

Business process 2

Business Scenario Business PI’s

GE SE GE SE

Decision system

Decisions (actions on decision variables)

Objectives

WP2

WP3WP3



11

2.3. Methodological Approach for Technical Indicators

Consistently with WP2 outcomes (D2.1 [1] and D2.2 [2]), a set of Technical Indicators has been

defined and instantiated both at Software Component and at Trial level for all the trials belonging to

different domains.

POLIMI has been the responsible of the interaction with the different Trial Support Partners in

order to guarantee the endorsement and the agreement on all the reported Technical Indicators.

A common list of Technical Indicators to be used in all the Trials has been established. The choice

of a unique framework has been motivated mainly by two concrete advantages:

To reduce the potential complexity related to the evaluation of the different Trial Integrated

Solutions. Specific choices for each Trial would lead to a confusing system without concrete

added value for our V&V purposes (WP2);

To allow an effective and efficient comparison of the final results of each Trial from the IT

side based on a common set of indicators.

The complete list includes eight Technical Indicators, i.e. three specific for the evaluation of the

single GEs/SEs and five for the evaluation of the whole Trial Integrated Solution.

The Technical Indicators derive directly from the IT V&V Criteria identified in D2.1 [1]. Those

Criteria derive in turn from six IT V&V Criteria Categories, in alignment with the ISO 9126

standard (International Organization for Standardization, 2001). However, even if the Criteria

Categories derive from the ISO 9126 standard, the Criteria selected include also additional

elements. The result is hence a mix between the strong foundations of the ISO 9126 standard and

the integration of ad hoc Criteria, enabling the formation of a complete and exhaustive Validation

process [1].

The related specific Technical Indicators have been subsequently selected and extracted from D2.2

[2] and further elaborated and integrated in order to be effectively implementable and useful in the

Trials environments.

For a complete list of the adopted Technical Indicators please refer to D2.2 [2] and for convenience

the aggregated list is reported in 9.1 Functional Technical Indicators, 9.2 Non-Functional Technical

Indicators.

The combination of these two different levels of Technical Indicators will hence guarantee the

systematic and complete Validation of all the IT aspects of a Trial. Two different perspectives are

taken into account, i.e. the one of the specific Software Component (i.e. GEs/SEs) and the one of

the final solution that results from the combination of these different elements and that will be

concretely used in the Trials environment.

2.4. Methodological Approach for Software Verification

Consistently again with WP2 (D2.1 and D2.3) outcomes, a Self-certification approach will be

supporting the Steps P1-P5 of the FITMAN V&V Methodology. Self-certification represents the

Verification of each Software Component directly by the Development Team which has been in

charge to develop it.

The Software Verification via Self-certification, will be carried out mainly on SEs (Specific

Enablers), as these are the only components that, for one side, are under the control of the FITMAN

project with involvement of actual developers and second can be utilized in multiple trials and

possibly in next instantiations of FITMAN platforms.

For a detail description of this verification process please refer D2.1 [1] and D2.2 [2]. A brief

summary is reported in 9.3 Software Verification (Self Certification).



12

3. FITMAN SMART FACTORY TRIALS TECHNICAL AND BUSINESS PERFORMANCE INDICATORS

This Chapter aims at describing the definition process of both Technical and Business Performance

Indicators for each trial included in the “Smart Factory” cluster. In particular, as Technical

Indicators are the same for all trials, in the following section the list of Software Components

assessed through them is reported. On the other side, Business Performance Indicators are strictly

related to the business objectives and their definition through the Simplified ECOGRAI

implementation is an iterative process described below.

3.1. Data Collection Form

In order to collect these data, a standard template called Data Collection Form has been developed

and distributed to trials in the domain. This form aims at supporting the trials as part of the Process

Assistant (Paragraph 4.1) in providing information for the online forms set up. It is following

represented:

Table 1 Data Collection Form

TRIAL [Trial Name]

Trial [Trial Number] Data Collection Form

Required Information

Business

Scenarios

List of business Scenarios composing the Trial (if possible specify

also envisaged business processes for each scenario.

SW components List of SW components (GE,SE, TSC) for each Business Scenario.

Business

Indicators

Complete list of finalized BI per Business Scenario.

For each indicator please describe using the template below(**).

Contacts List of names and e-mail address of :

A. TRIAL OWNER

B. people involved in the COMMUNITY based assessment

**Business Performance Indicators template: Indicator Name xxx

Purpose: PLEASE Specify why this indicator is relevant

Format : integer (min/max), %, alphanumeric, …

Information needed

(Source of data)

Where the data is available

Calculation Processing

(Formula)

If not directly available

Required evolution

(Target)

Target Value (PLEASE Provide rational for the value and what is the impact

coming from its achievement)

The owner

(Who measures)

Period PLEASE specify WHEN / HOW MANY TIMES it has to be measured and reported

Actions to react

depending on the value

of the PI

Description

***Contacts template (Recommendation: 6-8 persons for Community including final users, system

integrator and developer):

TRIAL OWNER E-MAIL ADDRESS



13

COMMUNITY BASED PANEL E-MAIL ADDRESS

1.

2.

3.

4.

5.

6.

7.

8.

Business Scenarios, Software Components and Business Performance Indicators (including detail

description for each of them) are included in this document

Contacts (including Trial Owner and Community based panel) will be collected in T2.5 for actual

deployment of web based survey.

3.2. Business Performance Indicators for Smart Factory Trials

In this section, for each Trial of Smart Factory, a list of Business Performance Indicators has been

developed, selected and mapped with its own reference processes. As previously mentioned, the

indicators definition is performed based on the Simplified ECOGRAI methodology, proposed in

D2.3 and summarized in Chapter 2.

During the deployment, the following important aspects have been analyzed to ensure significance

of the collected data:

- Significance of data: the observed values need to be significant in the period of

observation.

- Frequency: need to collect the data multiple times and compare them in different

phases of the implementation.

- “Background noise”: an appropriate evaluation of the direct connection of the

observed values change with the adoption of the FITMAN Trial platform, in order to

exclude possible effects coming from other causes.

- Confidentiality issues: some trial do not want to disclose absolute values of specific

indicators, in particular in relation to the current value, as following better described.

For each Business Performance Indicator, trial is required to measure and report AS-IS and TO-BE

values. According to the nature of the indicator and the expectations of the trials, it has to specify

when and how many times TO-BE value has to be reported. Furthermore, the trial has to identify

the TARGET value of each indicator, which represents the expected benefit coming from the

implementation of the solution.

The implementation of ECOGRAI methodology for each Trial and the resulting Business

Performance Indicators are following described. The detailed description of each Business

Performance Indicator using the template proposed in Data Collection Form (Chapter 3), are

reported in Annex III (Paragraph 7).

During the assessment period, the trials are responsible to monitor Business Performance Indicators

and collect values over a period of time; in particular, AS-IS, TO-BE and Target values have to be

identified. AS-IS value is the current value of the indicator before starting the assessment period;

due to the confidentiality issue previously anticipated, trials can decide to not provide the value

because are not allowed to share internal confidential data. For that reason, in some cases AS-IS

value is not reported. After the implementation, the trial has to measure what are the effects on the

business processes; in doing this, it has to identify how many times and when the measurement and



14

reporting of the indicator is planned to be performed. In order to understand if the solution allows

the trial to reach the objectives, a Target value need to be identified; it represents the expected

benefit and the trials are required to explain and justify the choice of this value.

For each Trial it is illustrated the Software Components’ structure in order to :

Identify the IT configuration of the Trial

3.2.1. Whirlpool Trial

The focus of the Whirlpool trial, which works in the appliance sector, is to provide a support to the

decision process at the shop floor level. As described in D3.1, workers have to take timely decision,

but the IT infrastructure is not enough adequate to support them in this activity. Furthermore, many

events are detected and recorded in a database, but they are not correlated and used in the decision-

making process. Therefore, the solution aims at implementing two Business Scenarios:

- Creating a correlation between events. (Big Data Scenario)

- Automatically identifying and communicating the event to the decision makers, with the

support of mobile devices. (Event Scenario)

The scenarios are actually interconnected since there is no meaning in creating a correlation

between events and not communicating it.

From the point of view of Performance Indicators, the effects of the two scenarios cannot be

distinguished, i.e. in both cases (events generated by a post elaboration of historical data vs. events

simply transferred from physical layer to decision makers) the tangible results are improvements of

product Quality due to an expected change of behavior of some decision makers (Quality Managers,

Maintenance Managers) who can hence better and faster improve many aspects of production

process.

The solution is applied to the Washing Unit production line, which is represented in the following

picture:

Figure 3 Whirlpool - WU Production line

Color of each box characterizes the type of the station according to the activity performed: light

blue block represents a process station, orange represents a quality control station, light yellow

represents an assembly station, and green block represents a marriage station.

Rear Tub

Thickness ControlBearing Insertion Seal Insertion

Read Data-Matrix

Tub Welding

(Branson)

Upper CW

Assembly

Screwing

Front CW

Assembly

Screwing

Pulley Wobbling

Control

Rear Tub

Dimensional and

Tolerance Control

Pulley

Assembly and

Screwing

Heating Element

Assembly

Screwing

Motor

Assembly and

Screwing

Final WU Visual

ControlCabinet and WU

Marriage

Pickup Drum

From Buffer

Marriage

Rear Tub –Drum

Marriage

Rear Tub

Front Tub

Pickup Drum

From Buffer

Cabinet (Foots,

Dampers, Springs)

Assembly Pallet

Changing

Exhaust Pipe

Assembly

Belt Assembly



15

Each station can potentially generate one or more events, which have to be detected and

communicated to the system and to the decision makers as a trigger for an action.

Two Business Scenarios have been identified according to the business objectives of the trial:

“Event Scenario”, which aims at creating correlation between events and users, and “Big Data

Scenario”, which aims at generating relevant events starting from a collection of a big quantity of

data such as measures, test results, counts of events, etc.

The overall Trial IT Solution includes on the other hand eight Software Components:

Table 2 Whirlpool SW components

Whirlpool SW components

GE IoT.Gateway.DataHandling

GE IoT.Backend.IoTBroker

GE IoT.Backend.ConfMan

GE Data.BigData

SE Secure Event Management

TSC Event Generator

TSC Notification Manager

TSC Notification Browser

The solution is integrated within the IT functional architecture of the company and the integrated

view results in four layers:

- ERP layer: manages the overall supply chain;

- MES layer: coordinates the execution of production plans;

- Shop Floor layer: includes the workstations of production line;

- GRACE layer: includes a set DBs and in particular Gra.Da.Co., which collects results of

Washing Unit line operations; it is the base for production events collection.

These layers and their logical links are described in [4] as well with the description of SW

Components and their interaction with the Shop Floor layer.:

Whirlpool trial aims at identifying possible events from a big amount of data collected, and

correlating them with the shop floor, in order to better manage and support the decision processes.

The starting point is to identify a list of events that can be generated and measured at process level

and, in particular, for each station of the line. These events have been listed and described within

the Table 3 Whirlpool relevant events, including the following information:

- STATION and STATION CODE, related to the considered event.

- RECEIPIENT, who receives the communication of the event through the device and has to

perform the decisions.

- MEASURE, which is the indicator to monitor on that station

- FEASIBILITY, which means the difficulty in finding data needed for measure and can be

low, medium or high.

- BUSINESS BENEFIT, which potentially can be generated by the implementation.

- CURRENT FREQUENCY, which refers to the frequency of the event occurrence; this kind

of information is not always available and there are some events which have never been

occurred.

- Reference SCENARIO for each event.



16

The first list defined contains 26 potential cases, but from a more detailed analysis has emerged that

14 of them should be discarded because of station unavailability or for unreliable event generation.

The other ones represent potential events to measure and for that reason has been ranked from 1

(less) to 4 (most) according to their relevance.

The events and the relative stations that are more significant for that process monitoring (with rank

equal to 4) are listed in the Table 3 Whirlpool relevant events:

Table 3 Whirlpool relevant events

Table 3 Whirlpool relevant events, depicts the list of significant events to monitor and their

association with one of the two identified business scenarios. (A comprehensive description of such

events is reported in [5])

An Analysis of relationship among each station and related events identified where events are more

significant, the results are described here:

- Fourth station – Bearing (WUBI): it checks the bearings insertion. In that case, the event of

interest is a sequence of defects detected on the station. The single defect is not

representative, but an historical series within a certain period can represent deterioration of

the process. For that reason, “Data-Handling” component has to identify the repetitive

defects and communicate it to the Quality and to the Team leader. This event has a high

relevance because can allow the monitoring of station performance and the process drifting,

allowing prevention of problems and maintenance activities.

- Fifth station – Seal insertion (WUSI): it add seal in the right quantity based on force

transducer. This case is quite similar to the previous one, where it is interesting to monitor

the occurrence of defects and communicate the repetitive ones.

- Tenth station – Tub welding (WUBR): it is responsible of welding rear tub and front tub

applying pressure and vibration. ….

- Functional tests station (ASFT): it measures the thermo-resistance and effective payload of

valves. In that case, the event generated is a pattern of defects on the products. The operators

causing the defect can have a feedback for potential improvements.

- Normative electrical test station (ASNT).

- ZHQ stations (ASZHA, ASZHBC): it includes a system that statistically performs a full test

of some washing machines sampled for an intensive testing cycle. It allows capturing

defects classified as type A, type B and type C, where defect A has a higher severity and for

sure if the product goes on the market the customer service will receive a call.

# Station CODEStation

Potential event

generated

Potential

Recepient Business benefit Measure

Feasibilit

y

Current

Frequency

Selecte

d for

TRIAL BS

4 WUBI

Station 4A and 4B

Bearing Insertion

Sequence of Defects;

Process Drifting (SPC)

Team Leader;

Quality Process

Manager

Anticipate problem resolution

(e.g. Maintenance

intervention): avoid

productions stop and reduce

defects.

OEE ,

FOR Medium 4 BigData

6 WUSI

Station 5 Seal

Insertion

Sequence of Defects;

Process Drifting (SPC)

Team Leader;

Quality Process

Manager


(e.g. Maintenance



defects. OEE, FOR High 0,055% 4 BigData

11 WUBR

Station 10 Tub

Welding (Branson)

Product Defect;

Machine stop;

SPC (Process Drifting)

Quality Manager;

IE, Maintenance


(e.g. Maintenance



defects. OEE, FOR High 0,01% 4 Event

23 ASFT

Functional test

(100%)

Pattern or sequence

of defects Quality process

Direct feedack to operators

causing defects lead to

improvement of their

operation FOR Low 4 BigData

24 ASNT

Normative

Electrical Test

Sequence of multiple

faults

Quality Process;

Quality Manager

Unsolicited verification

process: problem prevention

(e.g.epidemic problems) FOR Medium 4 BigData

26 ASZHA ZHQ (3%)

"A" defect (data from

DCS)

RDC Manager;

Quality Manager;

Factory Director Block potential faulty batch SIR Medium 4 Event

27 ASZHBC ZHQ (3%)

"B", "C" defects (data

from DCS) Quality Process;

Unsolicited verification

process: problem prevention

(e.g.epidemic problems) FOR High 4 Event



17

Events just identified in these stations can be classified in the three typologies following listed:

- Daily statistical analysis and process deterioration (events 4,6 e 11);

- Multiple faults events on functional and normative test (events 23 e 24);

- Communication of defects A, B and C on ZHQ station (events 26 e 27).

As emerged from the previous table, starting from the stations and the related potential events,

three main indicators have been identified at process level:

- Overall Equipment Effectiveness (OEE): is the total amount of time used to produce good

product versus the total available time.

- Fall Off Rate (FOR): measured as percentage, represents the internal defectiveness; is the

ratio between the number of defects detected along the production line and the total

production volume in a specified period (shift; day; month; YTD)

- Service Incident Rate (SIR): measured in parts per million [ppm], is the percentage of how

many calls received from the Customer Service on the overall production in a time period.

These indicators monitored on each stations will lead to business benefits. In order to measure these

benefits, is necessary to link them with higher level measures. For this reason, it’s necessary to map

the indicators identified in the previous step with the proposed PIs, developed thanks to the

ECOGRAI methodology and listed in the Table 5 . This means that the former should be analyzed

and eventually integrated in order to fit with the latter ones, shifting from the low level performance

indicators of the stations to the business ones. The Simplified ECOGRAI methodology

implementation is described below:

Phase 1 : in that phase, the trial is described identifying the elements, the process and the

objectives of the company.

Table 4 Whirlpool – Phase 1 ECOGRAI

In order to make the objectives more detailed and more coherent to the ECOGRAI definition, a

further decomposition is represented within the Table 5 .

Phase 2: the AV/DC identified is to use the Whirlpool platform in order to reach the pre-

defined objectives.

Phase 3: as previously said, the Whirlpool performance indicators identified at process level

have been mapped to the ECOGRAI objectives and listed in the Table 5

Elements of the system Functions (Static) and

Processes (Dynamic)

Whirlpool Objectives

Production, assembly,

delivery

Two identical parallel

production lines

Washing Unit Line,

Assembly Line, Testing

and Final Assembly

Shopfloor workers,

supervisors, managers

To produce different models

of washing machines

(Production, assembly,

delivery)

Obj.1: Improve the

communication

effectiveness along the

help chain organization

Obj.2: Improve the

effectiveness of decision

makers, their role, along

the help chain



18

The entire process of ECOGRAI methodology implementation for defining the Business

Performance Indicators, results in the following table, which reports the objectives (both trial and

ECOGRAI-based ones), the actions and the relative Business Performance Indicators:

Table 5 Whirlpool – Output from ECOGRAI

Whirlpool

Objective

FITMAN Relative

Objectives

Decision /Action

Variables (DV/AC)

Performance Indicators

Obj1: To improve the

communication

effectiveness along

the help chain

organization

Obj1.1. To improve the

product quality

To use Whirlpool trial

platform

PI 1.1.1: Ratio: FOR

after/before the DV/AV

implementation during a

period*

PI 1.1.2: Ratio: Service

Incidence Rate (SIR) after /

before the DV/AV

implementation during a period*

Obj1.2. To increase the

productivity


platform

PI 1.2: Ratio: Overall Equipment Efficiency (OEE)

after / before the DV/AV


period*

Obj2: To improve the

effectiveness of

decision makers along their role in

help chain

Obj2.1. To improve the

effectiveness of equipment

preventive maintenance


platform

PI 2.1.1: Ratio: Number of

breakdown between two

planned maintenances

(BBPM) after /before the

DV/AV implementation

during a period*

PI 2.1.2: Ratio: % of defective

parts to rework (DEFP) after

/before the DV/AV implementation during a

period*

Obj2.2To reduce the

production cost


platform

PI 2.2: Ratio: Conversion cost

per unit (CCPU) after /before

the DV/AV implementation

during a period*

Obj2.4.To reduce the Total

Cost of Quality


platform

PI: 2.4: Ratio: Total cost of

products scrapped (PSC) after

/before the DV/AV


period*

The final list of PIs has been achieved with a refinement exercise. In 8.1 it is described the

intermediate PIs proposed.

In order to reduce the effect of the “Background noise”, that means avoiding that elements not

considered in the assessment may affect the business results from the FITMAN solution

implementation, it has been decided to associate each business indicator to the related stations (as

reported within the Table 3 Whirlpool relevant events). Therefore, in doing this few main

assumptions have been considered:

- Business Performance Indicators which aim at measuring costs (in particular, Conversion

cost per unit and Total cost of products scrapped) are not linked to stations nor business

scenarios, but are related to whole solution; for that reason, they will be measured at trial

level but, for convenience, reported in the “Big Data Scenario” (which is the most sizeable);



19

- The new further indicators coming from the ECOGRAI implemented are allocated as

following:

o Number of breakdown between two planned maintenances is linked to the OEE,

therefore associated to the stations in which OEE is measured (i.e. WUBI, WUSI,

WUBR).

o Percentage of defective parts to rework is linked to the FOR, therefore, as in the

previous case, is associated to the station in which FOR is measured (i.e. WUBI,

WUSI, WUBR, ASFT, ASNT, ASZHBC).

The output of the above analysis results in the following table, which reports the Business

Performance Indicators, with their AS-IS and TARGET values. It has to be noted that the identified

indicators shown in Table 5 Whirlpool – Output from ECOGRAI represent PIs for the identified

Business Scenarios. They are then instantiated on the physical configuration of the system and

events identified in Table 3 Whirlpool relevant events. As a result we can see in Table 6 Whirlpool

Business Performance Indicators AS-IS and Target values the complete list of the PIs to actually

collect on the field.

Whirlpool identified 2 distinct Business Scenarios to monitor separately with an ad-hoc set of PIs:

BS 1 – Big Data Scenario

BS 2 – Event Management Scenario

Table 6 Whirlpool Business Performance Indicators AS-IS and Target values

BS 1 – Big Data Scenario

Business Performance Indicator AS-IS Value Target Value

Comments

WUBI – OEE na na Data are not available1

WUSI – OEE na na Data are not availableError! Bookmark not defined.

WUBI – BBPM na na Data are not availableError! Bookmark not defined.

WUSI – BBPM na na Data are not availableError! Bookmark not defined.

WUBI – FOR 0,24 0,22 no peaks

FOR measured at WUBI station in %. AS-IS value is the yearly value of 2013. Target is to reduce the number of peaks (i.e. Weekly rates exceeding 2sigma of the normal distribution)

WUSI – FOR 0,2 0,2 no peaks

FOR measured at WUSI station in %. AS-IS value is the yearly value of 2013. Target is to reduce the number of peaks (i.e. Weekly rates exceeding 2sigma of the normal distribution)

ASFT – FOR 4,49 4

Overall factory FOR measured in %. AS-IS value is the yearly value of 2013. The improvement of decision making process should allow a meaningful (10%)

1 Please consider that for some indicators, it is not specified neither AS-IS values not Target Values, that is

due to the fact that, at the date, in Whirlpool such values are not measured and there is not a clear expectation of possible improvement, nevertheless management intends to take the opportunity of FITMAN trial adoption to start measuring and monitoring them as they are perceived as critical production performance parameters.



20

reduction of the average Fall-off-rate and defective parts

ASNT – FOR 4,49 4

Overall factory FOR measured in %. AS-IS value is the yearly value of 2013. The improvement of decision making process should allow a meaningful (10%) reduction of the average Fall-off-rate and defective parts

ASFT – DEFP 31181 28000

Total number of defective parts recorded. AS-IS value is the yearly value of 2013. The improvement of decision making process should allow a meaningful (10%) reduction of the average Fall-off-rate and defective parts

ASNT – DEFP 31181 28000

Total number of defective parts recorded. AS-IS value is the yearly value of 2013. The improvement of decision making process should allow a meaningful (10%) reduction of the average Fall-off-rate and defective parts

CCPU 9,67 9

Variable conversion cost as percentage of average industrial cost. In the long run the improvement of efficiency in decision making can lead to a reduction of variable conversion cost from 9.67% to 9%.

PSC na na Data are not availableError! Bookmark not defined.

BS 2 – Event Management Scenario


Comments

WUBR – OEE na na Data are not availableError! Bookmark not defined.

WUBR – BBPM na na Data are not availableError! Bookmark not defined.

WUBR – FOR 0,03 0,03 no peaks

FOR measured at WUBR station in %. AS-IS value is the yearly value of 2013. Target is to reduce the number of peaks (i.e. Weekly rates exceeding 2sigma of the normal distribution)

ASZHBC – FOR 4,49 4

Overall factory FOR measured in %. AS-IS value is the yearly value of 2013. The improvement of decision making process should allow a meaningful (10%) reduction of the average Fall-off-rate and defective parts

ASZHBC – DEFP 31181 28000 Total number of defective parts recorded. AS-IS value is the yearly value of 2013. The improvement of decision making



21

process should allow a meaningful (10%) reduction of the average Fall-off-rate and defective parts

ASZHA – SIR 50000 47500

Overall factory Service Incidence Rate measured in ppm. AS-IS value is the 1st month in service yearly value of 2013. The improved capability of sharing detection of severe defect can lead in the long period (1 year) a reduction of 5% of present Service Incidence Rate measure on 12 months on service.

Overall comment on TO-BE value: the impact of FITMAN trial on the performance indicator will

be strongly biased by others external factors which are commonly and usually evolving along the

day-by-day activity of the factory. Among common activities to reduce FOR and SIR we can list:

product design changes, change in supplied goods, new measuring system introduced in the factory,

learning curve of operators, new organizational assets. As said FITMAN WHR trial is expecting to

influence only one aspect of this complex situation which is the prompt awareness of some decision

makers about the real status of part of the production process. In forecasting the impact of FITMAN

on TO-BE value, the experience of factory Quality expert has been used in order to make some

hypothesis on how the decision process can be improved and thus how this could be reflected on the

actual business indicators. Another thing which is not helping is the time factor: we are impacting

on people behavior using a novel way to communicate and interact with employee: presently there

are no certainty on how much time we need to change their approach and really got to positive

influence the decision process.

3.2.2. Piacenza Trial

Piacenza works in the textile sector and the main goal of this trial is to start going through a textile

and clothing “cloud production”, tracing the products and the machinery availability, sharing

production information and supporting interoperability.

The focus of this solution implementation is the fabric production phase, which starts from the yarn,

and goes through the yarn dyeing, weaving and finishing steps. Each of these step can be evaluated

singularly or together with the others.

The trial platform includes two different configurations:

- A set of cloud manufacturing services: which allows different internal actors to collaborate

on the same value chain. In that case, a monitoring system made of RFID technologies has

to be tested in order to verify the possibility to manage different data from different sources

(Smart configuration).

- A set of services which interact with the external customers cloud, in order to share with

them information about manufacturing capacity and availability (Virtual configuration).

The whole Trial IT Solution results in a complex architecture, with many components interacting

with the shop floor, as described in [4].:

There is not a complete division between SW components used for Piacenza virtual and smart

configurations. The Piacenza “smart configuration” is for sure based on the “virtual” one. On the

contrary the “virtual configuration” could be based on a “smart configuration” different from the

Piacenza one (for example in case of external companies/purchaser).

The overall Trial IT Solution includes in particular thirteen SW components:

Table 7 Piacenza SW components



22

Piacenza SW components




GE Apps.Marketplace

GE Apps.Repository

GE Apps.Mediator

SE Shopfloor Data Collection


SE Collaborative Assets Management

SE Supply Chain & Business Ecosystem Apps

TSC Event Management

TSC Order Management

TIC DB Trigger

Functional Technical Indicators will be evaluated for the GEs/SEs reported.

The Simplified ECOGRAI methodology previously described has been applied according to

Piacenza trial characteristics and objectives, in order to identify its own Business Performance

Indicators.

Phase 1: in order to describe the trial using the Modeling System, the following elements,

functions, boundaries and objectives have been listed.

Table 8 Piacenza – Phase 1 ECOGRAI

These objectives are detailed according to the ECOGRAI definition, decomposing them in further

lower levels, as shown in the Table 9 Piacenza .


Processes (Dynamic)

Boundary of

the system

Piacenza Objectives

Production, delivery

,yarn dyeing, weaving

(warping, weaving and

raw control) and

finishing (wet finishing,

raising and dry

finishing)

Production machineries,

labor force, Raw

material storage, yarn

storage, Production

manager, Production

operator, Sales

manager, Controller,

Sales operator

To produce textile

(Production, delivery

,yarn dyeing, weaving

(warping, weaving

and raw control) and

finishing (wet

finishing, raising and

dry finishing)

The external

stakeholders

(partners,

customers,

etc.). The

business

validation will

be performed

on a unique

machine,

which covers

all the

production

cycle

considered.

Obj.1: Better

exploitation of

internal and

external

production

capacity

Obj.2: Improved

monitoring of

production

capacity



23

Phase 2: after the definition of objectives, is necessary to identify the action to reach them, i.e. the

implementation of the Piacenza trial platform.

Phase 3: a set of AS IS trial indicators is defined:

Machinery exploitation: costs / produced unit

Production time: lead time from order to delivery

Energy for supporting systems per meter: Kwh per meter

Gas for supporting systems per meter: m3 per meter

Starting from these Business Performance Indicators, they have been mapped to the Simplified

ECOGRAI objectives previously defined as shown in the Table 9 Piacenza.

The entire process of Simplified ECOGRAI methodology implementation for defining the Piacenza

Business Performance Indicators results in the following table:

Table 9 Piacenza – Output from ECOGRAI

Piacenza Objective FITMAN Relative

Objectives

Decision /Action

Variables (DV/AC)


Obj1: Better

exploitation of

internal and external

production infrastructure

Obj1.1 To reduce the fixed

costs per machinery

To use the Piacenza trial platform

PI 1.1: Ratio: Machine fixed costs per produced unit after /

before the DV/AV


period*

Obj1.2 To reduce the

production time from order

to delivery

To use the Piacenza trial

platform

PI 1.2: Ratio: Average

production lead time per

meter produced from order to

delivery after / before the


during a period*

Obj1.3 To reduce the

quantity of energy for

supporting systems for

production


platform

PI 1.3: Ratio: The quantity of

energy spent per meter

produced after / before the


during a period*

Obj2: Improve the monitoring of the

production capacity


platform

PI 2.1: Ratio: Number of

production records including

machine identification



period*

P 2.2: Ratio: Percentage of

delivery forecast errors



period*

* to be defined according to the dynamic of the evolution of the system



AS-IS description and objective indication.



24

This analysis will be provided in relation with the situation of Piacenza production performance of

year 2013. Since Obj.1.1 and 1.3 are directly related, they will be described jointly.

AS-IS 2013

The market successful EU textile industries have moved towards high end and luxury market,

where design proposal, quality, flexibility in production and delivery, service and quick response to

customers’ need are critical to build the added value and overcome the pressure on prices. This

market is characterized by some very specific peculiarities, among which some ones are critical:

extremely high number of product variables in terms of style/material/color

deep customization of products

hardly predictable demand (i.e. shorter delivery requests)

length of production cycle (rigid deliveries, i.e. quality of service)

real prototyping (even if limited in the future) for style final choices

physical sampling for purchase choice (rigid quality of product)

fragmented distribution

un-efficient vertical information transfer

These factors, combined with these last years unpredictable fluctuations of global demand because

of macro-economic reasons (textile/clothing is typically a pro cyclic market), caused a fast decrease

of average lot dimension and a strong pressure for factory optimization instruments.

T/C product are object of a very fast renewal: each year at least fall winter and spring summer

season are presented, each one declined for man and for woman market, for a total of 4 brand new

design proposals per year. The needs of f/w and s/s seasons are different, like the ones of man and

woman markets, therefore each collection is brand new.

The timing of clothing main activities (raw material sourcing, design, wholesale order acquisition,

production, delivery and sales to consumers) can be summarized as follows:

Figure 4 Piacenza Timing of clothing main activities



25

Despite the short commercial lifecycle of the product (6 months), the length of each season

activities is 18-20 months. Fabric lifecycle is directly dependent from clothing one and ii precedes it

of 6 months.

Objective 1

Because of the abovementioned reasons the productivity of machineries is strictly dependent from

the kind of product which is realized (prototype, sample of large production). Machinery best

exploitation parameter is its time of use, which can be directly related with the directly operators

presence, depurated by supporting services.

Considering only production departments the exploitation per department in 2013 has been:

Figure 5 Piacenza exploitation per department in 2013

Positive and negative peaks are underlined in blue and orange. The periods when orders overcome

standard working hours are underlined in blue, in orange those ones when the structure is

underexploited. In the first ones external production sources are sought (Obj1.2), in the other ones

will be made available to third parties (obj.1.1).

Considering the present situation the average exploitation of present organization is negative (-7%),

ranging from -1% of raising and -10% of weaving. Per each department the range of exploitation is:

Weaving: from -27% to +10%

Dyeing: from -31% to +15%

Humid Finishing: from -14% to 14%

Raising: from -24% to +13%

Dry finishing: from -27% to +6%

Total structure exploitation ranges from -

20% to +9%. Exploitation range is related

to the specific production cycle, which can

be performed by numerous small

machineries (looms or dyeing machines for

example) or by large ones (dryer, fulling

machines) and in continuous or one shot

processes.

Weaving: small machines, continuous

Dyeing: small machines, one shot

Humid Finishing: large machines, one shot

Raising: small machines, continuous

Dry finishing: small and large Figure 6 Piacenza Production Process



26

machines, continuous

Obj.1.3 is to reduce the cost of energy consumption of Piacenza company per meter.

In fact the present distribution of energy consumption has shown a significant weight of supporting

systems, which goes beyond 20%:

Figure 7 Piacenza energy consumption - 1

Figure 8 Piacenza energy consumption - 2

The first objective of FITMAN is to reduce under exploitation in peak periods of 30% by

sharing (Obj.1.1) unexploited production to third parties, and proportionally to share the significant

energy consumption (Obj.1.3) dedicated to supporting systems. On the basis of above indicated

figures the potential benefit of 30% reduction of related cost is a conservative estimation, which

will be greatly affected by the level of diffusion of resource sharing model allowed by FITMAN

tools.



27

But another area of potential impact of FITMAN technology regards the reduction of delivery

time (Obj1.2) by the use of external resource use: in the month of April-May and November-

December production infrastructure is overexploited due to the overlapping of different production

activities.

For example in April and May 2014 machineries will satisfy the delivery of the sample fabrics for

2015 summer season (which will be presented in June 2014 fairs), the production of incoming

winter 2014/2015 and the first prototypes of the following one.

In November and December 2014 production must satisfy the demand of summer 2015 production,

samples of 2015/2016 winter season and prototypes of 2016 summer.

The above mentioned overlapping of activities leads to a consequent over exploitation of production

facilities and to the elongation of delivery times. Since the model of EU competition in fashion

production is based on service Piacenza is usually requested to give the best possible delivery in

relation with clothing industry requests. The fluctuations of demand are leading clothing industry to

retard orders as soon as possible and to ask fabric producers shorter deliveries, increasing the

seasonality of production and the overlapping of different seasons.

Usual production delivery timing is 12-15 weeks for fantasy fabrics (yarn dyed), which can be

shortened to 4-8 weeks for piece dyed fabrics, if ready to dye fabric is available.

But in April-May and November-December deliveries can be subject to even 4 weeks of delay, with

consequent delay of clothing production for customers.

This situation at present limits the sales of Piacenza, since it is not possible to increase resources

which are under exploited in other periods of the year and are related to fix costs (machineries,

employees). The availability of external production sources by FITMAN technology could lead to

potential a shortening (Obj.1.2) of delivery times up to 4 weeks (27-33%) in peak period of

April-May and November-December.

Objective 2

Improve the monitoring of the production capacity is a qualitative critical impact of FITMAN. At

present Piacenza production process is monitored by barcode technology. Each piece is

accompanied by a paper (named “carta pezza”) which resumes the fundamental characteristics of

the fabric, its item and serial number. After the passage in each machine the barcode of the “carta

pezza” is read and the ERP updated. This process is performed not contextually with the end of the

production step but in a second time, which can be hours or days (in case of mistakes) after it.

Expected delivery and production process scheduling are updated on the basis of the passages

through production steps, therefore present monitoring system inaccuracy is reflected into the

output date for production management and customer information. In order to provide a service to

third parties it is necessary to reduce or eliminate this inaccuracy and to provide precise and reliable

information as regards available or required production capacity. RF-ID technology

implementation in FITMAN will lead to an automated detection of piece entry into the machine (the

specific machine and not a generic one like now), its production time and its end. Present

information will be improved (Obj.2) in its quantity (3 times than now: entry, production, exit),

quality (specific fabric in the specific machine) and timing (contextual to production) providing a

full set of inputs to the IT infrastructure (MES, ERP). In addition to FITMAN benefits the increased

amount of data will greatly improve optimization of production and scheduling process and will

support a more accurate information about customer expected deliveries and, in general, of

Piacenza service.

Business Performance Indicators cross two different Business Scenarios, therefore are not easily

allocated; anyway, in general the following criteria has been adopted: Business Performance

Indicators which aims at measuring costs are more linked to the Production Capacity Seller



28

scenario, on the other hand, Business Performance Indicators which measure cycle time are more

related to the Production Capacity Purchaser one.

So, Piacenza identified 2 distinct Business Scenarios to monitor separately with an ad-hoc set of

PIs:

• BS 1 – Production Capacity Seller

• BS 2 – Production Capacity Purchaser

Starting from this consideration, they have been instantiated as in the following table:

Table 10 Piacenza Business Performance Indicators AS-IS and Target values

BS 1 – Production Capacity Seller


Comments

Machine fixed costs per produced unit

- 30 % sharing unexploited production to third

parties

The quantity of energy spent per meter

produced

- 30 % sharing significant energy consumption

with third parties

Percentage of delivery forecast errors

0 errors in reading of

tracing data

Information will be improved in quantity (3 times than now: entry, production,

exit), quality (specific fabric in the specific machine) and timing (contextual to

production)

BS 2 – Production Capacity Purchaser

Business Performance Indicator

AS-IS Value Target Value

Comments

Average production LT per meter produced from order to delivery

max 4 weeks

reduction

shortening (Obj.1.2) of delivery times up to 4 weeks (27-33%) in peak period of April-May and November-December

Common to both business scenarios

Number of production records including machine identification

None Many Qualitative indicator indicating the number of production records DA/AV, including machine identification

3.2.3. TRW Trial

The TRW trial is focused on the development of new prevention models and techniques through

risk detection and communication. Currently, TRW is based on a traditional risk prevention

strategy, where the prevention technician designs the plans thanks to shop floor regulations and

equipment self-diagnosed results, using partial and rigid approaches. The new prevention model

aims at:

- Empowering workers’ safety and security;

- Finding technologies and methodologies for managing risk modeling and detection;

- Improving decision-making process about safety and security in the production activities.

The developed trial solution allows addressing these goals, collecting and personalizing

heterogeneous data from different sources, monitoring communications and maintaining data

updated and consistent.

The trial is implemented on the assembly line of hydraulic steering for minivans, which is a

discontinuous line producing spare parts. The main problems that the prevention system has to

monitor in manufacturing and warehouse are the collision between machine and worker, and the

ergonomic problems of operating machines.



29

With the goal of monitoring the workers risk, the real-time data have to be collected using sensors,

control elements and communication systems. In particular, the solution will detect the risks of

manual load handling and awkward posture, identifying the inclination or deflection ranges adopted

and the numbers of times per minute they are performed, among others.

The overall Trial IT architecture is described in [4] and it includes the following sixteen SW

components:

Table 11 TRW SW components

TRW SW components




GE Apps.ApplicationMashup

GE Apps.Repository

GE Apps.Registery

GE Apps.Mediator


TSC Ergonomic Monitoring System

TSC Prevention Action Modelling

TSC Risk Modelling System

TSC Authorisation Policies Validation

TSC Map Service Widget

TSC Workflow Engine

TSC HMI

TSC Alert Notification Services

As in the previous cases, the process of TRW Business Performance Indicators definition through

the Simplified ECOGRAI methodology is summarized in this section.

Phase 1: according to the first phase of Simplified ECOGRAI methodology, the TRW trial

elements, processes, boundaries and objectives are modeled as follows.

Table 12 TRW – Phase 1 ECOGRAI


Processes (Dynamic)

Boundary of

the system

TRW Objectives

Prevention technician

Safety coordinator

Blue collar worker

Information systems

technician

Manager of the

company

Operation technician

Production line

Warehouse

TRW assemblies and

manufactures power

steering systems for

passenger cars and

commercial vehicles

Customers

Objectives for scenario 1:

Obj.1: Effective and

consistent prevention

strategy

Obj.2: Optimization of

prevention costs

Objectives for scenario 2:

Obj.1: Reduction of

accidents and incidents

Obj.2: Increase of the

productivity



30

In order to be more coherent with ECOGRAI, also in that case objectives have to be adjusted, re-

formulated and decomposed; the new ones are represented within the Table 13 TRW - Output from

ECOGRAI.

Phase 2: After the objectives definition, the TWR trial platform has to be used to reach them.

Phase 3: in this phase, for each Business Scenario a set of Business Performance Indicators has

been proposed by the trial. In particular, according to each Business Scenario:

Business Performance Indicators for Business Scenario 1:

PI1: Decrease of events: Reduction of the number of accidents and incidents in the factory

PI2: Optimization of cost of accidents and incidents: Increase of the profitability of the

investment in preventive strategy

PI3: Decrease of errors in the prevention strategy: Reductions of the human errors in the

design of the planning

PI4: Increase of the modelled risks and active preventions: Number of risks that has been

defined using the new system

Business Performance Indicators for Business Scenario 2:

PI1: Decrease of events: Reduction of the number of accidents and incidents in the factory

PI2: Decrease the rate of absenteeism: Reduction in the average number of lost days by

workers

PI3: Increase the number of alarms and alerts: Rise in the risk detections, alarms and

warnings

PI4: Increase the number of safety systems: Rise in the deployed monitoring systems

PI5: Decrease the number of workers with diseases: Reduction in occupational diseases

PI6: Increase the number of training sessions: Rise in the training sessions regarding H&S

PI7: Increase of the productivity: Rise in the produced units

The indicators need to be mapped according to the new objectives and the results of the PIs

definition are presented in the following Table 13 TRW :

Table 13 TRW - Output from ECOGRAI

TRW Objective FITMAN Relative

Objectives

Decision /Action

Variables (DV/AC)


Obj1: Effective and

consistent prevention

strategy &

Optimization of

prevention costs

Obj1-1: To increase the

standards and regulations

in the repository

To use the TRW trial

platform

BS1PI 1: Ratio: Number of

standards and regulations

added in the repository



period*

Obj1-2: To reduction the

number of accidents and

incidents in the factory


platform


accidents and incidents in the

factory after / before the DV/AV implementation

during a period*


modelled risks


platform


risks that has been defined

using the new system after /



31

before the DV/AV


period*


modelled preventive

actions


platform


preventive actions using the

new systems after /before the


during a period*

Obj1-5: To decrease the

errors in the prevention

strategy


platform


human errors in the design of

prevention strategy planning

after /before the DV/AV implementation during a

period*

Obj2: Reduction of accidents and

incidents & Increase

of the productivity

Obj2-1: To reduce the number of accidents and

incidents in the factory

To use the TRW trial platform

BS2PI 1: Ratio: Number of accidents and incidents in the

factory after / before the


during a period*


number of safety systems


platform


deployed monitoring systems

after / before the DV/AV


period*


number of risk detections,

alarms and warnings


platform


risk detections, alarms and

warnings set up after / before

the DV/AV implementation

during a period*

Obj2-4: To increase the number of training sessions

regarding safety


BS2PI 4: Ratio: Number of training sessions regarding

safety after /before the


during a period*



At the end of this process, the definitive Business Performance Indicators have been validated. Here

below there is their final list according to the two Business Scenarios. For the AS-IS values, the

value will not be provided because of confidentiality issues only % increments are provided.

TRW trial will use percentages of improvement and decrease of the Business Performance Indicator

as measuring unit, avoiding the usage of absolute values. The main reason for this choice is the

misuse that external users can do with current data of TRW, getting them out of context and

creating non-desirable image for a worldwide leader branch in the automotive sector. Due to this

unfortunate and possible situation, TRW will use percentages comparing current and future values

of each indicator.

Additionally, the most important target of TRW due to Business Performance Indicator is not only

to assess the impact of the FITMAN system instantiation, but also report and communicate this

impact in the manufacturing and production activities thanks to FI technologies deployment. In

order to reach these objectives of assessment and communication, percentage values of TRW

indicators are as useful as absolute values, since they are able to reflect the evolution of the business

processes in the factory.

TRW identified 2 distinct Business Scenarios to monitor separately with an ad-hoc set of PIs:

• BS 1 – Risk Modelling

• BS 2 – Risk Detection and Information

Table 14 TRW Business Performance Indicators AS-IS and Target values



32

BS 1 – Risk Modelling


Comments

Number of standards and regulations added in

the repository -

Increase of 5% Good Increase of 7% Very good Increase of 15% Excellent

TRW is currently using REBA, NIOSH and OCRA standards, which are the most important ones. With the new system, the time invested in the full application of these standards and the range of information controlled (parameters controlled) will be optimized, not changing the costs. If the target values are not achieved, the setting up process of the standards has to me redesigned.

Number of accidents and incidents in the factory

-

Reduction of 10% in the level of frequency and

gravity Good Reduction of 15% in the level of frequency and gravity Very good


gravity Excellent

If the expected values are not achieved, the whole system should be redesign; i.e. changing the configuration of the warning messages receivers or the location of the ergonomic monitoring systems.

Number of risks that has been defined using the

new system -

Increase of 30% Good Increase of 45% Very

good Increase of 60% Excellent

If the expected values are not achieved, the risk modelling TSC should be redesign to allow easier risk definition.

Number of preventive actions using the new

system -



If the expected values are not achieved, the preventive action modelling TSC should be redesign to allow easier action definition and risks prevention.

Number of human errors in the design of

prevention strategy planning

-

Reduction of 10% Good Reduction of 20% Very

good Reduction of 30%

Excellent

If the expected values are not achieved, the formulas associated to the risks should be redefined, detecting more level of risks.

BS 2 – Risk Detection and Information

Business Performance Indicator

AS-IS Value Target Value

Comments

Number of accidents and incidents in the factory

-


gravity Good Reduction of 15% in the level of frequency and gravity Very good


gravity Excellent


Number of deployed monitoring systems

- Increase of 55% Good Increase of 75% Very

good

The systems will provide the innovative aspect of the trial, but more sensors do not mean more detection, so the



33

Increase of 95% Excellent results should be carefully studied.

Number of risk detections, alarms and

warnings set up -


good Increase of 100%

Excellent


Number of training sessions regarding safety

-



Workers should receive training as a consequence of prevention actions. If the values are not achieved, the preventive actions design should be redefined.

3.3. Technical Indicators for Smart Factory Trials

As specified in Deliverable D2.3 [3] and resumed in Paragraph 2.1, a common set of functional and

non-functional Technical Indicators has been adopted for all the three Smart Factory trials. In

particular, openness, interoperability maturity and ease of application have to be measured for each

GE and SE of the solution; on the other hand, fulfilment of requirements, learnability,

understandability, attraction level and efficiency have to be evaluated for the whole Trial IT

Solution, on a crowd-based assessment. The set of GEs, SEs for each Trial, that has to be evaluated

using functional Technical Indicators, is listed.

Table 15 GEs and SEs for Smart Factory Trials

Type Name TRW Piacenza Whirlpool

GE Apps.ApplicationMashup X

GE Apps.Marketplace X

GE Apps.Mediator X X

GE Apps.Registery X

GE Apps.Repository X X

GE Data.BigData X

GE IoT.Backend.ConfMan X X X

GE IoT.Backend.IoTBroker X X X

GE IoT.Gateway.DataHandling X X X

SE Collaborative Assets Management X

SE Secure Event Management X X X

SE Shopfloor Data Collection X

SE Supply Chain & Business Ecosystem Apps X

3.4. Self-certification for Smart Factory Trials SW Components

As deeply explained in Paragraph 2.4, the Self-certification (Steps P1-P5 of the FITMAN V&V

Methodology) will be addressed just one time for each SE by the related Development Team, in

order to certify the correct execution of the main development activities.

For this reason, it is useful just to report the list of the different SEs used within Smart Factory

Trials and the name of the related Software Developer (with its leader):

Table 16 SEs and related Software Developers

SE SE Leader



34

SE – Collaborative Assets Management Mauro Isaja (ENG)

SE - Shopfloor Data Collection Jesus Benedicto (ATOS)

SE - Secure Event Management Domenico Rotondi (TXT)

SE - Supply Chain & Business Ecos. Apps Michele Sesana (TXT)

The abovementioned Software Developers will be in fact involved in the Self-certification of their

specific SEs, by means of the modalities deeply analyzed in Paragraph 2.4.



35

4. CONCLUSIONS & NEXT STEPS

In this last section the Technical and Business Performance Indicators for Smart Factory Trials are

summarized.. As previously said, the Technical Indicators, listed in the Table 17, are common and

therefore replicated for each Trial; three of them require just one value as assessment of the

GEs/SEs, the other five ones require the involvement of each Trial Team member using the Trial

Integrated Solution.

Table 17 Summary Technical PIs

Technical Indicators Description

GEs/SEs

Openness

Capability of ensuring that specific people groups may access the software for free with specified rights.

Interoperability Maturity The capability of the software to interact with other systems.

Ease of application The applicability of the software in the particular environment in terms of amount of work and extra actions or means.

Trial Integrated Solution

Fulfillment of requirements Capability of the solution fulfils the trial requirements.

Learnability Ease to start to use the solution and learn functionalities.

Understandability Ease of understanding concepts and terminology.

User’s attraction level Degree of attractiveness of the solution for the user.

Efficiency Capability of the solution to be fast enough and use reasonable resources.

On the other hand, Business Performance Indicators, reported within Table 18, have been developed

for each Trial according to its own business objectives, applying the Simplified ECOGRAI

methodology.

Table 18 Summary Business Performance Indicators

SMART

PIs N° TRIALS

Ratio: Number of standards and regulations added in the repository after/before the

DV/AV implementation during a period* 2 TRW

Ratio: Number of accidents and incidents in the factory after / before the DV/AV

implementation during a period* 2 TRW

Ratio: Number of risks that has been defined using the new system after/ before the DV/AV implementation during a period*

2 TRW

Ratio: Number of preventive actions using the new system after /before the DV/AV


Ratio: Number of human errors in the design of prevention strategy planning after

/before the DV/AV implementation during a period* 2 TRW

Ratio: Number of deployed monitoring system after / before the DV/AV implementation during a period*

2 TRW



36

Ratio: Number of risk detections, alarms and warnings set up after / before the

DV/AV implementation during a period* 2 TRW

Ratio: Number of training sessions regarding safety after / before the DV/AV


PIs N° TRIALS

Ratio: FOR after/before the DV/AV implementation during a period* 4 Whirlpool

Ratio: Service Incidence Rate (SIR) after / before the DV/AV implementation

during a period* 4 Whirlpool

Ratio: Overall Equipment Efficiency (OEE) after / before the DV/AV

implementation during a period* 4 Whirlpool

Ratio: Number of breakdown between two planned maintenances (BBPM) after

/before the DV/AV implementation during a period* 4 Whirlpool

Ratio: % of defective parts to rework (DEFP) after /before the DV/AV


Ratio: Conversion cost per unit (CCPU) after /before the DV/AV implementation

during a period* 4 Whirlpool

Ratio: Total cost of products scrapped (PSC) after /before the DV/AV


PIs N° TRIALS

Ratio: Machine fixed costs per produced unit after / before the DV/AV

implementation during a period* 5 Piacenza

Ratio: Average production lead time per meter produced from order to delivery

after / before the DV/AV implementation during a period*. 5 Piacenza

Ratio: The quantity of energy spent per meter produced after / before the DV/AV


Ratio: Number of production records including machine identification after / before the DV/AV implementation during a period*

5 Piacenza

Ratio: Percentage of delivery forecast errors after / before the DV/AV


The analysis of the PIs types for SMART factories shows that it is difficult to define a set of

reference PIs common to the three trials.

It is well known for Business Performance Indicators (BPIs) that the type of BPIs depends on the

considered level of management (strategic, tactical, operational) and also the nature of production.

At strategic level the PIs could be generic: there is no influence on the nature of production because

this level takes in consideration the global enterprise. It is the reason why the BSC (Balanced Score

Card) proposes generic KPIs.

Smart trials are more located at operational or tactical level. The type of products (or type of

services) influences the type of PIs.

But the nature of the objectives has also an influence on the type of PIs. Usually these objectives

concern time, cost, productivity and quality.

Examples:

Productivity: Ratio: Number of preventive actions using the new systems after /before the

DV/AV implementation during a period. (TRW)

Cost: Ratio: Total cost of products scrapped after /before the DV/AV implementation during

a period. (Whirlpool)

Time: Ratio: Average production lead time per meter produced from order to delivery after /

before the DV/AV implementation during a period. (Piacenza)



37

These data collected during the Trial implementation and measurement period will be extremely

important as an input for WP7 “Lessons learned, recommendations, best practices”, which will use

the main findings of WP2 “Verification and Validation Method” in order to merge together and

analyze the final results of the Trials experimentation developed in WP4 and provided in the present

document.

4.1. Process Assistant

The “Process Assistant” is the name that indicates the integrated business and technical model used

to guide and govern the correct application of the V&V Process in the different Trials. It is hence

constituted by the Business Performance and Technical Indicators designed for the Trial and by the

related methodological approach (Chapter 2) but also by all the complementary decisions and

actions needed to support their correct implementation and collection.

For each of the Trial, we can hence identify as elements of the overall Process Assistant the

following ones:

1. Identification of the Software Components and of the Business Scenarios according

to the most recent changes;

2. Implementation of the Business Performance and Technical Indicators and related

Data Collection;

3. Identification of the Trial Owner and of the Community Based Panel;

4. Collection of specific aspects related to the implementation of the Business

Performance and Technical Indicators system into the Trials.

As previously mentioned, a standard template called Data Collection Form was developed in order

to collect this information (Paragraph 3 Data Collection Form).

In relation to the fourth and last point, two specific aspects that the Process Assistant also wants to

address are:

The collection of the technical feedbacks on :

o the implementation of the Trial system;

o the operational resilience of the Trial (e.g. major bugs, blocking errors, etc.);

The collection and analysis of the most important operational issues faced in the

implementation of the system in the Trials, e.g. organizational and business difficulties,

degradation of the business system.

All the elements identified in relation to the four abovementioned points plus other specifically

addressed in WP7, will be elaborated and provided as reusable “Lessons Learned” in T7.1 –

Synthesis of Use Case Trials Experiences.

We anticipate as specific investigation areas in T7.1 the following:

1. V&V methodology and assessment package refinement based on experience;

2. Economic Impact of the Trials, with respect of FI PPP economic objectives and KPIs;

3. Social Impact of the Trials, with respect of FI PPP economic objectives and KPIs.

4.2. Specific Issues

During the interviews to the Trials, Data Confidentiality emerged as an issue for some of them. In

particular, these are their positions about this important topic:

Piacenza – No specific issues. Data can be represented and collected with their actual

values;



38

Whirlpool – Issues are related with the absolute values of production volumes and

production yields. No specific problems for other data, data related to performance of

production system will be represented in standard units;

TRW – Considering the focus of the Trial (oriented to security) all information are sensitive.

According to these considerations, Piacenza and TRW trials will NOT provide specific values for

AS-IS Indicators and TO-BE and target (desired values) are specified as percentage increment with

respect to these values.

Nevertheless this approach is not impacting at all significance of collected data and their usage.



39

4.3. Next steps

Considering that the three Trials and related Business Scenarios belong to different industrial

sectors (Textile, White Goods and Automotive components) and are focused on different processes

(Extended Production Systems, Internal Smart Production Processes and Security and Safety) and

considered processes are at operational level and not at strategic level, a consolidation and

comparison exercise is not easy to carry out. Nevertheless we can identify some communalities able

to identify possible cross trials evaluation of achieved impact and results.

To such purpose a clustered representation for Smart Factory trial is going to be presented in the

final release of this D.4.4 and it will be done also for trial belonging to Virtual and Digital Factory.

That will help (mainly in WP7 and WP8 as described below) to consolidate results from all the

trials.

It has to be noted that a refinement will take place in the next few months due to the adoption of

specific SW Components as result of the closure of “Open Calls” with the joining in the team

starting April 2 of new partners. With this respect new components will be included in trials’ IT

architecture and for that reason their (technical) assessment will be carried out.

T2.5 is in charge to assist the Trials in the instantiation of the V&V Process, by utilizing the

Methodology defined in WP2 and implementing the suggested set of Business Performance and

Technical Indicators.

T2.5 is in charge to support (utilizing data collected via the Data Collection Form Paragraph 3.1)

the creation of online forms to support data collection (see [3]).

Collected information will be stored in the repository implemented in T2.5 for the consolidation and

comparison of data. This task will take place in WP7 and WP8.

Moreover, all the data and elements of interest collected from the Trials will be elaborated and

provided as structured Lessons Learned in T7.1 – Synthesis of Use Case Trials Experiences.

T7.1 will be also in charge for collecting values (depending from the timing and frequency defined

in Data Collection Form paragraph 3.1 from each trial) of indicators after the implementation of the

solution.

Finally, the values gathered will be the input for T8.1 – FITMAN Use Case Trials comparative

evaluation and data consolidation.



40

5. ANNEX I : References

[1] FITMAN , "Deliverable D2.1 - FITMAN V&V Generic Method and Criteria Identification,"

2013.

[2] FITMAN, "Deliverable 2.2 - FITMAN V&V Business and Technical Indicators Definition,"

2013.

[3] FITMAN, "Deliverable D2.3 - Verification & Validation generic Assessment Package," 2013.

[4] FITMAN, "Deliverable D1.4 FI-WARE Platform Instantiation for FITMAN smart-digital-

virtual," 2013.

[5] FITMAN, "Deliverable D4.1 FITMAN System for Smart Factory," 2014.

[6] International Organization for Standardization, ISO/IEC 9126 Software engineering -- Product

quality, 2001.

[7] M. E. S. A. (MESA), "MESA Model," [Online]. Available:

http://mesa.org/en/modelstrategicinitiatives/MESAModel.asp.



41

6. ANNEX II : Glossary and Terminology

Term Description

AS-IS Value Performance Indicator Value of a Specific Indicator before the

adoption of FITMAN Trial

TO-BE Value Performance Indicator Actual Value of a Specific Indicator before

the adoption of FITMAN Trial

PI Target Value Value of a Specific Indicator as objective to

achieve after the adoption of FITMAN Trial

AC/DV A Decision variable is an element usually

used by a decision maker for reaching the

objectives. The DV modifies the states of

the controlled system.

An Action variable is the inductor of

performance, a variable which influences

the performance of an activity or a whole

process on which we can act to develop

the process to reach the goal better

In fact, the 2 variables represent very similar

concepts, the difference coming from the

human decision (D2.2).



42

7. ANNEX III: Business Performance Indicators Templates

7.1. Whirlpool – Business Performance Indicators Templates

PSC Business Performance Indicators template: Indicator Name PRODUCTS SCRAPPED COST (PSC)

Purpose: To reduce the Total Cost of Quality – Big Data Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

# Product Scrapped * Product Cost

Required evolution

(Target)

At medium long term the target is 0, meaning that the number of product to be

scrapped has to be reduced to 0

The owner

(Who measures)

Factory

Period Product scrapped is measured daily

Description mode % change from CURRENT and FUTURE VALUE(S) and % change from

CURRENT and TARGET VALUE(S)

CCPU Business Performance Indicators template: Indicator Name CONVERSION COST PER UNIT (CCPU)

Purpose: To reduce the production cost – Big Data Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

Directly available

Required evolution

(Target)

Reduction of Average Conversion Cost per Unit of 5%

The owner

(Who measures)

Factory

Period Calculated on a monthly basis

Description mode % change from CURRENT and FUTURE VALUE(S) and % change from

CURRENT and TARGET VALUE(S)

OEE Business Performance Indicators template: Indicator Name OVERALL EQUIPMENT EFFICIENCY (OEE)

Purpose: To increase the productivity – At station level (WUBI, WUSI, WUBR) - Big Data

Scenario and Event Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

-

Required evolution

(Target)

Improvement of each OEE of 5 points

The owner

(Who measures)

Production plant

Period Calculated on a monthly basis

Description mode CURRENT VALUE, FUTURE VALUE(S) and TARGET VALUE

BBPM Business Performance Indicators template: Indicator Name BREAKDOWN BEFORE PLANNED MAINTENANCE (BBPM)



43

Purpose: To improve the effectiveness of equipment preventive maintenance – At station

level (WUBI, WUSI, WUBR) - Big Data Scenario and Event Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

-

Required evolution

(Target)

see footnote Error! Bookmark not defined. at page 21

The owner

(Who measures)

To be identified in T2.5

Period M21


FOR Business Performance Indicators template: Indicator Name FALL OF RATE (FOR)

Purpose: To improve the product quality – At station level (WUBI, WUSI, WUBR, ASFT,

ASNT, ASZHBC) - Big Data Scenario and Event Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

-

Required evolution

(Target)

Improvement of 2 points in each station

The owner

(Who measures)

Production plant

Period Data available for Shift (M21)


DEFP Business Performance Indicators template: Indicator Name DEFECTIVE PARTS TO REWORK (DEFP)

Purpose: To improve the effectiveness of equipment preventive maintenance – At station

level (WUBI, WUSI, WUBR, ASFT, ASNT, ASZHBC) - Big Data Scenario and Event Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

-

Required evolution

(Target)

see footnote Error! Bookmark not defined. at page 21

The owner

(Who measures)

Production plant

Period M21


SIR Business Performance Indicators template: Indicator Name SERVICE INCIDENT RATE (SIR)

Purpose: To improve the product quality – At station level (ASZHA) - Event Scenario

Format : -

Information needed

(Source of data)

Internal Legacy


(Formula)

-

Required evolution see footnote Error! Bookmark not defined. at page 21



44

(Target)

The owner

(Who measures)

Production plant

Period M21


7.2. Piacenza – Business Performance Indicators Templates

P 1.1 Business Performance Indicators template: Indicator Name P 1.1 Machine fixed costs per produced unit

Purpose: Optimization of machine allocation and associated costs (mainly power services and

electricity)

Format : %

Information needed

(Source of data)

Improvement in utilization


(Formula)

-

Required evolution

(Target)

Reduction of 30% of fixed costs

The owner

(Who measures)

Trial Lead

Period M21

Description mode Percentage change of allocation of fixed costs

P 1.2 Business Performance Indicators template: Indicator Name P 1.2 Average production LT (Lead Time) per meter produced from order to

delivery

Purpose: Reduction of delivery time by the use of external resource use in the month of April-May and November-December when production infrastructure is

overexploited due to the overlapping of different production activities.

Format : %

Information needed

(Source of data)

Tracking of single lots and measuring of time from launch to delivery


(Formula)

-

Required evolution

(Target)

reduction of 30% of LT

The owner

(Who measures)

Trial Lead

Period M21

Description mode Percentage change of LT

P 1.3 Business Performance Indicators template: Indicator Name P 1.3 The quantity of energy spent per meter produced

Purpose: Optimization of allocation of fixed costs (power services and electricity)

Format : %

Information needed

(Source of data)

Improvement in utilization


(Formula)

-

Required evolution

(Target)

reduction of 30% of energy consumption costs

The owner

(Who measures)

Trial Lead

Period M21

Description mode Percentage change of machine allocation of energy costs



45

P 2.1 Business Performance Indicators template: Indicator Name P 2.1 Number of production records

Purpose: Present information will be improved in its quantity (3 times than now: entry,

production, exit), quality (specific fabric in the specific machine) and timing

(contextual to production) providing a full set of inputs to the IT infrastructure

(MES, ERP), that will ensure greater accuracy in product delivery forecasts

Format : %

Information needed

(Source of data)

At the moment these information are not available in automated fashion as they are

collected via a manual reading of bar-codes accompanying in loose way the product,

adoption of FITMAN technology will allow that


(Formula)

Qualitative

Required evolution

(Target)

All material flows (In, production and Out) are automatically traced

The owner (Who measures)

Trial Lead

Period M21

Description mode Percentage change of allocation of fixed costs

P 2.2 Business Performance Indicators template: Indicator Name P 2.2 Percentage of delivery forecast error

Purpose: As stated for P 2.1, the current method for production and material tracking is done

off line to the production and basically manually scanning a bar code.

Format : %

Information needed

(Source of data)

Automated reading of RFID


(Formula)

-

Required evolution

(Target)

0 errors in reading of tracing data

The owner

(Who measures)

Trial Lead

Period M21

Description mode

7.3. TRW – Business Performance Indicators Templates

BS1PI1 Business Performance Indicators template: Indicator Name Number of standards and regulations added in the repository after/before

the DV/AV implementation during a period

Purpose: To measure the time invested and the reduction of inefficiencies (time) in

the broad application of current regulations and standards

Format : %

AS IS value No reference value before FITMAN

Information needed The new time will be directly provided by the prevention technician,

regarding the average time he spends doing this task

Calculation

Processing

(Formula)

| |

Required evolution Increase of 5% Good



46

(Target) Increase of 7% Very good

Increase of 15% Excellent

The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

TRW is currently using REBA, NIOSH and OCRA standards, which are

the most important ones. With the new system, the time invested in the full

application of these standards and the range of information controlled

(parameters controlled) will be optimized, not changing the costs. If the

target values are not achieved, the setting up process of the standards has to

me redesigned.

Description mode % change from CURRENT and FUTURE VALUE

BS1PI2 & BS2PI1 Business Performance Indicators template: Indicator Name Number of accidents and incidents in the factory after / before the DV/AV

implementation during a period

Purpose: Ensure that the system is able to reduce the number of injured workers and

reduce the lost days in the production line

Format : %

AS IS value -

Information needed The data needed will be provided by H&S coordinator, due to TRW daily

activity recording on this issues.

Calculation

Processing

(Formula)

200.000 = 2.000*1.000, which comes from:

2.000 = number of hours performed by a worker in one year in the

United States (since TRW is an American group).

1.000 = ratio for number of workers. It is the basis for comparison

between the different facilities of TRW around the world.

Required evolution

(Target)

Reduction of 10% in the level of frequency and gravity Good

Reduction of 15% in the level of frequency and gravity Very good

Reduction of 20% in the level of frequency and gravity Excellent

The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

If the expected values are not achieved, the whole system should be

redesign; i.e. changing the configuration of the warning messages receivers

or the location of the ergonomic monitoring systems


BS1PI3 Business Performance Indicators template:



47

Indicator Name Number of risks that has been defined using the new system after / before

the DV/AV implementation during a period

Purpose: The system will allow setting up risks that can happen in the factory,

specifying concrete parameters and thresholds to detect them. The number

of risks will be the same, but the detailed configuration of the risks will be

the key for a better prevention.

Format : %

AS IS value -

Information needed The new FITMAN system will show it in its interface the new number of

risks defined

Calculation

Processing

(Formula)

| |

Required evolution

(Target)

Increase of 30% Good

Increase of 45% Very good


The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

If the expected values are not achieved, the risk modelling TSC should be

redesign to allow easier risk definition


BS1PI4 Business Performance Indicators template: Indicator Name Number of preventive actions using the new systems after /before the

DV/AV implementation during a period

Purpose: The system will allow setting up preventive actions, linked to the risks

detected. More preventive actions, more probability of risks prevention.

Format : %

AS IS value -


preventive actions defined

Calculation

Processing

(Formula)

| |

Required evolution

(Target)




The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

If the expected values are not achieved, the preventive action modelling

TSC should be redesign to allow easier action definition and risks

prevention.




48

BS1PI5 Business Performance Indicators template: Indicator Name Number of human errors in the design of prevention strategy planning after

/before the DV/AV implementation during a period

Purpose: Check that the human errors are reduced, which is one of the main problem

of current systems

Format : %

AS IS value -

Information needed

The new FITMAN system will have a registry to storage the modifications

in the system. Thus, the number of variations in the values of the formulas

and parameters of the risks will be shown there.

Calculation

Processing

(Formula)

Nº human errors = nº variations of the formula of the risks

Required evolution

(Target)

Reduction of 10% Good

Reduction of 20% Very good

Reduction of 30% Excellent

The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

If the expected values are not achieved, the formulas associated to the risks

should be redefined, detecting more level of risks


BS2PI2 Business Performance Indicators template: Indicator Name Number of deployed monitoring systems after / before the DV/AV


Purpose: Have an overview of the new IT equipment and infrastructures deployed in

the selected types of production lines

Format : %

AS IS value -

Information needed Direct information get when the systems are deployed in the selected types

of production lines.

Calculation

Processing

(Formula)

| |

Required evolution

(Target)




The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

The systems will provide the innovative aspect of the trial, but more

sensors do not mean more detection, so the results should be carefully



49

value of the PI studied


BS2PI3 Business Performance Indicators template: Indicator Name Number of risk detections, alarms and warnings set up after / before the

DV/AV implementation during a period

Purpose: Main PI: it determines the effectiveness of the systems to risk detection

and preventive actions deployment

Format : %

AS IS value -


detections

Calculation

Processing

(Formula)

| |

Nº detections = number of risks detected by the system + number of alarms

activated + number of messages & warnings send to the different actors

Required evolution

(Target)




The owner

(Who measures) TRW

Period M21

Actions to react

depending on the

value of the PI

If the expected values are not achieved, the whole system should be

redesign; i.e. changing the configuration of the warning messages receivers

or the location of the ergonomic monitoring systems


BS2PI4 Business Performance Indicators template: Indicator Name Number of training sessions regarding safety after /before the DV/AV


Purpose: Probe the increase in the awareness of the importance of H&S adoption in

the TRW factory

Format : %

AS IS value -

Information needed The new number of training sessions will be directly provided by the

prevention technician % safety manager

Calculation

Processing

(Formula)

| |

Required evolution

(Target)




The owner TRW



50

(Who measures)

Period M21

Actions to react

depending on the

value of the PI

Workers should receive training as a consequence of prevention actions. If

the values are not achieved, the preventive actions design should be

redefined.




51

8. ANNEX IV Business Performance Indicators refinement process

8.1. Whirlpool

Whirlpool Europe S.r.l. is the wholly owned European subsidiary of Whirlpool Corporation, the

world’s leading manufacturer and marketer of home appliances. The factory production line is

composed of two identical parallel lines designed to produce different models of washing machines.

This production line is organized in three main areas: Washing Unit Line, Assembly Line, Testing

and Final Assembly. In FITMAN, Whirlpool trial belongs to the category of SMART Factory. The

main objective of WHR trials is to demonstrate from the new technologies a better integration of

workforce in the decision phases of a production process: the decision process of shop floor

workers, characterized by fast reaction time, those of supervisors by medium reaction time and

those of managers by long time reaction.

The definition of Whirlpool Trial PI has been executed in three rounds. In following two table are

represented the intermediate iterations.

Table 19 Whirlpool Results after the First iteration

Objective DV/AV PI To improve the product quality To use

Whirlpool trial

platform

Ratio: First pass yield after/before the DV/AV


Ratio: Service Incidence Rate after / before the

DV/AV implementation during a period To increase the productivity To use

Whirlpool trial

platform

Ratio: Overall Equipment efficiency after / before the

DV/AV implementation during a period*

To improve the effectiveness of

equipment preventive maintenance To use

Whirlpool trial

platform

Ratio: Number of breakdown between two planned

maintenances after /before the DV/AV implementation

during a period*

% of defective parts to rework after /before the


To reduce the wastes of production

times To use

Whirlpool trial

platform

Ratio: Amount idle worker time after /before the


To reduce the production cost To use Whirlpool trial

platform

Ratio: Conversion cost per unit after /before the DV/AV implementation during a period*

To reduce the Total Cost of Quality To use

Whirlpool trial

platform

Ratio: Total cost of products scrapped after /before the


To increase the engagement level

of people To use

Whirlpool trial

platform

Absenteeism rate after /before the DV/AV



Table 20 Whirlpool Results after the Second iteration

Objective DV/AV PI To improve the product quality To use

Whirlpool trial

platform

Ratio: FOR after/before the DV/AV implementation

during a period*

Ratio: Service Incidence Rate after / before the

DV/AV implementation during a period To increase the productivity To use

Whirlpool trial

Ratio: Overall Equipment Efficiency after / before the




52

platform

To improve the effectiveness of

equipment preventive maintenance To use

Whirlpool trial

platform

Ratio: Number of breakdown between two planned

maintenances after /before the DV/AV implementation

during a period*

% of defective parts to rework after /before the

DV/AV implementation during a period* To reduce the production cost To use

Whirlpool trial

platform

Ratio: Conversion cost per unit after /before the


To reduce the Total Cost of Quality To use

Whirlpool trial

platform

Ratio: Total cost of products scrapped after /before the



The results of the second iteration have been validated by the Trial Owner. Despite a readily

available set of Business Performance Indicators, it turned quite difficult to identify a coherent and

meaningful set. This has been mainly due to two reasons: 1) WHR trial success will be dependent

on how people will adopt the new event communication system; 2) relevant Business Performance

Indicators are only partly dependent from the success of FITMAN.

In the first iteration a general situation has been depicted: in presence of a renewed and improved

decision process, the expected impact could actually be measured along many different directions.

In the second iteration a closer reflection on how the specific events selected for the trial could be

used by the DM and how the present measuring system is capable of capturing the variations has

been done and hence a renewed list has been developed.

Some of the proposed indicators also turned out to be presently not measured by the factory: the

possibility of establishing a new measuring system has been negatively evaluated since we cannot

compare a short period data (too much variability and dependence from external factors) to an

expected long term effect due to FITMAN introduction.

8.2. Piacenza

Piacenza is a manufacturer of fine woolen fabrics, leader in the top segment of noble fiber fabrics

for luxury market, and pure cashmere knitwear. The trial concerns the fabric production which is

part of Smart Factory. It will focus on yarn dyeing, weaving (warping, weaving and raw control)

and finishing (wet finishing, raising and dry finishing). With the adoption of FITMAN “cloud

production” model, the trial intends to maximize the benefits for industrial end users production.

Table 21 Piacenza Results after the First iteration

Objective DV/AV PI To reduce the machinery

exploitation cost To use the

Piacenza trial

platform

Ratio: Machine cost per produced unit after / before

the DV/AV implementation during a period*

To reduce the production time To use the

Piacenza trial

platform

Ratio: Average production lead time per meter

produced after / before the DV/AV implementation

during a period* To reduce the quantity of energy

supporting the production To use the

Piacenza trial platform

Ratio: The quantity of energy spent per meter

produced after / before the DV/AV implementation during a period*

To reduce the quantity of gas

supporting the production To use the

Piacenza trial

platform

Ratio: The quantity of gas spent per meter produced

after / before the DV/AV implementation during a

period* Improve the monitoring of the

production capacity

To use the

Piacenza trial

platform

Ratio: Capacity available at equal number of machines


period*




53

Table 22 Piacenza Results after the Second iteration

Objective DV/AV PI To reduce the fixed costs per

machinery To use the

Piacenza trial

platform

Ratio: Machine fixed costs per produced unit after /

before the DV/AV implementation during a period*

To reduce the production time from

order to delivery To use the

Piacenza trial platform

Ratio: Average production lead time per meter

produced from order to delivery after / before the DV/AV implementation during a period*

To reduce the quantity of energy

for supporting systems for

production

To use the

Piacenza trial

platform

Ratio: The quantity of energy spent per meter

produced after / before the DV/AV implementation

during a period* Improve the monitoring of the

production capacity To use the

Piacenza trial

platform

Ratio: the number of production records after / before

the DV/AV implementation during a period*

Ratio: Percentage of forecast error after / before the



Status after second iteration: I-VLab has sent a first proposition of PIs. The trial Owner has sent a

feedback by improving the content. These results have been validated by Piacenza and I-VLab and

have been inserted in D4.4.

8.3. TRW

TRW Automotive is a worldwide reference (Tier 1 provider) in manufacturing (machining,

handling and assembly) of active and passive systems. It is one of the world’s largest automotive

suppliers. As leader in automotive safety, TRW produces active systems in braking, steering and

suspensions and sophisticated occupant safety systems, as seat belts, airbags and steering wheels. In

FITMAN, the TRW trial takes place in the Smart Factory application domain and seeks to improve

the health and safety of workers in production workplace through the adoption of FI-WARE

technologies.

Table 23 TRW Results after the First iteration

Objective DV/AV PI To decrease number of accidents

and incidents in the factory To use the TRW

trial platform

Ratio: Number of accidents and incidents in the


during a period* To increase the modelled risks and active preventions


Number of risks that has been defined using the new system after/ before the DV/AV implementation

during a period* To decrease the number of errors in

the prevention strategy To use the TRW

trial platform

Ratio: Number of human errors in the design of the

planning of prevention strategy after/ before the

DV/AV implementation during a period* To increase the profitability of the

investment in preventive strategy To use the TRW

trial platform

Ratio: Rate of profit of the investment in preventive

strategy after/ before the DV/AV during a period* Reduction of accidents and

incidents

To use the TRW

trial platform



during a period*

To increase the number of

deployed (H&S:Hygien &Security)

monitoring systems

To use the TRW

trial platform

Ratio: Number of deployed H&S monitoring system


period*



54

Ratio: Number of risk detectors, alarms and warnings

set up after / before the DV/AV implementation during

a period* To increase the number of H&S

training sessions To use the TRW

trial platform

Ratio: Number of training sessions regarding H&S


period* Increase the productivity To use the TRW

trial platform

Ratio: Number of produced units after / before the


Ratio: Number of permanent working employees after

/ before the implementation of the DV/AV during a period*


Status of this first iteration: this iteration was not validated by the Trial owner.

Table 24 TRW Results after the Second iteration

Objective DV/AV PI To increase the standards and

regulations in the repository To use the TRW

trial platform

Ratio: Number of standards and regulations added in

the repository after/before the DV/AV implementation

during a period* To reduction the number of

accidents and incidents in the

factory

To use the TRW

trial platform



during a period* To increase the modelled risks To use the TRW

trial platform

Ratio: Number of risks that has been defined using the

new system after / before the DV/AV implementation

during a period* To increase the modelled

preventive actions

To use the TRW

trial platform

Ratio: Number of preventive actions using the new

systems after /before the DV/AV implementation

during a period* To decrease the errors in the

prevention strategy

To use the TRW

trial platform

Ratio: Number of human errors in the design of

prevention strategy planning after /before the DV/AV


To reduce the number of accidents and incidents in the factory


Ratio: Number of accidents and incidents in the factory after / before the DV/AV implementation

during a period*

To increase the number of safety

systems

To use the TRW

trial platform

Ratio: Number of deployed monitoring systems after /

before the DV/AV implementation during a period*

To increase the number of risk

detections, alarms and warnings To use the TRW

trial platform

Ratio: Number of risk detections, alarms and warnings

set up after / before the DV/AV implementation during

a period* To increase the number of training

sessions regarding safety

To use the TRW

trial platform

Ratio: Number of training sessions regarding safety

after /before the DV/AV implementation during a

period*


Status of this second iteration: This information has been validated by the Trial Owner.



55

9. ANNEX V: Technical Indicators

9.1. Functional Technical Indicators

The first three Technical Indicators require the definition of a common unique value at Trial level

for each of the GE/SE used in the Trial Integrated Solution:

Table 25 Functional Technical Indicators

Technical Indicators for

GEs/SEs (P5)

Levels for the selection

Openness Level 0: Open specifications –Developers can view &

study the requirements posed and implement them as they

wish.

Level 1: Enablers as a Service – Developers can utilize

software provided as a service through open interfaces.

Level 2: Releasing code as open source - Developers can

inspect, download, run and improve the open source code

according to their needs.

Level 3: Consulting with the use cases about their needs

and collaboratively contributing to the source repository,

design documents, and bug reports.

Interoperability maturity Level 0: Isolated Approach: No API exposing the GE / SE

functionalities is available.

Level 1: Baseline Unified Approach (International

Standards exists): Offering an API exposing main part of

the GE / SE functionalities, in its own format.

Level 2: Open Unified Approach (No International


the GE / SE functionalities, in its own format.

Level 3: Standardized Integrated Approach (International


the GE / SE functionalities, following international

standards.

Ease of application Level 0: ”no applicability in our environment without extra

applying actions or means”.

Level 1: ”applicable with significant amount of work”.

Level 2: “applicable with limited amount of work”.

Level 3: “Easily applicable in our environment”.

Four different predefined levels have been defined for each Indicator, according to its features and

to the specific FITMAN needs.

As it is evident in Figure, the three Indicators are related to the Step P5 of the FITMAN V&V

Methodology, i.e. “Product Validation”, whose main aim is to understand if the product satisfies

intended use and user needs [1].

The specific meaning of each of the three Technical Indicators is explained below:

Openness: “A measure of defining the level of openness” [2], where openness is “Ensuring

that specific people groups may access the software for free with specified rights (depending

on the level of openness)” [1];



56

Interoperability Maturity: “A measure of how mature in terms of interoperability the

software is” [2], where interoperability is “The capability of the software to interact with

other systems” [1];

Ease of application: A measure of the applicability of the software in the particular

environment in terms of amount of work and extra actions or means.

9.2. Non-Functional Technical Indicators

The other five Technical Indicators require the personal evaluation of each of the participant to the

Trial Integrated Solution, i.e. Trial Support Partner, Trial Owner and the members of the Trial

Team. As a consequence, the final result will be the collection of the different perceptions of the

beneficiaries of the Trial Integrated Solution:

Table 26 Non-functional Technical Indicators

Technical Indicators for the Trial

Integrated Solution (T1)

Statements to be assessed

Fulfilment of requirements “The solution fulfils the Trial requirements”

Learnability “It is easy to start to use the solution and learn

functionalities.”

Understandability “The solution is easy and self-clear to

understand and the concepts and terminology

are understandable.”

User’s attraction level “The solution is attractive to the user. I feel

satisfied and comfortable when using it.”

Efficiency “The time and resources required to achieve

the objectives of the solution are reasonable,

the solution is fast enough and does not

require too many steps.”

As it is depicted in the Figure, for each of these Technical Indicators a specific statement should be

assessed by the user. For each of the sentences, he/she should express his/her own level of

agreement according to his/her experience and by choosing one option among the following ones:

I strongly agree

I agree

I disagree

I strongly disagree

These five indicators are related to the Step T1 of the FITMAN V&V Methodology, i.e. “Trial

Solution Validation”, whose main aim is to understand if the overall trial solution satisfies intended

use and user needs [1].

The specific meaning of each of the five Indicators is explained below:

Fulfilment of requirements: The capability of the software product to fulfil in a satisfying

way the requirements established by the Trial;

Learnability: “The capability of the software product to enable the user to learn its

applications” [1];

Understandability: “The capability of the software product to enable the user to understand

whether the software is suitable, and how it can be used for particular tasks and conditions

of use” [1];

User’s attraction level: “The capability of the software product to be liked by the user” [1];



57

Efficiency: “The capability of the software product to provide appropriate performance,

relative to the amount of resources used, under stated conditions” [1].

9.3. Software Verification (Self Certification)

The approach will hence address the first five Steps of the Methodology, i.e. Code Verification,

Model Verification, Backlog Verification, Release Verification and Product Validation. The reasons

of the adoption of this mechanism beyond the definition of Technical and Business Performance

Indicators are mainly the following:

the will not to prevent the different consolidated methods and procedures followed by the

different Development Teams in the creation of the different SEs(D2.3);

the extreme difficulty to create a general common procedure for the Verification and

Validation of extremely various and complex components (D2.3);

the will to define a V&V Process as simpler as possible but at the same time an effective

tool for the proper collection of different kinds of data (D2.3).

By means of Self-certification, the different Development Teams will be able to certify the correct

definition of the SEs, guided by specific procedures.

In particular, for each of the first five Steps a Recommended V&V Technique will be proposed,

together with proper V&V Success Conditions:

Table 27 Self-certification Methodology

Step

“Self-certification”

approach

Recommended V&V

Technique

V&V Success

Conditions Mandatory Step for

P1 - Code Verification White Box Testing

- The development

team has written tests

for all the code and

believes there are no

other tests to be

written for specific

functionalities.

- All bugs reported

during the tests are

fixed.

SEs

P2 - Model

Verification Traceability Analysis

- All requirements

trace correctly and

sufficiently to design

and implementation.

SEs

P3 - Backlog

Verification Regression Testing

- No old bugs reappear

- All new bugs

reported are fixed

SEs

P4 - Release

Verification Regression Testing

- All failures occurred

during testing are

resolved.

- Internal functions and

SEs



58

interfaces work as

expected.

P5 - Product

Validation

Black Box Testing for

Validation

- All failures reported

are resolved.

- Specification or the

implementation has no

defects or

incompliances

- The release

sufficiently provides

its intended

functionality to the

users.

- The Technical

Indicators have been

assessed.

SEs

For each SE and for each of the first five Steps, the Development Team will hence be able to

precise the result of the specific V&V Test:

V&V Technique – Positive result

V&V Technique – Negative result

V&V Technique – NA [if not needed or not performed yet]

or, in case, to point out the Alternative V&V Technique used and its final result:

Alternative V&V Technique – Positive result

Alternative V&V Technique – Negative result

Alternative V&V Technique – NA [if not needed or not performed yet]

future internet technologies for manufacturing 24/04/2014 … · 2017-04-25 · project id 604674...

Documents