vrije universiteit brussel validated crisp methodology

Vrije Universiteit Brussel

Validated CRISP MethodologyHempel, Leon; Hirrschman, Nathalie; Von Laufenberg, Roger; Wurster, Simone; Sveinsdottir,Thordis; Kamara, Irene; De Hert, Paul

Publication date:2015

License:Unspecified

Link to publication

Citation for published version (APA):Hempel, L., Hirrschman, N., Von Laufenberg, R., Wurster, S., Sveinsdottir, T., Kamara, I., & De Hert, P. (2015).Validated CRISP Methodology: Deliverable 5.1 for the CRISP project. CRISP project.

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portalTake down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

Download date: 08. Jan. 2022

https://researchportal.vub.be/en/publications/5e91abf4-aa73-417f-89f1-1da657c7ad0c

Project acronym: CRISP

Project title: Evaluation and Certification Schemes for Security Products

Grant number: 607941

Programme: Seventh Framework Programme for Security

Objective: Topic SEC-2013.5.4-1 Evaluation and certification schemes for security products

Contract type: Capability project

Start date of project: 01 April 2014

Duration: 36 months

Website: www.crispproject.eu

Deliverable D5.1: Validated CRISP Methodology

Authors: Leon Hempel & Nathalie Hirschmann (TUB-CTS);

Tatsiana Haponava (NEN);

Roger von Laufenberg (VICESSE); Simone Wurster (TUB-INNO); Thordis Sveinsdottir (Trilateral); Paul de Hert & Irene Kamara (VUB)

Contributors: Cristina Pauner, Jorge Viguri, Artemi Rallo & Rosario García (UJI), Jelena Burnik & Andrej Tomšič (IP-RS)

Dissemination level:

Public

Deliverable type: Final

Version: 1

Submission date: 30 October 2015

D5.1: Validated CRISP Methodology CRISP project

2

CRISP has received funding from the European Union’s Seventh Framework Program for research, technological development and demonstration under grant agreement no 607941. Re-use of information contained in this document for commercial and/or non-commercial purposes is authorised and free of charge, on the conditions of acknowledgement by the re-user of the source of the document, not distortion of the original meaning or message of the document and the non-liability of the CRISP consortium and/or partners for any consequence stemming from the re-use. The CRISP consortium does not accept responsibility for the consequences, errors or omissions herein enclosed. This document is subject to updates, revisions and extensions by the CRISP consortium. Questions and comments should be addressed to: [email protected]


3

Table of Contents

List of Selected Abbreviations ................................................................................................... 5

List of Figures ............................................................................................................................ 7

List of Tables ............................................................................................................................. 7

Executive Summary ................................................................................................................... 8

1. Introduction ........................................................................................................................ 9

2. Synthesis of Prior CRISP Work Packages ....................................................................... 12

2.1 WP 1: ‘Categorising Security Equipment, Systems and Services’ ........................... 12

2.2 WP 2: ‘Review of Standards, Certification and Accreditation for Security Products, Systems and Services’ ............................................................................................... 15

2.3 WP 3: ‘Security Certification Stakeholder Analysis’ ............................................... 19

2.4 WP 4: ‘The S-T-E-Fi-approach: Analysis of Core Dimensions – Security, Trust, Efficiency, Freedom Infringement’ ........................................................................... 23

3. CRISP’s Basis: Forming the First Draft of a Multidimensional Methodology ................ 27

4. Validation Activities of the First Methodology Draft ...................................................... 31

4.1 Outlining of the Market Evaluation and Certification .............................................. 31

4.2 Validation Workshop ................................................................................................ 33

4.3 Advisory Board Members ......................................................................................... 35

5. Using Standards in the CRISP Evaluation and Certification methodology ..................... 37

6. Evaluation and Certification Methodology ...................................................................... 42

6.1 Evaluation Process Within CRISP’s Methodology ................................................... 43

6.1.1 Evaluation Configuration and Assessment ........................................................ 44

6.1.2 Actor Roles and Functions ................................................................................. 49

6.1.3 Response to Particular Validation Workshop Feedback .................................... 51

6.2 Audit and Attestation as a Certification Part of the CRISP Methodology ................ 53

6.2.1 Connecting Standard Functions of the Certification Process to CRISP Evaluation and Certification Methodology ........................................................ 53

6.2.2 Requirements Based on Evaluation Criteria as an Input for the Certification Part of the CRISP Methodology ................................................................................ 54


4

6.2.3 Certification: What is Needed? .......................................................................... 56

7. Summary: Towards Certification within the CRISP Evaluation and Certification Methodology ..................................................................................................................... 61

List of References .................................................................................................................... 64

Appendix .................................................................................................................................. 66

Appendix 1: On the Market Evaluation and Certification Models ...................................... 66

Appendix 2: Validation Workshop Invitation and Agenda ................................................. 89

Appendix 3: Feedback Questionnaire (Validation Workshop) ............................................ 90


5

LIST OF SELECTED ABBREVIATIONS CBRNE Chemical, Biological, Radiological, Nuclear and Explosive

CCTV Closed-circuit television

CEN Comité Européen de Normalisation

CENELEC Comité Européen de Normalisation Electrotechnique

CoESS Confederation of European security services

DAkkS Deutsche Akkreditierungsstelle GmbH

DIN Deutsches Institut für Normung

DPAs Data Protection Authorities

DEL Deliverable report

EA European co-operation for Accredition

EN European Norm

ETSI European Telecommunications Standards Institute

IEC International Electrotechnical Commission

ISO International Organization for Standardization

NEN Netherlands Standardization Institute

PSS Product – System – Service

R1|R2: Output from one evaluation activity which work as input to another activity

RPAS Remotely Piloted Aerial Systems

SIAM Security Impact Assessment Measures

S-T-E-Fi Security – Trust – Efficiency – Freedom infringement

SWOT Strengths-Weaknesses-Opportunities-Threats model

TC Technical Committee

TRL Technology Readiness Level

TUB-CTS Technische Universität – Center for Technology and Society

WP Work Package

WP 1 CRISP Work Package on ‘Categorising Security Equipment, Systems and Services’

WP 2 CRISP Work Package on ‘Review of Standards, Certification and Accreditation for Security Products, Systems and Services’

WP 3 CRISP Work Package on ‚Security Certification Stakeholder Analysis‘

WP 4 CRISP Work Package on ‘The S-T-E-Fi Approach: Analysis of Core Dimensions – Security, Trust, Efficiency, Freedom Infringement’

WP 5 CRISP Work Package on the ‚Validated Certification Methodology’

WP 6 CRISP Work Package on ‘Developing a Roadmap’


6

WP7 CRISP Work Package on ‘Enhancing Confidence in the New Certification Measures‘

WP 8 CRISP Work Package on ‘Dissemination’


7

LIST OF FIGURES

Figure 1: CRISP’s core dimensions and exemplary stakeholder classification ....................... 28 Figure 2: Four level structure of the S-T-E-Fi approach ......................................................... 29 Figure 3: Schematic representation of using existing standards as a ‘hard’ condition ............ 38 Figure 4: Schematic representation of using existing standards as a ‘soft’ condition ............ 39 Figure 5: CRISP evaluation and certification methodology .................................................... 42 Figure 6: Proposed configuration and assessment stages during evaluation ........................... 43 Figure 7: Mock-up of proposed evaluation criteria questioning .............................................. 46 Figure 8: Example mockup of a conflict matrix per dimension .............................................. 47 Figure 9: The difference between standards and certification schemes .................................. 56

LIST OF TABLES

Table 1: WP 1 recommendations and reference to upcoming CRISP work ............................ 14 Table 2: WP 2 recommendations and reference to upcoming CRISP work ............................ 18 Table 3: WP 3 recommendations and reference to upcoming CRISP work ............................ 22 Table 4: WP 4 recommendations and reference to upcoming CRISP work ............................ 26 Table 5: On the market evaluation and certification ................................................................ 31 Table 6: Possible assessment findings reported in R2 .............................................................. 48 Table 7: Stakeholder groups and intended allocation to S-T-E-Fi dimensions ....................... 49 Table 8: Intended actor roles and functions during evaluation ................................................ 50 Table 9: The main requirements regarding the content of a certification scheme ................... 57


8

EXECUTIVE SUMMARY

This project report is the deliverable for CRISP’s Work Package 5 (WP 5), ‘Validated Certification Methodology’. The objectives of this WP are as follows:

to develop and describe a harmonised approach and certification methodology for security products and systems in Europe, to develop and outline policy and certification procedures for security certification, and to test drive the proposed certification model, evaluate its actual working and, accordingly, revise and refine the certification scheme. Here, the revision of the certification scheme will be focused on the main content of the certification scheme and will be refined by the recommendations from experts during a number of scenario based workshops. The results of the revision and refinement of the certification scheme will be included in the second report of WP 5, which is about the documentation of the scenario based workshops to be undertaken in the next months of the CRISP project (DEL 5.2).

The focus of this WP 5 report is on the concept development and validation of the project’s evaluation and certification methodology, not on presenting a final certification scheme. Thus, this report is entitled ‘Validated CRISP Methodology’. A further testing of the CRISP methodology – as scheduled in the shape of scenario based workshops – is needed in order to:

1. Draft guidelines on the way the evaluation criteria related to the S-T-E-Fi dimensions could be translated into requirements for a normative document which will be the basis for the certification scheme.

2. Draft recommendations on the content of a certification scheme for security PSS based on the S-T-E-Fi dimensions.


9

1. INTRODUCTION

In theory, an optimal security measure should be safe and secure, trustworthy, efficient, and should not violate rights. In practice, to achieve these dimensions all at once seems difficult as they may sometimes even conflict with each other. There is a strong market fragmentation, legal uncertainty, divergent national standards and irritations between market participants and additional expenses for both producers and users which interfere with the conformity assessment and certification environment of security products, systems and services.1

“At EU-level there is no common (single) framework that applies to security products and the market for security products as a whole. Rather, there are a multitude of different rules and regulations that have been adopted to cover security concerns related to different sectors and activities”.2

Furthermore, the agenda setting is often dominated by strong stakeholder groups, which leads to inconsistency between security and safety requirements and security and safety expectations.

The EU FP7 funded project “CRISP” (Evaluation and Certification Schemes for Security Products; Grant Agreement No: 607941) aims at facilitating a harmonised playing field for the European security industry by developing a robust methodology for security products, systems, and service certification. To achieve the most optimal solution approximately possible, CRISP will enhance existing security evaluation and certification schemes by offering certification based on a four-dimensional approach.

CRISP’s starting point is the European Commissions’ claim that the European security market is highly fragmented and a harmonised European certification scheme is eligible and hence should be fostered. Responding to this fragmentation, CRISP follows a tailored strategy by investigating the state of the art in standards and certification of security products, systems and services (PSS3), analysing the EU security market, identifying and analysing relevant stakeholders and developing an innovative multidimensional methodology that will address the gaps in the current certification landscape and strive to meet the recommendations of the European Commission particularly. The key outcome of the CRISP project shall be a certification manual and the proposition of a certification scheme. The

1 Reigel, Markus, ‘Ansätze zur Angleichung der unterschiedlichen Konformitätsbewertungsansätze‘;

presentation at the DKE-Workshop Konformitätsbewertungsbedarf Informationssicherheit, 7. October 2015, Frankfurt am Main, Germany.

2 ECORYS, Security Regulation, Conformity Assessment & Certification. Final Report – Volume 1: Main Report, Brussels, 2011. http://ec.europa.eu/dgs/home-affairs/e-library/documents/policies/security/pdf/secerca_final _report_volume__1_main_report_en.pdf, p. 18.

3 As defined by Fritz et al. (2014) in the context of the CRISP project, security PSS “are all those PSS which serve a security function […], or which, in other words, give operators the capability to perform such functions. This means that it is the context that allows us to decide whether a PSS should be framed as ‘security-related’ or not.” As ‘products’ are interpreted as providing “a security function or capability as part of a given security measure or intervention […] ‘systems would typically refer to more complex combinations of such building blocks, their integration to a higher degree of complexity, coupling several ‘products’, possibly also with a service” (Fritz, Florian, Reinhard Kreissl, Roger von Laufenberg, Paul de Hert, Alessia Tanas, Rosamunde van Brakel, Simone Wurster, “Glossary of Security Products and Systems”, DEL 1.1 CRISP Project, 31 July 2014, p. 7).


10

challenge is to base the variety of security PSS on a corporate approach which satisfies all needs of all stakeholders involved, enhances existing security evaluation and certification schemes and contributes foremost to a harmonisation in security certification.

As will be presented in this report, a two-part evaluation and certification methodology has been developed during the research work of this WP 5 that allows to encompass a variety of security PSS, taking into account the varying roles of a diverse stakeholder community and containing a set of evaluation criteria that have been developed alongside four core dimensions – Security, Trust, Efficiency and Freedom infringement (S-T-E-Fi).

This S-T-E-Fi approach emerged in the course of empirical research work of the EU FP7 funded SIAM project and has been enlarged in the CRISP project.4 Within SIAM, it was recognised that by means of a certain stakeholder constellation, particular criteria and attributes related to security measures and technologies occurred at all times, which were structured into the above introduced dimensions. Numerous notions of security and safety are in use, often depending on the community a stakeholder is associated with. An engineer responsible for the computer infrastructure at an airport, for instance, has another understanding of security than a police officer responsible for border control security might have. Hence, the S-T-E-Fi approach attempts to systematise assessment dimensions by encompassing as many perspectives as possible in an easily comprehensible manner, which allows the integration of different perspectives and activities in one approach and thus increases the level of inter-subjectivity during an assessment process.5 Implementing the S-T-E-Fi approach into the CRISP methodology shall help to avoid acceptance problems that challenge current certification schemes by offering certification applicants to go beyond usual self-assured methods of simplification and confronting them with complexity and possible effects that a security PSS might have in the S-T-E-Fi context. On this account, the S-T-E-Fi dimensions, as the highest aggregation level, represent the core of CRISP’s evaluation methodology of security PSS, which covers the first two functions of certification (‘selection’ and ‘determination’6). To achieve an optimum solution by approximation an innovative evaluation methodology of security PSS should take into account its appropriate environment.

“Security is a process, not a product. Products provide some protection, but the only way to effectively do business in an insecure world is to put processes in

4 The S-T-E-Fi model is a multilevel bottom-up approach and followed from the EU FP7 funded project

“Security Impact Assessment Measures” (SIAM), February 2011 until March 2014; see project website: www.siam-project.eu. Hempel, Leon, Lars Ostermeier, Tobias Schaaf and Dagny Vedder, “Towards a social impact assessment of security technologies: A bottom-up approach”, Science and Public: Policy, Vol. 40, 2013, pp. 740-754.

5 Hempel, et al., op. cit., 2013, p. 748. 6 As defined by ISO/IEC 17067:2013 Conformity assessment - Fundamentals of product certification and

guidelines for product certification schemes.


11

place that recognize the inherent insecurity in the products. The trick is to reduce your risk of exposure regardless of the products or patches.”7

In more detail, this report is structured as follows:

Chapter 2 presents the previous CRISP WP findings translated into concrete recommendations which have been – were appropriate – considered for CRISP’s evaluation and certification methodology.

Chapter 3 illustrates the base of CRISP’s methodology, a multidimensional approach on which the methodology is formed at.

Chapter 4 describes validation activities which have been performed to review first drafts of the methodology and to refine it accordingly to feedback suggestions and recommendations.

Chapter 5 is devoted to the aspect of how to use existing standards in CRISP’s evaluation and certification methodology.

Chapter 6 describes the evaluation and certification methodology by explaining the relevant stages during an evaluation and certification of a security PSS.

Chapter 7 highlights main outcomes and the steps needed to present and finalise specific elements of the evaluation and certification methodology and finishes with recommendations which emerged during this WP’s research.

7 Schneier, Bruce, “Computer Security: Will We Ever Learn?”, 15. May 2000,

https://www.schneier.com/crypto-gram/archives/2000/0515.html. Although this statement was made in the context of IT-security, it seems legitimate for other security areas as well.


12

2. SYNTHESIS OF PRIOR CRISP WORK PACKAGES

2.1 WP 1: ‘CATEGORISING SECURITY EQUIPMENT, SYSTEMS AND SERVICES’

The objective of WP 1 was to provide a common understanding within the broad field of security products, systems and services (PSS), as well as application areas and concepts of operations. In two steps, first a glossary of security PSS was developed, which then served as a foundation for a taxonomy. This also allowed for a better comparison and evaluation of security PSS, which was part of the second objective of this WP.

The findings of WP 1 were disseminated in two deliverables: DEL 1.1 Glossary of Security Products and Systems8 and DEL 1.2 Taxonomy of Security Products, Systems and Services9.

DEL 1.1 offers the terminological foundation for the CRISP project. A common understanding of the terminology used in CRISP is an important aspect, and is thus the main objective of the glossary. The CRISP glossary for security PSS has a unique characteristic, which is the focus on the functionality of security PSS instead of the technique itself. With the function of a security PSS, the intended result of security PSS in operation were described. This approach was chosen, since defining a product, system or service in a technological way often involves a discretionary component, as “each technological system is comprised of several interconnected elements and each of these elements can itself be analysed as a system of its own.”10 By analysing the intended function of a product, system or service, this can be prevented and allows an easier identification of security PSS. The security aspect of the PSS is always context related, as a technology in itself is seldom considered to own a security feature. Only when applied in such a context and with the specific desired function(s), it becomes a security technology.

Thus based on these considerations, the glossary of security functions for PSS was established for the CRISP project. On the basis of relevant literature, the following functions have been identified, without claiming that they are complete:

Locate, identify, verify, control, track, assess, authorise, create situational awareness, information collection, storage and management to produce intelligence, detain, prevent/protect

As already mentioned, the context in which a technology is deployed attributes the aspect of ‘security’ to that specific technology. A similar consideration needs to be made for the security function, since depending on the context where they are required, the function might aim at different objects or subjects, or be fulfilled by different products, systems or services. By introducing the area of security, the different security contexts are reflected and the function can be attributed to different security PSS depending on the area where it is required. 8 Fritz, Florian, Reinhard Kreissl, Roger von Laufenberg, Paul de Hert, Alessia Tanas, Rosamunde van

Brakel, Simone Wurster, “Glossary of Security Products and Systems”, DEL 1.1 CRISP Project, 31 July 2014.

9 Sveinsdottir, Thordis, Rachel Finn, Rowena Rodrigues, Kush Wadhwa, Florian Fritz, Reinhard Kreissl, Roger von Laufenberg, Paul de Hert, Alessia Tanas, Rosamunde van Brakel, “Taxonomy of Security Products, Systems and Services”, DEL 1.2 CRISP Project, 31 July 2014.

10 Fritz, et al., op. cit., 2014, p. 37.


13

For the CRISP glossary, four areas of security were identified: (1) security of the citizen; (2) critical infrastructures; (3) border security; (4) crisis management.

Finally, in order to classify and also compare security functions, the existence of different levels have been considered on which the functions operate in security PSS, based on the dependency functions have on other functions. As an example, the function ‘to identify’ is only possible if the function ‘information collection’ has been performed beforehand. There is thus a strong interdependency between the functions, which is also expressed in the specific security PSS, as they seldom serve only one function. This results in three levels of functions, with an increase in dependency (and often complexity), the levels being primitive, connective and performative functions. The final result of the deliverable 1.1 and the main output is a dictionary-like listing of the functions with an extensive definition (along the application areas) and an exemplary sample of security PSS performing this function.

Based on the glossary of the first deliverable, in DEL 1.2 a categorisation of security PSS was established. In form of taxonomies for security products, security systems and security services, this deliverable examines the wide diversity in the domain of security PSS. The categorisation of the taxonomy is done on three levels: On the first level are the security application areas. Similar as in the glossary, the taxonomy categorises security PSS, depending on where the intended application is based. While the glossary consists of a more condensed approach towards the security areas, since they only slightly influence the definition of the function, the application areas for the taxonomy need to be more extensive, as these solely serve classification purposes. The extended application areas include:

Border management, critical infrastructure, emergency preparedness centres/crisis management, households and individuals, industry and retail, public and semi-public venues, transport

The second level of categorisation of taxonomy are the security demands. The demands are mainly based on reviews of commercial security company websites, European policy literature and related security research projects and further specify the areas where the security PSS can operate while already refining the intended security function. For the taxonomy, the following security demands were identified:

Access control, asset / freight / cargo security, cyber security, employee / visitor / passenger safety, loss prevention / shrinkage, perimeter / area / building security, point of transaction security, situation awareness

On the third and last level of categorisation are the security needs, as they are called for the taxonomy (as a follow-up of the demand category), which are equivalent to the security functions of the glossary. By creating the taxonomy, three additional security needs (or security functions) were identified and added to the existing ones: communicate, respond and detect. Following the security needs the specific security PSS is added, depending on their application within that classification. The security PSS is of course not exclusively classified within one categorisation, but can, similar to the security function, be part of different application areas, security demands, and needs. As such, the taxonomy is a very flexible tool for the categorisation of security PSS.


14

One peculiarity of the taxonomy of security services is the addition of an extra category in the taxonomy, reflecting the amount and quality of the training necessary by the security professional (by standards or other legal/compliance codes) to perform a specific security service. This facilitates a better understanding of the service with regard to its nature – is it more manpower-oriented or is it more of an intellectual service? – that also plays an important role for the certification and standardisation of the training. Although the output of WP 1 is divided into two deliverables, it is important to see them in their totality in order to understand security PSS from a function-based approach and to be able to categorise and compare them along the different levels as introduced in this chapter.

The table below summarises main recommendations of WP 1 outcomes which are related to the continuing project work of CRISP. Those recommendations of Table 1 referring to WP 5 are highlighted and have been considered more closely for the development of the two methodology parts – evaluation and certification – which will be explained in chapter 6 of this report in more detail.

Table 1: WP 1 recommendations and reference to upcoming CRISP work WP 1 recommendations WP relation WP 5 referring Certification scheme should not only consider the technical aspects of security PSS, but also their intended functionality.

WP 5 WP 6* WP 7**

security PSS functionality is considered and introduced in the configuration stage of the evaluation part; see chapter 6.1.1

Functions depend on the context / application area of the security PSS and should be considered by the certification scheme.

WP 5 WP 6 WP 7

context / application area of a security PSS is considered and introduced in the configuration stage of the evaluation part; see chapter 6.1.1

The interconnectivity of PSS are responsible for the emergence of higher-level functions, and should thus also be considered in the certification scheme.

WP 5 WP 6 WP 7

see above

Taxonomies of security PSS as a mean for comparison of PSS – which can be included into certification systems.

WP 7

Security services are difficult to classify as they represent a large amount of functions, but should nonetheless be considered by the certification scheme, as they have a wide range of applicability.

WP 5 WP 6 WP 7

considered and included in CRISP’s evaluation and certification methodology

Source: Fritz, et al., op. cit., 2014; Sveinsdottir, et al., op. cit., 2014. Note: Numbers put in parentheses refer to the respective task of the CRISP WP: *WP 6: ‘Developing a Roadmap’, starting month 21; **WP 7: ‘Enhancing Confidence in the New Certification Measures’, starting month 27.


15

2.2 WP 2: ‘REVIEW OF STANDARDS, CERTIFICATION AND ACCREDITATION FOR SECURITY PRODUCTS, SYSTEMS AND SERVICES’

The work on WP 2 in particular showed the importance of 13 framework conditions for CRISP’s work:

x The European security certification landscape is complex.

x Hundreds of security certification bodies exist, most of them with various security certification services.

x Different guidelines as well as national, European and international standards are used for certification, even for identical topics.

x Privacy aspects are also considered differently between EU countries.

x The number of new certificates/certification schemes based on the standards of the new security TCs at CEN and CENELEC is limited.

x In addition to existing standards, common documents with more specific guidelines for certification are missing.

x Different auditing practices in accreditation are hindering the implementation of comparable certification services.

x Besides existing databases of EA members, there are no databases of organisations, which use documents with restricted access for security certification (for example governmental organisations).

x The number of opportunities to certify new, innovative security solutions is limited in the EU, which is a barrier regarding market access.

x There are also good practice examples, for example ARGE DIN 14675 and accreditation services, which consider specific security requirements.

x International country studies unveiled additional good practice for instance The U.S. Safety Act, which provides certification opportunities for various types of security PSS11.

x Experts in particular formulated the following suggestions: one-stop certification, formulation of performance-based requirements, development of use cases for selected security standards, interoperable solutions, accreditation, and regular follow-up controls of certified solutions as well as implementation of common auditing measures at accreditation bodies.

11 Criteria of the U.S. Safety Act were, for example, used to enrich the S-T-E-Fi criteria list to be developed in

CRISP’s WP 4. For more information on WP 4 see chapter 2.4 of this report.


16

The findings of WP 2 were disseminated in two deliverables: DEL 2.1 Report on security standards and certification in Europe – A historical/evolutionary perspective12 and DEL 2.2 Consolidated report on security standards, certification and accreditation – best practice and lessons learnt13.

DEL 2.1 offers an analysis of the state of the art of security standards and certification in Europe. It also explains the economic benefits of conformity assessment and the value of common solutions and / or mutual recognition in the security field. Specific security-related standards and technical committees are described, and current and potential interrelations between standards and certification are shown.

An important aspect is the interrelation of standardisation and certification. Many security-related TCs are quite new and new standards are under development. Therefore, it is recommended to seek collaborations with certification bodies in early stages.

Security areas in which the use of open standards is limited were also analysed. Usually, several different governmental authorities and security authorities are responsible for these topics in a Member State, making the European landscape very complex in this regard. Although databases exist that show all national certification bodies which are accredited by a national EA member, databases of non-EA members are not available. Therefore, an extension of current databases or creating an additional database is recommended. An additional area in which standards are not used for certification is related to innovative security solutions, for which standards do not exist yet. Besides illustrating the state of harmonisation and mutual recognition in Europe, DEL 2.1 also described suggested concepts of “one-stop testing” and “multiple certification”. There are fields in which appropriate certification solutions are missing in general, for example in areas of complex security systems. With regards to fields with existing certification services in which common solutions are missing, it was expressed that the key issue is often not the use of alternative documents for certification instead of common standards. Differences between the certificates are rather caused by documents which are used in addition to standards. Therefore, needs for additional standards in these fields as well as their potential usability for certification must be analysed in further research. Furthermore, security issues, shaped by different national preferences in Member States have been named. EN 50131, which includes specific national amendments provides an example for that. It shows that there are areas which should not be covered by general harmonised solutions, but by complementary certification. The number of these areas is to be kept as small as possible. Several European

12 Wurster, Simone, Tim Pohlmann, Patrick Murphy, Florian Fritz, Roger von Laufenberg, Jolien van Zetten,

Cristina Pauner, Artemi Rallo, Rosario García Mahamut, Rosamunde van Brakel, Alessia Tanas, “Report on security standards and certification in Europe – A historical/evolutionary perspective”, DEL 2.1 CRISP Project, 30. August 2014.

13 Wurster, Simone, Tim Pohlmann, Nathalie Hirschmann, Patrick Murphy, Jolien van Zetten, Ying Ying Lau, Tatsiana Haponava, Thordis Sveinsdottir, Rachel Finn, Rowena Rodrigues, Kush Wadhwa, Reinhard Kreissl, Florian Fritz, Roger von Laufenberg, Cristina Pauner, Artemi Rallo, Rosario García Mahamut , Jorge Viguri, Irene Kamara, Paul de Hert, Eva Kalan, Jelena Burnik, Igor Kolar, “Consolidated report on security standards, certification and accreditation – best practice and lessons learnt”, DEL 2.2 CRISP Project, 30. June 2015.


17

countries are perceived as providers of high quality products and solutions. There are specific concerns that collaborations with providers of ‘other’ certificates whose requirements are less advanced bear the risk of diluting the image of their own certificate. Therefore, reaching a common high quality level of EU security certificates is important, for example based on the development of a common auditing guide.

DEL 2.2 presents specific national findings by conducting ten European and six international case studies, which show the varying and complex framework conditions for security standards and certification in the Member States but also good practices. Differences of the security certification landscape include, for example, the different country size and protection needs, the different size of the national markets for security solutions and of the security industries, different structures of this industry and its market segments as well as different needs regarding security certification. Another observation was that Europe’s security industry includes many security service companies. In the CRISP context, services for security systems are particularly important. The new standard EN 16763 Services for fire safety systems and security systems will provide a foundation to offer harmonised certification services for these companies, but the need for additional standards remains as the document can only be used in conjunction with additional documents.

The findings of WP 2 also highlight supplementary need for action. There are Member States in which a significant number of the security certification bodies are not accredited despite the importance of this proof of qualification.

Based on the analysis of the framework conditions in Europe, of multinational standards, certification schemes, and the European and international country studies, specific recommendations were derived. As shown at the beginning, they consider, for example, the socio-technical contexts in the Member States and the need for performance-based requirements. They also highlight the importance of interoperable solutions, of accreditation, regular follow-up controls of certified solutions, as well as the implementation of common auditing measures at accreditation bodies. In summary, CRISP’s WP 2 provides substantial information for the project’s further activities, particularly, for CRISP’s roadmap and certification manual in WP 6 (Developing a Roadmap). Moreover, the information obtained from stakeholder interviews was valuable and, in several cases, fundamental, for WP 3 (Security Certification Stakeholder Analysis), WP 4 (The S-T-E-Fi Approach: Analysis of Core Dimensions – Security, Trust, Efficiency, Freedom Infringement) and the extension of WP 8’s stakeholder list (Dissemination). WP 2 also helped to identify certification schemes for the present WP 5.



18

Table 2: WP 2 recommendations and reference to upcoming CRISP work WP 2 recommendations WP relation WP 5 referring Facilitate one-stop certification. WP 5

WP 6* considered in certification

Consider the work of CEN and ISO TCs on privacy, data protection and privacy by design.

WP 5 WP 6 WP 7 (5)

considered in WP 5 preparation and in certification

Based on the example of the U.S. Safety Act: also consider new, innovative PSS and the development of solutions to certify them.

WP 5 WP 6 WP 7** (5)

the methodology presented in chapter 6 of this report may provide specific advantages in this regard. In the context of evaluation, the ‘Technological Readiness Level’ is introduced in the configuration stage; see chapter 6.1.1

Analyse and consider needs for flexible parts in the certification scheme to consider important differences in the Member States, e.g. reflected by EN 54-13.

WP 5 the methodology presented in chapter 6 might avoid many problems.

In addition to existing standards: develop common documents with more specific guidelines for the certification of selected security PSS.

WP 6 WP 7 (5)

Formulate performance-based requirements. WP 5 To be considered in the upcoming WP 5 work (scenario based workshops); see reference in chapter 7

Provide use cases for selected areas. WP 5 (3) WP 6 WP 7 (5)

To be considered in the upcoming WP 5 work (scenario based workshops); see reference in chapter 7. The results of these workshops can be used as a foundation for such activities.

Focus on interoperable solutions. WP 5 This aspect has been already considered in CRISP’s WP 4 in the context of the Efficiency dimension and is therefore included in the assessment stage of the evaluation part; see


19

WP 2 recommendations WP relation WP 5 referring chapter 6.1.1

Strengthen the role of accreditation in the European security certification landscape.

WP 6

Provide solutions to ensure that the auditing practices in security-related certification and accreditation are as harmonised as possible.

WP 6 WP 7 (5)

Ensure regular follow-up controls of certified solutions.

WP 6

Source: Wurster, et al., op. cit., 2014; Wurster, et al., op. cit., 2015. Note: Numbers put in parentheses refer to the respective task of the CRISP WP: *WP 6: ‘Developing a Roadmap’, starting month 21; **WP 7: ‘Enhancing Confidence in the New Certification Measures’, starting month 27.

2.3 WP 3: ‘SECURITY CERTIFICATION STAKEHOLDER ANALYSIS’

The findings of WP 3 were disseminated in the deliverable: DEL 3.1 Stakeholder Analysis Report14. The objectives of WP 3 were as follows:

x To identify all direct and indirect stakeholders in the security products and services standardisation and certification sector and understand stakeholder motivations underlying security standardisation and certification

x To gauge stakeholder views on security certification challenges and determine whether and how a new improved scheme for security certification could be implemented across Europe

x To develop and outline recommendations for standards and evaluation policies for security certification.

In order to reach these objectives, WP 3 first conducted a review of the relevant policy, grey and academic literature for the purpose of identifying key stakeholder groups in security certification. The literature review also helped to identify key challenges and good practice within this field. Secondly, it undertook three case studies of specific technology areas to understand the general and field-specific issues of security certification and key stakeholder views on and needs from certification. The case studies included stakeholder interviews, which aimed to identify specific needs of different stakeholder groups (for example, manufacturers, end-users, and certification bodies) when it comes to the development of standards and certification schemes, and also how they perceive the notion of a European certification scheme for security PSS. The three case study technologies were as follows: Video Surveillance Cameras (CCTV), Remotely Piloted Aerial Systems (RPAS) and Alarm Systems (Fire and Intrusion Alarms). Thirdly, the findings on stakeholder needs and views were verified and discussed at a stakeholder workshop in December 2014 and then used to

14 Sveinsdottir, Thordis, Rachel Finn, Kush Wadhwa, Rowena Rodrigues, Jolien van Zetten, Simone Wurster,

Patrick Murphy, Nathalie Hirschmann, Artemi Rallo, Rosario García, Cristina Pauner, Jorge Viguri, Eva Kalan, Igor Kolar, “Stakeholder Analysis Report”, DEL 3.1 CRISP Project, 28. February 2015.


20

develop two web surveys (for demand- and supply-side stakeholders), which helped to ascertain the prevalence of views expressed among a broader set of stakeholders in Europe.

The review of literature identified the following key groups of stakeholders:

Security product manufacturers, suppliers and system integrators (henceforth referred to as security industry), conformity assessment and certification bodies, standardisation organisations, accreditation bodies, data protection authorities and other regulators, end-users

Secondary stakeholders (affected by and indirectly involved in security certification):

Watchdogs and civil society organisations, Individuals, Academics

The bullet points below give a brief summary of key stakeholder groups’ needs from security certification, as they were presented to the CRISP consortium through the research:

x For the security industry, certification is important for accessing different national markets, as well as providing requirements to which products and services are designed and developed. The key need of the security industry is increased harmonisation of certification across borders for the purposes of avoiding re-certification which is costly, time-consuming, and causes delays to the market.

x Certification bodies expressed the need for a certification scheme, founded on robust standards and clear evaluation criteria. It is also of key importance that a certification scheme is well known, accepted and trusted, thus offering clear added value for the industry. In the absence of the aforementioned qualities, certification bodies will have difficulties justifying and selling the scheme to security industry stakeholders.

x Accreditation and standardisation bodies, while sharing concerns regarding the lack of transparency and the complexity of the current certification landscape, view their processes for accreditation and developing standards as robust and fit for the purpose of supporting a new certification scheme. Their needs regarding any new certification scheme is that its purpose and evaluation criteria are clear and transparent so that the development of standards or accreditation is consistent and straightforward.

x Out of the group of regulators relevant to certification, Data Protection Authorities (DPAs) were identified as very relevant in light of the social evaluation of the CRISP scheme. Currently, DPAs are usually not involved in standards development nor certification (with the exception of Germany) but due to their focus on privacy and data protection aspects of surveillance and other security technologies, they function as a stakeholder which will be necessary in the further development of the certification scheme and in ensuring acceptance and trust.

x End-users feel that the current certification system is complex and opaque, that there are too many schemes / seals available, and the difference between them is not clear. Increasing transparency regarding evaluation criteria for different schemes is thus imperative to better meet end-users needs regarding certification.


21

Regarding the CRISP certification scheme and its focus on the evaluation of social dimensions, WP 3 found that overall stakeholders were positive towards this type of scheme. The research findings however revealed that stakeholders were uncertain about what such a scheme would look like and how it would be applied. There were some concerns that social dimensions would be too vague to serve as efficient evaluation criteria. This indicates that, beyond the design and development of the scheme, the CRISP consortium must raise awareness about the prospective scheme amongst relevant stakeholder groups, to ensure acceptance, even implementation and take-up across Europe. This, in addition to further stakeholder consultation and road mapping exercises, is the focus of the subsequent work packages of CRISP.

Working towards the harmonisation of security certification in Europe is a complex task, given the myriad products, systems and services available, the pace of technological development and consequent social concerns, and last but not least the differing national regulatory, legal, social and cultural aspects within the European countries. These drew our attention to how flexibility must be inherent in the CRISP scheme to account for different technologies, means of operation, national cultures, as well as legal and regulatory frameworks in each member state. The case studies furthermore revealed that while some fields are well on their way towards a harmonised landscape (e.g., alarm systems), newer technologies, such as RPAS, are very much at the starting point of setting up a system of certification across Europe.

National differences emerged as key barriers to harmonisation of security certification. These need to be fully understood and tackled in order to successfully work towards harmonisation. Consequently, the CRISP scheme should be flexible to the extent of including national requirements as supplementary to the overall minimum requirements stated in the scheme. The aim here, however, would be to keep national requirements to a minimum to avoid continuing fragmentation in certification. Further to raising awareness and meeting the needs of key stakeholders, mechanisms for ensuring consistency and acceptance across Europe were identified as key success factors for implementing the CRISP scheme. As identified in the literature review and analysis of stakeholder roles, accreditation bodies and European cooperation for Accreditation (EA), here emerge as strong organisations to ensure a consistent accreditation and of the scheme across Europe. To ensure that the scheme complies with national regulation, a collaboration between DPAs and accreditation bodies and their respective Europe-wide bodies were suggested as useful endeavour to ensure consistency and compliance, whilst avoiding fragmentation and uneven application. The WP 3 report ends with listing minimum requirements for a harmonised approach which are derived from data analysis and all research tasks. These recommendations are intended as guidance for the consequent WP’s as the project embarks on designing and developing the scheme. The recommendations clearly outline which steps must be taken in order to develop, implement, and encourage the take-up of the new scheme.

The table below summarises main recommendations of WP 3 outcomes which are related to the continuing project work of CRISP. Those recommendations of Table 3 referring to WP 5 are highlighted and have been considered more closely for the development of the two


22

methodology parts – evaluation and certification – which will be explained in chapter 6 of this report in more detail.

Table 3: WP 3 recommendations and reference to upcoming CRISP work WP 3 recommendations WP relation WP 5 referring The certification system should be based on robust European or international standards.

WP 5 WP 6*

partly considered in chapter 5 and considered in certification; see chapter 6.2

The certification schemes and the underlying requirements should be transparent and clear in what they evaluate and certify.

WP 5 referring to evaluation & certification; see chapter 6

The certification system must be accepted throughout Europe.

WP 6

Certification bodies should ensure that their evaluation and standards interpretation, is consistent throughout Europe.

WP 7**

The certification system should be endorsed (or enforced) by regulators.

WP 7

The certification scheme should be operated under accreditation.

WP 6

The certification system should provide one recognisable European seal.

WP 5 WP 6 WP 7

to be considered in certification; see chapter 6.2

The CRISP scheme should take national specific requirements into consideration.

WP 5 WP 6

considered in certification; see chapter 6.2

An appropriate implementation and awareness-raising process is necessary for the certification system to be successful.

WP 7

Source: Sveinsdottir, et al., op. cit., 2015. Note: Numbers put in parentheses refer to the respective task of the CRISP WP: *WP 6: ‘Developing a Roadmap’, starting month 21; **WP 7: ‘Enhancing Confidence in the New Certification Measures’, starting month 27.


23

2.4 WP 4: ‘THE S-T-E-FI-APPROACH: ANALYSIS OF CORE DIMENSIONS – SECURITY, TRUST, EFFICIENCY, FREEDOM INFRINGEMENT’

The findings of WP 4 were disseminated in three deliverables: DEL 4.1 ‘Legal Analysis of Existing Schemes’15, DEL 4.2 ‘Ethical Expert Report on Freedom Infringement Evaluation’16 and DEL 4.3 ‘S-T-E-Fi Based SWOT Analysis of Existing Schemes’17.

WP 4 focused on the analysis of existing evaluation and certification schemes and standards to identify evaluation criteria for security PSS based on the four core dimensions of S-T-E-Fi and coming up with requirements for the further development, enhancement, adaptation, and integration of evaluation and certification schemes of products used for security purposes. Furthermore, it was aimed to identify and analyse core issues associated with certification within WP 4. The three deliverables approached the objective of the WP from three different perspectives, namely legal, ethical, and S W O T (which, among others, includes technical and market / efficiency related aspects). They form a comprehensive multi-dimensional analysis of the evaluation and certification schemes of security PSS.

DEL 4.1 identified that the lack of EU legislation for certification schemes is covered by the international and European standards and the guidance from standardisation bodies to a certain extent. With regard to the security legal study, the EU and the Member States share competence in the area of security. The security legislation in the EU is sector-specific and provides useful insight as to requirements for security products and services. Demands relate to physical controls and training of personnel, as well as the performance and the functioning of the security equipment. The multi-layered risks of physical and digital nature urge for accountability, security, and risk assessments. Access to information systems and prevention from illegal interception and interference to the data and systems are also significant requirements incorporated to the EU legislation. In terms of standardisation and certification, trust means that the security PSS, amongst others, respects legislation and fundamental rights, is technically reliable, efficient, transparent, and responds in a predictable and acceptable manner. The role of evaluation and certification schemes in enhancing the needs of the citizens can be crucial if stakeholders are involved, if it is reviewed regularly, operated by an independent body and includes requirements that address the main concerns of the scrutinised in an auditing or evaluation procedure. Looking at efficiency from a legal perspective, the case studies on drones, CCTV and alarm systems revealed significant challenges as into how the security PSS should operate and perform in order to balance investments and achieve the objective of security. Procedural economy, collective redress mechanisms, energy efficiency and adaptability to new technologies with the minimum cost possible are efficiency criteria

15 Kamara, Irene, Paul de Hert, Alessia Tanas, Ioulia Konstantinou, Rosamunde van Brakel, Cristina Pauner,

Jorge Viguri, Artemi Rallo, Rosario García, Florian Fritz, Roger von Laufenberg, Eva Kalan, Jelena Burnik, “Legal Analysis of Existing Schemes”, DEL 4.1 CRISP Project, 30. April 2015.

16 Neyland, Daniel, Irene Kamara, Paul de Hert, “Ethical Expert Report on Freedom Infringement Evaluation”, DEL 4.2 CRISP Project, 30. April 2015.

17 Kamara, Irene, Paul de Hert, Rosamunde van Brakel, Ioulia Konstantinou, Alessia Tanas, Simone Wurster, Tim Pohlmann, Nathalie Hirschman, Leon Hempel, Barbara Bossert, Cristina Pauner, Jorge Viguri, Artemi Rallo, Rosario García, Reinhard Kreissl, Florian Fritz, Roger von Laufenberg, “S-T-E-Fi based SWOT analysis of existing schemes”, DEL 4.3 CRISP Project, 30. June 2015.


24

for security PSS. The freedom and fundamental rights of individuals are potentially at risk from the establishment and / or operation of security PSS in different environments by different entities such as public authorities, private legal persons, or individuals. The impact of drones and CCTV systems in public spaces for crime prevention and detention on the right to privacy and data protection may include extended surveillance, panoptic effect, profiling, excessive collection of personal data, lack of notification to data subjects, lack of possibility to exercise data subject rights, and others. Accordingly, biometric alarm systems pose risks to the rights to data protection and privacy, as well as other freedom and rights. Equal treatment and prohibition of discrimination, bodily integrity, and presumption of innocence, due process and fair trial might be infringed by the security measures, either from the functionalities of the equipment itself or the use / abuse of the security product. Similarly, security service providers might also infringe the above rights when the safeguards for their protection are not respected. The study identifies core criteria for security PSS to respect rights, freedom, and be in line with the protective framework.18 Furthermore, DEL 4.1 examined to what extent the existing evaluation and certification schemes respond to the criteria identified in the legal studies of its previous chapters. Drawing lessons from the analysis of existing schemes, the type of entity operating the scheme and the type of the scheme (certification scheme, certification system, code of practice, technical specification) play a crucial role in determining the quantitative and qualitative integration of the S-T-E-Fi requirements identified in the legal studies. Public authorities tend to prioritise trust and freedom infringement requirements while certification and standardisation bodies, as well as the industry, focus on security and safety requirements (without this being an absolute rule). The impact of the S-T-E-Fi dimensions on the function and performance, user acceptability, legal compliance of the security PSS, and the difficulty to locate a scheme with a comprehensive approach to the core aspects of all four dimensions highlight the importance of the CRISP objective to develop an innovative evaluation methodology that integrates the security, trust, efficiency, and freedom infringement assessment dimensions.

DEL 4.2 highlights that ethical and other concerns have to be considered in the evaluation of freedom infringement in the course of certification schemes for security PSS. The ethical expert stressed that a key task for CRISP would be to develop a set of evaluation criteria relevant for a broad range of security products by overcoming relevant risks such as making the criteria too broad or too narrow. Also, the details and characteristics of the technology under consideration need to be taken into account in the assessment of the security PSS. Supplying a matrix of assessment criteria, encompassing different types of technologies and the related freedom infringement risks, the risk-affected actors, the protective compliance and accountability measures against such risks, as well as redress mechanisms would be a recommended approach for the CRISP methodology. The complexity of freedom infringement, as analysed in DEL 4.1, is advised to be tackled with the assessment procedure and criteria.

18 See Kamara, et al., op. cit., April 2015, p. 129f.


25

DEL 4.3 accomplished a twofold aim: Firstly, it contributed to the identification of core criteria for evaluation and certification schemes of security PSS, based on the four dimensions of the S-T-E-Fi model; secondly, it examined and analysed the strengths and weaknesses, opportunities and threats of existing schemes in order to further use, enhance and develop evaluation and certification schemes for the assessment and certification of PSS used for physical security of people and infrastructures. The SWOT (strengths-weaknesses-opportunities-threats) model and the S-T-E-Fi perspective were adapted to CRISP objectives. In order to further identify requirements based on the S-T-E-Fi model, DEL 4.3 drew lessons on security problems from a showcase and was concentrated in selected potential security solutions such as CCTV, police patrols, community policing and others. Furthermore, an S-T-E-Fi based analysis of CCTV, alarm systems and drones underlined and elaborated the core issues of the three security measures. Issues such as false alarm rates, performance, and interoperability are discussed for the security dimension; reliability, safety, and transparency for the trust dimension; deployment and lifecycle costs for the efficiency dimension and lack of awareness, big data, social sorting, and discrimination for the freedom infringement dimension. The outcomes of the case studies are suggestions for S-T-E-Fi criteria and attributes for evaluating security PSS. The deliverable elaborated on the S-T-E-Fi criteria for evaluation and the enhancement of existing schemes, as developed by the CRISP partners participating in WP 4.

The S-T-E-Fi criteria and their attributes provide a useful tool for existing evaluation and certification schemes.1 The results of the analysis of existing schemes are primarily based on general indicators such as the issuing body, the validity period, the normative references of the schemes and others. The analysis explored the landscape and identified different practices and common denominators (“clustering of schemes”). This part also offered clarity to the results of the second level analysis. Among the key findings of the analysis is the diversity of types of bodies that issue evaluation or certification schemes: public authorities, standardisation bodies, certification bodies, industry and not-for-profit organisation synthesise the landscape of security certification.19 Secondly, the S-T-E-Fi based SWOT analysis of the existing schemes, based on the four dimensions, is presented. Best practices and recommendations for further enhancement, adaptation, and development of the existing schemes in order to include societal, legal, and ethical security aspects are outlined. The table below points out the main shortcomings and opportunities / recommendations for the enhancement of existing evaluation and certification schemes.


19 Kamara, et al., op. cit., June 2015, p. 78.


26

Table 4: WP 4 recommendations and reference to upcoming CRISP work WP 4 recommendations WP relation WP 5 referring Evaluation of security PSS that includes security criteria such as accuracy, performance, risk, robustness, data/system interference and others.

WP 5 WP 6* (3) WP 7**

considered in the context of the evaluation criteria questionnaire regarding Security dimension, see chapter 6.1.1

Evaluation of security PSS than includes trust criteria such as observability, awareness, perception, physiological invasiveness, transparency, reliability, safety & maintenance and others.

WP 5 WP 6 (3) WP 7

considered in the context of the evaluation criteria questionnaire regarding Trust dimension; see chapter 6.1.1

Evaluation of security PSS than includes efficiency criteria such as general efficiency aspects, redress mechanisms, unintended economic effects, use/utilisation, interoperability, lifecycle costs, portability, usability and others.

WP 5 WP 6 (3) WP 7

considered in the context of the evaluation criteria questionnaire regarding Efficiency dimension; see chapter 6.1.1

Evaluation of security PSS than includes freedom infringements criteria such as due process, equal treatment, non-discrimination, protection of vulnerable groups, freedom from unlawful detention, freedom of movement, bodily integrity, privacy, personal data protection, presumption of innocence and others.

WP 5 WP 6 (3) WP 7 (1)

considered in the context of the evaluation criteria questionnaire regarding Freedom infringement dimension; see chapter 6.1.1

Evaluation that takes into account the impact and risks of security PSS on data protection and privacy rights on the including chilling effect from surveillance, profiling, mishandling of personal information and others.

WP 5 WP 6 (3) WP 7 (1)

considered in the context of the evaluation criteria questionnaire Freedom infringement dimension; see chapter 6.1.1

Involvement consumer protection associations in the evaluation of the security PSS

WP 5 WP 6 WP 7 (3)

considered in CRISP’s basis; see chapter 3

Independence of the certification body from the security product manufacturer/ the security service provider

WP 5 WP 6 (1) WP 6 (4) WP 7 (5)

considered in certification

Source: Kamara, et al., op. cit, April 2015; Kamara, et al., op. cit., June 2015; Neyland, et al., op. cit., 2015. Note: Numbers put in parentheses refer to the respective task of the CRISP WP: *WP 6: ‘Developing a Roadmap’, starting month 21; **WP 7: ‘Enhancing Confidence in the New Certification Measures’, starting month 27.


27

3. CRISP’S BASIS: FORMING THE FIRST DRAFT OF A MULTIDIMENSIONAL METHODOLOGY

In order to provide a robust evaluation and certification methodology, (non-exhaustive) societal, economic and legal aspects are included into one assessment approach presented by core issues of Security, Trust, Efficiency and Freedom infringement. This approach has been introduced as the S-T-E-Fi approach.20

As described in chapter 6 of this report in more detail, CRISP’s methodology is composed of five main functions of certification: ‘selection’ and ‘determination’ together with a participatory (inter-subjective), systemic and iterative ‘assessment’ which are covered by the evaluation part of the CRISP methodology, while a third-party ‘review’, ‘decision’ and ‘attestation’ represented in the certification part of the methodology.21

The S-T-E-Fi approach is embodied by the evaluation part of the CRISP methodology, more precisely used as an assessment tool for security PSS. The S-T-E-Fi dimensions are defined as follows:22

x Security: This dimension involves different aspects of security and safety.23 It describes the functionality of a security PSS in countering threats and reducing risks. It also covers questions of whether the security PSS fulfils promises and expectations regarding its performance. Among others, it contains the detection rate and the false alarm rate as well as the impact of intended interference.

x Trust: This dimension encompasses the experience of the users of the technology, such as employees, as well as those scrutinised by the technology, for example passengers at an airport. Beside the experience, the subjective perception defines in which way a security PSS reaches an appropriate acceptance level. Requirements for trust include transparency, openness, fairness and accountability or by using a more practical perspective, habitus (e.g. in the context of usability), emotions and cognition (e.g. the degree of discrimination regarding the use of technology, as well as the potential physiological and psychological invasiveness, for example the impacts body scanning has on users suffering from claustrophobia).

x Efficiency: This dimension implies the economical dimension of the security PSS. Assessment criteria for this perspective are the product life cycle costs, such as the purchasing costs, the implementation costs, the operating costs and dismissal costs, the quality and quantity of training necessary for the use of a security product or questions connected with the infrastructure integration. It also contains derivative criteria like opportunity costs and the impact on business processes.

20 Kamara, et al., op. cit., June 2015. 21 Third party in the context of certification refers to an independent body; see chapter 6.2 of this report. 22 Hempel, et al., op. cit., 2013, p. 748; Kamara, et al., op. cit., June 2015, pp. 15f. 23 Hempel, Leon, Hans Lammerant, Lars Ostermeier, Tobias Schaaf, Christian Geminn, “SIAM Methodology

Handbook”, DEL 12.2 SIAM Project, no date.


28

x Freedom infringement. This dimension depicts the impact of a product on the freedoms and rights of persons. One of the main impacts of security products and services is enhanced personal data collection, processing, sharing and retention. This affects the rights to privacy and data protection. Additionally, security products have a tendency to affect other rights such as bodily integrity, equal treatment and non-discrimination, freedom of movement, freedom of unlawful detention, presumption of innocence, fair trial and due process; all these must be taken into account in the evaluation of security PSS.

Primarily, these four dimensions have a systematisation function as they allow to structure the field of a diverse stakeholder community on a first level by assembling the differences between related aspects or criteria, notions or concepts as they occur in the field.

„These dimensions may provide some systematisation of a socio-technical security regime, but they are no separated boxes. On the contrary, they mutually overlap. Above all it is true that they involve different, often contesting perspectives and activities. However, instead of defining them in very abstract terms, the respective systematisation approach here is different, not distinctive by definition but by resemblance. […] their conceptual indeterminacy shall allow the widest range of involvement possible to discuss assessment criteria and attributes as well as their mutual relationships.”24

Thus, each stakeholder (group) can be allocated exemplarily at least to one of the four dimensions. The allocation of stakeholders presented in Figure 1 derived from findings of the SIAM project and have been inherited for the CRISP approach accordingly.25

Figure 1: CRISP’s core dimensions and exemplary stakeholder classification

Source: @ Hirschmann, September 2015. According to Hempel, et al., op. cit., 2013; SIAM project. 24 Hempel, et al., op. cit., no date, pp. 5f. 25 For references see footnote 4.

employee (internal) and citizen (external), political actor... TRUST FREEDOM iNFRINGE-MENT political actor, NGO‘s, data protection experts...

EFFICIENCY municipality, infrastructure

operators, political actor…

public authority, private security, political actor...

SECURITY


29

The denomination of the presented dimensions is comparative and modifiable. They must be considered as umbrella terms showing a certain semantic proximity or ‘family resemblance’. What characterises them is their mutual delineation of each other.26

As pictured in Figure 2, the following requirements (level structure) apply for all S-T-E-Fi dimension: Each dimension can involve a multitude of criteria and associated attributes concerning organisational, economic, technological, legal, societal, trust- and security-related issues. Associated attributes allow to clearly distinguish the criteria from each other. This means that the S-T-E-Fi dimensions may have equal sounding criteria but they must be defined differently which is made possible by using attributes. Consequently, one criterion can have more than one attribute. For instance, the criterion ‘non-discrimination’ in the Freedom infringement dimension is subdivided into attributes such as ‘categorisation’ or ‘exposure of disabilities’. The four level structure and an overview of a first set of defined criteria including attributes and related questions per S-T-E-Fi dimension have been introduced in CRISP’s DEL 4.3 (part of WP 4).27

Figure 2: Four level structure of the S-T-E-Fi approach

Source: @ Hirschmann, October 2015. Hempel et al., op. cit., 2013; Kamara, et al., op. cit., June 2015.

26 Kamara, et al., op. cit., June 2015, p. 16. 27 See chapter 6.2 in Kamara, et al., op. cit., June 2015.

CRISP’s WP 5: add distinct 1.) simple y/n question 2.) qualitative question

LEVEL 1

Dimensions

LEVEL 2

Criteria

LEVEL 3

Attributes

LEVEL 4

Questions


30

Relevant S-T-E-Fi criteria for security PSS have been collected during CRISP’s WP 4 which derived from literature review, legislation and treaties, standards and technical specifications, research results of other EU funded projects and other sources.28 Some examples of the S-T-E-Fi criteria as stated in CRISP’ DEL 4.3 are ‘accuracy’, ‘performance’, and ‘risk’ for the security dimension, ‘awareness’, ‘observability’, ‘safety’, and ‘transparency’ for the trust dimension, ‘unintended economic effects’, ‘interoperability’, and ‘lifecycle cost’ for the efficiency dimension, and ‘equal treatment’, ‘bodily integrity’, ‘privacy’, and ‘presumption of innocence’ for the freedom infringements dimension.29

During the development phase of the methodology, a refinement of Level 4 (Questions) started and needs to be continued.30 This refinement affects all questions related to attributes of all S-T-E-Fi criteria and means that they need to be distinctive formulated allowing little room for interpretation (see Figure 2).

To achieve an optimal solution by approximation for every conceivable security PSS (being safe and secure, trustworthy, efficient, and not violating rights), it will depend on how the four dimensions are reasonably related to each other during an evaluation, which then needs to be linked to certification appropriately.

The four level structure of the S-T-E-Fi approach has been translated into an evaluation criteria questionnaire which will be introduced in chapter 6.1 of this report as it belongs to the evaluation part of CRISP’s methodology.

28 Kamara, et al., op. cit., June 2015, p. 59. See also appendix 1 in Kamara, et al., op. cit., June 2015. 29 See Kamara, et al., op. cit., June 2015. 30 A first practical application revealed that the formulation of distinctive evaluation criteria questions is a

time-consuming project task.


31

4. VALIDATION ACTIVITIES OF THE FIRST METHODOLOGY DRAFT

The CRISP project follows a tailored strategy and aims at involving stakeholder feedback at different stages in the project in order to foster greater acceptance and usability of the developed methodology. To achieve this objective when it comes to the validation and refinement of the developed but still work in progress CRISP methodology, three main activities have been proceeded apart from internal consortium web meetings and internal presentations and discussions in respective CRISP consortium institutions: inspecting evaluation and certification approaches within the realms of market possibilities, conducting a Validation Workshop and consulting CRISP’s Advisory Board members.

4.1 OUTLINING OF THE MARKET EVALUATION AND CERTIFICATION

Table 5 illustrates an overview of fourteen selected – because accessible – procedures, methods, and codes of practices of evaluation and certification processes, compiled differently via desk research and conducting interviews with experts from certification bodies.31 While desk research analysis focused on a specific standard, certification scheme, seal or code of practice of security PSS, the short expert interviews provided a more general insight in how certification schemes apply for security PSS.32

Table 5: On the market evaluation and certification Model1 ISO/IEC 27001 conformity (information security management) Model2 EuroPriSe - The European Privacy Seal Model3 JIS Q 15001:2006 conformity (Personal Information Protection Management

System - Requirements) Model4 TÜV Rheinland Spain Model5 AENOR Model6 Underwriters Laboratories of Canada (UL) Model7 ISO/IEC 17067 Conformity assessment - Fundamentals of product certification

and guidelines for product certification schemes Model8 ISO/IEC TR 17026 Conformity assessment - Example of a certification scheme

for tangible products Model9 EN-ISO 22301 Societal security - Business continuity management systems -

Requirements Model10 CertAlarm Scheme Rules Part 1-4; in addition: CertAlarm Scheme Rules Parts

2+3 regarding the stages and process descriptions Model11.1 SSAIB Rules 9. (there are many additional, general rules, in particular Rules 1-8) Model11.2 SSAIB Code of practice for Access Control Systems Model12 BSIA code of practice planning, design, installation and operation of CCTV

surveillance systems code of practice and associated guidance Model13 Warwick District Council Control Centre Code of Practice for CCTV Scheme

(BS 7958, BS 7858) Model14 Code of Practice for the Cambridge City Council’s Public CCTV Scheme

Source: Contributions by Burnik, Tomšič, Haponava, Pauner, Wurster and Viguri 2015.

31 Special thank goes to the interviewed experts for their contributions and project support. 32 The explained procedure does not refer to a concrete scheme or standard.


32

Analysing these models did not apply for identifying strengths or weaknesses of one or the other as it is, above all, a small sample size. This sample rather helps to point out aspects of how certification schemes are designed and which requirements might be crucial when it comes to the development and the refinement of the proposed CRISP evaluation and certification methodology in the consequent project work. Furthermore, looking back on existing approaches should avoid running the risk of being too theoretical and causing acceptance problems of the proposed methodology ‘on the market’.

Regardless of the different compilation formats, these fourteen models have been analysed more systematically according to the following aspects.

(1) What subjects/objects are focused at?

(2) Is the certification process pictured (if so, how many stages does the process contain and what does it imply?

(3) Which terms are used (for instance, certification, validation, assessment, evaluation etc.)?

(4) Which stakeholders are addressed (for instance, is there a main stakeholder and are there different roles)?

(5) What resources are used for the certification process (money, power=law, quality or valuation, evidence, reputation, social status etc.)?

(6) Is the time frame for the certification process specified (for instance, how long does the certification process take)?

(7) Any other specifications?

Appendix 1 of this report presents all aspects per model. Only some aspects of the overall analysis are presented in the following section:

The examined models again illustrate the variety of evaluation and certification procedures and processes of security PSS, as already highlighted in the context of CRISP’s WP 2 and WP 433. The terms and the number of formulated steps or processes differ greatly (between two ‘main’ components up to ten ‘detailed’ steps). Nevertheless, it becomes apparent that for evaluation, any kind of documentation or assessment of a products or system shown in Table 2 is preceded by a validated output statement. As presented in chapter 6, the CRISP methodology follows this process of already existing schemes in the security certification by implementing two main components (comparable to those certification processes of Model1 and Model2), using the terms ‘evaluation’ and ‘certification’ in one approach.

Although the certification process of Model2 is based exclusively on data protection legislation, its two-stage certification process is worth mentioning in more detail in reference to the CRISP methodology: In the first stage (evaluation), legal and technical experts,

33 Wurster, et al., op cit., August 2014; Wurster, et al., op. Cit., June 2015; Kamara, et al., op. cit., June 2015.


33

admitted by a certification body, evaluate IT products and technologies, such as hardware or software, according to evaluation criteria and associated open questions specified for intended usage, legal framework and technical environment of the product or technology. They report their findings in an evaluation report. The evaluation criteria, amongst others, include: the overview of fundamental issues, legitimacy of data processing, technical-organisational measures and the data subject’s rights. In the second stage (validation), the certification body checks the evaluation report with respect to completeness, plausibility and comparability with other certifications. After a product passed the validation stage successfully, a certification report is published. Additionally, a short public report summarising the evaluation findings is published. Named main stakeholders involved in an evaluation and certification process of the analysed models are foremost applicants / certificate holder and certification / accreditation bodies. Model2 and Model7 both stress additional actors such as legal and technical experts or end-users. The CRISP approach, however, goes even further by encouraging participation of a diverse stakeholder community as described in chapter 3 of this report. Not all presented models in this chapter specify a timeframe, as this depends heavily on the product, system or service to be certified. But a minimum of two month has been identified (Model4).

4.2 VALIDATION WORKSHOP

The CRISP Validation Workshop has been planned as a one day event and was held on 3rd September 2015 at the Center for Technology and Society of the Technische Universität Berlin (TUB-CTS).34 The workshop aimed to present and discuss the first draft of the developed CRISP evaluation and certification methodology. 37 participants from ten European35 countries attended the event, forming a variety of stakeholder groups, such as end-users from the security industry, standardisation, certification and accreditation bodies, Federal Ministry and public administration as well as researchers and academics who granted valuable feedback on the methodology draft and raised questions and requirements in order to formulate a clearer roadmap for the proposed CRISP certification manual (Validation Workshop agenda see Appendix 2 of this report).36

The workshop was opened by Ronald Boon, CRISP’s project coordinator from the Netherlands Standardization Institute (NEN), who presented the overall objectives and main findings of CRISP’s WP’s 1 to 4. The forenoon of the 3rd consisted of short input presentations by Marco Pagels, Business Development Manager and Laboratory Manager at DIN CERTCO Gesellschaft für Konformitätsbewertung mbH, Daniel Neyland, Professor in Sociology of the Goldsmiths University London and Martin Scheinin, Professor in International Law and Human Rights of the European University Institute in Florence, Italy. These short inputs aimed for reminding stakeholders of their expectations towards evaluation and certification in general and in the context of the CRISP project. After lunchtime, Nathalie 34 The workshop was organised by the German CRISP team of the TUB-CTS, leader of CRISP’s WP 5. 35 From Austria, Belgium, Germany, Great Britain, Italy, the Netherlands, Slovenia, Spain, Sweden and

Switzerland. CRISP consortium members included. 36 The development of a clear roadmap – which will include the vision for the proposed certification scheme –

is a matter of CRISP’s upcoming WP 6.


34

Hirschmann, CRISP partner from TUB-CTS, presented the first draft of the CRISP evaluation and certification methodology followed by a presentation on using existing standards within CRISP methodology by Tatsiana Haponava, CRISP partner from NEN. Thereafter, a mock-up of the evaluation assessment was introduced by Nathalie Hirschmann, which provided a basis for the feedback session afterwards.

Throughout the workshop, external experts and CRISP consortium partners discussed the CRISP methodology and the STEFI dimensions. The following theme complex emerged during the workshop:

Context and use specificity

Participants commented on the potential difficulty in assessing and evaluating security technologies for every potential function and use and also on how the context of use would complicate matters. An example was that for assessing a security camera which has numerous uses already, it also depends upon where it is pointing and what it is capturing. The camera itself may be manufactured according to the strictest principles and privacy by design guidelines but however, as soon as it is used in a specific context (retail or airport), its application can be intrusive and unsafe. As CRISP partner Leon Hempel from TUB-CTS pointed out, standards / certification are perhaps not always context neutral. It is impossible to assess a technology / application for all possible use scenarios. Also, assessing the context of use could be part of the certification process. Furthermore, it was pointed out that standards can be developed and applied contextually. As an example in the field of Explosive Environments, a laptop, for instance, while it might be certified for home use, if it is taken to a different context, it needs to be certified for this context and against a different standard. A possibility would be to assess in accordance with the level of risk – determining the level of risk determines the level of conformity assessment. An idea would be to do this in a similar manner as insurers calculate for risk.

Subjectivity

The issue of subjectivity came up after the CRISP assessment mock-up was demonstrated. Concerns were raised over the evaluation questions and how they would be answered differently, depending on who was carrying out the assessment. Also, questions about the concepts of Security and Trust and how these could have many different meanings depending on context, person, background, and cultural and social differences were raised. It was pointed out that there are already issues of mistrust between certification bodies in different countries; this uncertainty and subjectivity could potentially further add fuel to the debate. Also, customers have to be able to rely on the neutrality, independence and competence of the certification bodies and schemes. In order to guarantee this, certification bodies should go for accreditation, which could serve as a guaranty of the above criteria. Accreditation with respect to the CRISP methodology could help with their legitimisation.

The role of standards for the CRISP methodology

A controversial question was initially posed by CRISP partner Tatsiana Haponava from NEN, regarding the role of standards in the CRISP methodology. Three conditions for existing standards have been addressed:


35

x Require existing standards as part of the certification.

x Existing standards which directly link to the security PSS that can be a ‘plus’ for the certification.

x Leave existing standards out of CRISP’s certification part.

The discussion that followed was by trend in agreement that leaving out existing standards would be like ignoring decades of work and effort of setting standards for security PSS. Also, not including standards would basically mean to leave out the whole technical aspect of security PSS. More details on this discussion can be found in chapter 5 of this report. A question was also raised with respect to the European Commission requirements for standardisation in relation to CRISP’s certification part as the European Commission will only accept European Standards. A balance needs to be found between a model that is possibly too ‘soft’ and / or too ‘subjective’, and the often perceived usual self-assured methods of simplification. For example, aspects of privacy currently do not exist in technical or performance standards. With regard to the standard’s development, the comment that standards convey a level of legitimacy was uttered. Standards are developed by the relevant stakeholder community and become legitimate with use and acceptance from the market. There are different participatory models but it is important to have all relevant stakeholders / representatives in the committee. With regard to covering different security cultures, an idea could be to invite members of national standardisation committees, and try to cover all areas of the economy and society. The ISO and IEC are in the process of opening up to comments from the general public, so perhaps this is something that CRISP could consider, too.

The feedback questionnaires collected at the Validation Workshop were analysed in order to use them for the refinement of the CRISP methodology (see chapter 6; for feedback questionnaire see Appendix 3 of this report). In total, 16 feedback questionnaires could be analysed. Not every question was answered by each of the survey participants.37 The questionnaire was subdivided into four main questions referring to evaluation and certification of the CRISP methodology and the possible link between both parts. The provided feedback was not only used for the refinement of the CRISP methodology. It will also be taken into account for the upcoming CRISP work.

4.3 ADVISORY BOARD MEMBERS

The Advisory Board is a group of important security and certification stakeholders. Currently, there are five experts on the CRISP Advisory Board representing: materials research testing and certification, testing and evaluation of security products, aviation security, and security manufacturers. An Advisory Board with relevant expertise was incorporated right from the project’s beginning with members reflecting a range of expertise and interests – scientific, technical, academic, standards setting, ethical and so forth. The

37 The number of to be filled in feedback questionnaires was N=37. Three participants quoted they cannot give

any recommendations during the event. Furthermore, not all participants attended the workshop until the end of the event. 16 out of 22 distributed feedback questionnaires were returned.


36

Advisory Board supports the project to guarantee both scientific excellence and policy relevance of the project. In this regard, Advisory Board members have been consulted during the conduct of this report’s tasks by keeping them informed about the draft methodology and by inviting them to the Validation Workshop.

From the beginning of WP 5, the Advisory Board members were well informed on the project and its progress. A number of phone meetings were planned to:

x get more information on the background and the work focus of the Advisory Board members in order to be able to use their knowledge and expertise in an efficient way;

x know their opinion on the current situation of the European market in the area of certification of security products, systems and services;

x know their motivation of participation in the project;

x provide the Advisory members with the required details on the project;

x ask for their general feedback on the project and specific recommendations on the steps made and results achieved; and

x receive their input for a number of the planned activities.

In general, all Advisory Board members highlighted the current difficulties regarding the European certification of security products, systems and services. Along with the existing European standards, many European countries use their own country specific standards, documents and marks which are prescribed by law but due to the strong marketing, are very popular. National certification bodies follow their own procedures and labels. This forms a market barrier as manufacturers have to get the exact same product certified in every country they want to market their product anew to make sure their product, systems or services gets the national certificate. This situation leads to an extra effort in time, costs and unsatisfied industries and consumers. Furthermore, the European market is not very free and open to the external organisations: non-European manufacturers have difficulties putting their products on the market because of the national differences.

The Advisory Board members see the project as a good step towards a common European market for certification of security products, systems and services. They want to share their expertise on the current situation and their vision of how it should be. Besides, they want to advise on the practical implementation of the methodology to be proposed by the project. So far, the Advisory Board members see a lot of challenges in the project concerning the holistic and very broad nature of the project, the S-T-E-Fi dimensions which are hard to test in order to get a security product, system or service certified and the practical implementation of the results of the project. Their specific recommendations towards the drafted methodology have been considered for the refinement of the CRISP methodology presented in chapter 6 of this report.38

38 Due to anonymity, a differentiation of feedback from Advisory Board members and other participants cannot

be made.


37

5. USING STANDARDS IN THE CRISP EVALUATION AND CERTIFICATION METHODOLOGY

The majority of the existing standards cover the technical aspects and requirements of security PSS. For the sake of better understanding, in this project a distinction between technical and non-technical requirements for security PSS is made. The CRISP requirements as defined and described in CRISP’s DEL 4.139 and presented in chapter 3 of this report cover the dimensions of Security, Trust, Efficiency, and Freedom infringement.

During the project, the question regarding the way and the extent to which existing standards on technical requirements could be used within the CRISP evaluation and certification methodology has been raised. The following three options have been discussed:

a. Using existing technical standards as a ‘hard’ condition

At the configuration stage of the evaluation part of the methodology a number of existing technical standards have to appear in relation to the security PSS under evaluation. Then, a question regarding whether a security PSS is already certified based on existing standards or on other European normative documents needs to be asked. In case this question is answered with a ‘yes’, then a certificate needs to be provided as evidence. So, the condition for proceeding with the evaluation and certification, based on the CRISP methodology, is a certificate based on existing technical standards for security PSS. Schematically, the description of this process is given in Figure 3.

Advantage

Considering existing technical standards in the CRISP evaluation and certification methodology will lead to an integrated approach between technical requirements and requirements based on the S-T-E-Fi dimensions. In this case, the certification based on existing technical standards forms a basis for the certification according to the S-T-E-Fi dimensions and allows to evaluate a security PSS in its complexity from both technical and non-technical perspectives.

Disadvantage

Using the certification based on existing technical standards as a ‘hard’ condition to continue with the CRISP methodology creates a barrier the shape of being strongly dependent from certification based on the technical standards.

39 Kamara, et al., op. cit., April 2015. See also chapter 2.4 of this report.


38

Figure 3: Schematic representation of using existing standards as a ‘hard’ condition for CRISP methodology

Assess securityPSS based on the

CRISPmethodology

Is a security PSScertified based on a

standard?

Provide evidence/certificate to continue

with CRISPmethodology

Are there any existingstandards on this

security PSS?yes yes

no

Certify securityPSS based on the

CRISPmethodology

Get certificate tocontinue with CRISP

methodology

no

Security PSS tobe evaluated

Configuration

Assessment

Source: @ Haponava, October 2015.

b. Using existing technical standards as a ‘soft’ condition

In this option, the certification of the security PSS based on existing technical standards is seen as a desirable but not obligatory step in the whole CRISP methodology. The question whether a security PSS is already certified based on existing standards or other European normative documents with the provision of a certificate is still valid. If the security PSS is not certified based on existing standards, it is still possible to proceed with the CRISP methodology. In this case a warning will appear, stating that the total evaluation score of the security PSS will be lower because of the absence of a certificate on technical standards. Schematically, the description of this process is given in Figure 4.


39

Figure 4: Schematic representation of using existing standards as a ‘soft’ condition for CRISP methodology

Assess securityPSS based on the

CRISPmethodology

Is a security PSScertified based on a

standard?

Provide evidence/certificate to continue

withCRISPmethodology

Are there any existingstandards on this

security PSS?yes yes

no

Certify securityPSS based on the

CRISPmethodology

Security PSS tobe evaluated

Configuration

Assessment

Warning:low

certificationscore

Are you intended toget certified based on

standards?Get

certificateyes

no

no


Advantage

In this option, existing standards are still taken into account but, in comparison with the first option, on a voluntary basis. A security PSS can be certified based on the S-T-E-Fi dimensions without being obligatorily certified according to existing technical standards. This way, the importance of certification based on the S-T-E-Fi dimensions is equal to certification based on technical standards and is not restricted by the last one.

Disadvantage

In absence of a certificate, there is uncertainty about the quality of a security PSS to be evaluated and certified based on the CRISP methodology.


40

c. Not using existing technical standards at all

The third option is to not consider existing technical standards in the CRISP evaluation and certification methodology at all and only focus on the evaluation criteria related to S-T-E-Fi dimensions.

Advantage

The evaluation criteria, as defined in CRISP’s DEL 4.140 related to the S-T-E-Fi dimensions are emphasised. There is no confusion about the level of importance of certification based on existing technical standards and based on the S-T-E-Fi dimensions.

Disadvantage

There are three main disadvantages of this option:

1. Ignoring certification based on existing standards separates the technical and non-technical part of the same security PSS.

2. Existing standards represent best practices and accumulated expertise and experience. If the CRISP methodology doesn’t take into account the existing standards, it would try to re-invent the work around best practices.

3. This option does not serve as a motivational factor for the European market to be certified based on the CRISP methodology. The European market is familiar with certification based on technical standards and unfamiliar with certification based on the S-T-E-Fi dimensions. This fact might result in a barrier while introducing to the market certification based on the CRISP methodology.

DISCUSSION

These three options were presented during the Validation Workshop of the CRISP methodology in Berlin (see chapter 4.2) and individually discussed with the Advisory Board members.

In general, the Advisory Board members commented that many product standards produced by CENELEC, ETSI or CEN cover specific technical requirements for the corresponding products. The current certifications cover those technical aspects and have no relevance for the CRISP methodology. However, some CEN or CENELEC standards dealing with systems that are interacting with each other or with elements outside of the systems, like remote communications, remote functions or controls performed by third parties, could be considered as candidates under the scope of CRISP. They may contain personal information and require a certain degree of trust and responsibilities from people who are performing or handling these tasks, be it from public or private sources.

40 Kamara, et al., op. cit., April 2015.


41

Specifically related to the discussed three options, the stakeholders and the Advisory Board members have clearly communicated that they would not consider proposed option c. as a possible option. This is a confirmation of the earlier conclusions of this project stating that the CRISP methodology should be based on the existing robust European and international standards (see first recommendation of Table 3, page 22).

However, there was a slight difference in opinions about using existing standards as a ‘hard’ or a ‘soft’ condition:

x The supporters of the ‘hard’ condition stated that certification of security PSS only based on the S-T-E-Fi dimensions will raise uncertainty in a certified PSS without a confirmation of fulfilment of technical requirements.

x On the other hand, the supporters of the ‘soft’ condition highlighted that the inability to proceed with the evaluation and certification based on the CRISP methodology will create great limitations in using the proposed methodology.

Besides, the Advisory Board members highlighted that the CRISP methodology should not burden certification of simple products with no relation to one of the S-T-E-Fi dimensions, with a full evaluation. Therefore, the CRISP methodology should allow skipping the evaluation of any of the dimensions if not applicable. The examples for such simple products are relay contacts, Passive InfraRed detectors, dualtech motion sensors, Carbon Monoxide CO detectors, heat sensors, power supplies and sirens (internal and external).


42

6. EVALUATION AND CERTIFICATION METHODOLOGY

The overall methodology can be seen from Figure 5. CRISP’s approach is composed of:

(1) Evaluation consisting of two main stages: configuration and S-T-E-Fi assessment. (2) Certification as third-party attestation related to products, processes, systems or

persons41 and consisting of two stages: audit42 and attestation43, which consider the results of the evaluation part. The certification part of the CRISP methodology is more described in detail in chapter 6.2 of this report.

Figure 5: CRISP evaluation and certification methodology

Source: @ Haponava & Hirschmann, October 2015. Refined methodology. Note: R1|2|3: outputs from one activity which work as input to another activity.

This two-part – evaluation and certification – methodology emerged during the research work of the CRISP project taking into account existing evaluation and certification processes and expert recommendations of different validation activities (as introduced in chapter 4 of this report).

In order to address significant gaps in the current certification landscape44, the methodology pictured above claims to imply two important aspects which have been hardly or only scarcely considered in previous evaluation and certification activities: First, it includes social criteria for the evaluation and certification of security PSS. And second, it integrates the complexity of different assessment dimensions in one approach by enabling and encouraging an early participation of diverse stakeholders representing both the supply and demand side

41 According to ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles. 42 Audit: systematic, independent, documented process for obtaining records, statements of fact or other

relevant information and assessing them objectively to determine the extent to which specified requirements are fulfilled according to ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles.

43 Attestation: issue of a statement, based on a decision following review, that fulfilment of specified requirements has been demonstrated according to ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles.

44 Such as the inclusion of social / human factors in the evaluation of security PSS stated by the European Commission; see European Commission, A strategic vision for European standards: Moving forward to enhance and accelerate the sustainable growth of the European economy by 2020, Communication from the Commission to the European Parliament, the Council and the European Economic and Social Committee, COM (2011)311 final, Brussels, June 2011, http://eurlex. europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0311:FIN:EN:PDF.

Information provider

Audit Review & Decision

Attestation

Auditor (third party)

R3 Configuration Selection and Determination

Assessment S-T-E-Fi

R1 R2

EVALUATION CERTIFICATION


43

of security PSS. As already emphasised, the S-T-E-Fi approach offers a structure of how to organise stakeholder arguments as well as on how to identify possible interrelations between these different assessment perspectives.45 Including the S-T-E-Fi approach into the CRISP methodology shall ensure that all relevant stakeholder perspectives will be addressed or at least considered during an assessment of a security PSS.

6.1 EVALUATION PROCESS WITHIN CRISP’S METHODOLOGY

As illustrated in Figure 6, the first part of CRISP’s methodology involves two stages which cover the first two functions of certification as defined by ISO/IEC 17067:201346: a configuration stage and an assessment stage. Both stages demand (an early and partially broadly based) stakeholder involvement and need to be reflected by different actor roles. In total, three main actor roles are intended during the evaluation part: ‘project leader’, ‘project participants’ and ‘information provider’. Their intended functions during the configuration and / or assessment stage are described in the following chapters of this report.

Figure 6: Proposed configuration and assessment stages during evaluation

Source: @ Hirschmann, October 2015. Note: R1|2: outputs from one activity which work as inputs to another activity.

45 See Hempel, Leon, Tobias Schaaf, Dagny Vedder, and Lars Ostermeier, Towards a multi-dimensional

technology impact assessment: Security, Trust, Efficiency, and Freedom Infringements. White paper in the context of a paper presentation at the International Conference of the PRESCIENT project – Privacy and Emerging Technologies, Berlin, November 27-28 2012; Hempel, et al., op. cit., 2013.

46 See ISO/IEC 17067:2013 Conformity assessment - Fundamentals of product certification and guidelines for product certification schemes.

‘Information provider’

-Assessment

Configuration

R1 R2

T Fi E

S

‘project participants‘ including ‘project leader‘

1. Create application scenarios

4. Add documents

‘Project leader’ creates scenario

2. Add narrative

3. Add extension

C E R T I F I C A T I O N


44

In the course of evaluation three questioning formats are designated and posed to one or more of the introduced actor roles above:

1. Scenario set up questions during the configuration stage: to be answered by a ‘project leader’ and, if needed, an ‘information provider’.

2. Technology specification questions belonging to the configuration stage: to be answered by a ‘project leader’ and, if needed, an ‘information provider’.

3. Evaluation criteria questionnaire which belongs to the assessment stage: to be answered by a ‘project leader’, the ‘project participants’ and, if needed, ‘information provider’.

6.1.1 EVALUATION CONFIGURATION AND ASSESSMENT

CONFIGURATION STAGE: SELECTION AND DETERMINATION

In the configuration stage, a specific scenario is created (‘selection’) by the ‘project leader’ of a security PSS who has all relevant information about a given security PSS (‘determination’).47 As a scenario can have more than one application, the context of each application of a specific scenario shall be defined and assessed afterwards.

The configuration set up, done foremost by the ‘project leader’, shall include the following aspects:

x Specification of the security application area such as ‘border security’, ‘security of the citizens’, ‘critical infrastructure’, ‘crisis management’48, and ‘IT-security’.

x Specification of the functionalities of a security PSS (‘primitive’; ‘connective’; ‘performative’ functions49) referring to questions as highlighted in CRISP’s DEL 1.1 such as: “What is / are the intended result(s) of the security PSS?”, “How does the security PSS reach this result?”, “What functions are needed for the performance of the security PSS?” and “Can there be any unintended results of a security PSS and what function could be the reason of this?”50

x Specification of the Technology Readiness Level (TRL 0 up to TRL 951). The TRL question shall basically enable evaluation and certification of different development stages of security PSS.

47 ‘Selection’ and ‘determination’ are functions of certification as defined by ISO/IEC 17067:2013, but are

already implemented during the evaluation part of the CRISP methodology; for more information see also chapter 6.2 of this report.

48 Based on the results of WP 1; see summary in chapter 2.1 of this report. 49 Based on the results of WP 1; see summary in chapter 2.1 of this report. 50 Fritz, et al., op. cit., 2014, p. 7. 51 TRL is a method of estimating the maturity level of a particular technology and is based on a scale from 0 to

9, 9 being the most mature technology. TRL 0: unproven concept, no testing has been performed; TRL 1: principles postulated and observed but no experimental proof available; TRL 2: technological concept and application have been formulated; TRL 3: first laboratory tests completed; proof of concept; TRL 4: small scale prototype built in a laboratory environment; TRL 5: large scale prototype tested in intended


45

x Basic information of application scenario(s) which allows to identify scenario boundaries of a security PSS: for example objective(s), content and context, retention period/duration, space needed for running, stakeholders involved, a narrative of the process, scenario extension by indicating best and worst case examples, document supply for explanation (to be accessible by an auditor in the certification part).

x Indication of technology specifications and a technological description of a security PSS.

x Introduction / Invitation of a first set of stakeholders to be involved in the assessment stage after the project set up has been closed. The first set of stakeholder can later be extended.

A ‘project leader’ shall be able to consult an ‘information provider’, of its own company for instance, who is familiar with the security PSS in case configuration questions cannot be answered. The determinations specified in this stage shall help to ensure applicability of specified use cases and evaluation criteria. Applicability is defined by the type, security area, functionality and technology specification of a security PSS.

The configuration stage will be closed with a first output (R1). This first output (summary of basic information) works as input for the assessment stage and shall be reviewed by ‘project participants’, and if needed by the ‘information provider’, once invited by a ‘project leader’.

environment; TRL 6: prototype system tested in intended environment close to expected performance; TRL 7: demonstration system operating in operational environment at pre-commercial scale; TRL 8: first of a kind commercial system (manufacturing issues solved); TRL 9: Full commercial application, technology available for consumers. Schild, Philippe, “Horizon 2020”, no date. http://ec.europa.eu/research/conferences/2013/energy_infoday/pdf/session_3_summary_of_the_calls_open_in_2014_-_philippe_schild.pdf.


46

ASSESSMENT STAGE: REVIEW OF SECURITY PSS BY THE USE OF S-T-E-FI

In the assessment stage, each application scenario of a security PSS shall be evaluated by ‘project participants’ (including the ‘project leader’) on the basis of an evaluation criteria questionnaire. In this stage, the ‘project participants’ shall be able to consult an ‘information provider’ in case evaluation criteria questions cannot be answered again. The evaluation criteria questionnaire not only aims towards acquiring information on a specific use case of a security PSS. It also helps to identify interrelations and uncover potential conflicts within or between S-T-E-Fi criteria – and hence between different stakeholders and social groups52.

For the purpose of illustrating the evaluation criteria questionnaire derived from the four level structure introduced as S-T-E-Fi approach on the highest aggregation level, there are the four dimensions. “Each dimension can involve a multitude of criteria and associated attributes concerning organisational, economic, technological, legal, societal, trust- and security-related issues”, as described on page 29 of this report. In order to facilitate the retrieval of S-T-E-Fi criteria, each associated attribute is introduced by a first question with the simple choice of a ‘yes’ or ‘no’ answer and, if applicable, followed by a qualitative answer type question allowing more detailed information. Both question formats are essential when it comes to the identification of interrelations within and between S-T-E-Fi criteria. Again, the ‘selection’ and ‘determination’ of a security PSS shall ensure that ‘project participants’ will only be confronted with applicable evaluation criteria (mock-up example of evaluation criteria questions see Figure 7).

Figure 7: Mock-up of proposed evaluation criteria questioning

Source: @ Hempel & Hirschmann, September 2015.

52 Hempel, et al., op. cit., 2013, p. 751.


47

As pointed out in the third deliverable of CRISP’s WP 453, from a methodological point of view it is important to maintain “some conceptual flexibility as new notions might become relevant while old ones lose their meaning”.54

Hence, it is intended that the conceptual structure of the evaluation criteria questionnaire can be extended to incorporate supplementary resources (e.g. uploads; verification uploads to be used within the certification part), relevant information (e.g. consulting third parties), and additional tags (e.g. quoting of relevant references or sources). To identify interrelations, S-T-E-Fi criteria will be transmitted into a matrix structure. The matrix structure as shown exemplary in Figure 8 allows a more systematic analysis of interrelations and potential conflicts. Concerning this matter, conflict rules need to be defined for all relations. Below a basic example of a conflict rule is given, charged on the basis of simple yes/no-question answers of the evaluation criteria questionnaire:

Figure 8: Example mockup of a conflict matrix per dimension

S T E Fi

Sens

itivi

ty

Circ

umve

ntio

n

Aut

hent

icat

ion

Obs

erva

bilit

y

Tran

spar

ency

Ease

Test

ing

Thro

ugh-

put

Mai

nten

ance

cos

t

Non

-dis

crim

inat

ion

Tran

spar

ency

S

Sensitivity 1 Circumvention Authentication

T Observability Transparency 1 Ease

E Testing Through-put Maintenance cost

Fi Non-discrimination Personal data

Note: 1 = potential conflict if ‘people are constantly observed by a security PSS’ (criterion observability - Trust) & ‘the security PSS is not clear on what it offers’ (criterion: transperancy - Trust); potential conflict if ‘people are constanlty observed by a security PSS’ (criterion: observability - Trust) & ‘the sensitivity of a security PS(S) is not given so that the maximum of security can be achieved’ (criterion: sensitivity – Security).

53 See Kamara, et al., op. cit., April 2015. 54 Hempel, et al., op. cit., 2013, p. 751. This also refers to the notion, that the naming of the dimensions is

variable.

If Criterion ‘A’ = yes & Criterion ‘B’ = no Æ then potential conflict

If Criterion ‘A’ = yes & Criterion ‘B’ = yes Æ then no conflict


48

To give an example by using the criteria ‘observability’ and ‘transparency’ both of CRISP’s Trust dimension: The questions, if people are or are not constantly observed by a security PSS and if the security PSS is clear or is not clear on what it offer will be asked. Depending on the particular condition of the responses, a conflict may or may not arise.

Once all applicable evaluation criteria have been answered by all introduced stakeholders (‘project leader’, ‘project participants’) or consulted third parties (‘information provider’), the assessment stage will be initially closed with a second output (R2). As confidentiality of information was highlighted as a crucial aspect in the context of security PSS during the Validation Workshop, two output versions of R2 reporting reduced and extended assessment findings are intended depending on the stakeholders function during evaluation (see Table 6 for more detail).

1. R2 Partial Evaluation Report primarily provided to ‘project participants’ and if applicable to ‘information provider’.

2. R2 Overall Evaluation Report primarily provided to a ‘project leader’ and an auditor for certification.

The evaluation outputs R1 and R2 Overall Evaluation Report will serve as base for a third-party ‘review’, ‘decision’ and ‘attestation’ (certification), but do not lead to certification itself.55 Table 6: Possible assessment findings reported in R2

R2 Partial Evaluation Report R2 Overall Evaluation Report x configuration stage output R1

x personal contributions per actor role during the assessment stage

x if applicable, assessment analysis of potential conflicts within the same S-T-E-Fi criteria.

x configuration stage output R1 plus further information such as uploaded (confidential) information, evidences, stakeholder participation

x listing of potential conflicts within and between S-T-E-Fi criteria

x determination of a ‘reflexivity score’ (how many applicable evaluation criteria have been answered)

x determination of a ‘conflict score’ (how many potential conflicts within or between S-T-E-Fi criteria have been uncovered)

x possible proposals for solutions and / or recommendations for improvement actions.

55 Nor shall it be understood as an validation report as, for example, provided by ‘The European Privacy Seal’

(see chapter 4.1 of this report: Model2).


49

6.1.2 ACTOR ROLES AND FUNCTIONS

Three main actor roles are intended within the scope of the evaluation part of the CRISP methodology:

1. ‘Project leader’: acts as a coordinator within the whole evaluation process providing all relevant information regarding a security PSS. The ‘project leader’ must be capable of giving a review of the security PSS objectives as well as general and specific information. This could be, a manufacturer or supplier of a security PSS – eventually a purchaser in case of new security PSS.56

2. ‘Project participants’: are involved in the assessment stage one invited by the ‘project leader’. There are different stakeholders groups such as presented in Table 7 (no exhaustive list). The group of ‘project participants’ can, for instance, consist of manufacturers or suppliers, (end-)users or purchasers57, or other (professional) parties interested or affected by a security PSS such as NGO’s, data protection experts, politicians.

3. ‘Information provider’: in case configuration questions and assessment criteria questions cannot be answered exclusively and / or adequately, a ‘project leader’ / ‘project participant’ shall be able to consult third persons’ both for the configuration and assessment stage (see actor role functions in Table 8).58

Table 7 illustrates a first proposal about which S-T-E-Fi evaluation criteria shall be answered by which ‘project participant’. The testing of the evaluation part in the upcoming project work will show the applicability of this determination.

Table 7: Stakeholder groups and intended allocation to S-T-E-Fi dimensions Stakeholder group S T E Fi security manager employee/facility personnel facility managers privacy advocate police politician data protection expert manufacturer service provider supplier local authority end-user emergency organisations end-user

56 According to ISO/IEC 17007:2009: described as ‘first party’. 57 According to ISO/IEC 17007:2009: described as ‘second party’. 58 Consultancy in case more information is needed in order to answer a criteria-attribute question correctly.

The calculation of criteria interrelation will be postponed until the question is answered by the one project participant who consulted an information provider.


50

Stakeholder group S T E Fi transport operator end-user law enforcement authority end-user retail organisation end-user health organisation end-user educational organisation end-user NGO academics (depending on the academic discipline) individuals

Note: user groups in dependence on CRISP’s WP 3 results (Sveinsdottir, et al., op. cit., 2015, pp. 19-34). No exhaustive list; allocation to S-T-E-Fi will be refined after the first set of scenario based workshops of CRISP’s WP 5; see chapter 7.

It is recommended that the introduced actor roles above perform different functions during the configuration and assessment stage as they reflect certain interests and responsibilities. Table 8 illustrates the actor roles and their intended functions.

Table 8: Intended actor roles and functions during evaluation actor roles function involved in ‘project leader’ (coordinator)

x set up a new (application) scenario (including general information, technology specifications

x specify and invite other stakeholders relevant for the assessment stage

x answer configuration questions and evaluation criteria questionnaire

x provide evidence (such as standards or other forms of certifications) when requested

x delegate questions to an ‘information provider’ (for verification)

x access to evaluation outputs: R1 and R2 (both versions)

configuration &

assessment

‘project participants’ x access to application scenarios (once invited by the ‘project leader’)

x answer the evaluation criteria questionnaire x consult ‘information provider’ for evaluation

criteria questionnaire x access to evaluation outputs: R1 and R2

(primarily Partial Evaluation Report)

assessment

‘information provider’ x answer only the configuration questions which have been delegated by the ‘project leader’

x if applicable, access to evaluation output: R1

configuration


51

actor roles function involved in x answer only those questions of the evaluation

criteria questionnaire which have been delegated by the ‘project leader’ and / or ‘project participants’

x if applicable, access to evaluation outputs: R1 and R2 (primarily Partial Evaluation Report)

assessment

Source: Based on Grau, Ronald R., “Models and Tools for the Computational Support of Technology Impact Assessments, Applied in the Context of Mass Transportation”, in Serge Gutwirth, Ronald Leenes and Paul de Hert (eds.), Reforming European Data Protection Law, Springer, p. 98.

6.1.3 RESPONSE TO PARTICULAR VALIDATION WORKSHOP FEEDBACK

To finish the presentation of the evaluation part of the CRISP methodology, some feedback questions of Validation Workshop participants highlighted and attributed to the ongoing fine tuning of the evaluation process will be featured below.

Validation Workshop feedback: “How many analyses are needed?”

The number of iteration of the assessment process is not determined yet but must be addressed in the testing phase of the upcoming project work. It might possibly depend on the ‘conflict score’ level if another assessment is recommended.

Validation Workshop feedback: “Who and how resolves the conflicts? Can all [of them] be resolved?”

For an optimised security solution it should be important in particular for a ‘project leader’ and / or those stakeholders involved in security PSS development to uncover all (potential) conflicts, and where appropriate, solve them accordingly. For instance, there might be a conflict between the criterion ‘observability’ and the criterion ‘transparency’, but legal conditions may lower the conflict accordingly. Hence, to avoid exaggerations or systematic responses when only referring to simple yes/no-questions during the S-T-E-Fi assessment, all uncovered conflicts need to be surveyed by taking into account relevant qualitative questions. This science-based assessment analysis is an ongoing task which needs to be refined in the course of the CRISP project.

Validation Workshop feedback: “How long does an assessment take? Is the assessor able to answer all questions?”

The assessment duration cannot be specified at this state but will likely depend on the type of the assessment procedure (workshop format or the use of software), the number of application scenarios, the applicable questions of the evaluation criteria questionnaire, and


52

the number of participants taking part during the assessment stage.59 As mentioned before, in case a ‘project participant’ is not able to answer (all) applicable evaluation criteria questions it is proposed to consult third parties (‘information provider’, see chapter 6.1.2 of this report) which will extend the assessment duration accordingly. In response to feedback statements by Validation Workshop attendees, it could be optional to perform only the evaluation part of the CRISP methodology or both evaluation plus certification. In either case, evaluation is seen as a condition for certification in the context of the CRISP project. In regard to the last-mentioned option, evidence for a third-party attestation will be needed. Providing evidence of any kind (for example, detailed information, already existing certificates etc.) by involved stakeholders already during evaluation again links both parts of the CRISP methodology in order to ensure that standards’ requirements will be met when it comes to the certification of a security PSS.

59 As outlined in chapter 4 of this report, already existing evaluation and certification procedures often do not

have formal constraints to the length of an evaluation and certification procedure (e.g. Model3 see Appendix 1).


53

6.2 AUDIT AND ATTESTATION AS A CERTIFICATION PART OF THE CRISP METHODOLOGY

6.2.1 CONNECTING STANDARD FUNCTIONS OF THE CERTIFICATION PROCESS TO CRISP EVALUATION AND CERTIFICATION METHODOLOGY

Certification is one of the subject fields of conformity assessment and is defined by ISO/IEC 17000:2004 as a “third-party attestation related to product, processes, systems or persons”, where “a service is covered by the definition of a product”.60 In order to provide attestation, which is an “issue of a statement, based on a decision following review, that fulfilment of specified requirements has been demonstrated”61, an independent process of verification should take place to determine the extent to which specified requirements are fulfilled. Given the non-technical nature of the S-T-E-Fi dimensions, this verification process is covered by an audit, defined as a “systematic, independent, documented process for obtaining records, statements of fact or other relevant information and assessing them objectively to determine the extent to which specified requirements are fulfilled”62.

In chapter 4.1 of this report, the main conclusion on the analysed evaluation and certification models was that their certification procedures are different. Moreover, in CRISP’s DEL 4.3, the SWOT S-T-E-Fi based analysis of existing schemes showed that not only certification bodies develop certification schemes for security products, systems and services, but that public authorities, standardisation bodies, the industry and other stakeholders63 do, too. On the other hand, the basic functions of the certification appear across the analysed models. These functions are selection, determination, review, decision, attestation and surveillance64.

In the proposed CRISP evaluation and certification methodology selection and determination functions are covered by the evaluation part of the CRISP methodology, while the review and decision functions fall within the audit phase in the certification part of the CRISP methodology, as shown in the Figure 5, page 42.

During the audit, the collected evidence (R1) and the assessment results from the evaluation part of the CRISP methodology (R2) are to be examined and reviewed in order to confirm that security PSS fulfil the specified requirements. In case of acceptable non-conformities discovered during the audit, an organisation such as a manufacturer or any other organisation which the PSS is to be attested by has an opportunity to eliminate the non-conformities within an agreed period of time in order to further proceed with the certification process.

The decision is to be made by the auditor, based on the fulfilment of the specified requirements demonstrated and is to be confirmed during the attestation by issuing a statement of conformity.

60 See Note 2, definition of conformity assessment in ISO/IEC 17000:2004. 61 See, ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles. 62 See, ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles. 63 See Kamara, et al., op. cit., June 2015, p. 78. 64 See ISO/IEC 17067:2013 Conformity assessment - Fundamentals of product certification and guidelines for

product certification schemes.


54

Surveillance as a function is not always performed within the certification process. This is the reason why surveillance is not included as a standard function within the CRISP evaluation and certification methodology.

6.2.2 REQUIREMENTS BASED ON EVALUATION CRITERIA AS AN INPUT FOR THE CERTIFICATION PART OF THE CRISP METHODOLOGY

The requirements for products, systems, services or persons are specified in a standard or other normative documents. In WP 4, a number of relevant criteria have been identified to evaluate the security PSS related to the S-T-E-Fi dimensions. Based on these criteria, the requirements for the security PSS will be formulated later in the project, and will be used for drafting a standard and/or other normative document(s).

ISO/IEC 17007:2009 describes the main principles to be followed while developing a normative document with the specified requirements. These principles are65:

PRINCIPLE 1 Separation of specified requirements for the object of conformity assessment from specified requirements related to conformity assessment activities

This principle highlights the necessity to separate specified requirements for objects of conformity assessment, such as product/service, system or person, and their characteristics from the requirements related to conformity assessment activities, such as certification except sampling and testing methods related to the specified characteristics. The requirements related to certification should be laid down in a separate document: a certification scheme.

PRINCIPLE 2 Neutrality towards parties performing conformity assessment activities

Normative documents for objects of conformity assessment should be written so that the conformity of the objects to the specifications can be assessed by a manufacturer or supplier of the object (first party); a user or purchaser of the object (second party); an independent body (third party).

PRINCIPLE 3 Functional approach to conformity assessment

Normative documents that specify conformity assessment activities should consider the “functional approach to conformity assessment”, consisting of the functions of ‘selection’, ‘determination’, ‘review’ and ‘attestation’; and ‘surveillance’ (if needed).

65 See ISO/IEC 17007:2009 Conformity assessment - Guidance for drafting normative documents suitable for

use for conformity assessment.


55

PRINCIPLE 4 Comparability of conformity assessment results

The requirements for the objects of conformity assessment and the requirements for the conformity assessment activities should be specified in a clear and unambiguous manner, with sufficient detail to ensure that the conformity assessment results will be comparable and reproducible.

PRINCIPLE 5 Good practice in conformity assessment

In normative documents for conformity assessment activities, international standards and guides should be considered as a source of good practice in conformity assessment.

While describing specified requirements based on the evaluation criteria within the CRISP evaluation and certification methodology, a special attention should be paid to the following aspects:

x the focus should only be on the criteria or performance characteristics, rather than the design or descriptive characteristics of the object;

x the test methods may be specified to determine whether the criteria or characteristics have been met;

x specified requirements should be generated in terms of results or outcomes, together with values and tolerances, where applicable;

x specified requirements should be related to the object, and not to the production process for the object;

x specified requirements should be divided into consistent and easily identifiable sections, in order to permit their incorporation by reference in codes, regulations and other standards. This structure permits selected clauses to be identified separately in a code or regulation when only part of the normative document is referenced.

More details and other requirements for normative documents can be found in ISO/IEC 17007:2009. As mentioned earlier in this chapter, the requirements based on the S-T-E-Fi evaluation criteria will be specified in a standard and/or other normative document. This normative document will be used to carry out the certification of security PSS.


56

6.2.3 CERTIFICATION: WHAT IS NEEDED?

In section 6.2.1 and 6.2.2, the main functions of the certification process have been discussed and the importance of standards as the important input for the certification has been highlighted. But what is actually the difference between standards and certification schemes and what should a certification scheme contain? The answers to these questions are given in the sections below.

6.2.3.1 DIFFERENCE BETWEEN STANDARDS AND CERTIFICATION SCHEMES

While a standard “provides requirements, specifications, guidelines, or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose”66, a certification scheme describes “rules, procedures and management for carrying out certification related to specified products, to which the same specified requirements, specific rules and procedures apply”67.

The schematic difference between a standard and a certification scheme is given below:

Figure 9: Schematic representation of the difference between standards and certification schemes


66 ISO, “Standards: What is a standard?”, no date. http://www.iso.org/iso/home/standards.htm. 67 See ISO/IEC 17065:2012 Conformity assessment - Requirements for bodies certifying products, processes

and services

Normative document (standard): Certification scheme:

Requirements for: 9 Product/service 9 System 9 Person

Requirements for: 9 test/assessment method 9 evaluating results and

decision making 9 monitoring regime 9 certification/inspection

body 9 competence

auditors/assessors


57

6.2.3.2 CERTIFICATION SCHEME

The main requirements to consider while developing a certification scheme is briefly summarised in the table below, based on the information from ISO/IEC 17067:2013.

Table 9: The main requirements regarding the content of a certification scheme adopted from ISO/IEC 17067:2013

Chapter Element Description DEVELOPMENT OF CERTIFICATION SCHEMES: GENERAL

CONSIDERATIONS 6.4.1

Purpose Can be used for different purposes, the purpose, however, should be stated clearly.

6.4.3

Scheme owner

Should have a clear understanding of the objectives of the scheme and the assumptions that underlie the need and the acceptance of the scheme.

6.4.5 should ensure that:

x the information about the scheme is made publicly available to ensure transparency, understanding and acceptance

x the scheme is regularly reviewed, including the confirmation that it is fulfilling its objectives, in accordance with a process that includes stakeholders

6.4.4

Fundamental scheme principles

should be agreed among the stakeholders and may include:

x confirmation of the ownership,

x confirmation of the governance and decision making mechanisms that may or may not provide direct involvement of stakeholders,

x confirmation of the underlying business and funding model, and

x providing an outline for monitoring and periodic review of the scheme

CONTENT OF A SCHEME 6.5.1

Elements of a scheme

x the scope, including the type of products covered

x the requirements against which the products are evaluated by a reference to standards or other normative documents

x the selection of the activities appropriate to the purpose and the scope of the scheme

x other (specific) requirements to be met by the client

x the requirements for certification bodies and other conformity assessment bodies involved in the


58

certification process

x whether conformity assessment bodies involved in the scheme

x are to be accredited, participate in peer assessment or qualified in another manner

x the methods and procedures to be used by the conformity assessment bodies and other organisations involved in the certification process

x the information to be supplied to the certification body by an applicant for certification

x the content of the statement of conformity (e.g. certificate) which unambiguously identifies the product to which it applies

x the conditions under which the client may use the statement of conformity or marks of conformity

x where marks of conformity may be used, the ownership, use and control of the marks; the requirements of ISO/IEC 17030 should be applied

x the resources required for the operation of the scheme, including impartiality and competence of the personnel, the evaluation resources, and the use of subcontractors

x how non-conformities with the certification requirements are to be dealt with and resolved

x surveillance procedures, where surveillance is part of the scheme

x the criteria for access of conformity assessment bodies to the scheme and for the access of clients to the scheme

x content, conditions and responsibility for publication of the directory of certified products by the certification body or the scheme owner

x the need for, and content of, contracts, e.g. between scheme owner and certification body, scheme owner and clients etc.

x general conditions for granting, maintaining, continuing, and extending the scope of, reducing the scope of, suspending and withdrawing certification: this includes requirements for discontinuation of advertising and return of certification documents and any other action if the certification is suspended, withdrawn or terminated

x the way in which the clients’ complaints records are to be verified if such verification is part of the scheme


59

x the way in which the clients make reference to the scheme in their publicity material

x retention of records by scheme owner and certification bodies

6.5.2 Sampling The scheme should define when sampling is required and who is permitted to undertake it.

6.5.3 Acceptance of conformity assessment results

The scheme should define whether and under what condition conformity assessment results can be considered in the certification process.

6.5.4 Outsourcing of the conformity assessment activities

If applicable, the scheme should require outsourcing to meet the applicable requirements of the relevant international standards. The scheme should state the degree to which prior agreement to outsourcing needs to be obtained from the scheme owner or the client whose products are being certified under the scheme.

6.5.5 Complaints and appeals to the scheme owner

The scheme owner should define the complaints and appeals process and who is responsible for undertaking this process. Appeals against the decision of the certification body and complaints about the certification body should be addressed to the certification body in the first instance.

6.5.6 Licensing and control of the mark

Where the scheme provides for the use of certificates, marks or other statements of conformity, there should be a license or other form of enforceable agreement to control such use.

6.5.7 Surveillance If surveillance is included, the scheme should define the set of activities that make up the surveillance functions.

6.5.8 Non-conforming products

The scheme should define requirements that apply when a product no longer fulfils certification requirements, such as product recall or providing information to the market.

6.5.9 Reporting to the scheme owner

When reporting to the scheme owner is required, the content and frequency of reporting should be defined.

6.5.10 Subcontracting of the operation of the scheme

If the scheme owner subcontracts all or parts of the operation of the scheme to another party, it should have a legally binding contract defining the duties and responsibilities of both parties.

6.5.11 Marketing

The scheme should define the policies and procedures related to marketing, including the extent to which certification bodies and clients can make references to the scheme.

6.5.12 Fraudulent claim of

Actions and responsibilities for situations where certification under the scheme is being claimed


60

certification fraudulently should be described.

SCHEME DOCUMENTATION 6.7 The scheme owner should create, control and maintain an adequate

documentation for the operation, maintenance and improvement of the scheme. The documentation should specify the rules and the operating procedures of the scheme and in particular the responsibilities for the governance of the scheme.

Source: ISO/IEC 17067:2013 Conformity assessment — Fundamentals of product certification and guidelines for product certification schemes.


61

7. SUMMARY: TOWARDS CERTIFICATION WITHIN THE CRISP EVALUATION AND CERTIFICATION METHODOLOGY

As shown in WP 2 different country size and protection needs, the different size of the national markets for security solutions and the security industries, different structures of this industry and its market segments, and different needs regarding security certification as well make it necessary to base an evaluation of security PSS on a corporate approach.

In conclusion, this WP 5 report proposes a two-part evaluation and certification methodology for assessing security PSS. This two-part methodology emerged during the research work of the project, taking into account existing evaluation and certification processes and procedures and expert recommendations of different validation activities. The evaluation part of the CRISP methodology is participatory, systematic, and iterative by nature enabling the determination, selection and assessment of security PSS according to the four S-T-E-Fi dimensions. It is participatory due to the encouragement of an (early) involvement of different stakeholders. It is systemic as a variety of differently dimensional criteria will be brought into a matrix structure. And it is iterative as the evaluation process is repeated until each potential conflict uncovered is addressed to relevant / involved stakeholders and, where appropriate solved.

Regardless of the issue that there might be security PSS that do not need to go through all proposed S-T-E-Fi dimensions, the overall benefit of the participatory approach is about increasing the level of inter-subjectivity, releasing stakeholders “from their usual self-assured methods of simplification and [confronting] them with the complexity and the effects of their [previous] decisions outside their professional or common view”68. This fact might seem to conflict with standardisations’ logic, which seeks to reduce complexity. Consequently, the challenge is to test how the theoretical complex appearing part of the CRISP approach fits in with the complex world of certification and standardisation.

Finally, some main recommendations / requirements are outlined below which derived from this WP 5 research work to be considered in the upcoming research work of the CRISP project.

x At the earliest developmental stage, security PSS should be confronted with S-T-E-Fi criteria. ‘S-T-E-Fi by design’.

x Proposing security PSS evaluation based on S-T-E-Fi as requirement for certification.

x Proposing a diverse stakeholder involvement at an early development stage of a security PSS.

x Clearly identifying those security PSS that can be evaluated and certified according to the CRISP methodology

x Defining the role of an auditor more precisely: what is feasible, recruitment of an auditor, one auditor or one auditor per dimension.

68 Hempel, et al., op. cit., 2013, p. 751.


62

So far, this report has a detailed focus on the development and the validation of the evaluation part of the CRISP methodology as it works as a base for the proposed CRISP methodology. With regard to the ongoing development of the complete CRISP methodology, the testing of the developed evaluation part in form of scenario based workshops and ongoing input by experts will help the fine-tuning of the methodology. Beyond that, consulting the experts will help in further developing the questions by adding more relevant ones and subtracting the ones that are too general or not relevant.

Until January 2016, a series of scenario based workshops will be conducted, including multi-functional stakeholders. Within each workshop, one specific scenario – such as on ‘drones’, ‘alarm systems’, ‘border control’ and ‘security of the citizens’ – will be presented to illustrate how the chosen security PSS will perform within the proposed methodology. They are envisioned as different workshops, reflecting varying needs of all user groups involved in the application of a security PSS in practice. The planned scenario based workshops allow an active exchange between practitioners and researchers and shall not only contribute to the validation of the CRISP methodology but also ensure implementable research results to be included in a second WP 5 report.

Thus, the scenario based workshops and its outcome aims are as follows:

x To systematically review which user group responds to which interrelation of S-T-E-Fi criteria.

x To test the evaluation part with the engagement and practical feedback from the participants for possible modifications and revisions of the evaluation part in relation to certification.

x To test if users / stakeholders can be guided properly through the process.

x To test how the evaluation outcome (scoring) can be implemented and serve as base for certification.

x To refine the evaluation reports (R1|2) on the basis of the scenario based workshops.

x To obtain comparative values of how long an assessment will take in reference to the amount of the assessed criteria.

x To discuss the way the evaluation criteria related to S-T-E-Fi dimensions are to be translated into the requirements for a normative document. Based on this discussion the guidelines will be drafted to be used later on in the CRISP project.

x To discuss and to draft recommendations on the content of a certification scheme for security PSS based on the S-T-E-Fi dimensions.


63

In the course of the empirical (field) work, some recommendations and practical advises emerged: ‘S-T-E-Fi by design’: In order to avoid acceptance problems of a security PSS, it is recommended to consider all four S-T-E-Fi dimensions in an early development stage. The same applies to the involvement of diverse stakeholders concerned with a security PSS (see different actor roles, chapter 6.1.2). This goes beyond current design implementations and can be crucial when it comes to certification of even single security PSS components. It is expected that the sooner S-T-E-Fi criteria are considered in the development of a security PSS, the safer, more secure, trustworthy, efficient, and legitimate they will be perceived.

Innovative aspects of the presented methodology are: the evaluation part based on S-T-E-Fi which is designed to allow appropriate improvements / measures of a security PSS at different development stages which can act as incentives for certification as it allows a product, system or service optimisation in the first place. Consequently, a ‘project leader’ (e.g. manufacturer or vendor) should be motivated to start the S-T-E-Fi assessment not only when it comes to attestation. Especially when it comes to a newly developed / designed security measure, an evaluation of a security PSS based on S-T-E-Fi might help to reduce costs in the long run.


64

LIST OF REFERENCES

ECORYS, Security Regulation, Conformity Assessment & Certification. Final Report – Volume 1: Main Report, Brussels, 2011. http://ec.europa.eu/dgs/home-affairs/e-library/documents/policies/security/pdf/secerca_final _report_volume__1_main_report_en.pdf.

European Commission, A strategic vision for European standards: Moving forward to enhance and accelerate the sustainable growth of the European economy by 2020, Communication from the Commission to the European Parliament, the Council and the European Economic and Social Committee, COM (2011)311 final, Brussels, June 2011, http://eurlex. europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0311:FIN:EN:PDF.

Fritz, Florian, Reinhard Kreissl, Roger von Laufenberg, Paul de Hert, Alessia Tanas, Rosamunde van Brakel, Simone Wurster, “Glossary of Security Products and Systems”, DEL 1.1 CRISP Project, 31 July 2014.

Grau, Ronald R., “Models and Tools for the Computational Support of Technology Impact Assessments, Applied in the Context of Mass Transportation”, in Serge Gutwirth, Ronald Leenes and Paul de Hert (eds.), Reforming European Data Protection Law, Springer, pp. 91-123.

Hempel, Leon, Hans Lammerant, Lars Ostermeier, Tobias Schaaf, Christian Geminn, “SIAM Methodology Handbook”, DEL 12.2 SIAM Project, no date.

Hempel, Leon, Lars Ostermeier, Tobias Schaaf and Dagny Vedder, “Towards a social impact assessment of security technologies: A bottom-up approach”, Science and Public: Policy, Vol. 40, 2013, pp. 740-754.

Hempel, Leon, Tobias Schaaf, Dagny Vedder, and Lars Ostermeier, Towards a multi-dimensional technology impact assessment: Security, Trust, Efficiency, and Freedom Infringements. White paper in the context of a paper presentation at the International Conference of the PRESCIENT project – Privacy and Emerging Technologies, Berlin, 27-28 November 2012.

ISO, “Standards: What is a standard?”, no date. http://www.iso.org/iso/home/standards.htm.

ISO/IEC 17067:2013 Conformity assessment — Fundamentals of product certification and guidelines for product certification schemes, 2013.

ISO/IEC 17007:2009 Conformity assessment — Guidance for drafting normative documents suitable for use for conformity assessment, 2009.

ISO/IEC 17000:2004 Conformity assessment - Vocabulary and general principles, 2004.

Kamara, Irene, Paul de Hert, Alessia Tanas, Ioulia Konstantinou, Rosamunde van Brakel, Cristina Pauner, Jorge Viguri, Artemi Rallo, Rosario García, Florian Fritz, Roger von Laufenberg, Eva Kalan, Jelena Burnik, “Legal Analysis of Existing Schemes”, DEL 4.1 CRISP Project, 30 April 2015.


65

Kamara, Irene, Paul de Hert, Rosamunde van Brakel, Ioulia Konstantinou, Alessia Tanas, Simone Wurster, Tim Pohlmann, Nathalie Hirschman, Leon Hempel, Barbara Bossert, Cristina Pauner, Jorge Viguri, Artemi Rallo, Rosario García, Reinhard Kreissl, Florian Fritz, Roger von Laufenberg, “S-T-E-Fi based SWOT analysis of existing schemes”, DEL 4.3 CRISP Project, 30 June 2015.

Neyland, Daniel, Irene Kamara, Paul de Hert, “Ethical Expert Report on Freedom Infringement Evaluation”, DEL 4.2 CRISP Project, 30 April 2015.

Reigel, Markus, ‘Ansätze zur Angleichung der unterschiedlichen Konformitätsbewertungsansätze‘; presentation at the DKE-Workshop Konformitätsbewertungsbedarf Informationssicherheit, 7. October 2015, Frankfurt am Main, Germany.

Schild, Philippe, “Horizon 2020”, no date. http://ec.europa.eu/research/conferences/2013/energy_infoday/pdf/session_3_summary_of_the_calls_open_in_2014_-_philippe_schild.pdf.

Schneier, Bruce, “Computer Security: Will We Ever Learn?”, 15 May 2000, https://www.schneier.com/crypto-gram/archives/2000/0515.html.

Sveinsdottir, Thordis, Rachel Finn, Rowena Rodrigues, Kush Wadhwa, Florian Fritz, Reinhard Kreissl, Roger von Laufenberg, Paul de Hert, Alessia Tanas, Rosamunde van Brakel, “Taxonomy of Security Products, Systems and Services”, DEL 1.2 CRISP Project, 31 July 2014.

Sveinsdottir, Thordis, Rachel Finn, Kush Wadhwa, Rowena Rodrigues, Jolien van Zetten, Simone Wurster, Patrick Murphy, Nathalie Hirschmann, Artemi Rallo, Rosario García, Cristina Pauner, Jorge Viguri, Eva Kalan, Igor Kolar, “Stakeholder Analysis Report”, DEL 3.1 CRISP Project, 28. February 2015.

Wurster, Simone, Tim Pohlmann, Patrick Murphy, Florian Fritz, Roger von Laufenberg, Jolien van Zetten, Cristina Pauner, Artemi Rallo, Rosario García Mahamut, Rosamunde van Brakel, Alessia Tanas, “Report on security standards and certification in Europe – A historical/evolutionary perspective”, DEL 2.1 CRISP Project, 30 August 2014.

Wurster, Simone, Tim Pohlmann, Nathalie Hirschmann, Patrick Murphy, Jolien van Zetten, Ying Ying Lau, Tatsiana Haponava, Thordis Sveinsdottir, Rachel Finn, Rowena Rodrigues, Kush Wadhwa, Reinhard Kreissl, Florian Fritz, Roger von Laufenberg, Cristina Pauner, Artemi Rallo, Rosario García Mahamut , Jorge Viguri, Irene Kamara, Paul de Hert, Eva Kalan, Jelena Burnik, Igor Kolar, “Consolidated report on security standards, certification and accreditation – best practice and lessons learnt”, DEL 2.2 CRISP Project, 30 June 2015.


66

APPENDIX

APPENDIX 1: ON THE MARKET EVALUATION AND CERTIFICATION MODELS

Model1 ISO/IEC 27001 conformity (information security management) Aim of the certification

Proof to the public in general and business partners in particular that the certified organisations meets the requirements of the international information security management standard ISO/IEC 27001. Certified organisation is able to identify and mitigate information security risks to desired levels, improve trust into its services and manage information security processes.

Subjects/Objects Certification methodology focuses on information security processes to develop an information security management system (ISMS). Information security processes include information assets, procedures, policies and human resources. The object of certification can be a single business process (e.g. HR), a particular service or the whole business process of an organisation that is being certified.

Number & naming of process stages

Certification process is pictured. The procedure consists of two main stages:

(1) Documentation audit

(2) Certification audit, conducted in two parts: (2a) documentation audit and (2b) on-site audit.

Process: What does the process description at which stage imply?

The applicant/certificate holder first completes a questionnaire and then receives a quotation form the certification body. The applicant/certificate holder then completes the application form. Certification body organizes the certification procedure, conducts a pre-audit and then conducts the certification audit (stage 1), and drafts the audit report.

The applicant/certificate holder implements the required actions/corrections and drafts a report. The certification body then conducts the second stage of the audit and drafts a report which is again sent to the applicant/certificate holder to implement the required actions/corrections. The certification body conducts a post-audit and drafts the audit report (only in case non-conformities are detected during the audit). If there are no non-conformities the certification body issues a certificate, organises the certificate maintenance details. Within a three-year period the certification body conducts two surveillance audits and one recertification audit (if the applicant wishes to prolong the value of its certificate) and drafts an audit report each time. The applicant/certificate holder implements the required actions/corrections and drafts a report. The certification body conducts a post-audit and drafts an audit report or confirms the report. After the recertification the certification body issues a new certificate. The first part of a certification audit, focussing on the establishment and documentation of the information security management system which is usually not conducted on-site, encompasses:


67

Model1 ISO/IEC 27001 conformity (information security management) x A review of the security policy and objectives; x A review of the certification scope, supporting procedures and

controls; x A risk assessment report, implemented programmes and actions to

reduce risks; x A statement of applicability. The second part of certification audit is carried out on-site and focuses on implementation of the system and its effectiveness, meeting the ISO/IEC 27001 standard requirements as well as legal and customer requirements. After awarding ISO/IEC 27001 certificate, the certification body annually conducts a surveillance audit of individual parts of the system to test whether the organisation still meets the requirements of the standard. A reassessment of the entire system and its effectiveness is conducted once every three years. The certificate is valid for three years.

Terms used certification audit, surveillance audit, recertification audit

Stakeholder involvement in certification procedure

There are two main stakeholders in the certification process. x Applicant/certificate holder - the organisation that is (to be)

certified. Applicant/certificate holder appoints a team of employees that will participate in the certification procedure.

x Certification body - the organisation performing the certification. Certification body appoints a lead auditor and other auditors that will perform the certification audit, surveillance audit or recertification audit.

Resources Both main stakeholders need to devote adequate human resources in the certification process. Applicant/certificate holder invests money into certification process and needs to have human resources, documentation and premises at the disposal of auditors from certification body.

Time: specifications on the time frame of the certification process

Certification audit is usually conducted within 6 months after the initial document assessments. The length of the certification procedure as a whole depends on the scope of certification.

After awarding ISO/IEC 27001 certificate, the certification body annually conducts a surveillance audit of individual parts of the system to test whether the organisation still meets the requirements of the standard.

A re-certification of the entire system and its effectiveness is conducted once every three years. The certificate is valid for three years. Any discovered irregularities or deficiencies should be eliminated within 3 months after the audit was conducted.

Any Other Business

None


68

Model2 EuroPriSe - The European Privacy Seal Aim of the certification

To facilitate an increase of market transparency for privacy relevant products and an enlargement of the market for Privacy Enhancing Technologies and finally an increase of trust in IT by certifying privacy compliance with European data protection regulations.

Subjects/Objects IT products such as hardware (e.g., a hardware firewall) and software (e.g., a database application in a hospital). IT-based services and automated processing of data (e.g., commissioned data processing) can be subject to a certification. The evaluation of such a service includes auditing of live performance of data processing and will include process auditing. Evaluated is either the complete product (e.g., a piece of software) or a part of a product.


Certification process is pictured (https://www.european-privacy-seal.eu/EPS-en/Fact-sheet). The procedure consists of two main stages:

(1) Evaluation: admitted experts evaluate product or service

(2) Validation: impartial certification authority checks evaluation and awards a European Privacy Seal.


Evaluation: Legal and technical experts (admitted by a certification body) evaluate the product according to evaluation criteria specified for intended usage, legal framework and technical environment of the product. They report their findings in an evaluation report. The evaluation criteria include: overview of fundamental issues, legitimacy of data processing, technical-organisational measures, and data subject’s rights.

Validation: The certification body checks the evaluation report with respect to completeness, plausibility and comparability with other certifications. After a successful certification, a certification report is published. Additionally, a short public report summarizing the evaluation findings is published.

Terms used certification, evaluation, admitted experts, validation, certification report


The process of certification involves four main stakeholders: (1) the applicant, who chooses two experts for initial discussion on evaluation. The applicant is involved in discussion also at the stage of validation by Certification Body. The applicant approves the final reports, compiled by experts. (2) The legal and technical experts conduct evaluations, inspections and examinations and report their findings in a confidential evaluation report (for validation by a certification body) and in a short public report. Experts are subject to admission by a certification body depending on their proficiency, reliability and independence; they are admitted either for legal, technical or both types of evaluation. Experts can either be single experts (such as lawyers or IT consultants) or organized as an evaluation centre. (3) The Certification Body: the organisation responsible for admission of experts, validation of evaluations, compiling an internal certification report, awarding of the seal and publication of the short public report. (4) European Privacy


69

Model2 EuroPriSe - The European Privacy Seal Seal Board: The coordination of certification bodies is to be the responsibility of the European Privacy Seal Board, due to be established during the EuroPriSe project. The coordination aims at joint interpretation of criteria. The board will also decide on the accreditation of further certification bodies.

Resources Two types of costs arise in the course of the certification: Costs for the evaluation by the experts and fees for certification by the certification bodies. The evaluation costs are negotiated between applicant and expert. The costs for certification are set by certification body. All stakeholders devote human resources to evaluation and validation process. In case of re-certification the amount of effort involved depends on three factors: The extent of product changes, the extent of legal changes (either legal norms or legal interpretations) and the technical developments (are the applied security measure still reliable and state of the art/up-to-date?).


The timeframe of the certification process (evaluation and validation) is not specified. The certification is valid for two years. After this period, re-certification in a simplified certification process is possible. The simplified certification process can range from an extension of validity requiring only a few, minor measures to new evaluations of parts or even the complete product.

Any Other Business

In the description of processes there is no reference to the foreseen procedures and timeframes when the applicant must adapt its product or service because an insufficient level of conformity to criteria has been discovered in the evaluation period or at the validation level.

Model3 JIS Q 15001:2006 conformity (Personal Information Protection Management System - Requirements)

Aim of the certification

The “PrivacyMark System” (http://privacymark.org/) is a conformity assessment system set up by the Japanese Information Process Development Center (JIPDEC) in 1998 in order to allow private enterprises to assess and prove whether they take appropriate measures to protect personal information. If so, they are granted the right to display “PrivacyMark” in the course of their business activities. The system is based on JIS Q 15001:2006 (Personal Information Protection Management System - Requirements), a Japanese industrial standard. PrivacyMark requires that a third-party organisation (a conformity assessment body, 17 currently in total) to objectively evaluate the compliance of private enterprises with all relevant laws and regulations. It is viewed as an effective tool that allows private enterprises to demonstrate that they are in compliance with the law and that they have voluntarily established a personal information protection management system with a high level of protection.

Subjects/Objects The subjects are private enterprises that have business establishments in Japan and which use personal information during the course of those establishments. They may be small, medium or large in size. They


70

Model3 JIS Q 15001:2006 conformity (Personal Information Protection Management System - Requirements) must have a pre-existing “Personal Information Protection Management System” (PMS) complying to JIS Q 15001: 2006 (Personal Information Protection Management Systems - Requirements) up and running. The object of the assessment is the said PMS system, which is a JIS Q 15001:2006 term describing their technical and organisational measures undertaken to protect personal data against accidental or unlawful destruction or accidental loss, alteration, unauthorized disclosure or access, or other unlawful forms of processing.


The certification process is pictured, see http://www.privacymark.org/application/new/image/FlowChart.jpg.

The steps are:

(1) Provision of application.

(2) Registration of the application.

(3) Arrangement of on-site screening data.

(4) Document screening, on-site screening.

(5) Find items to improve.

(6) Decision on accreditation.

(7) Notification of accreditation.

(8) Entering into contract.

(9) Issuing license.

(10) Announcement of newly accredited entity.


There are initial eligibility requirements (establishment in Japan, an existing Personal Information Protection Management System set up according to JIS Q 15001, not having failed the application in the previous 3 months, accreditation not being revoked in the preceding 2 years, no major data breaches or leaks in the preceding 2 year, top managerial staff not having served a prison sentence or suspended sentence in the preceding 2 years). The Application Documents are to be submitted either to the Accreditation Body (JIPDEC) or to one of the Conformity Assessment Bodies. They are first checked to determine if all required documents are filed and if all forms have been fully completed. The Application fee will have to be paid at this point. Then, the Documents are to be screened from the perspective of compliance with JIS Q 15001. Particular points to look for would be:

x whether a specific person (a manager) has been hired to oversee data protection, and whether that person has set clear internal responsibility and role-sharing arrangements in regard to data protection;

x whether the staff is being properly trained, by (at the very least) yearly workshops etc.;

x whether there is an appropriate IT system in place;


71


x whether there is an auditing system in place; x whether there is a customer complaints system in place; x whether there are appropriate measures against data leakage and

cyber-attacks in place; x whether any third parties processing the data for the corporation

have been subjected to an appropriate data processing and confidentiality agreement. If a question arises during the document screening, file of additional documents may be requested.

After the completion of document screening, on-site screening will be executed. It serves to clarify any question arising during document screening, as well as to confirm whether the system is structured and managed according to PMS. This will include:

x an interview with a company representative; x further interviews with all the personnel in charge of the PMS,

including managers, staff, subcontractors, auditors, etc., x a thorough on-site inspection of the PMS, including its

information security policy, physical and logical access controls, backups, record keeping, and online procedures.

If the PMS is deemed as conformant to the JIS Q 15001 standard, the company will then be issued a Privacy Mark certificate. Appropriate fees for its use must be paid at this point. After two years, the certificate can be extended, and after that, renewed.

Terms used conformity assessment, accreditation.


There are three main stakeholders in the certification process. x Applicant - the organisation that is to be assessed. The applicant

appoints a team of employees that will participate in the conformity assessment procedure.

x Conformity Assessment Body - organisations that process, screen and assess the application, and then issue certification. There are currently 17 such organisations, see http://www.privacymark.org/agency/member_list.html.

x Accreditation body - JIPCDEC, the organisation in charge of the Privacy Mark system, and the one performing the accreditation of conformity assessment bodies.

Resources The cost of the accreditation process varies from about 2200 to 9000 euros for new applications, depending on the size of the enterprise. It consists of the application fee, screening/assessment fee, and the privacy mark use fee. Renewal costs roughly 2/3 of that. The applicants must have a pre-existing "Personal Information Protection Management System" (PMS) complying to JIS Q 15001: 2006 (Personal Information Protection Management Systems - Requirements) up and running.


72



There are no formal constraints to the length of the certification procedure. It will mainly depend on the size of the applicant and the scope of their data processing activities. However, in general, both the document audit and the on-site audit take around 3 months to complete. The effective period of a PrivacyMark certification is 2 years. A 2-year extension can be applied for after the 2-years effective period. After that, renewals may be applied every 2 years. The renewal application must be made between 3 to 4 months prior to the end of the effective period.

Any Other Business

In the description of processes there is no reference to the foreseen procedures and timeframes when the applicant must adapt its product or service because an insufficient level of conformity to criteria has been discovered in the evaluation period or at the validation level.

Model4 TÜV Rheinland Spain Aim of the certification

TÜV Rheinland aims to satisfy the Spanish, EU and international legal requirements to demonstrate market demands and regulatory requirements. It ensures consistency of production for a product, service or system requirements in a specific sector safeguarding the position in the European retail market. Certified companies benefit from “one-stop service” delivered by highly qualified and experienced TÜV auditors with a presence in many countries building confidence with current and potential customers in order to gain a lead against the competition with a neutral audit seal and to reduce risk of company liability with documented security standards.

Subjects/Objects TÜV comprises a wide range of products, services and systems, specially the following: Construction and real state, consulting, information Security, education and personnel, management systems, materials testing and inspection, occupational safety and health, plants and machinery, product testing and vehicles and traffic.


Common process is defined in the following stages:

(1) Preliminary Audit (optional)

(2) Documentation Review.

(3) Certification Audit.

(4) Issue of Certificate.

(5) Surveillance Audits.

(6) Certification Renewal.


Once the company is accredited, TÜV issued an accreditation document. In addition, a label that certifies compliance with a particular standard is issued by TÜV. The information delivered by stakeholders is always examined. The exchange of information / communication (between TÜV and stakeholders) are always carried out. Accredited


73

Model4 TÜV Rheinland Spain certification processes required to inform stakeholders if required. An exception to this rule is confidential information or technical know-how about the product or service. Certification processes are always conducted between the requesting client and TÜV. Private and confidential business information is assessed and only a part of this information is public according to legislation.

Terms used Certification is the document that a company get once the evaluation process, the audit or conformity assessment and/or testing has been conducted by TÜV.


The customer is the only stakeholder in the accreditation process without prejudice to the users and the administration can exercise by themselves their rights to have public information and consultation.

Resources Any source to provide information on the certification process is relevant.


The duration of the certification process depends on the product, size of the company, maturity of the company in the process. These aspects are always regulated by the accreditation bodies (ENAC).

Any Other Business

None

Model5 AENOR Aim of the certification

AENOR aims to provide a stand out element in the market, improving the image of products and services offered and building trust between clients and consumers. Trust in the organisation and to customers, shareholders, employees, governments and the social environment of the company. Trust in the quality and safety of their products and services

Subjects/Objects AENOR comprises a wide range of products, services and systems.


Certification seals and marks:

(1) Preliminary questionnaire and request: a) audit planning process and study of documentation; b) the audit is performed; c) plan of corrective actions from the company

(2) The certification is granted by AENOR.

(3) Results are recorded

(4) Certification Maintenance Audit Procedures

Process: What does the process description at which stage

According to ISO 17021 the process of certification consists of two phases (phase 1 and 2). Once granted, the certificate ISO 27001 ISMS is valid for three years. Annual monitoring audits are performed two years after the certification audit (See chart below). There is an


74

Model5 AENOR imply? information assessment: All assets of information systems that support

the ISMS are valued, assessing appropriate risk analysis. AENOR requests all documentation / information on the ISMS. Among the information requested, the implementation document and risk action plan are the basic documentation of the ISMS. (Other documents are: Annual ISMS policy, objectives, processes, procedures, etc.). In addition to this information in written or digital form, auditors exchange oral information with the audited interviews). The audit of the certification process consists of two significant parts:

x Revision by the ISMS audit team (see ISO standard 27001) x Revision by the ISMS audit team of the implemented controls

according to scope and risks and subsequent testing compliance with them.

Terms used The certification audit is the conformity assessment according to ISO 27001 standard and following guidelines of the ISO 17021 and ISO 27006 standards.


Stakeholders must place their confidence in the heads of the organisations who are involved in the audit. An official of the ISMS, which in turn is authorised to perform their duties by the Executive Committee of the organisation, is mandatory. The Executive Committee (or delegate), involved business units, ISMS technical personnel in charge, technical staff that supports ICT, etc. are involved.

Resources The following Spanish legal legislation is the main source: Act– LSSICE – Law on Information Society Services and Electronic Commerce –, LPI – Law on Industrial Property (in any case, (laws that apply in the ICT sector, depending on the country).


The duration of ISMS audit certification is regulated in ISO 27006. With regard to the implementation of an ISMS, it will depend on the size of the organisation, mature management systems, scope of ISMS, etc. (its implementation will never exceed the period of one year (because the risk analysis of the information systems may become outdated). 1st year: monitoring audits. 2nd year: monitoring audits. 3rd year: renewal audit.

Any Other Business

Since 2006, AENOR has designed an ICT government and management model where the information security systems cuts across DevOps areas (Development & Operations). - See Figure 2) by linking the ISO 27001 (ISMS) to the (Spanish) National Insurance Scheme (NHIS) and also assessing ISO 27001 with SCADA systems control. Also in the area of ICT innovation, AENOR is designing data quality and security models, where not only the quality of the data but their intrinsic security will be studied.


75

Model6 Underwriters Laboratories of Canada (UL) Aim of the certification

UL aims to facilitate global trade and deliver peace of mind. It certifies, validates, tests, inspects, audits, and advises and trains. In addition, it provides the knowledge and expertise to help customers navigate growing complexities across the supply chain from compliance and regulatory issues to trade challenges and market access.

Subjects/Objects Physical products such as security equipment, access control systems, safes and also some software. All are addressed by standards that apply to the product area.


Is the certification process pictured? ISO 17067 Type 4 Scheme is closest to this. There are 6 key steps. It is outlined in ISO/IEC 17067.


(1) Agenda setting. It has been understood as a project plan or outline. This would be in Steps I and II.

(2) Consultation/Involving of stakeholders. This would occur throughout the process.

(3) Information assessment? Step II.

(4) Exchange of information/communication.

Step VI of ISO/IEC 17067 covers the requirements for testing and inspection of samples from the open market; testing or inspection of samples from the factory and assessment of the production, the delivery of the service or the operation of the process.

Terms used What terms are used in the certification process: certification, assessment, evaluation etc.? We use terms such as certification, assessment and listing.


Are stakeholders involved in the certification process? Stakeholder is understood to be the client and the staff of the certification body. No other stakeholders are involved at that time.

Resources Could you provide to us what kind of information (technical or legal sources) is needed for the certification process? An agreement for the work and the ongoing services, applicable standards have to be identified, marking and labelling requirements apply. Anything what implies information on resources initiated or needed for the certification process could be a valuable. For instance, in regard to the chosen technology selection the respective status of the relevant stakeholders; market success; acceptance by the public and policy makers.


Specifications on the time frame of the certification process (month, years etc.). This varies according to the product type and the standards being covered but it would be likely between 2 up to 6 months.


76

Model6 Underwriters Laboratories of Canada (UL) Any Other Business

None

Model7 ISO/IEC 17067 Conformity assessment - Fundamentals of product certification and guidelines for product certification schemes


1.) To provide the assessment and impartial third-party attestation that fulfilment of specified requirements has been demonstrated. 2.) To provide confidence to consumers, regulators, industry and other interested parties that products conform to specified requirements, including for example product performance, safety, interoperability and sustainability. 3.) To facilitate trade, market access, fair competition and consumer acceptance of products on a national, regional and international level.

Subjects/Objects Product certification (inclusively processes and services); certification system; certification scheme


(1) Selection;

(2) Determination;

(3) Review;

(4) Decision;

(5) Attestation;

(6) Surveillance


Selection: planning and preparation activities, specification of requirements, e.g. normative documents, and sampling, as applicable.

Determination: by testing, inspection, design appraisal, assessment of services or processes, other determination activities, e.g. verification.

Review: examining the evidence of conformity obtained during determination stage to establish whether the specified requirements have been met.

Decision: granting, maintaining, extending, reducing, suspending, withdrawing certification.

Attestation: issuing a certificate of conformity or attestation; granting the right to use certificates or other statements of conformity; issuing a certificate of conformity for a batch of products; grating the right to use marks of conformity.

Surveillance: testing or inspection of samples from the open market; testing or inspection of samples from the factory; assessment of the production, the delivery of the service or the operation of the process; management system audits combined with random tests or inspections

Terms used conformity assessment, product certification, certification system, certification scheme, scheme owner


77

Model7 ISO/IEC 17067 Conformity assessment - Fundamentals of product certification and guidelines for product certification schemes


The scheme owner, the certification body/bodies, the manufacturers of certified products, users of the certified product and entities that rely on certification, regulatory authorities, purchasers and users of certified products, conformity assessment bodies, such as testing laboratories and inspection bodies, involved in the product certification process, accreditation bodies and peer assessment groups, international certification schemes that facilitate the recognition of certification status from one scheme owner to another, consumers; more generally, all parties interested

Resources one or more samples of the product, a hole batch of products, periodically tested samples of the product; assessment/control of processes and resources; rules, procedures, management to be used for a scheme; fundamental scheme principles to be agreed among stakeholders; resources for the operation of the scheme, including impartiality and competence of the personnel, the evaluation resources


depending on a scheme and its supporting documentation; maintenance and improvement should be defined by the scheme owner

Any Other Business

Scheme owners: a) certification bodies which develop a product certification for the sole use of their clients; b) organisations such as a regulatory body or a trade association not being a certification body, which develop a product certification scheme in which one or more certification bodies participate. NOTE: A group of certification bodies, perhaps in different countries, can together set up a certification scheme. In that case, it would be necessary for the certification bodies, as joint owners of the scheme, to create a management structure so that the scheme could be operated effectively by all participating certification bodies

Model8 ISO/IEC TR 17026 Conformity assessment - Example of a certification scheme for tangible products


Identify type of the product, the product requirements and other requirements specified by the certification scheme and the geographical areas within which it operates

Subjects/Objects A type 5 product certification scheme


(1) Selection;

(2) Determination;

(3) Review;

(4) Decision;

(5) Licensing and control of the mark;


78

Model8 ISO/IEC TR 17026 Conformity assessment - Example of a certification scheme for tangible products (6) Surveillance;

(7) Suspending or withdrawing a certification and license;

(8) Managing changes affecting certification.


Selection: specified requirements for the products covered by the scope of the scheme, elements of the production process to be assessed and of the management system to be audited, determination activities, and the basis on which those activities be undertaken, sampling methods and frequency, requirements which the client has to fulfil in order to gain and maintain certification of the product, any other certification requirements.

Determination: evaluation of the product, assessment of the production process and audit of other elements of the client's management system critical to managing product conformity through document review and onsite assessment.

Review of the evaluation results.

Decision on certification and attestation of conformity

Licensing and control of the mark: mark of conformity, publicity to clients, misuse of certificate and marks of conformity.

Surveillance: testing and inspection of product samples, assessment of the production process and audit of the management system.

Suspending or withdrawing a certification and license.

Managing changes affecting certification: changes to product requirements, changes to other scheme requirements, changes by clients.

Terms used certification scheme, certification requirements, product requirements


the scheme owner, the certification body, the organisation that has a certification agreement with the certification body, or that has applied for (the manufacturer)

Resources product requirements, other requirements for the client (manufacturer) to fulfil; samples to be used for evaluation, procedures to be used for determination activities


no timeframe

Any Other Business

Confidentiality: the certification body is responsible for ensuring that confidentiality of information is maintained by its employees and those of its subcontractors concerning all information obtained as a result of their contacts with the client; this applies also to information obtained


79

Model8 ISO/IEC TR 17026 Conformity assessment - Example of a certification scheme for tangible products at the application stage.

Model9 EN-ISO 22301 Societal security - Business continuity management systems - Requirements


Specify requirements to Business Continuity Management Systems (BCMS) to protect against, reduce the likelihood of occurrence, prepare for, respond to, and recover from disruptive incidents when they arise.

Subjects/Objects Business Continuity Management Systems (BCMS)


(1) Establish processes;

(2) Implement and operate;

(3) Monitor and review;

(4) Maintain and improve


Establish processes: establish BCMS requirements considering the organisations’ mission, goals, internal and external obligations and legal and regulatory responsibilities, identify products and services and all related activities within scope of the BCMS, take into account interested parties' needs and interests; define the scope of the BCMS in terms of and appropriate to the size, nature and complexity of the organisation, determine leadership in top management, management commitment, policy, organisational roles, responsibilities and authorities, plan actions to address risks and opportunities, business continuity objectives and plans to achieve them, resources, competence, awareness and communication, documentation of information.

Implement and operate: operational planning and control, business impact analysis and risk assessment, business continuity strategy, business continuity procedures, exercising and testing.

Monitor and review: monitoring, measurement, analysis and evaluation, internal audit, management review.

Maintain and improve: nonconformity and corrective action, continual improvement of the suitability, adequate or effectiveness of the BCMS.

Terms used Business continuity management system, audit, conformity, measurement, performance evaluation, risk assessment, work environment


customers, investors, shareholders, the supply chain, public and/or community authorities

Resources all assets, people, skills, information, technology, premises and suppliers and information that an organisation has to have available to use, when needed, in order to operate and meet it objective


80

Model9 EN-ISO 22301 Societal security - Business continuity management systems - Requirements


no specific timeframe

Any Other Business

None

Model10 CertAlarm Scheme Rules Part 1-4; in addition: CertAlarm Scheme Rules Parts 2+3 regarding the stages and process descriptions


x Providing a 3rd party certification mark, applied voluntarily to PSS. x Showing that compliance with the requirements of the standard(s)

specified in the “CertAlarm System: Certification Rules - Part 2” was demonstrated.

x Providing assurance that the PSS consistently meets all requirements of the relevant European or other specified standards.

Subjects/Objects Based on the definitions in ISO/IEC Guide 67, it provides “certification type 5” applicable to products, and “type 6” applicable to services. Specific PSS:

x components for electrical and electronic Intruder Alarm Systems and Fire Detection & Alarm Systems;

x electrical and electronic Intruder Alarm Systems and Fire Detection & Alarm Systems;

x other Security or Fire protection related components and systems (e.g.: Access Control CCTV, Fire extinguishing systems, Smoke control systems, etc.);

x design, installation, commissioning and maintenance services relating to the above;

x alarm receiving, monitoring and other remote services relating to the above.


This is specified in Part 2+3. The processes depend on the specific PSS. Certification: see Part 2. Surveillance Period: see Part 3. In general: 7 checks in the surveillance re-test interval are conducted. In addition, the following additional tests are relevant:

x Fire Detection & Alarm Systems and Products: tests based on 19 standards,

x Intrusion & Hold Up Alarm Systems and Products: tests based on 10 standards,

x Alarm Transmission Systems: tests based on 2 standards, x Social Alarm Systems: tests based on 4 standards (see next cell).

All standards: several tests per document are possible.

Process: What does the process

Certification: In general: the standards that are referenced by Part 2 shall apply. Surveillance Period: On all samples the following checks


81


description at which stage imply?

shall be carried out: x Technical Documentation, x PCB layout version, x Firmware / software version(s), x Bill of Material, -Documentation, -Marking / labelling, x Visual inspection of the sample with respect to build, assembly

of the unit.

If the above does not identify any problems requiring specific attention, specific tests will be carried out at the specified surveillance re-test interval. Specific tests based on the content of concrete standards exist for:

x Fire Detection & Alarm Systems and Products: tests based on 19 standards,

x Intrusion & Hold Up Alarm Systems and Products: tests based on 10 standards,

x Alarm Transmission Systems: tests based on 2 standards, x Social Alarm Systems: tests based on 4 standards.

Additional tests may be required if the product has been modified or other problems are identified. Additional processes depend on the specific PSS, see cell “Number & naming of process stages”

Terms used system, certification rules, quality mark, mark, system rules and scheme rules, certification mark, compliance


CertAlarm was created by the industry, CertAlarm AISBL is a Not-for-Profit Association. CertAlarm has four groups of members: Certification Bodies, Test laboratories & Inspection Bodies; -Industry; -Users & Customers; -Insurers & Authorities.

Source: http://ec.europa.eu/competition/consultations/2009_insurance/certalarm_en.pdf; A list of CertAlarm's partners is provided a: http://www.certalarm.org/ca/partners; Complaints and appeals procedure is defined

Resources The schemes are operated by “partners” under a license contract* and the fees may differ

*Source: http://ec.europa.eu/competition/consultations/2009_insurance/certalarm_en.pdf


4 years


82


Any Other Business

A solution for the inclusion of national requirements exists, see CertAlarm System Certification Rules - Part 1, Annex B, p. 21. Part 3 also refers to additional (national) standards for products which are not considered by a European standard. The supplier shall establish a Quality Management System (QMS). “It is anticipated that ISO 9001 certification will provide the basis for this, but may not be sufficiently rigorous in isolation, so that additional product-specific factory production control may be required.” The concept includes specific rules for dealing with innovation

Model11

SSAIB Code of practice for Access Control Systems: SSAIB Rules 9. (there are many additional, general rules, in particular Rules 1-8)


To promote and encourage high standards of service and equipment and to endeavour to procure the protection of purchasers, hirers and users of Security and Safety Systems, and/or Services against defective systems and services

Subjects/Objects organisations providing security and safety systems and services


Specific chapters regarding:

(1) Application for Registration,

(2) Pre-assessment,

(3) Appraisal of applications for Registration,

(4) Making and Amending Conditions of Registration (this seems to be optional),

(5) Certificate of Registration,

(6) Routine Inspections (described after the reassessment),

(7) Reassessment (additional chapters provide, for example, information on fees and charges).

Information on the Certificates of Conformity follows later. Additional information is given in separate files.


Extensive information is given chapters 3ff

Terms used Registration schemes, registered firms, Certificates of Conformity, reassessment, the document provides a list of different definitions (as many documents, which are analysed in this CRISP file)

Stakeholder involvement in

Chief Police Officers, UKAS. see AOB


83

Model11

SSAIB Code of practice for Access Control Systems: SSAIB Rules 9. (there are many additional, general rules, in particular Rules 1-8)

certification procedure

Resources Pre-Assessment Æ at the request and cost of the applicant. Each applicant and Registered Firm shall pay fees and charges as are required by SSAIB in accordance with the Fee Schedule including:

x a non-refundable application fee and any additional payments under Rule 3;

x a non-refundable annual registration fee payable in advance; x any reassessment or inspection fees incurred pursuant to these

Rules; x the cost of purchasing Certificates of Conformity in accordance

with Rule 15.5.

Each applicant and Registered Firm “shall obtain and maintain at all times third party liability insurance of a type and to a level appropriate to … its obligations under the SSIAB Rules and relevant Criteria for Registration” or establish adequate self-insurance reserves for general comprehensive public liability and professional indemnity risks and product liability risks.


3 years

Any Other Business

SSAIB “holds UKAS Accreditation (...). It is also recognised by the Association of Chief Police Officers (England, Wales, and Northern Ireland) and the Association of Chief Police Officers (Scotland) in relation to the certification and inspection services that it carries out.” Specific aspect: “registered firms”.

Model11 SSAIB Code of practice for Access Control Systems: SSAIB Code of practice for Access Control Systems


The goal is to provide guidance to those responsible for specifying, designing, installing, commissioning and where required, repairing temporary alarm systems and equipment (this shows that the key goal is not certification). The document is supplementary “to the SSAIB Rules and Criteria documents as published from time to time”. It focusses on contexts “where there is no customer’s requirement for full compliance with European Standards”, i.e. BS EN 50133.

Subjects/Objects services/systems


Stages are not formulated in the document but it considers 10 areas/processes in the context of security services: 1 The Company, 2 Design and Specification of systems, 3 Installation of systems, 4


84

Model11 SSAIB Code of practice for Access Control Systems: SSAIB Code of practice for Access Control Systems Commissioning of systems, 5 Power Supplies, 6 Handover of system to customer, 7 Service and Maintenance – routine attention A1, 8 Service and Maintenance – emergency service, 9 Records ,10 Cessation of Maintenance.

Information in the web: The Application Process (1) Register with the SIA, (2) Complete the Self-Assessment Workbook, (3) Submitting an Application, (4) SSAIB Verification Services. Source: https://ssaib.org/page/the-application-process/


Each chapter includes a long list of requirements, one example regarding the company is: “Companies should be aware of their obligations under Data Protection legislation and comply with these at all times”.

Terms used SSAIB Certificate, SSAIB Certificate of Conformity, re-certification, Code of Practice, compliance, inspections, Certificate of Conformity


Indirect: the buildings’ designers1, standardisation bodies2, regulators3, no accreditor 1quote: “When a system is designed for use in a new building, it is essential to involve the building’s designers with a view to producing a system which has a minimum disruptive effect on the use of the finished building.” 2“All parts of the installation should conform to lEE Wiring Regulations and British Standards/ European Norms as applicable.”, BS7671. 3“all wiring and connections should be installed in accordance with the latest edition of 'Regulations for Electrical Installations', issued by the Institution of Electrical Engineers.”

Resources There is an application fee “in accordance with the Fee Schedule”. Specific financial requirements: “Before an applicant company can be registered as Approved by SSAIB under this scheme it will be required to satisfy the SSAIB that it is financially sound, operating from suitable premises, and able to meet all of its obligations to its customers and to SSAIB”. Opportunities to extend the system are required: “In all cases, the power supplies, both mains and standby, should have 30% spare capacity when the system is installed”.


3 years, see “SSAIB rules issue 9”: reassessment at intervals not exceeding 3 years, “a new SSAIB Certificate of Conformity should be issued and supplied to the customer ... in accordance with SSAIB Rules and Criteria”.

Any Other Business

“Companies should be aware of their obligations under Data Protection legislation and comply with these at all times” (see cell “Process: What does the process description at which stage imply?”). The document


85

Model11 SSAIB Code of practice for Access Control Systems: SSAIB Code of practice for Access Control Systems does not describe a UKAS accredited Certification scheme.

Model12

BSIA Code of Practice planning, design, installation and operation of CCTV surveillance systems code of practice and associated guidance


To assist in the process of designing and using CCTV systems by taking account of the various standards for CCTV systems, and presenting them in a useable framework showing the “building blocks” necessary to achieve an effective security surveillance solution.

Subjects/Objects services/systems


Key areas: planning, design, operation, maintenance. The guide builds on 12 principles/requirements, example: 1. Use of a surveillance camera system must always be for a specified purpose which is in pursuit of a legitimate aim and necessary to meet an identified pressing need. Example of topics in general (here regarding planning): Capturing user need; threat, vulnerability, risk assessment (to ensure the design of the system results in an installation that adequately addresses the threats and reduces the security risks); establishing operational requirements and agreeing with the customer (the requirements are not fixed); Target Capture and Image Detail (a table 'Levels of image detail' is provided); Environmental considerations.


The document shows only the service processes it refers to (see p.9).

Terms used Recommendation: “The content of this Code of Practice should assist with compliance with the Home Office Surveillance Camera Code of Practice.”


There is only the following information. “This document will be of use to many key stakeholders, all of which need to be considered at the planning and design stages, such as: insurers, specifiers, owners / operators, public, police & justice system, monitoring, installer /maintainer, inspection” (p.6). This shows that the concept aims to pay attention to the different stakeholder needs. In addition, the document builds on standards, which were created by different stakeholder groups complaints are for example considered by this requirement: “There must be as much transparency in the use of a surveillance camera system as possible, including a published contact point for access to information and complaints”.

Resources There are specific criteria for BSIA membership in general, see http://www.bsia.co.uk/join-the-bsia/membership-criteria.aspx

Another very interesting example given by BSIA regarding resources is


86

Model12

BSIA Code of Practice planning, design, installation and operation of CCTV surveillance systems code of practice and associated guidance given here: http://www.bsia.co.uk/sections/cash-and-valuables-in-transit/section-criteria.aspx#

The Code of Practice provides guidance for the “Training of operators to gain effective use”.


Additional BSIA documents should be checked in this regard

Any Other Business

The document provides interesting information regarding privacy by design (see chapter “Regulations and Legal Requirements relevant to CCTV”). “Recommendations for the maintenance of CCTV systems are outside the scope of this document and can be found in BSIA Form 120.”

Model13 Warwick District Council Control Centre Code of Practice for CCTV Scheme (BS 7958, BS 7858)


To detail the management, administration and operation of the CCTV system in a specific region and the associated Control and Monitoring Facility. The document also describes indirect goals: (a) reducing the fear of crime, (b) deterring and preventing crime, (c) assisting in the maintenance of public order and reducing offences involving vandalism and nuisance, (d) providing high quality evidence which may assist in the detection of crime and the apprehension and prosecution of offenders, (e) protecting property, (f) providing assistance with civil claims, (g) providing assistance with issues relating to public safety and health, (h) providing assistance and reassurance to the public in emergency situations.

Subjects/Objects products, in particular cameras; CCTV systems and associated safety and security equipment connected to the Control, Monitoring and Recording facility


12 principles + 16 privacy principles in chapter 8.2


-

Terms used CCTV scheme, problem orientated process to assess the appropriateness of CCTV in (the specific region), Code of Practice


87

Model13 Warwick District Council Control Centre Code of Practice for CCTV Scheme (BS 7958, BS 7858)


Data Protection Commissioner plus indirect involvement of stakeholders based on the standardisation of BS 7958 and BS 7858. An annual assessment of the scheme will be undertaken by an independent consultancy appointed by the owner to evaluate the effectiveness of the system. Regular independent random audits (see next cell). A member of the public wishing to make a complaint about the system may do so through Warwick District Council's complaint procedure.

Resources "The Information Commissioner’s CCTV Code of Practice stipulates that the system should be reviewed annually to determine whether CCTV continues to be justified. It further states that it is necessary to establish the system’s effectiveness to ensure that it is still doing what it was intended to do." Regular independent random audits "will check the operation of the scheme and the compliance with the code of practice". They will consider: "-The level of attainment of objectives and procedures, -Random audits of the data log and release of information, -The review policy, -Standard costs for the release of viewing of material, -The complaints procedure, -Compliance with procedures".


no information but their system is reviewed annually (see the two previous cells)

Any Other Business

The Code of Practice will be supplemented by a CCTV Operations Procedural Manual and an Operators Equipment manual. The scheme “will be managed in accordance with the principles of the Data Protection Act 1998”, which encompasses eight Data Protection Principles. It is registered with the Data Protection Commissioner. Registration Number Z623925X. All personnel employed to control/operate or manage the scheme will be security screened in accordance with British Standard 7858: Code of practice for screening of personnel in a security environment. All operators are or will be trained to the criteria required by the private Security Industry Act 2001 and licensed by the Security Industry Authority for Public Space Surveillance systems.

Model14 Code of Practice for the Cambridge City Council's Public CCTV Scheme


The text is more related to the objectives of the CCTV systems: x Protecting areas and premises used by the public; x Deterring and detecting crime; x Assisting in the identification of offenders leading to their arrest

and successful prosecution; x Reducing anti-social behaviour and aggressive begging; x Reducing fear of crime;


88

Model14 Code of Practice for the Cambridge City Council's Public CCTV Scheme

x Encouraging better use of city facilities and attractions; x Maintaining and enhancing the commercial viability of the city

and encouraging continued investment.

Subjects/Objects Only selected cells of this additional table are filled to provide additional and interesting input-


Only selected cells of this additional table are filled to provide additional and interesting input-


Only selected cells of this additional table are filled to provide additional and interesting input-

Terms used Code of practice, scheme


Directly: “Copies of the Code of Practice as agreed following public consultation will be made available for public inspection at all Council reception points, public libraries and on the City Council’s Website”, According to cell “Any Other Business” below, the Information Commissioner’s Office is also involved and a formal complaints procedure exist. Indirectly: Human Rights Act 1998, the Freedom of Information Act 2000, the Regulation of Investigatory Powers Act 2000 and the Protection of Freedoms Act 2012. Where any doubts exist, legal advice or advice from the Surveillance Commissioner’s Office will be sought before the Council agrees to undertake action under this Act. (page 3). Cambridge City Council will be responsible for the evaluation of the Scheme, which will be conducted at regular intervals. This evaluation will be conducted both: -internally (Police and City Council staff) and -independently by a body appointed by the Council. The following areas will be examined: Assessment of the impact on crime; Assessment of neighbouring areas without CCTV (Displacement); The views of the public; Operation of the Code of Practice.

Resources The Control Centre Duty Operators will daily confirm the operational efficiency of the system and the link to the Police. As shown in the previous cell, Cambridge City Council will be responsible for the evaluation of the Scheme. The costs of the evaluation programme will be built in to the annual running costs of the Scheme.


No time specification.

Any Other Business

The scheme has been registered with the Information Commissioner’s Office


89

APPENDIX 2: VALIDATION WORKSHOP INVITATION AND AGENDA


90

APPENDIX 3: FEEDBACK QUESTIONNAIRE (VALIDATION WORKSHOP)


91