the cyber capability development centre (ccdc) concept · 2019. 9. 19. · defence research and...

Defence Research andDevelopment Canada

Recherche et developpementpour la defense Canada

CAN UNCLASSIFIED

The Cyber Capability Development Centre(CCDC) ConceptAn infrastructure for cyber research, experimentation, testing and evaluation,demonstration and training

Maxwell DondoJonathan RistoDarcy SimmelinkDRDC – Ottawa Research Centre

Paul WorthIBISKA Telecom, Inc.

Defence Research and Development CanadaReference DocumentDRDC-RDDC-2018-D071May 2019

CAN UNCLASSIFIED

CAN UNCLASSIFIED

IMPORTANT INFORMATIVE STATEMENTS

This document was reviewed for Controlled Goods by DRDC using the Schedule to the Defence Production Act.

Disclaimer: This publication was prepared by Defence Research and Development Canada an agency of the Department ofNational Defence. The information contained in this publication has been derived and determined through best practice andadherence to the highest standards of responsible conduct of scientific research. This information is intended for the use of theDepartment of National Defence, the Canadian Armed Forces (“Canada") and Public Safety partners and, as permitted, may beshared with academia, industry, Canada’s allies, and the public (“Third Parties"). Any use by, or any reliance on or decisionsmade based on this publication by Third Parties, are done at their own risk and responsibility. Canada does not assume anyliability for any damages or losses which may arise from any use of, or reliance on, the publication.

Endorsement statement: This publication has been published by the Editorial Office of Defence Research and DevelopmentCanada, an agency of the Department of National Defence of Canada. Inquiries can be sent to:[email protected].

c© Her Majesty the Queen in Right of Canada, Department of National Defence, 2019

c© Sa Majesté la Reine en droit du Canada, Ministère de la Défense nationale, 2019

CAN UNCLASSIFIED

Abstract

The vision for the Cyber Capability Development Centre (CCDC) is to facilitate scienceand technology (S&T) research, experimentation, testing, evaluation, demonstration andtraining activities within the Cyber Operations and Signals Warfare (COSW) section ofthe Defence Research and Development Canada (DRDC)-Ottawa. The CCDC conceptis driven by identified research and development (R&D) requirements solicited from thescientific community in the COSW section of DRDC-Ottawa [1]. It was determined, througha business case analysis, that the best way to meet such requirements could be to construct avirtualised infrastructure based on VMware technology. The resulting implementation couldprovide an agile and effective cyber research infrastructure to meet the CCDC’s originalvision. This report documents the overall concept that addresses the DRDC’s cyber S&Tresearch infrastructure needs and functional (business) requirements. The report’s focus ison how the CCDC’s technologies could meet researchers’ needs and in turn provides detailedsteps on how the researchers could use the capabilities provided by the CCDC to supporttheir S&T activities. Throughout the report, the CCDC is shown to have the potential forsuperior time and resource saving capabilities compared to the classical experimentationapproaches. The report also outlines how the current CCDC vision could be extended toincorporate classified level II capabilities as well as connectivity with local and internationalpartners.

Significance for defence and security

This concept document, whose targeted audience are the science and technology (S&T)decision makers, outlines the Cyber Capability Development Centre (CCDC)’s envisionedcapabilities and how its infrastructure could support the high level S&T deliverables, asoutlined in the CCDC charter. The report specifically addresses how the CCDC concept couldmeet the infrastructure requirements for cyber research, experimentation, testing, evaluation,demonstration and training as identified in the business case and options analyses [2].

DRDC-RDDC-2018-D071 i

Résumé

La vision du Centre de développement des cybercapacités (CDC) est la suivante : faciliter larecherche, l’expérimentation, la mise à l’essai, l’évaluation, la démonstration et les activitésde formation en S & T au sein de la Section Cyberopérations et guerre des transmissions(COGT) de Recherche et développement pour la défense Canada (RDDC). Le concept duCDC se fonde sur les besoins établis en recherche et développement (R & D) émanant dela communauté scientifique de la Section des COGT du Centre de recherches d’Ottawade RDDC [1]. On a déterminé, au moyen d’une analyse de rentabilisation, que le meilleurmoyen de répondre à ces besoins pourrait être de bâtir une infrastructure virtualisée baséesur la technologie VMware. Une fois mise en place, elle pourrait constituer une cyberinfra-structure de recherche souple et efficace qui serait conforme à la vision initiale du CDC.Ce rapport vient documenter le concept général portant sur les besoins fonctionnels (etopérationnels) de RDDC en matière de cyberinfrastructure de recherche en S & T. Dansle présent rapport, on décrit surtout : comment faire en sorte que les technologies du CDCpuissent répondre aux besoins des chercheurs et aussi comment fournir à ces derniers unplan détaillé pour pouvoir utiliser les capacités du CDC afin de soutenir leurs activités deS & T. Dans l’ensemble du rapport, on démontre que le CDC pourrait disposer de capaci-tés supérieures permettant d’épargner temps et ressources comparativement aux approchestraditionnelles d’expérimentation. Dans le rapport, on souligne également que la vision ac-tuelle du CDC pourrait englober les capacités classifiées de niveau II, de même que lescapacités de connectivité avec les partenaires locaux et internationaux.

Importance pour la défense et la sécurité

Ce document de concept, qui s’adresse aux décideurs en S & T, présente sommairement lescapacités envisagées au CDC. On explique également comment l’infrastructure pourrait sou-tenir les produits livrables de haut niveau de S & T, tel que stipulé dans la charte du CDC.Dans ce rapport, on aborde tout particulièrement la façon dont le concept du CDC pourraitrépondre aux besoins en infrastructure pour la cyberrecherche, l’expérimentation, la mise àl’essai, l’évaluation, la démonstration et la formation, besoins qui ont été déterminés dansl’analyse de rentabilisation et celle des options [2].

ii DRDC-RDDC-2018-D071

Acknowledgements

We would like to thank Geoffrey Delage, LCdr Robin Moll, Dr. Natalie Nahkla, Dr. TrishWillink, Maj. Laetitia Richard, Dr. Martain Salois, Dr. Adrian Taylor and Dr. KathrynPerrett for their valuable contributions to the CCDC project.

iii DRDC-RDDC-2018-D071

Table of contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Significance for defence and security . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Résumé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

Importance pour la défense et la sécurité . . . . . . . . . . . . . . . . . . . . . . . . ii

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Table of contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 The CCDC concept and need . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Report Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Quick document guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 The CCDC concept and its technologies . . . . . . . . . . . . . . . . . . . . . . . 5

2.1 The CCDC objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 CCDC as a private cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Physical layer hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3.1 Hardware for unclassified S&T activities . . . . . . . . . . . . . . . 9

2.3.2 Hardware for classified S&T activities . . . . . . . . . . . . . . . . 9

2.4 The CCDC as a software-driven cloud . . . . . . . . . . . . . . . . . . . . . 10

2.4.1 CCDC’s cloud catalog . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4.2 CCDC’s cloud blueprints . . . . . . . . . . . . . . . . . . . . . . . 11

2.4.3 CCDC’s cloud entitlements . . . . . . . . . . . . . . . . . . . . . . 11

2.4.4 CCDC’s “zero-touch” cloud automation . . . . . . . . . . . . . . . 12

2.4.5 CCDC’s cloud orchestration . . . . . . . . . . . . . . . . . . . . . . 13

DRDC-RDDC-2018-D071 iii

2.4.6 The CCDC’s zero-trust and micro-segmentation . . . . . . . . . . 13

2.4.7 Interaction with the physical layer . . . . . . . . . . . . . . . . . . 14

2.4.8 CCDC’s software packages . . . . . . . . . . . . . . . . . . . . . . 15

2.5 Overall design layout of the CCDC . . . . . . . . . . . . . . . . . . . . . . 15

2.6 Communication between domains . . . . . . . . . . . . . . . . . . . . . . . 16

2.7 Support personnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 CCDC as a flexible and scalable test and development environment . . . . . . . . 19

3.1 Use of affordable COTS products . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.1.3 Miscellaneous COTS features . . . . . . . . . . . . . . . . . . . . . 21

3.1.3.1 Secure multi user . . . . . . . . . . . . . . . . . . . . . . 21

3.1.3.2 One-way access and internal data sharing . . . . . . . . 22

3.1.3.3 Cyber security experimentation at scale . . . . . . . . . 23

3.1.3.4 Security domains . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Reuse, revive and support interoperability across projects . . . . . . . . . . 23

3.2.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.2.1 The CCDC concept for infrastructure re-use . . . . . . . 25

3.2.2.2 The CCDC concept for interoperability . . . . . . . . . . 26

3.2.2.3 The CCDC concept for day-2 data sharing . . . . . . . . 26

3.2.2.4 The CCDC concept to archive or revive projects . . . . . 28

3.3 Infrastructure to facilitate testing and experimentation . . . . . . . . . . . 28

3.3.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

iv DRDC-RDDC-2018-D071

3.4 Infrastructure to facilitate training and demonstrations . . . . . . . . . . . 29

3.4.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.4.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4.2.1 The CCDC concept for demonstration capabilities . . . . 30

3.4.2.2 The CCDC concept’s Training capabilities . . . . . . . . 31

3.4.2.3 Automated access control for transient demonstrationand training . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.5 Infrastructure to provide local desktop connectivity to the lab . . . . . . . 33

3.5.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5.3 General examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4 The setting up and tearing down of experiments, tests and demonstrations . . . 36

4.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.3 Application example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.4 Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5 Project lifecycle management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.3 User interface and display example . . . . . . . . . . . . . . . . . . . . . . 44

6 Classified (level II) and other future CCDC capabilities . . . . . . . . . . . . . . 46

6.1 Level II processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.1.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.2 Connecting to other level II networks . . . . . . . . . . . . . . . . . . . . . 48

6.2.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

DRDC-RDDC-2018-D071 v

6.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.2.2.1 Local partners . . . . . . . . . . . . . . . . . . . . . . . . 49

6.2.2.2 International partners . . . . . . . . . . . . . . . . . . . 49

6.3 Other future capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Acronyms and abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Annex A: CCDC Software and examples . . . . . . . . . . . . . . . . . . . . . . . . 59

A.1 Current software packages that could be used in CCDC . . . . . . 59

A.2 Application examples . . . . . . . . . . . . . . . . . . . . . . . . . . 60

A.2.1 Service deployment example . . . . . . . . . . . . . . . . . 60

A.2.1.1 Researcher’s perspective . . . . . . . . . . . . . . 60

A.2.1.2 The lab administration perspective . . . . . . . . 63

A.2.2 Steps in the wireless testing and experimentationdeployment example . . . . . . . . . . . . . . . . . . . . . . 69

A.2.3 Experimental testbed design examples . . . . . . . . . . . . 76

A.2.3.1 Example 1: Complex corporate network design . 77

A.2.3.2 Example 2: Wireless design . . . . . . . . . . . . 79

A.2.3.3 Example 3: SDK design example . . . . . . . . . 81

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

vi DRDC-RDDC-2018-D071

List of figures

Figure 1: The original CCDC design. . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Figure 2: An illustration of cloud computing. . . . . . . . . . . . . . . . . . . . . . 6

Figure 3: Envisioned CCDC solution architecture including software and hardware. 7

Figure 4: HCI hardware platform provides a vendor agnostic scale-out platform. . . 8

Figure 5: Hardware platform for unclassified research. . . . . . . . . . . . . . . . . 10

Figure 6: An illustration of the assignment of entitlements to users and groups. . . 12

Figure 7: CCDC could support a “zero touch” deployment for experimental testbeds. 13

Figure 8: Micro-segmentation and zero-trust. . . . . . . . . . . . . . . . . . . . . . 14

Figure 9: Envisioned CCDC’s unclassified network layout and configuration. . . . . 15

Figure 10: Envisaged communication between unclassified and classified level II sites. 17

Figure 11: Administrative roles within the CCDC. . . . . . . . . . . . . . . . . . . . 17

Figure 12: Access control and entry into the CCDC through a user portal. . . . . . . 21

Figure 13: Secure one-way access from Operational Zones into the CCDC. . . . . . . 22

Figure 14: CCDC Security Domain deployment. . . . . . . . . . . . . . . . . . . . . 24

Figure 15: CCDC shared infrastructure reuse. . . . . . . . . . . . . . . . . . . . . . . 25

Figure 16: vRealize Orchestrator workflow control. . . . . . . . . . . . . . . . . . . . 27

Figure 17: CCDC’s controlled data sharing on shared infrastructure. . . . . . . . . . 27

Figure 18: CCDC informal demonstration capability. . . . . . . . . . . . . . . . . . . 31

Figure 19: CCDC formal demonstration capability. . . . . . . . . . . . . . . . . . . . 31

Figure 20: CCDC training capability. . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Figure 21: CCDC dynamic access control through XaaS capability. . . . . . . . . . . 32

Figure 22: CCDC XaaS blueprint capability. . . . . . . . . . . . . . . . . . . . . . . 33

Figure 23: CCDC published catalog XaaS service. . . . . . . . . . . . . . . . . . . . 33

DRDC-RDDC-2018-D071 vii

Figure 24: Local desktop access to unclassified CCDC. . . . . . . . . . . . . . . . . . 35

Figure 25: Local desktop access to CCDC (Level II). . . . . . . . . . . . . . . . . . . 35

Figure 26: The CCDC’s entitlements configuration. . . . . . . . . . . . . . . . . . . . 38

Figure 27: A new CCDC entitlement for user groups. . . . . . . . . . . . . . . . . . 38

Figure 28: CCDC entitlement options. . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Figure 29: CCDC’s automated reclamation of resources. . . . . . . . . . . . . . . . . 40

Figure 30: CCDC provides portability of blueprints. . . . . . . . . . . . . . . . . . . 41

Figure 31: vROp Operational and Management toolset. . . . . . . . . . . . . . . . . 44

Figure 32: vRO centralized console propagates up events based on policy. . . . . . . 45

Figure 33: Envisioned CCDC level II processing capability. . . . . . . . . . . . . . . 47

Figure 34: Envisioned CCDC classified (level II) collaboration capability. . . . . . . 48

Figure A.1: An overview of the CCDC deployment scenario. . . . . . . . . . . . . . . 61

Figure A.2: Service Catalog item for SDK deployment. . . . . . . . . . . . . . . . . . 62

Figure A.3: Available service deployment portal. . . . . . . . . . . . . . . . . . . . . 63

Figure A.4: Additional tabs provided to administrators. . . . . . . . . . . . . . . . . 63

Figure A.5: Administrative approval policies configuration and other options. . . . . 64

Figure A.6: Options for the automation of self-service. . . . . . . . . . . . . . . . . . 66

Figure A.7: Steps involved in creating a new service. . . . . . . . . . . . . . . . . . . 66

Figure A.8: New service request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Figure A.9: Multi-machine blueprint interface. . . . . . . . . . . . . . . . . . . . . . . 68

Figure A.10: Operational user’s login screen. . . . . . . . . . . . . . . . . . . . . . . . 69

Figure A.11: Operational user tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Figure A.12: Operational user Catalog Tab. . . . . . . . . . . . . . . . . . . . . . . . 70

Figure A.13: Operational user’s view of the services catalog. . . . . . . . . . . . . . . 70

viii DRDC-RDDC-2018-D071

Figure A.14: Operational user’s new service request form. . . . . . . . . . . . . . . . 71

Figure A.15: User’s view of request status. . . . . . . . . . . . . . . . . . . . . . . . . 72

Figure A.16: Workflow of a successful deployment request. . . . . . . . . . . . . . . . 72

Figure A.17: List of VMs in a successful deployment. . . . . . . . . . . . . . . . . . . 73

Figure A.18: Entitlements granted to the operational user. . . . . . . . . . . . . . . . 74

Figure A.19: Operational user accessing remote console access to resources. . . . . . 74

Figure A.20: Remote console login screen. . . . . . . . . . . . . . . . . . . . . . . . . 75

Figure A.21: Remote console IP address. . . . . . . . . . . . . . . . . . . . . . . . . . 75

Figure A.22: Connecting to remote VM through Putty. . . . . . . . . . . . . . . . . . 76

Figure A.23: Requesting a change to the leased period. . . . . . . . . . . . . . . . . . 76

Figure A.24: Submitting a change to the leased period. . . . . . . . . . . . . . . . . . 77

Figure A.25: CCDC complex experimental testbed service deployment. . . . . . . . . 77

Figure A.26: CCDC’s envisioned external zone security rules. . . . . . . . . . . . . . 78

Figure A.27: CCDC’s envisioned DMZ zone security rules. . . . . . . . . . . . . . . . 78

Figure A.28: CCDC’s envisioned internal zone security rules. . . . . . . . . . . . . . . 79

Figure A.29: CCDC’s micro-segmentation default security rule. . . . . . . . . . . . . 79

Figure A.30: Traffic flow steps from West to East (Virtual to Physical integration). . 80

Figure A.31: Wireless experimental testbed blueprint design. . . . . . . . . . . . . . . 81

Figure A.32: SDK experimental testbed blueprint design. . . . . . . . . . . . . . . . . 81

Figure A.33: Blueprint view of embedding on-the-fly software components. . . . . . . 82

List of tables

Table 1: A quick guide to the contents of the concept document. . . . . . . . . . . 4

Table A.1: Current software components that could support the CCDC. . . . . . . . 59

DRDC-RDDC-2018-D071 ix

This page intentionally left blank.

x DRDC-RDDC-2018-D071

1 Introduction

The need for a “real world” environment for cyber security researchers to test and validatetechnologies and innovations cannot be overstated. Researchers need networks that can betested to scale, with traffic generation that emulates real network characteristics, topologies,host machines, services and protocols. Conducting such activities on operational networksor “live” on the Internet provides excellent “real world” network alerts, metrics, falsepositive and false negative sensor readings and accurate end user metrics. However, suchactivities could expose the organisation to tremendous risks by subjecting its infrastructureto experimentation and testing while simultaneously attempting to keep the networksoperational.

To avoid such risks while undertaking science and technology (S&T) activities, DefenceResearch and Development Canada (DRDC) has traditionally conducted cyber securityS&T activities using mostly small private research lab infrastructures that have been specificto an experiment and project. Such infrastructures may not represent the large operationalnetworks or Internet elements that emulate “real world” cyber security scenarios. This couldresult in inadequate solutions that might require additional research and development (R&D)effort to represent reality. To address this S&T deficiency in the short- and long- term,DRDC needs an agile and effective infrastructure for cyber research, experimentation,testing and evaluation, demonstration and training. The Cyber Capability DevelopmentCentre (CCDC) is envisioned to meet infrastructure requirements that could provide such acapability.

1.1 The CCDC concept and need

The CCDC is envisioned as a centralised lab infrastructure, built on virtual technologiesand concepts, to support DRDC’s computer network operations (CNO) S&T activitiesundertaken by the Cyber Operations and Signals Warfare (COSW) section on behalf of theDepartment of National Defence (DND)/Canadian Armed Forces (CAF). Its design aims toprovide the DRDC with service capabilities for research, experimentation, testing, validation,demonstration and training. The CCDC’s automated framework could enable S&T activitiesthat represent “real-world” network environments, including deployed defence technologies,attack behaviors and network topologies, without endangering its own infrastructure, theInternet or other deployed operational networks. Its original vision, which addressedpreviously identified research infrastructure requirements [1, 2], is illustrated in Figure 1.

As shown in the figure, the CCDC concept aims to establish the infrastructure and part-nerships needed to enhance cyber S&T research, training, demonstration and collaboration.The design envisioned research enclaves in both the unclassified and classified (level II)domains. The CCDC was designed to support research enclaves such as wireless or electronicwarfare (EW) as well as communications between domains, other DRDC research centersand DND networks to establish partnerships and engage in S&T collaboration activities.While the core unclassified CCDC domain would be supported in the conceptual design,full classified activities as well as all connectivity with local and multinational partners are

DRDC-RDDC-2018-D071 1

Figure 1: The original envisioned CCDC design as adapted from [1].

deferred to possible future concept enhancements. A more detailed discussion of the CCDCrequirements can be found in the literature [1, 2].

In an earlier business case analysis [2] it was determined that a VMware-based virtualisedinfrastructure would provide the best low-cost solution for the CCDC. Therefore the CCDCconcept is envisioned to use the VMware-based virtualised infrastructure presented inSection 2 to address specific business and user requirements [1, 2]. Those requirements couldbe met through the following CCDC capabilities:

• Multi user and secure separation: The CCDC concept supports multiple usersand groups of users. As described in Section 2, it is designed to use policy-drivensoftware defined networks (SDNs) and their supporting capabilities to separate orisolate experimental testbeds in the multi-user shared infrastructure. Secure separationprevents cross-pollination across enclaves, avoiding possible disruptions of activities inother enclaves, insuring accurate, repeatable and controlled experimental environments.

• Cyber security experimentation at scale: As described in Section 3, the CCDCconcept could provide the necessary infrastructure, automation, networks, tools andmethodologies to support the DRDC’s advanced cyber security research at enterprisescale.

• Self-service and automation: On-demand secure virtual testbeds, which use re-peatable blueprints of complete network infrastructures, could provide researchers withself-service and automation in self-contained environments as described in Sections 3and 4. Researchers could run experiments on network testbeds that emulate Internetelements, enterprise and operational networks.

2 DRDC-RDDC-2018-D071

• Centralised management monitoring and service assurance: As described inSection 5, the CCDC could provide analytics for an integrated approach to perfor-mance, capacity and configuration management that could proactively ensure efficientmonitoring, running and allocation of virtual resources.

• Collaboration: As described in Section 6, the ability to share and exchange data,experiments, demonstrations and training activities would be supported by the CCDC’scollaboration capability. Although this capability is not part of the current CCDCconcept, current collaboration capabilities could allow researchers to carry out limitedcollaborative S&T activities through the centralised lab infrastructure.

• Security uniformity and compliance: The CCDC concept uses a uniform securitybaseline across its hardware and virtual infrastructure. This ensures cyber securityexperimentation employs a hardened security standard that aligns with the followingDND guidance:

1. Director of Information Management Security (DIM Secur) Security Assessment& Authorization Guidance (SAAG);

2. Communications Security Establishment Canada (CSE) information technology(IT) security guidelines;

3. Director Information Management Engineering and Integration (DIMEI) CyberSecurity Architecture and Engineering guidance.

The CCDC could provide technologies to support all the stated capabilities to allow theresearcher to quickly and efficiently put together, execute, manage, dismantle and reassembleexperiments. The time-consuming classical experimentation approach of requesting ITservices and waiting for them to be provided could now be replaced with an automatedself-service experimentation environment that would require minimal technical support.

The current CCDC concept design does not support classified (level II) S&T activities, whichresearchers also need. Additional capabilities and connectivity could be put in place to allowaccess to the Classified Test and Development Centre (CTDC), which would enable accessto other collaborative spaces such as the Canadian Forces Exercise and ExperimentationNetwork (CFXNet). Connectivity enhancements could enable collaborative activities anddata exchange through partnerships with other labs such as the Test and DevelopmentCentre (TDC) or with international partners. An example of those international partnershipsis through The Technical Cooperation Program (TTCP) Mission Assurance and SituationalAwareness (MASA) activity that the COSW section of the DRDC is an active participant1.

1.2 Report Layout

The rest of the report is laid out as follows: Section 6 presents classified (level II) and otherfuture CCDC capabilities. A summary presented in Section 7 concludes the report.

1At the time the document was written.


1.3 Quick document guide

In addition to the index provided on page 83, this section gives a summary guide of wherethe reader can quickly find CCDC concepts. The quick guide, which is shown in Table 1, liststhe sections where the CCDC capabilities, as outlined in the previously defined requirementsand business case analyses [1, 2], can be found.

Table 1: A quick guide to the contents of the concept document.Section ContentsSection 2 Virtual infrastructure architecture and con-

cepts.Sections 3-5 CCDC capabilities:

1. Section 3: Secure multi-user enclaves,flexible, and scalable test and develop-ment environment,

2. Section 4: Setting up and tearing downof experiments, tests and demonstra-tions,

3. Section 5: Project lifecycle management.

Section 6 Classified (level II) activities, collaboration,and future capabilities.

Readers can go directly to the sections that address capabilities of their interest withoutloss of generality. However, for readers that are not familiar with virtualisation, it isrecommended that they read Section 2 in its entirety before proceeding to subsequentsections.


2 The CCDC concept and its technologies

This section presents a general overview of the CCDC concept and the technologies thatcould support it. A high-level discussion of the technologies and concepts that could enablethe CCDC is presented.

2.1 The CCDC objectives

The concept of the CCDC is based on a virtualised infrastructure solution that couldprovides an enterprise-class, scalable platform to enable agile and effective cyber research,experimentation, testing, evaluation, demonstration and training. The solution is envisagedto provide on-demand access and control of shared infrastructure resources and securitywhile allowing researchers to maximize time and asset utilization. Specifically, the solutionis envisaged to integrate the following key capabilities from a private cloud concept:

• Self-service portal and automated provisioning;

• Security;

• Collaboration;

• Resource elasticity;

• Automated monitoring;

• Lifecycle management;

• Unified management of hardware and software.

This section presents the CCDC concept design as a capability driven by VMware2 conceptsand technologies [3]. To provide an understanding of the technologies, the presentation startswith a high level introduction of the cloud computing concepts followed by components thatcould make up the virtual infrastructure.

2.2 CCDC as a private cloud

The CCDC concept is built on cloud computing technology. The cloud computing concept,which is illustrated in Figure 2, is a convergence of physical and virtual technologies thatsupport the sharing of resources, software and data over a network.

Cloud computing, as illustrated in Figure 2, can be implemented as a public, private orhybrid (public and private) network. Public providers of cloud computing services, such asApple [4], own, maintain and control the physical servers that store information. Users can,through the connecting network, access the stored information or any other services. Theycan also access computing services through the cloud, as if they are using their desktop

2An earlier options analysis [2] identified VMware as the best solution for the CCDC.


Figure 2: An illustration of cloud computing.

computers, and simultaneously perform routine tasks with other users such as editing files.Depending on the type of cloud, the supporting network could be the Internet (for publiccloud), a private network or a combination of the two. In the case of the CCDC, thesupporting network is envisioned to be a private network.

Cloud computing can be as simple as catering to individual users who want to store personalphotos, files and videos to sophisticated enterprise services for large organisations. Suchcomplex services can support the development of IT applications and other services. Publicproviders such as Google [5], Amazon [6] or Apple [4] offer advanced services that provide,among other capabilities, remote access, storage, easy setup and automatic updates. Cloudcomputing providers offer services at reduced costs and often make significant investmentsto safeguard users’ privacy and security. Further reading on cloud computing can be foundin the documentation [4–9].

Unlike the public cloud providers, the CCDC is envisioned as a private cloud that usesVMware virtualisation infrastructure [7–9]. It would acts as a service governor that providesaccess to preconfigured experimental environments such as infrastructure as a service (IaaS),storage as a service (SaaS) and platform as a service (PaaS). Such a capability wouldconstitute the CCDC’s private cloud automation designed to function with minimal to nohuman middleware. The capability could also allow the CCDC to enforce business andIT policies throughout the infrastructure’s service life cycle. CCDC connectivity could berestricted to the lab and remote links from researchers’ offices or conference rooms. Furtherenhancements to the CCDC capabilities could extend CCDC’s connectivity to external


networks such as the General Purpose Network (GPNet) or classified networks such as theCTDC, capabilities that are not available in the current concept design.

To enable the technology illustrated in Figure 2, both software and hardware componentsneed to be integrated to achieve a common purpose. Such an integration is illustrated inFigure 3.

Figure 3: An envisioned CCDC solution architecture is made up of the virtual and physical (hardware) layers. The hardware layer is explained in detail in the next section. User connectivity to the experimental enclaves (Malware and Wireless in this case) would be achieved through an edge services gateway (ESG), a VM device that functions as a gateway.

The figure illustrates how a researcher located in an operational zone could connect to the CCDC’s virtualised infrastructure. The researcher could use the virtual desktop infrastruc-ture (VDI) capability to connect to the CCDC through the ESG.

VMware’s VDI is a virtualised desktop approach in which the desktop operating system (OS) runs from the data center and is presented to the end user. It is a variation of the client-server computing model, sometimes referred to as server-based computing – a term that was coined by VMware. While using the VDI, an image of the desktop is available to the user to interact with the OS and its applications as if they were running on a local host. Detailed discussion of the VDI can be found in the VMware documentation [10, 11].


The most important concepts in Figure 3 are the envisioned CCDC virtual and physicallayers. The virtual layer would be software defined as will be discussed later. The physicallayer would be composed of computing, storage and networking hardware that characteriseclassical enterprise networks. This infrastructure convergence, termed the hyper convergedinfrastructure (HCI), is described in the next section. The technologies that could make upthe physical and virtual layers are presented in the next sections, starting with the hardwarethat supports both layers.

2.3 Physical layer hardware

The physical layer consists of all of the necessary hardware required to provide computing,networking and storage services. It also provides technologies to support network andinformation security. The hardware allows cyber security experimentation to emulate realworld, enterprise networks without endangering existing operational networks.

The CCDC’s physical layer could be built on a HCI solution [12], which is illustrated inFigure 4. It is an emerging platform for data center infrastructure that integrates computing,storage and virtualisation into one appliance [13].

Figure 4: HCI hardware platform provides a vendor agnostic scale-out platform.

Figure 4 shows a conceptual layout of the HCI, which is sold as a pre-engineered, certifiedsolution that supports server, storage and network components. The HCI scales out, notup3, which makes for a more modular and agile approach for expanding deployment modelswhere computing and storage must scale cohesively [14]. The following components makeup the HCI:

3Scale up refers to adding more processors and random access memory (RAM), which requires buying amore expensive and robust server. Scale out refers to a cheaper and easily scalable option of adding moreservers with less processors and RAM.


• The direct-attached storage (DAS): HCI utilizes local storage, rather than sharedstorage such as a storage area network (SAN) and/or network attached storage (NAS);DAS can be local hard disks, flash storage, just a bunch of disks (JBOD) or dynamicrandom-access memory (DRAM);

• The software defined storage (SDS): Pools DAS into logical, abstracted virtual storage;

• Server virtualisation: Separates computing resources from the underlying hardwareusing a hypervisor or container technology;

• Data services: Provides built-in data services such as auto-provisioning, deduplication,replication, encryption, failure tolerance methods (redundant array of independentdisks (RAID));

• Central management interface: Supports both storage and computing resources;

• Network switching equipment: Provides connectivity to the HCI nodes and resourcesfor software defined data center (SDDC) to utilize.

CCDC’s HCI-based physical layer could allow for a vendor agnostic hardware4. Detaileddiscussions on the HCI can be found in the documentation [8, 12–14].

As will be discussed shortly, the hardware underlay would be managed, controlled andsecured from the software overlay. Such a design could provide significant capital expenditure(CAPEX) savings since users would not be bound to use just one type of hardware [12].

2.3.1 Hardware for unclassified S&T activities

Figure 5 illustrates the envisioned application of the CCDC in an unclassified environment.The figure, which is similar to Figure 3, shows that the hardware and software layersare not tightly coupled. The hardware represents the underlay solution. The softwarerepresents the overlay solution. Unclassified CCDC activities would require connectivity toa DND backbone network in order to support collaboration, training and demonstrationrequirements.

2.3.2 Hardware for classified S&T activities

The current CCDC concept does not support classified activities. It is envisaged that itsfuture enhancements would include such a capability. The envisioned application is similarto that shown in Figure 5, except that it would use the CTDC backbone network.

The CTDC would be an important component of the envisioned CCDC’s classified network.Remote connectivity to the CTDC would be needed as this facility would not be hosted onsite within the CCDC. Such connectivity would require certification to become operational.

4Vendor agnostic refers to coupling products from heterogeneous manufacturers. A vendor agnosticsolution can be implemented with any off the shelf hardware server.

9

Figure 5: Hardware platform for unclassified research.

2.4 The CCDC as a software-driven cloud

The CCDC concept is based on the VMware’s SDDC architecture [3]. A data centre isa facility composed of a large group of networked computer and storage servers used toorganise, store, process or distribute large amounts of data. The SDDC is an architecturalframework driven by software where all of its data centre domains (servers, storage andnetworking) are virtualised. It extends virtualisation concepts such as abstraction, poolingand automation to all of the data center’s resources and services to achieve IT as a service(ITaaS). ITaaS is an approach that treats IT services as a commodity and delivers it toenterprises for a fee [15].

A core capability of the SDDC is its policy-based automation that drives operationalefficiencies [3]. In contrast to manual tasks or script-based automation, policy-basedautomation utilizes defined rules and policies to ensure consistent deployment of services.This automation increases the reliability and resiliency of infrastructure by reducing manualtasks and potential human errors. Complex processes can be simplified and manual labor-intensive tasks that are prone to error can be entirely automated in software.

A key benefit of the SDDC is its ability to leverage the power of virtualisation for abstractingthe underlying physical infrastructure to create logical pools of resources that can beefficiently managed and secured. Resources, such as computing, networking and storage canbe created, replicated and backed up within an SDDC, improving infrastructure resiliency.Logical network constructs, such as firewalls, are reproduced in software and deployed on a

10

per VM level. This significantly improves the security posture of the deployed VMs andenables the provisioning of east-west5 network security traffic in the research environments.Some of the important capabilities supported by VMware software are described in the nextsections.

2.4.1 CCDC’s cloud catalog

VMware vSphere software can be used as a platform to run applications and/or VMs. Suchapplications, which can be packaged in a format called VMware vApp, can be configuredto run on top of VMware vSphere. VMware defines such a vApp as a logical collection ofpre-configured VMs. Each VM is separately configured with OSs, applications, networksand networking rules. The vApp shares functionality with disparate VMs allowing themto work together. It can be powered on or off and can be replicated if required. vAppscan be nested within other vApps. New vApps that are based on existing ones can also bedeployed. vApps can be nested within other vApps and can also be deployed as new vAppsbased on existing ones. VMware also defines a vApp template as a copy of a vApp thatcontains one or more VM images. It may also contain one or more vApp networks [16, 17].

VMware defines the concept of a catalog as a container for an organisation’s vApp templatesand media files that can be made available to users. The templates, which are sharableamong the organisation’s users, can be deployed as new vApps. The media files can beinstalled to existing VMs [8].

2.4.2 CCDC’s cloud blueprints

An important part of the CCDC’s virtual capabilities would be the concept of a blueprint [18,19]. As the name implies, VMware defines a blueprint as a fully drawn up specificationfor the provisioning and management of services such as computing, storage or networking.A blueprint can be created and replicated to define generic resources to be provisionedand used by the requesting users. It specifies each resource’s components, provisioninginformation and defines options available to the requesting user. Such options include VMspecifications or lease durations, which are lengths of time the user books to use the resource.Disparate blueprints are pooled together as catalog items. Once a blueprint is created, itcan be published as a catalog item that can be made available to users [7, 18–21]. Detailedsteps to create blueprints are outlined later in the document.

2.4.3 CCDC’s cloud entitlements

VMware uses the concept of entitlements to assign the usage of catalog items to users orbusiness groups [7,20,21]. Entitlements are equivalent to permissions, assigning access rightsto users or groups. As illustrated in Figure 6, the entitlements are created by administratorsand group managers.

5East-west traffic refers to communications within a data center or between servers.


Figure 6: An illustration of the assignment of entitlements to users and groups.

The figure shows how entitlements allow users or groups of users to access service categories(such as applications), catalog items (such as blueprints) and action items (such as reboot,power-off, destroy). The process of adding and removing entitlements is discussed indetail in Section 3. Additional information on entitlements can be found in VMwaredocumentation [7, 20–22].

2.4.4 CCDC’s “zero-touch” cloud automation

VMware’s vRealize Automation (vRA) implements automation through its software thatcould unify cloud management to provide for an agile, customizable, scalable and efficientCCDC [7,21,23]. The CCDC could implement this capability through a resource and timesaving “zero touch” approach. The “zero touch” automation setup is illustrated in Figure 7(reproduced with VMware’s permission).

The figure illustrates how the combination of the computing, networking and storagevirtualisation with cloud automation software could allow testbeds or experimental enclavesto be pre-engineered with design specification blueprints. The blueprints could then bepublished as catalog items that can be deployed within minutes with little or no manualintervention from the requestor. Researchers could monitor the status of their request froma provided web portal.

The traditional complexity of setting up experiments whereby researchers would waste asignificant amount of time waiting for IT services to be set up for an experiment could bereplaced by the CCDC’s “zero touch” setup. It could enable the setting up and tearing downof experimentation and testing infrastructure designed to be scalable and repeatable, andwould include the sophisticated tools and topologies needed to address DRDC’s advancedcyber R&D activities [7, 21,23].


Figure 7: CCDC could support a “zero touch” deployment for experimental testbeds.

2.4.5 CCDC’s cloud orchestration

VMware’s orchestration tool helps to automate the management of virtualised systems andprocesses from a single point [24]. VMware’s vRealize Orchestrator (vRO) achieves thatthrough the use of a workflow engine, which is a chain of tasks and decisions that can beexecuted sequentially [8]. These workflows, which vary in complexity, can be pre-built orcustomised to the environment. They simplify the automation of very complex IT tasks and,as will be explained later, they could facilitate the CCDC’s setting up and tearing down ofexperimental testbeds. In addition to the setup of vRO discussed later on in this report,more information can be found in the VMware documentation [8, 24].

2.4.6 The CCDC’s zero-trust and micro-segmentation

The classical defence approaches that mostly relied on perimeter protection to keep threatsat bay has been found to be mostly ineffective in handling new types of threats such asadvanced persistent threats (APTs) and coordinated attacks. One approach to this securityshortcoming is to remove the default assumption of trust from any network entity, includingusers, devices, services or network traffic. All entities and their interactions must always beverified through the use of micro-perimeters of control around a network’s important assets,leading to the term “zero-trust” security as it was introduced by Forrester [25, 26]. Thisconcept is illustrated in Figure 8(b).

In the figure, the comparison of a classical defence approach against the zero-trust approach isshown. The zero-trust approach’s micro-perimeters of control around a network’s importantassets ensures that networks that are automatically generated during deployment of therequired service blueprints do not allow for any cross pollination between the enclaves.

Another approach to address the perimeter-centric defence shortcomings is through micro-segmentation [25, 27, 28]. An example of micro-segmentation using vMware is illustrated


(a) Perimeter defence (b) Zero-trust. (c) Micro-segmentation.

Figure 8: A comparison of the classical perimeter defence approach against micro-segmentation and zero-trust(adapted from [27]).

in Figure 8(c). The figure shows how the traditional defence relied on perimeter defences. However, with vMware’s micro-segmentation, fine-grained security policies are assigned to devices, OSs, applications, workloads or other properties as shown in the figure. The micro-segmentation approach allows security implementations using software defined networking components.

2.4.7 Interaction with the physical layer

The CCDC’s SDDC is envisioned to use physical hardware devices as an underlay and transport mechanism. As mentioned earlier, all the management and control functions could be abstracted from the physical device and centralised in the software. Control decisions could be made on an up-to-date global view of the complete shared infrastructure. The CCDC infrastructure, which would not be entirely virtualised, could enable integration with non-virtualised physical devices. As a result, the CCDC could support virtual-physical (V-P) and physical-virtual (P-V) data flows.

The current vision of the CCDC experimentation and testing components would be provided by physical devices6. An example of this is the Ixia BreakingPoint traffic generator [29]. Although the CCDC’s SDDC could integrate with existing physical devices, it is important to note that physical devices lack the automation and orchestration capabilities required to allow complete buildup and tear down capabilities for experimental testbeds. An enhancement to the CCDC would be to virtualise the physical devices. In this example, the traffic generator could be virtualised through its replacement with the BreakingPoint Virtual Edition. Such

6Experimental networks under the CCDC could function as real enterprise networks, complete with network configurations and p rovisioning. Traffic flows, which could be monitored through sniffers or other traffic loggers, would be a true representation of real network tr affic. The only exception would be that the current CCDC concept would not have internet connectivity.


a replacement would then allow full automation and orchestration as part of blueprintedservice deployments that supports the setup and tear down capabilities on the sharedinfrastructure.

2.4.8 CCDC’s software packages

The CCDC software stack that could support the above capabilities includes the vCloudSuite, network virtualization and security platform (NSX), vSAN and Horizon View. Theseare the software that are available at the time of writing, which may change when thisCCDC vision is realised. The complete list of the individual products and services currentlysupported is listed in Table A.1.

2.5 Overall design layout of the CCDC

The overall layout of the CCDC is shown in Figure 9. The figure shows users in an operational

Figure 9: Envisioned CCDC’s unclassified network layout and configuration.

zone, network connectivity and the CCDC cloud infrastructure in a restricted zone. Thecloud infrastructure could host a number of experimental enclaves, e.g., Wireless and softwaredevelopment kit (SDK). It also shows the isolated management network that could be usedto manage the experiments and the cloud as a whole. The demilitarised zone (DMZ) andESG could facilitate user connectivity to the experimental enclaves using the VDI. Thewhole cloud management and control would be overseen by different policies as shown inthe figure.


From the figure, users in the operational zone could connect to the CCDC lab from physicallab computers or remotely through office desktops. Network connectivity could be added toS&T desktops that require access to the CCDC. Backbone connectivity to other networksis not yet envisioned in the current the unclassified CCDC. Future plans envision itsconnection to other networks such as the GPNet. This connectivity deficiency wouldrestrict collaboration and demonstration activities to the local S&T community. The HCIinfrastructure, which is illustrated in Figure 4, could provide a vendor agnostic hardwareplatform for all management and control to take place from a centralised software definedadministration interface.

The current CCDC design does not support classified activities. It is envisioned that furtherenhancements and policy changes would upgrade it to support S&T research at classifiedlevel II. The overall virtual infrastructure layout of the envisioned level II CCDC resemblesthat shown in Figure 9.

A component of the classified CCDC could be hosted in the CTDC which is connected toa backbone network. The classified (level II) CTDC would allow remote access through adesktop unit (DTU) which could be installed at any required DND location. This portionof the classified (level II) CCDC lab would be able to provide remote collaboration anddemonstration capabilities. The HCI and SDDC platforms could remain consistent with theunclassified CCDC Lab. The unclassified CCDC could provide a proving ground environmentfor experimentation before it is moved up to the Level II CCDC if necessary.

2.6 Communication between domains

DND’s technology roadmap includes cross domain solutions that will be approved throughthe Security Authorisation and Accreditation (SA&A) process to provide the Authority toOperate (ATO) on DND networks. Such authorizations could allow the CCDC to exportexperimental blueprints as code between its unclassified and classified (level II) instances.Until a solution is approved, the CCDC would have to manually move experimental testbedsfrom the low to high security domain. The vRA in the CCDC could allow blueprints to beextracted as code, which would allow the reuse of existing testbed designs established in theunclassified CCDC lab to be moved to the high side CCDC lab for further level II testingas illustrated in Figure 10.

2.7 Support personnel

Administration over the private cloud and users would be the responsibility of a CCDCLab Administration Team (LAT) or Cloud Administrator. These administrative roles of theCCDC are illustrated in Figure 11, which is reproduced with permission from VMware.

From the figure, the LAT would be responsible for creating VM, network and storagetemplates. They could also create blueprints based on service deployment requirements. Inthe current CCDC design, the team could be divided into three roles, a lab administrator,

16

Figure 10: Envisaged communication between unclassified and classified level II sites.

Figure 11: Administrative roles within the CCDC.


technical authority and researcher (marked red). The team could assign roles (e.g. adminis-trative), catalog services and infrastructure resources to business groups. The team couldalso track resource usage by users and could initiate a process to reclaim them. Granularroles could be possible through vRA, which could be used to provide additional or separationof duties within the CCDC [23]. A full breakdown of the vRA roles is available in theVMware documentation [20, 23].


3 CCDC as a flexible and scalable test anddevelopment environment

The CCDC’s vision is to provide a flexible and scalable development and testing environmentto support cyber S&T activities. This capability could be achieved through the adoption ofinfrastructure that is flexible and scalable to allow the rapid deployment and provisioning ofdevelopment and testing environments for cyber research.

This section presents the CCDC’s capabilities that support this CCDC vision as defined inthe original CCDC requirements [1]. The capabilities are as follows:

• Use of affordable commercial off-the-shelf (COTS) products;

• Infrastructure to re-use, revive and support interoperability across projects;

• Infrastructure to facilitate testing and experimentation;

• Infrastructure to facilitate training and demonstrations;

• Infrastructure to provide local desktop connectivity to the lab;

The presentation of each technology starts with its application, followed by its descriptionusing examples. Where applicable, its use in the creation, sharing and customisation ofdatasets is also discussed.

3.1 Use of affordable COTS products

The CCDC is envisioned to support the use of affordable COTS products7. This capabilitycould allow researchers to use a broad range of research tools within an environment. Thistechnology is envisioned to be provided through the VMware’s vCloud suite [8].

3.1.1 Application

DRDC’s cyber S&T staff need an environment that enables them to run experiments onnetwork models that can emulate Internet elements, enterprise and operational networkswithout endangering other research enclaves, operational networks or the larger Internet.Such activities can be supported by the integration of tools developed in-house throughresearch activities, acquired as COTS products or shared through partners such as theNorth Atlantic Treaty Organization (NATO) or the TTCP. The CCDC could support theserequirements through self-service automation that provides researchers with repeatableblueprinted snapshots of experimental infrastructure. That infrastructure could incor-porate VMs, configurations and networking services to utilise or integrate with multipleinstantiations of other COTS technologies.

7COTS products are considered affordable compared to custom products that incur significant developmentand maintenance costs.

19

Automated deployments of experimental testbeds, including preinstalled tools, allow re-searchers to conduct faster cyber security experiments than they would using classicalenterprise setups. For example, a VM with preinstalled Anaconda Python software libraries,which are a popular source of data science tools, could be made available for researchers touse in the CCDC [30]. Instead of a new installation and configuration of such research tools,researchers could simply deploy the preinstalled VM for their experiments, significantlysaving on time and resources. Thus, a CCDC integrated COTS software suite would becomea foundation for automation and flexibility of the test and development environment.

In the past, when researchers had to request new COTS test and development resourcesfrom IT, they had to wait for them to be provided (see Figure 7). Instead of taking thistime-consuming approach, the CCDC would use the VMware vRealize suite to create anew private cloud infrastructure that could provide timely delivery of such IT services ondemand [19].

One of the VMware tools, the vRA, provides development and test infrastructure through aself-service portal that allows new service requests and interactions with deployed servicessuch as VMs [19, 23]. The private cloud could allow researchers to get resources on demandas required, rather than wait for days or weeks for the request to be fulfilled. An example,presented in Annex A.2.1, walks through such a service deployment from a researcher’s ordeveloper’s perspective. That section also walks through the administrator’s interface andthe requirements to provision service deployments.

3.1.2 Description

The CCDC concept’s web based portal, which would be provided through the vRA, wouldenable researchers to deploy catalog items [19, 23]. It would also enable them to configureand interact with VMs, networks and the storage resources required to provide completeend to end service deployments. A service deployment could be as simple as a single VM oras complex as an experimental environment that emulates attackers and/or defenders on areal network topology.

Development groups could request a service deployment from the services catalog and thenmanage its lifecycle through the use of a “lease”. The lease could grant the developer controlover their entitlements within the virtual infrastructure for a specified duration. An approvalmechanism could be built into the user’s lease request workflow, which could be used to:

• Allow the scientific authority to control, through granting approval, the use of theshared infrastructure for experimentation;

• Allow auto approval of requests that remain with the default configurations;

• Disable workflows if desired.

A user portal could provide access control and entry into the CCDC as illustrated in Figure 12.From this portal, researchers would be provided with development and testing servicesthrough catalog items that administrators would have previously configured. The catalog

20

Figure 12: Access control and entry into the CCDC through a user portal. The figure shows a user accessing a catalog item that allows for the deployment of a specific blueprint.

items would enable researchers to deploy blueprinted development and testing environments. The deployments would be automated and orchestrated to make use of any of the SDDC’s computing, networking and storage components required to produce the required testing and development environment [3].

3.1.3 Miscellaneous COTS features

This section addresses additional technologies that could support the CCDC’s affordable COTS capabilities.

3.1.3.1 Secure multi user

The CCDC concept supports secure multi-user environments that allow researchers to safely test advanced defence mechanisms against “live” threats without endangering other research enclaves, operational networks or the larger Internet. SDNs would be automatically generated during the service deployment based on blueprint designs that include COTS products. All deployed enclaves would be, by default, isolated to protect data against the cross pollination. Controlled environments would be created using a micro-segmentation policy that uses a whitelisted security model [25, 28].

In the CCDC, A zero trust network policy would enforce a deny trust policy for all entities, including users, devices, applications and packets [25]. Zero trust boundaries

21

compartmentalize different segments of the shared infrastructure into isolated, self-containedexperimental testbeds. Using the CCDC’s testing framework, researchers could experimentwith a variety of COTS and shared technologies representing the network environment,including deployed defence technologies, attack behaviors and configurations [26, 31].

3.1.3.2 One-way access and internal data sharing

The CCDC concept combines the VDI with the NSX identity firewall (IDFW) to enforce asecure one-way traffic flow into the lab. As illustrated in Figure 13, the VDI controls thesharing of data between enclaves.

Figure 13: Secure one-way access from Operational Zones into the CCDC concept. Data leaks into other deployed enclaveswould beprevented by the VDI and the NSX micro-segmentation.

To illustrate this capability with an example, consider a researcher that develops a wireless experiment prototype in their SDK enclave and wants to deploy it in the wireless experiment enclave. The VDI would allow the researcher to move files b etween t he t wo enclaves. The VDI security policy would also uses micro-segmentation to control which resources or enclaves the researcher has access to. This policy and a perimeter firewall would ensure that the CCDC enforces a secure one-way access. Data loss prevention would ensure that the CCDC’s private cloud data is contained within the dedicated virtual infrastructure. The only external interface into the private cloud would be the VDI client configured to ensure that no data can be stored on external end point devices.

22

3.1.3.3 Cyber security experimentation at scale

The CCDC concept provides the necessary automation, infrastructure, networks, tools,methodologies and supporting processes to enable the S&T testing of emerging and advancedsecurity technologies at enterprise scale. Enterprise scale emulation would be achievedthrough the use of virtualisation and software defined network and storage technologiesacross the computing, networking and storage layers of an enterprise network.

Techniques such as SDNs and network cloning would be provided through the virtualreplication of network services such as firewall, load balancing, virtual private network (VPN),network address translation (NAT), etc. They would allow cyber security testbeds to providesimilar accuracy and dependability to large cyber farms8. But, they would avoid the expenseof large amounts of hardware and software maintenance costs, or the power, space andventilation costs that is a characteristic of cyber farms [32, 33].

3.1.3.4 Security domains

Separate security domains, as illustrated in Figure 14, would require the deployment ofcompletely separate CCDC infrastructures, leading to unique instances of the CCDC. Theseparate CCDC private cloud infrastructures would be deployed on HCI that could belocated within the required unclassified or classified security domain per data center. Allother configuration and deployment capabilities described above would function in eithersecurity domain.

3.2 Infrastructure to reuse, revive and support interoperabilityacross projects

Classical experimentation infrastructure is limited to providing support to only one exper-iment at a time. The infrastructure is tied to an experiment for the lifetime of a projectand may be discarded at the end of the project. Fortunately, the CCDC concepts providesa capability that could reuse and revive interoperable infrastructure across experiments.This capability could support S&T activities while savings on resources and time could beachieved if infrastructure could be re-used by other experiments or researchers. Instances ofcomputing, networking and storage resources could be made available for re-use when theyare not tied to ongoing activities.

3.2.1 Application

Researchers need a capability that allows for the easy archiving and reviving of experimentswithout tying down infrastructure that could be used by other activities. In addition, suchinfrastructure should support interoperability to allow instances of computing, networkingand storage resources to be used in different combinations of service deployments. Researchers

8Large cyber farms often consist of thousands of computers that have high power consumption requirementsneeded for running and ventilation. At the optimum performance level, a server farm has enormous costs(both financial and environmental) associated with it.

23

Figure 14: CCDC security domain deployment. A separate instance of the CCDC would be required per security domain.

could also benefit from the ability to share data among deployed projects through data shares that protect against data cross-contamination across project enclaves.

24

3.2.2 Description

The CCDC concept realises researchers’ needs through a capability that could allow forthe following: infrastructure re-use, interoperability, data sharing and the archiving andreviving of experiments. This section describes each of these capabilities as realised withinthe CCDC.

3.2.2.1 The CCDC concept for infrastructure re-use

The CCDC concept has the capability to simultaneously deploy and re-use distinct experi-ments on a shared infrastructure. This setup, which is illustrated in Figure 15, represents ascenario where the physical infrastructure is being treated as an underlay of resources forconsumption and re-use.

Figure 15: CCDC shared infrastructure reuse.

In the figure, the CCDC infrastructure is treated like a shared pool of resources. When anexperimental enclave is deployed, the infrastructure requirements would be removed fromthe available pool and be dedicated to an experimental instance. That instance would becontrolled through the metering of resources based on design specifications contained in theblueprint. Multiple instances of a research enclave could be deployed as required providedresources would be available from the shared infrastructure pool [21].

25

Reservations could allow the infrastructure to be partitioned and metered into allocatedresource pools to ensure controlled experimentation. The automation framework couldinclude end to end lifecycle management of the infrastructure, which allows for eitherscheduled or on-demand recapture of resources based on a lease concept. As soon as aresource instance is released, the resource pool would be available for re-use by anotherinstance of an experiment.

For the re-use application, the number of experiments that could be run at any one timewould be limited by the underlay capabilities [21]. In a purely virtualised environment, there-use of computing, storage or networking infrastructure would be limited to the capabilitiesof that infrastructure. For example, a service deployment experiment that would be acombination of physical and virtual infrastructure must ensure that multiple instances wouldnot be deployed simultaneously. This is because the physical infrastructure would notsupport multiple instance deployments and could skew experimental data.

3.2.2.2 The CCDC concept for interoperability

Infrastructure interoperability is achieved through the creation of service deploymentsbased on modeled components that could consist of blueprints, software components oranything as a service (XaaS) (custom IT services). Software components and XaaSwouldbe used by blueprints and XaaS could be independently exposed as published catalog items.Multiple instances of a blueprint could be used in different experiments and could also beindependently created or destroyed if needed [23, 24].

The experimental and SDK service deployments would require changes to existing blueprintsas new versions of software are developed and life cycled, new datasets are captured,components are added or removed, and as infrastructure requires updates. Updates, suchas VM sizing, security settings and network topology changes, could be rolled out asnew service deployments or availed to existing deployments. This process could involveestablishing a release process where blueprints would be tested prior to publishing in theservice catalog [23, 24].

As illustrated in Figure 16, vRA and vRO allow complete workflow control over day-29

operations that would ensure interoperability across templated and deployed instances [8,21, 23, 24]. In the figure, the workflow control shows how day-2 operational workflows couldbe designed using custom scripts. Scripts could be designed to update virtual machines orguest operating systems as required.

3.2.2.3 The CCDC concept for day-2 data sharing

As stated earlier, instances of experiments or service deployments run in securely separatedenvironments to ensure there is no cross pollination of data. An example of this separationis illustrated in Figure 17. The figure shows that enclaves would never communicate directly

9Day-2 operations refer to instances when the system is operational and functioning normally within theenvironment.

26

Figure 16: vRealize Orchestrator workflow control.

Figure 17: CCDC’s controlled data sharing on shared infrastructure. Although experiments share the repository, data is prevented from crossing enclaves to prevent interfering with

other CCDC activities.

because they would have been isolated by design. Experiment 1 could be running a wireless experiment and Experiment 2 could be running a separate malware analysis experiment. There is no direct communication among the enclaves and experiments such as malware analysis that could infect other experiments would be completely contained. Any required updates could be pushed and pulled from a shared repository as shown in the figure.

Interoperability between SDK and experimental service deployments could also be achieved by allowing SDK deployments to push software updates into the shared repository. All

27

deployed experiments could then access the read-only shared repository for manual updates.This setup could be accomplished by creating a shared repository security zone that controlsthe flow of shared data across deployments [21].

3.2.2.4 The CCDC concept to archive or revive projects

The CCDC lab administrator could remove projects from the activity service catalogue bydeactivating existing entitlements from the users. The archived service deployments andblueprints would be backed up and could be revived by re-entitling them. Blueprints couldbe exported as code, which could allow archived components to be broken down into buildingblocks for re-use in future projects as needed. Blueprints could also provide a means ofsharing testbed designs across security domains such as classified or unclassified enclaves.

3.3 Infrastructure to facilitate testing and experimentation

The CCDC concept could provide researchers with infrastructure that could facilitate testingand experimentation. The classical testing and experimentation infrastructure requiressignificant time and resource commitments to efficiently support advanced research activities.

3.3.1 Application

Researchers could benefit from infrastructure that could speed up the emulation of simpleto complex enterprise networks for advanced network research. They could also benefit frombeing able to save, share and later redeploy instances of experiments, thus saving on timeand infrastructure.

One challenge in advanced cyber security research is the ability to test and experiment onmalicious network activities without risking them spilling over to other research enclaves orthe Internet as a whole. Researchers could benefit from infrastructures that provide testbedsthat would be isolated from each other. Such infrastructure could prevent experimentsfrom cross-contamination and facilitates controlled testing and experimentation. Micro-segmentation could further prevent experimental data or activities spilling to the Internetas a whole.

3.3.2 Description

The CCDC concept’s private cloud is built on a software defined architecture that facilitatestesting and experimentation through blueprinting testbeds. The testbeds could range fromreplicating complex enterprise network designs to small implementations such as providing afew VMs for simple experiments. Multiple enclaves could be provisioned and deployed thoughthe NSX technology supported by the CCDC [34–36]. Micro-segmentation, as describedearlier, could ensure that testbeds would be isolated from each other, preventing cross-contamination and facilitating a controlled testing and experimentation environment [25, 28].vRealize could provide the cloud management platform and automate blueprinted IT

28

deployments for enclaves, supports development capabilities and monitors and reports oninfrastructure resources [23].

A significant part of the design of testbeds could involve the ability to provision separatenetwork topologies and enforce security between the networks. The key technologies tosupport this capability in the CCDC concept are as follows [34–36, 36]:

1. The NSX ESG provides secure access and protection to isolated networks. The NSXprovides a replication of the following data center network services:

• Routing and NAT;• Firewall;• Load balancing;• VPN;• dynamic host control protocol (DHCP)/domain name system (DNS) relay.

2. The NSX distributed logical router (DLR) provides east-west traffic routing.Logical routing enforces security by providing isolated network enclaves that would bebased on defined security policy. Such routing could improve performance by removinghairpins to the physical layer. The DLR enables each vSphere hypervisor host toperform layer 3 (L3) routing between virtual and physical subnets in the kernel at linerate.

3. The NSX distributed firewall (DFW) and identity firewall (IDFW): TheNSX centralized logical firewall, which inspects north-south traffic10, could residewithin the NSX ESG and protects east-west traffic. NSX DFW is directly connectedto virtual machines and ensures traffic and security policieswould be decoupled fromInternet protocol (IP) addressing.The IDFW could control network access associated with a user’s identity. For example,from the VDI infrastructure, the IDFW could enforce a zero trust white-listed modelbased on a user’s identity [10, 11, 25, 26].

Application of these capabilities are presented through the following illustrated examples inSection A.2.3.

3.4 Infrastructure to facilitate training and demonstrations

The CCDC concept’s infrastructure is designed to facilitate training and demonstrations insupport of S&T activities.

3.4.1 Application

Cyber security researchers have a regular need for environments that support formal andinformal training and demonstrations. They need a demonstration capability to share project

10North-south traffic refers to client-server communications.

29

outputs and technologies with colleagues, clients and partners. Such a capability allowsresearchers to engage with potential capability users in order to solicit feedback that can beused to improve the work.

Instead of taking away significant time and resources to set up training and demonstrationinfrastructures, researchers could benefit from a capability that enables them to easilyand quickly deploy unique service deployments of training or demonstration environments.Research groups such as the COSW Section of DRDC could, for example, have a regularrequirement to perform technology demonstrations to a number of CAF clients. The CCDCcould allow for that number of unique service deployments of the same CAF demonstrationenvironment to be simultaneously deployed on the shared infrastructure with completeisolation between each environment. When a training or demonstration session is completed,researchers could also benefit from the ability to save and later redeploy instances of trainingor demonstration service deployment environments.

3.4.2 Description

The CCDC concept provides S&T staff members with a cyber training environment, capableof supporting multiple users at a time. The environment could be formalized into a blueprintof VMs, software tools, datasets, storage and network security requirements. The trainingenvironment could be configured with the appropriate entitlements to provide a controlledenvironment with role based access providing the desired level of access per user.

3.4.2.1 The CCDC concept for demonstration capabilities

The demonstration capability could be provided through two options, the formal andinformal, as discussed below:

1. Informal demonstrations: An R&D demonstration option could be accomplishedthrough the publishing of artifacts in a shared repository as shown in Figure 18. Fromthe figure, the shared repository could be accessed by other research enclaves in theCCDC. Artifacts could be downloaded and reviewed to provide feedback. This couldbe considered an informal demonstration11 capability that could be achieved throughcontrolled sharing of artifacts.

2. Formal demonstrations: A formal S&T demonstration option could be accom-plished through established baseline blueprints that have been formally tested toensure repeatable results. Such results would be available for review and feedbackfrom a broader community of interest. The formalized blueprint could be publishedthrough a demonstration service deployment offered through the vRealize CatalogService Items [19], which could be available from the web-based portal as illustratedin Figure 19. External access could be granted, through the backbone network, to anytrusted parties that would host the CCDC.

11Formal demonstrations are given to the CAF client and project sponsors as formal deliverables. Informaldemonstrations are given to follow researchers and any other interested parties as capability showcases.

30

Figure 18: CCDC informal demonstration capability. Updates could be pushed and pulled through a shared repository mechanism.

Figure 19: CCDC formal demonstration capability. A Service deployment would be pre-engineered to provide a formalized demonstration capability that could be

deployed as requred.

3.4.2.2 The CCDC concept’s Training capabilities

The CCDC supports both formal and informal training capabilities similar to those presented in demonstration capabilities. Training blueprints could be created and deployed. Created training blueprints could be published through the vRealize Catalog Services Items as shown to the left of the demonstration capability in Figure 19. The selection is also highlighted in Figure 20, an image clip from the catalogue services window similar to Figure 19. This example provides a formalized training capability through a pre-engineered service deployment. A specific service catalog could be created to filter the DRDC-CCDC training enclaves, as shown in the bottom left vertical tabs.


Figure 20: CCDC training capability.

3.4.2.3 Automated access control for transient demonstration and training

The user-base requiring demonstration or training capabilities could be dynamic. Usersmay require access for a period of training or demonstration and then access is removed.Manually adding and removing users to and from active directory (AD) groups within thelab, as shown in Figure 21, is a good example of a management and operations processthat could be automated through custom resources available through the vRA and vROcapabilities [8, 21,23,24]. AD capabilities would be provided through an embedded plugin

Figure 21: CCDC dynamic access control through XaaS capability.

made available through vRA’s custom resources. This could make adding and removing usersa process that could be automated and delivered through a service deployment entitlementthat would be based on role access.

A configured custom resource such as the AD could be used by XaaS blueprinting. CCDCadministrators could then create blueprints that provide service deployment catalog items.This could allow researchers to add and remove transient users that require training ordemonstration capabilities.

XaaS allows repeatable programmable tasks to be automated through the use of blueprints.The CCDC could design XaaS blueprints for common tasks, such as creating a branch in arevision control system when performing development tasks or for automating access control(creation and deletion of temporary user accounts), for training and demonstration servicedeployments. Such an implementation is shown in Figure 22.

The steps taken are as follows (with numbering as reflected on the figure):

1 On the left hand side menu, select XaaS blueprints in the Advanced Services menu;

32

Figure 22: CCDC XaaS blueprint capability.

2 Press the + New button next to the XaaS blueprints header.

Once the required AD configuration information is provisioned in the blueprint, an entitlementcould be published and made available to the users. These users could then be responsiblefor providing access to demonstration or training entitlements. Figure 23 shows an exampleof such a blueprint. In the figure, the Create an AD User (circled in red) has been published

Figure 23: CCDC published catalog XaaS service.

as a catalog item. XaaS provides a plugin for AD services.

3.5 Infrastructure to provide local desktop connectivity to the lab

The CCDC could provide researchers with a capability to connect to the lab from theiroffices or conference rooms. Such a capability could remove the possibility of congesting thelab with users. It could also help manage possible usage clashes when multiple researchersneed to use the same lab infrastructure at the same time.


3.5.1 Application

Researchers need to access and execute their experiments or conduct demonstrations andtraining from their desktops. They could also require to carry out demonstrations andtraining sessions in on-site conference rooms though local lab connectivity. The researcherscould, therefore, require local desktop connectivity to the CCDC.

3.5.2 Description

Currently the unclassified CCDC is envisioned to be located on campus at Shirley’s Bay wherethe researcherswould be mostly located. The unclassified site would support connectivitythrough local desktop network drops wired directly to the CCDC infrastructure. Connectingto the CCDC from a local desktop would make use of a VDI client that ensures a secure oneway access to the lab [10, 11]. The current unclassified CCDC concept has no connection tothe DND backbone network. The longer term vision would require the unclassified CCDCto be connected to a backbone network such as the GPNet so as to provide support forexternal collaboration with partners and other government departments (OGDs).

The classified (level II) CCDC could use the CTDC lab, which would not co-located withDRDC operational users. The classified CCDC instance site would support limited localdesktop connectivity from the DRDC campus through a remote access VDI. The CTDCwould provide the VDI infrastructure required to connect to the CCDC. The CTDC providesremote access capabilities from DND controlled classified (level II) networks which wouldallow further collaboration and demonstration capabilities.

3.5.3 General examples

Desktop connectivity to the unclassified CCDC is illustrated in Figure 24. From the figure,researchers could connect to the CCDC from their desktop environment using a VDI-basedclient through the ESG. Once connected researchers could access the CCDC web portalto manage or request new deployments or connect to any of their deployed experiments.If connectivity to a different security domain is desired a different VDI instance would berequired. It should be noted that the user cannot connect to a higher security domain fromthe unclassified instance of the CCDC.

Researchers could, however, connect to the classified (level II) CCDC from their classified(level II) desktop environment with a VDI-based client as illustrated in Figure 25. Onceconnected, researchers will be able to perform activities similar to the those described underthe unclassified connectivity if such activities would be available.


Figure 24: Local desktop access to unclassified CCDC. The figure also shows the connectivity policies that could allow connectivity while ensuring data separation among the different

enclaves.

Figure 25: Local desktop access to CCDC (Level II).


4 The setting up and tearing down of experiments,tests and demonstrations

This section presents the CCDC’s vision of a capability to set up and tear down experiments,tests and demonstrations. This vision could be realised through automation and orchestra-tion technologies that could be provided though the use of a software defined computing,networking and storage virtual infrastructure. The capability is presented in this sectionthrough the following topics: application, discussion and examples.

4.1 Application

In an R&D environment, researchers need to easily set up and tear down experiments andmanage their infrastructure at the end of the projects’ lifetimes. Such a capability couldbring the following benefits to the researcher:

• Lab support could be easily available (e.g. for setting up and tearing down of toolsand experiments);

• Mechanisms to easily migrate experimental instances of computing, networking andstorage resources for use in other unrelated projects;

• Mechanisms and processes to move or copy data, tools and technical models to otherexperimental environments such as the TDC, or with partners such as through theTTCP MASA project;

• Reduced dependency on contractors;

• Facilitate repurposing of hardware and software;

• Provide common and familiar testbeds.

The CCDC’s private cloud could provide researchers with these benefits through self-serviceand automation for the lifecycle of experiments and the test infrastructure. On-demandsecure virtual testbeds supported by these capabilities could enable cyber security researchersto run experiments on networks that emulate Internet elements, enterprise and operationalnetworks. Repeatable blueprinted snapshots of complete infrastructure setups that includesVMs, configurations and networks, could allow multiple instantiations of uniquely designedexperimentation environments. Automated deployment of experimental testbeds, includingpre-installed tools, could allow researchers to expedite experiments to quickly meet S&Tobjectives. As explained in the previous section, a lease concept could reserve deploymentsuntil an expiration period at which time the reserved resources would be recaptured andmade available for the next lease.

4.2 Description

The CCDC could provide the benefits listed above through a resource and time saving “zerotouch” approach, which is illustrated in Figure 7. The automation of “zero touch” setup


and tear down of experimentation and testing infrastructure would be the responsibility ofthe CCDC’s LAT. Before automation can take place the CCDC private cloud should bepopulated with templates of licensed OSs, software applications, VMs, networks, securityzones and orchestration scripts. Through integrated monitoring systems, the LAT would ex-ercise management oversight to the infrastructure’s performance, capacity and configuration.They could use analytics to provide the knowledge and visibility required to proactivelyensure healthy service levels in the virtual and physical environments.

Before vRealize can be used for automation, the shared infrastructure must be organized intousable segments [20]. This is accomplished by first establishing endpoint resources, whichare anything that vRA needs to complete its provisioning processes. In the CCDC’s case,these resources could be vSphere, vCenter and NSX [16, 35]. Fabric groups could provide away of segmenting the endpoints into different types of resources or to separate them byintent. A business group could then be established to associate a set of resources with a setof users. Business categories could allow groupings based on the type of user consumingresources in the CCDC, for example researchers, training, demonstration, external partnersand OGDs.

Reservations could be used to grant a percentage of the fabric group resources to a businessgroup. Once the infrastructure is ready for use, the LAT could design blueprints basedon the scientific experimentation and testing requirements. A building block or modularapproach should be taken to ensure re-use across current and future deployments. Blueprintswould then be made available to the users through entitlements. For example, researcherscould use entitlements to create snapshots of their VMs. They could use those snapshots ifthe components of their environment break and they would wish to rollback. They couldalso extend their lease period or destroy their service deployment should they no longerrequire it.

An optional approval workflow could be automated to ensure compliance and to helpestablish control over the self-service deployments. The LAT could ensure that day-112

deployments of experiments are backed-up and available should an experiment require a“break or fix” scenario as a result of a test. This would only allow the required componentsto be replaced, rather than a new instance to be deployed. At the end of a lease period, theservice deployment would be destroyed along with any instances of its snapshots and theresources would be made available in a shared pool. At any point in time during a lease,data could be captured from an experiment and updated or repurposed for future testingthrough snapshots and cloning.

4.3 Application example

The application of this technology is described through an example. Entitlements are a keymechanism to enabling end-user self-service over the setup and tear down of experimentation,testing and demonstration infrastructure. As explained in Section 3, new entitlements

12Day-1 refers to the setup, installation and configuration of a system in the environment. This is alsosometimes referred to as setup and commissioning of the system.


are located under the Administration tab → Catalog Management → Entitlements options.Click the “New” button to add a new entitlement as shown in Figure 26.

Figure 26: The CCDC’s entitlements configuration. This configuration controls user access to deployed catalog items and the day-2 self-service options provided to the researchers.

Under the General tab, enter a name for the entitlement and a description. Change the status to “Active” and select a business group. Business groups allow role based groupings to catalog service items. The CCDC would require business groups for roles such as experimenter, collaborator, demonstrator and trainee. Note that if only a single business group is created, this selection would not be possible since it would default to the only available group. Then select the users that would be part of this entitlement as shown in Figure 27.

Figure 27: A new CCDC entitlement for user groups. The dialog maps entitlements toaccess control using information sources such as AD groups. Expiration dates can be set toremove an entitlement. The status determines whether an entitlement is currently active ornot.

Next, under the “Items & Approvals” tab, the CCDC LAT could pick the objects the user(s)would have access to on day-2 operations. Once the entitlements are set up the blueprintwould then be published into the services catalog and made available to the researchers


for deployment. The destruction or tear down could either be manually performed by theresearcher or it could happen automatically when the lease expires.

Figure 28: CCDC entitlement options. The administrator could control which entitlement options are enabled for the researchers. These options could allow researchers to control the

day-2 operations in which an enclave is deployed.

4.4 Miscellaneous functionalities: Portability (Blueprintsexported as code)

For the automation of administration tasks, the CCDC LAT could also use automationand orchestration through workflows and vRealize Operations (vROp) [19, 22]. This couldensure efficient setup and tear down capabilities of the shared infrastructure and whennecessary reclaim unused resources in the infrastructure as shown in Figure 29 (reproducedwith permission from VMware).

From the figure, vRO could automate administrative tasks and allow resources to beautomatically reclaimed based on the service deployment policies such as inactivity andexpiration of lease time. The vRO could also provide intelligent alarms that would allowadministrators to identify bottlenecks and health alerts to help improve the performance ofthe shared infrastructure.

Portability of experimental blueprints between unclassified and classified instances of the


Figure 29: CCDC concept’s automated reclamation of resources. The image shows the resource reclamation workflow.

CCDC could be made possible by exporting the blueprint design as code. Programmatically exported content from one vRA environment could be imported into another instance by using the vRA API or by using the vRealize CloudClient.

For example, the CCDC could develop and test blueprints in an unclassified environment and then move them up to a classified ( level I I) e nvironment o r i mport t hem i nto the classified (level I I) vRA i nstance. This is i llustrated in Figure 3 0. Clone template data must be included in the package.


Figure 30: CCDC could provide portability of blueprints. Blueprints could be exported as code and transferred between unclassified and level II CCDC instances.


5 Project lifecycle management

This section presents the CCDC’s capability that could allow researchers to focus theirattention on the design, testing and evaluation efforts over projects’ lifecycles, rather than theprocurement (or licencing) and manual configuration of infrastructure per experiment. Asdiscussed here, the CCDC’s private cloud could meet that capability by providing re-useablebuilding block components (provisioning, operations and management, maintenance andoptimization) within the project management lifecycle.

5.1 Application

To effectively manage their projects, researchers need an agile platform that facilitatesproject lifecycle management. Researchers could gain the following lifecycle managementbenefits from such a capability:

• Provisioning could be facilitated by providing an automation framework for on-demand self-service access to the necessary infrastructure resources and pre-templatedblueprints required for experimentation and testing. Micro-segmented experimentationon shared infrastructure and the re-assignment of resources based on priority of taskscould also be supported.

• Operations and management could be supported by providing monitoring andmanagement of physical and virtual resources. This could allow operational processesto be unified and standardized across the CCDC. Real-time event-driven processautomation and job scheduling, backup and restore capabilities are supported.

• Maintenance could be enabled by providing continuous license management andpatching from a centralized management interface. The CCDC could provide for aninfrastructure that supports dynamic maintenance and updates to deployed enclaves.

• Optimization could be provided through a development operations environmentthat automates the process of software delivery and infrastructure changes. Theenvironment could also support the exploitation of developed capabilities, re-usedshared infrastructure, rolled-back capabilities, streamlined compliance and securityhardening and the automation and orchestration of computing, networking and storagefunctions. It could also enable physical to virtual integration of different technologiesthat support advanced research activities in cyber security.

• Management of old projects could be provided in the following ways: infrastructurefor a completed project could be released for use by other projects; completed projectscould be saved as code and revived whenever there is need to run them again, nophysical infrastructure would be tied up to an old project as is the case with theclassical approach of acquiring experiment-specific resources that become disused atthe end of the project.


5.2 Description

Converged blueprints could provide the provisioning of infrastructure resources for end userexperimentation. The blueprints could be assembled from reusable virtual componentsacross the computing, networking and storage software defined layers. NSX could allowdeployed blueprints to be placed into micro-segmented security zones for multidisciplinaryexperimentation on a shared infrastructure, preventing cross-contamination [8, 21, 23, 24].

Micro-segmentation, combined with secure one-way access into the CCDC, could allowtesting and evaluation of advanced defence mechanisms against “live” threats withoutendangering other research enclaves, or operational networks [25,28]. Automated provisioningof resources could allow for a large-scale13 experimentation capability, as well as a safemeans to qualitatively evaluate cyber security solutions against actual malware and otherthreats in a realistic environment. Converged blueprints could provide reusable designs forscience-based hypothesis testing.

The vROp could provides the Operational and Management toolset for the CCDC [19,22]as illustrated in Figure 31 (reproduced with permission from VMWare).

As shown in the figure, vROp could ensure quality of service, operational efficiency as wellas control and compliance. The figure shows how the vROp could provide a managementconsole with complete visibility across computing, storage and networking infrastructure. Italso provides predictive analytics, health alerts and the remediation of emerging performance,capacity and configuration issues. Policy-based automation could allow custom enforcementactions to proactively address infrastructure issues.

The vRA could provide “backup and restore” functionality through entitlements of on-demand snapshots or day-1 “backup and restore” capabilities of the initial blueprint compo-nents [8,21]. This could allow for “break or fix” experimentation without having to redeploycomplete enclaves. The CCDC’s vSphere data protection, through its backup/restorefunction, could allow for the following [16, 35]:

• Backup and restore VMs;

• Store data according to organizational policies;

• Inform administrators about “backup and restore” activities through reports;

• Backup of individual components that is required for vRA and NSX.

Centralized licensing for the shared infrastructure could include the software defined datacenter components, guest OSs and any other COTS product licensing that might be usedthrough the virtual infrastructure. Patching could be handled centrally and dynamicallypushed out to enclaves through the VDI infrastructure [10,11], shared repository and throughconverged blueprint updates.

13The size of these large-scale networks are determined by the available hardware implementing the virtualenvironment.


Figure 31: vROp Operational and Management toolset. The vRO toolset is focused on ensuring quality of service, operational efficiency and control and compliance over the SDDC

shared infrastructure.

Physical infrastructure resources could be acquired and utilized though the NSX Edge, which would allow re-use and optimization of existing toolsets such as traffic generators or wireless devices [34–36]. An example of this setup is shown in the wireless experimentation presented in Section A.2.3.2.

5.3 User interface and display example

VMware’s vROp has analytic capabilities that could be used to monitor the CCDC’s IT private cloud infrastructure based on defined p olicies f or t he e nvironment [19, 2 2]. For example, a default policy, created during the installation of vROp, would delete all snapshots after a 180-day retention. Similar policies designed for workload monitoring could be configured based on specific resource metrics as those shown in Figure 32.


Figure 32: vRO centralized console propagates up events based on policy.

The figure shows a centralized console view with a heat map that could allows user todrill down into issues based on a configurable policy framework [19, 22]. The managementinterface could also allow control over the 1 configuration of threshold settings and 2selection of resources to monitor. The centralized console could provide a global view acrossthe virtual and physical infrastructure and could be summarized by categories such asHealth, Risk and Efficiency (as shown in the bottom half of Figure 32.

The optimization of the shared resources pool, such as CPU, memory, and storage, couldbe achieved through concepts such as the over-commitment [37]. Over-commitment occurswhen the physical central processing unit (CPU) resources are exceeded by the totalconfigured virtual CPU resources of all powered-on VMs. When the virtual infrastructure isover-committed, vROps would institute a fair redistribution of physical resources amongpowered-up VMs. Monitoring could allows the scheduling of both CPU and memory inorder to prioritise resources to the most needy VMs, while taking them away from VMsthat need less.


6 Classified (level II) and other future CCDCcapabilities

This section presents the CCDC’s level II capabilities as well as other technologies thatcould be of further benefits for conducting S&T activities in cyber security. As discussedhere, the software defined computing, networking and storage infrastructure supporting theCCDC concept could be extended to provide these added benefits if there were no limitingDND/CAF security policies.

6.1 Level II processing

To support researchers’ need to undertake S&T activities classified at level II, the CCDCaimed to support that through its level II processing capability. Such a capability wasenvisioned to be realised through the following level II collaborative activities:

1. Telecon/video teleconference (VTC) discussion;

2. Sharing data and information;

3. Collaborative testbed.

6.1.1 Application

The current CCDC concept does not support this capability. But researchers still need tocarry out a variety of simple to complex research experiments at level II. Such experimentswould require setting up and processing cyber security experiments covering computing,networking and storage infrastructures. The most common applications would be tosimultaneously conduct computations, discussions and documentation at level II.

The environments would allow classified (level II) discussions by telephone or VTC in away that would not be possible through classical telephone discussions. The complexityof sharing data at level II could be alleviated through the CCDC’s level II processing. Itwould also be possible to run collaborative experimental testbeds to support common goalssuch as the mission assurance activities under MASA.

6.1.2 Discussion

Extensions to the current CCDC concept could include a classified (level II) CCDC instancethat would share the same HCI and SDDC platforms used in the unclassified CCDC.Experimental testbeds developed in the unclassified CCDC could be moved up into theclassified CCDC environment by exporting data and manually moving the information intothe classified CCDC lab.

The envisioned CCDC level II processing capability is illustrated in Figure 33. The figureshows CCDC’s possible multiple network security zones. Network security zones could


Figure 33: Envisioned CCDC level II processing capability.


ensure that research testbed enclaves are isolated per enclave to ensure separation of dataand research activities. NSX’s micro-segmentation capabilities could be used to providea zero trust white listed security model. The management of the virtual infrastructurecould be assigned to a management restricted zone (MRZ) as shown in the figure. TheNSX’s IDFW, integrated with the VDI, could provide a dynamic perimeter zone interfacepoint (ZIP), where AD groups could associate user identities to logical security zones. Thiscould ensure that a whitelisted security model is enforced for user access into the CCDC.

Due to the nature of the CCDC lab research enclaves, remote access to the CCDC lab wouldneed to be highly secured. This could be achieved through a secure one-way access, whichcould be provided through the use of VDI and NSX policy. A ZIP combined with the VDIcould ensure that data is contained within the CCDC. The CCDC’s private cloud data lossprevention could ensure that data is contained within the dedicated virtual infrastructure.As shown in the figure, the only external interface into the CCDC would be through theVDI client configured to ensure that no data can be stored on external end point devices. Inaddition to the usual level II processing policy measures, additional security measures suchas the prevention of USB redirection or remote desktop connectivity should be enforced.

Figure 34: Envisioned CCDC classified (level II) collaboration capability.

6.2 Connecting to other level II networks

To effectively support the level II processing discussed above, connectivity to other classifiednetworks is very important. The connectivity could enable S&T activities with other DRDC


research centres, OGD and international partners such as 5-Eyes or NATO.

6.2.1 Application

Researchers require level II connectivity to carry out activities at classified levels. Fordemonstrations and training, researchers need to remotely connect with collaboratingpartners and to share results or receive feedback. In some instances, researchers need “live”leveraging of other international researchers’ capabilities without installing instances ofthe capabilities in their countries, which comes with its own sharing complications. Forinstance, researchers could remotely connect to Automated Computer Network Defence(ARMOUR) [38,39] but cannot install an instance of that framework in their own labs. Withthe envisioned CCDC connectivity, researchers would be able to achieve that up to classifiedlevel II. That way, remotely collaborating partners could review each other’s technologieswithout having to install their instances in their labs. With remote connectivity, researcherscould participate in or observe joint exercises such as Cyber Defense Exercise (CDX) [40].

6.2.2 Discussion

As illustrated in Figure 25, the CTDC could provide a backbone network supporting remoteaccess capabilities to the CCDC from Shirley’s Bay and any other DND sites that requireaccess to the CCDC for collaboration activities. Access could support both local andinternational partnerships.

6.2.2.1 Local partners

As originally envisioned, the CCDC could support local partners in OGD, industry andacademia [1]. Connectivity with OGD could be achieved through the CTDC using asecure one-way access supported by the use of VDI and NSX policy as described above.Collaboration activities with industry and academia could be supported through connectivityto the CCDC using other level II spaces such as Bell’s Test and System Integration Lab(TSIL). Such connectivity could support demonstrations or collaboration activities thatrequire level II processing supported by the CCDC. Similar to the OGD connectivity, VDIand NSX policy would facilitate the connectivity to the enclaves as illustrated in Figure 25.

6.2.2.2 International partners

The CCDC could also facilitate collaboration with international partners such as 5-Eyes orNATO through the CTDC connectivity to the CFXNet14 or CFBLNet [41].

6.3 Other future capabilities

The CCDC’s architecture is designed to be flexible. Such an architecture can, therefore,adapt to future capability extensions such as the following:

14The Canadian Forces Warfare Center (CFWC) maintains the connection between the CFXNet theCombined Federated Battle Laboratories Network (CFBLNet) through Fort Meade.


1. Expansion: The HCI is a scaled architecture of what is being used by major cloudproviders. When the HCI is combined with the SDDC, more hardware nodes canbe added to the architecture as needed. Such an expansion could provide for moreprocessing power for software defined computing, networking and storage infrastructure.

2. Collaboration and demonstration: The unclassified CCDC could be hosted on aShared Services Canada (SSC) backbone and Internet-connected network such as theGPNet. This connectivity would allow unclassified collaboration and demonstrationcapabilities with remote DND locations, OGD and trusted partners.

3. Cross domain solution: In the future the DND will adopt an enterprise crossdomain solution. The CCDC could be able to use the cross domain solution to furtherenhance the sharing between the CCDC’s unclassified and level II instances.

4. Software defined components: The SDDC, whose architecture the CCDC is basedon, is the next generation data center architecture that can provide specificallycustomised software defined components [3]. Thus, the CCDC could use the SDDC tooffer a cyber security experimentation environment on a the next generation data centertechnology. Specific experimental testbeds could be designed to perform advancedand specialised cyber security activities based on the SDDC offerings. For example,the SDDC provides an architecture which is capable of providing an advanced threatdetection platform. Threats can be dynamically intercepted and moved into containedisolated environments for further forensic investigation and evaluation.

5. Provides ability to transmit wirelessly at level II: Currently, the DND does notsupport wireless transmission of data at level II. However, existing wireless technologiescould be integrated at the physical infrastructure level. Since the envisioned CCDCarchitecture supports both physical to virtual (P-V) and virtual to physical (V-P)integrations, it could fully support wireless technology once the DND has establishedthe ATO at level II.


7 Summary

This report has outlined the vision of the Cyber Capability Development Centre (CCDC).The report presents the CCDC as an agile and effective cyber science and technology (S&T)infrastructure that could support the research, experimentation, test and evaluation, demon-stration, exercise and training needs of the Department of National Defence (DND)/CanadianArmed Forces (CAF).

The CCDC’s virtualisation-based experimental testbed infrastructure could provide anenterprise-class, scalable, multi-user platform for full service automation and lifecyclemanagement. Its envisioned capabilities could meet the previously identified requirements ofproviding an agile and effective infrastructure for cyber research, experimentation, testingand evaluation, demonstration and training [1]. The CCDC could provide the necessaryautomation, infrastructure, networks, tools, methodologies and supporting processes thatcould enable the S&T testing of emerging and advanced security technologies at enterprisescale. The enterprise scale emulation could be achieved through the use of computing,networking and storage virtualisation technologies. Techniques such as network cloning thatuse the virtual replication of network services, could allow for the provision of low cost cybernetwork testbeds.

Through examples, this report also shows how experimental testbeds could provide containedenvironments that could allow researchers to safely test defence techniques against “live”threats without endangering other research enclaves, operational networks or the largerInternet. The report shows how the CCDC could support large-scale experimentation thatcould allow researchers to safely evaluate cyber security solutions in a realistic environment,with a capability to test malware and other threats. The testbeds could also facilitatescientific experimentation and validation against established baselines of attack behaviorand could support innovative approaches that involve “breaking” the network infrastructurewhile allowing the testbed to be reset through automation and be broken again and again.

If the current CCDC vision is implemented, the next step would be to create an identicalclassified and collaborative infrastructure to test and experiment with our partners and allies.Connection to existing test networks, such as the Test and Development Centre (TDC),could allow cyber research and development (R&D) testing to baseline operational networkswithout affecting operations. This network could be grown as the experiment needs of theCyber Operations and Signals Warfare (COSW) section change. Extending this testinginfrastructure to tactical wireless networks and combining strategic networking, tacticalwireless and electronic warfare cyber testing would be the desired end state for this capability.


Acronyms and abbreviations

AD active directoryAPT advanced persistent threatARMOUR Automated Computer Network DefenceATO Authority to OperateCAF Canadian Armed ForcesCAPEX capital expenditureCCDC Cyber Capability Development CentreCDX Cyber Defense ExerciseCFBLNet Combined Federated Battle Laboratories NetworkCFWC Canadian Forces Warfare CenterCFXNet Canadian Forces Exercise and Experimentation NetworkCNO computer network operationsCOSW Cyber Operations and Signals WarfareCOTS commercial off-the-shelfCPU central processing unitCSE Communications Security Establishment CanadaCTDC Classified Test and Development CentreDAS direct-attached storageDFW distributed firewallDIMEI Director Information Management Engineering and IntegrationDHCP dynamic host control protocolDLR distributed logical routerDMZ demilitarised zoneDND Department of National DefenceDNS domain name systemDRAM dynamic random-access memoryDRDC Defence Research and Development CanadaDTU desktop unitDIM Secur Director of Information Management SecurityESG edge services gatewayEW electronic warfareGPNet General Purpose NetworkGSM global system for mobile communicationHCI hyper converged infrastructureIaaS infrastructure as a serviceIDFW identity firewallIP Internet protocolIT information technologyJBOD just a bunch of disksLAT Lab Administration Team


LDAP lightweight directory access protocolL2 layer 2L3 layer 3MASA Mission Assurance and Situational AwarenessNAS network attached storageNAT network address translationNATO North Atlantic Treaty OrganizationNSX network virtualization and security platformNTP network time protocolOGD other government departmentOS operating systemPaaS platform as a serviceRAID redundant array of independent disksRAM random access memoryRDP remote desktop protocolRFID radio frequency identificationR&D research and developmentSaaS storage as a serviceSDS software defined storageSSC Shared Services CanadaSSH secure socket shellSA&A Security Authorisation and AccreditationSAAG Security Assessment & Authorization GuidanceSAN storage area networkSDDC software defined data centerSDK software development kitSDN software defined networkSMTP simple mail transport protocolS&T science and technologyTDC Test and Development CentreTSIL Test and System Integration LabTTCP The Technical Cooperation ProgramURL uniform resource locatorVDI virtual desktop infrastructureVIM virtual interface managerVM virtual machineVPN virtual private networkvRA vRealize AutomationvRO vRealize OrchestratorvROp vRealize OperationsVTC video teleconferenceXaaS anything as a service


ZIP perimeter zone interface point

References

[1] Magar, A. (2013), Cyber Capability Development Centre (CCDC) Science &Technology (S&T) equirements, Logical Architecture & Reference Design, (CR2013-057) Defence Research and Development Canada.

[2] Dondo, M., Simmelink, D., and Worth, P. (2017), The Cyber Capability DevelopmentCentre (CCDC): A business case, (DRDC-RDDC-2017-D003) Defence Research andDevelopment Canada.

[3] VMware (2017), Lab Overview - HOL-1706-SDC-6 - Guide to SDDC: VMwareValidated Designs (online),http://docs.hol.vmware.com/HOL-2017/hol-1706-sdc-6_html_en/ (Access Date:3 April 2017).

[4] Apple (2017), iCloud (online), http://www.apple.com/icloud/ (Access Date: 25April 2017).

[5] Google (2017), Google Cloud (online), https://cloud.google.com/ (Access Date: 25April 2017).

[6] Amazon (2017), Amazon Cloud (online), http://daws.amazon.com/cloud (AccessDate: 25 April 2017).

[7] VMware (2017), Lab Overview - HOL-1706-SDC-1 - Cloud Management Platform:Integrate vRealize and NSX (online),http://docs.hol.vmware.com/HOL-2017/hol-1706-sdc-1_html_en/ (Access Date:3 April 2017).

[8] VMware (2017), vCloud Suite (online),http://www.vmware.com/products/vcloud-suite.html (Access Date: 4 April2017).

[9] VMware (2017), Cloud computing (online),http://www.vmware.com/ca/solutions/cloud-computing.html (Access Date: 25April 2017).

[10] VMware (March 2017), VMware Validated Designs Documentation (online),https://www.vmware.com/support/pubs/vmware-validated-design-pubs.html(Access Date: 3 April 2017).

[11] VMware (2017), VMware Horizon 7. What’s new: Desktop and app virtualizationreimagined (online), http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/horizon/vmware-horizon-7-whatsnew.pdf (AccessDate: 5 April 2017).


http://docs.hol.vmware.com/HOL-2017/hol-1706-sdc-6_html_en/

http://www.apple.com/icloud/

https://cloud.google.com/

http://daws.amazon.com/cloud


http://www.vmware.com/products/vcloud-suite.html

http://www.vmware.com/ca/solutions/cloud-computing.html

https://www.vmware.com/support/pubs/vmware-validated-design-pubs.html

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/horizon/vmware-horizon-7-whatsnew.pdf

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/horizon/vmware-horizon-7-whatsnew.pdf

[12] VMware (2017), VMware hyper-converged infrastructure (HCI) (online),http://www.vmware.com/products/hyper-converged-infrastructure.html(Access Date: 5 April 2017).

[13] DeLL EMC (2017), Hyper converged infrastructure appliance (online),https://www.emc.com/en-ca/converged-infrastructure/learning-center.htm(Access Date: 1 May 2017).

[14] VCE (2016), Providing enterprise performance, capacity, and data services for splunkenterprise, (Technical Report) VCE.

[15] VMware (2016), Delivering IT as a Service with a software-defined data center (online),VMWare, https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/include/microsite/sddc/delivering-it-as-a-service-with-a-sddc-white-paper.pdf (Access Date: 13October 2017).

[16] VMware (2016), VMwareő NSX for vSphere End-User Computing Design Guide(online), https://communities.vmware.com/docs/DOC-31982 (Access Date: 4 April2017).

[17] VMware (2017), vSphere API and SDK documentation (online),https://www.vmware.com/support/pubs/sdk_pubs.html (Access Date: 5 April2017).

[18] Soeldner, G., Soeldner, C., and Soeldner, J. (2016), Mastering vRealize automation 7.1:Implementing cloud management in the enterprise environment, CreateSpaceIndependent Publishing Platform.

[19] VMware (2016), Lab Overview HOL–1701–USE–2 vRealize Operations and vRealizeBusiness: Optimize Compute Utilization (online),http://docs.hol.vmware.com/HOL-2017/hol-1701-use-2_html_en/ (Access Date:3 April 2017).

[20] VMware (2016), Reference architecture: vRealize automation 7.0.1, (TechnicalReport EN–001847–03) VMware.

[21] VMware (2016), Lab Overview - HOL-1721-USE-2 - vRealize Automation 7: Advanced(online), http://docs.hol.vmware.com/HOL-2017/hol-1721-use-2_html_en/(Access Date: 5 April 2017).

[22] VMware (2016), Lab overview - HOL-1706-USE-4 - vRealize Operations: Advanced usecases (online), http://docs.hol.vmware.com/HOL-2017/hol-1706-use-4_html_en/(Access Date: 5 April 2017).

[23] VMware (2016), Lab Overview HOL–1721–USE–1 vRealize Automation 7: Basics(online), http://docs.hol.vmware.com/HOL-2017/hol-1721-use-1_html_en/(Access Date: 4 April 2017).


http://www.vmware.com/products/hyper-converged-infrastructure.html

https://www.emc.com/en-ca/converged-infrastructure/learning-center.htm

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/include/microsite/sddc/delivering-it-as-a-service-with-a-sddc-white-paper.pdf



https://communities.vmware.com/docs/DOC-31982

https://www.vmware.com/support/pubs/sdk_pubs.html

http://docs.hol.vmware.com/HOL-2017/hol-1701-use-2_html_en/




[24] VMware (2016), vRealize Orchestrator coding design guide, (Technical Report)VMware.

[25] Networks, P. A. (2017), Network Segmentation/Zero Trust (online), https://www.paloaltonetworks.com/solutions/initiatives/network-segmentation(Access Date: 11 April 2017).

[26] Forrester (2013), Developing a Framework to Improve Critical InfrastructureCybersecurity, (Technical Report) The National Institute of Science and Technology.

[27] Holmes, W. (2016), VMware NSX Micro-segmentation, (Technical Report) VMware.

[28] Central, S. (2017), How Does Micro-Segmentation Help Security? Explanation (online),https://www.sdxcentral.com/sdn/network-virtualization/definitions/how-does-micro-segmentation-help-security-explanation/ (Access Date: 11April 2017).

[29] Ixia (2017), Breaking Point: All-in-one applications and security testing platform(online), https://www.ixiacom.com/products/breakingpoint (Access Date: 3April 2017).

[30] Continuum Analytics (2017), Anaconda (online), https://www.continuum.io/(Access Date: 1 May 20017).

[31] VMware (2016), Lab Overview HOL–1706–SDC–3 Secure Your Software Defined DataCenter (online),http://docs.hol.vmware.com/HOL-2017/hol-1706-sdc-3_html_en/ (Access Date:4 April 2017).

[32] Gandhi, A., Harchol-Balter, M., Das, R., and Lefurgy, C. (2009), Optimal powerallocation in server farms, In Proceedings of the Eleventh International JointConference on Measurement and Modeling of Computer Systems.

[33] Yao, F., Wu, J., Subramaniam, S., and Venkataramani, G. (2017), WASP: WorkloadAdaptive Energy-Latency Optimization in Server Farms Using Server Low-PowerStates, In , 2017 IEEE 10th International Conference on Cloud Computing (CLOUD),pp. 171–178, IEEE.

[34] VMware (2016), Lab Overview - HOL-1703-USE-3 - VMware NSX: Operations andVisibility (online),http://docs.hol.vmware.com/HOL-2017/hol-1703-use-3_html_en/ (Access Date:3 April 2017).

[35] VMware (2016), Reference Design: VMware NSX for vSphere (NSX) NetworkVirtualization Design Guide, (Technical Report) VMware.

[36] VMware (2016), Lab Overview - HOL-1703-USE-2 - VMware NSX: DistributedFirewall with Micro-Segmentation (online),http://docs.hol.vmware.com/HOL-2017/hol-1703-use-2_html_en/ (Access Date:3 April 2017).


https://www.paloaltonetworks.com/solutions/initiatives/network-segmentation

https://www.paloaltonetworks.com/solutions/initiatives/network-segmentation

https://www.sdxcentral.com/sdn/network-virtualization/definitions/how-does-micro-segmentation-help-security-explanation/

https://www.sdxcentral.com/sdn/network-virtualization/definitions/how-does-micro-segmentation-help-security-explanation/

https://www.ixiacom.com/products/breakingpoint

https://www.continuum.io/




[37] Banerjee, I., Guo, F., Tati, K., and Venkatasubramanian, R. (2013), Memoryovercommitment in the ESX server, VMware Technical Journal.

[38] General Dynamics Canada (2014), System Concept of Operations (CONOPS) for theAutomated Computer Network Defence (ARMOUR) Technology Demonstration (TD)Contract, (DRDC-RDDC-2014-C73) Defence Research and Development Canada.

[39] Sawilla, R. E. and Wiemer, D. J. (2011), Automated computer network defencetechnology demonstration project (ARMOUR TDP): Concept of operations,architecture, and integration framework, In 2011 IEEE International Conference onTechnologies for Homeland Security (HST), pp. 167–172.

[40] NSA (2016), Cyber Defense Exercise (CDX) (online),https://www.iad.gov/iad/programs/cyber-defense-exercise/ (Access Date: 24April 2017).

[41] CFBLNet (September 2010), Combined Federated Battle Laboratories Network(CFBLNet) (online),http://www.disa.mil/~/media/Files/DISA/CFBLNet/trifold4a.pdf (Access Date:24 April 2017).

[42] VMware (2016), NSX End-User Computing Design Guide, (Technical Report)VMware.

[43] Davoli, M. V. (2016), The beginner’s guide to vRealize automation infrastructure ascode (online), Nuvoli Systems, http://www.nuvolisystems.com (Access Date: 9 May2017).

[44] VMware (2016), Distributed execution managers (online),https://pubs.vmware.com/vCAC-60/index.jsp?topic=%2Fcom.vmware.vcac.install.doc%2FGUID-1C5EE1A9-1464-4AB9-B14F-001738D8597F.html (AccessDate: 3 April 2017).


https://www.iad.gov/iad/programs/cyber-defense-exercise/

http://www.disa.mil/~/media/Files/DISA/CFBLNet/trifold4a.pdf

http://www.nuvolisystems.com

https://pubs.vmware.com/vCAC-60/index.jsp?topic=%2Fcom.vmware.vcac.install.doc%2FGUID-1C5EE1A9-1464-4AB9-B14F-001738D8597F.html

https://pubs.vmware.com/vCAC-60/index.jsp?topic=%2Fcom.vmware.vcac.install.doc%2FGUID-1C5EE1A9-1464-4AB9-B14F-001738D8597F.html

This page intentionally left blank.


Annex A: CCDC Software and examplesA.1 Current software packages that could be used in CCDC

Current software that could be used to implement the CCDC.

Table A.1: Current software components that could support the CCDC.

Software components Description

Cloud Automation• vRealize Automation

(vRA) 7.x

• vRealize Orchestrator(vRO) 7.x

VMware vRA and vRO could provide a secure CCDCportal for users to request information technology (IT)services. The software also manages cloud and IT re-sources that could enable CCDC to deliver services forexperimental testbeds.

Secure Backend Platform,software defined data center(SDDC) Computing and Stor-age (hyper converged infras-tructure (HCI)):

• VMware vstorage areanetwork (SAN) 6.x

• VMware vSphere 6.x

• VMware vCenter

The CCDC’s Core Virtual foundation of the SDDC in-cludes the following components:

• VMware vSphere ESXi hosts the virtual machines(VMs);

• VMware vCenter Server, as the virtual interfacemanager (VIM), provides management of resourcesincluding virtual desktop infrastructure (VDI), com-puting, networking and storage;

• vSAN provides an software defined storage (SDS)layer, which uses the HCI hardware architecture toprovide technology that adds automation, orches-tration, centralized management and control over ashared pool of storage resources abstracted from thephysical layer;

Software Defined Networking• VMware NSX 6.x

The VMWare network virtualization and security plat-form (NSX) provides software defined network (SDN) vir-tualisation services to replicate physical network functions.NSX provides network security services including layer2 (L2) isolation, network address translation (NAT), fire-wall, dynamic host control protocol (DHCP), virtual pri-vate network (VPN) and micro-segmentation. Coupledwith vRA and vSAN, it provides the automation and ser-vice blueprints as well as integration to physical devices.

Continued on next page


Table A.1 – continued from previous pageSoftware components Description

Cloud Operations Manage-ment

• VMware vRealize Op-erations Manager 6.x

• VMware vRealize LogInsight 3.x

VMware vRealize Operations Management Suite and LogInsight:

• vCenter Operations Manager provides infrastructuremonitoring and operations management;

• vCenter Log Insight provides for real-time log man-agement and analysis, search and troubleshootingacross the physical and virtual environments.

Secured End-UsersWorkspace

• VMware Horizon 7.x

Horizon VDI could provide secure one-way remote accessto the shared infrastructure, security and management,traceability of external connectivity to the CCDC as wellas ensuring data is contained within the CCDC privatecloud.

Cloud Common Services The CCDC would also require enterprise resources such asdomain name system (DNS), simple mail transport proto-col (SMTP), lightweight directory access protocol (LDAP),network time protocol (NTP) and database servers.

A.2 Application examplesA.2.1 Service deployment example

In this detailed application example that envisions the application of a fully functioningCCDC, both the researchers’ and administrators’ perspectives are presented.

A.2.1.1 Researcher’s perspective

In this example, a researcher deploys a wireless service experiment from the private cloud’sservice catalog. The service deployment is composed of the experiment’s blueprint designthat includes services, datasets, data sources and the scale required to emulate a real worldnetworking environment. The researcher also requires a new software development kit (SDK)service deployment. The SDK service deployment includes all the software developmenttools and repositories required to develop prototypes that can then be deployed into thewireless testing environment [16, 17].

A test and development scenario walk through is used to show the user interface, displayand workflow provided by the private cloud. The scenario is presented through the followingactivities:

• Wireless network experimentation setup;


• Wireless testing and experimentation deployment;

• Development in the SDK environment

It should be noted that the use of a wireless network deployment is arbitrary, any networkdeployment can be used instead.

Activity 1: Wireless network experiment setup: An overview of such a deployment isillustrated in Figure A.1. The figure illustrates security zones (shown in blue dashes)

Figure A.1: An overview of the CCDC deployment scenario showing the deployments of experimental and developmental enclaves.

constructed using an NSX security group [16, 35, 42]. Each zone has VMs as members. An NSX security policy is then associated with a security group, where access to or from that group is required. The numbers in red circles represent the following activities:

1 Complex wireless testing and experimentation deployment:The commercial off-the-shelf (COTS) wireless service experiment has been modeledto deploy the infrastructure as a service (IaaS) design required to perform wirelessexperiments.


2 Development SDK required for prototyping:The COTS SDK deployment has been modeled to develop and prototype applicationsto use with the wireless experiment.

The COTS wireless and SDK deployments are isolated on private networks to ensure thatexperimentation is controlled, eliminating the possibility of data spilling across enclaves.The shared infrastructure’s metering ensures that each enclave is configured with resourcesnecessary for accurate and consistent experimentation.

Activity 2: Wireless testing and experimentation deployment: The detailed steps onhow a researcher could deploy a wireless testbed are presented in Annex A.2.215. Thesteps show how the researcher can remotely log into the CCDC and request the “CCDCWireless Experiment” catalog item from available services. The researcher can then configureenvironment to conduct the desired experiments.

Activity 3: Development SDK deployment: The procedure to deploy the developmentSDK is the same as the experimental setup just described. The CCDC SDK is selected fromthe available service deployments as illustrated in Figure A.2.

Figure A.2: Service Catalog item for SDK deployment. The CCDC SDK request button is highlighted in RED.

The figure shows the service catalogue item for the SDK deployment (middle of second row). On clicking the request button, which is highlighted in red, the service deployment can be submitted. Then a “zero touch” deployment provisions the IaaS components based on the designed blueprint. The researcher is then able to view and connect to the deployed

15The detailed steps are too long to include in the main body of the text.


components as described in the previous wireless experiment example. The workflow for theSDK deployment is illustrated in Figure A.3.

Figure A.3: Available service deployment portal.

The figure shows a researcher choosing the deployment SDK Service listed in the Catalogitems 1 , submit a request 2 and then deploy the SDK service 3 .

A.2.1.2 The lab administration perspective

This part of the example outlines the administrator’s roles. In the CCDC, the lab ad-ministrator has system wide responsibilities for the management of the cloud platform,virtual infrastructure and the self-service automation of experimentation platform. Theadministrator’s role can be further subdivided across team members as required, vRealizesupports granular user administration roles [23]. For this example, the administrator’sperspective is best understood by walking through the responsibilities of this role within theCCDC.

A. Cloud platform administration: When the CCDC lab administrator logs into vRealize,the administration portal displays two additional tabs as shown in Figure A.4. The figureillustrates the 1 Administration and 2 Infrastructure tabs [23]. The options available

Figure A.4: Additional tabs provided to administrators.

from each tab are as follows:


1. Administration tab optionsThe Administration tab lists all of the functions that are available to perform theadministration of users. As illustrated in Figure A.5 (reproduced with permission fromVMware), the functions are as follows:

Figure A.5: Administrative approval policies configuration as well as other options are listed in the leftmost vertical tab.

• Approval Policies - Create and manage approval policies;• Directories Management - Connect to and manage authentication sources such

as Microsoft active directory (AD), LDAP and VMware Horizon;• Users and Groups - Manage users, directory and business groups;• Catalog Management - Services, catalog items, actions and entitlements can be

configured;• Property Dictionary - Users can use the property dictionary to define new custom

definitions and groups;• Reclamation - Under-utilized machines within the user group can be identified

and reclaimed;• Branding - Allows users to change the look and feel of vRA by adjusting the

color scheme and logos;• Notifications - Enable and disable alert scenarios which can be sent by email (e.g.

submitted or pending requests);• Events - Review event logs generated within the tenant;• Configuration - Manage vRO endpoints and configure either an internal or external

instance of vRO to be used by vRA [18,24];• Artifact16 Management - Configure a connection to user’s artifactory server and

file repository.16VMware defines an artifact as a script (e.g. Bash, Windows Cmd, JavaScript) or the output from a

software build process (such as compiled source code) [20, 43].


2. Infrastructure administrationThe CCDC lab administrator uses options available through the Infrastructure tab,shown in Figure A.5, to manage and maintain the virtual infrastructure. The manage-ment activities include the following:

• Recent Events - Observe any recent deployments or changes to the infrastructurethrough the display shown here;

• Endpoints - Configure and manage the CCDC cloud endpoints, vCenter forvSphere servers are examples of endpoints [35];

• Administration - Implement global settings;• Monitoring - Review logs and view the status of Distributed Execution Man-

agers, which are features for executing business logic to enable the interaction ofdatabases [44].

B. Automation of self-service: The CCDC’s Lab Administration Team (LAT) is responsi-ble for the self-service automation, which allows users to perform zero touch deploymentson a software defined virtual infrastructure. This dynamic and fully orchestrated infrastruc-ture provides secure separation of experimental enclaves, supports domains of applicability,models real world deployment topologies, supports multidisciplinary experimentation, pro-vides a framework and the building blocks for extensibility, allows designs for science-basedhypothesis testing and provides flexible user-friendly applications.

The automation of self-service is illustrated in Figure A.6. From the figure, the service cata-logs contain items that researchers are entitled to deploy from the web portal. Catalog itemsare linked to pre-engineered blueprint designs for experimental enclaves and developmentSDKs.

As shown in the figure, the automation involves the creation of the following:

• Services to provide a logical container to support sharing blueprints with user groups;

• Blueprints to design service deployments, which can be as simple as a VM or as complexas an experiment that includes many networks (Internet, demilitarised zone (DMZ),corporate network) with all of the embedded software, network zoning and enterpriseservices that are required to establish a real world deployment topology;

• Entitlements to control the level of self-service applied to catalog items available toresearchers.

C. Service authoring: The CCDC’s LAT is responsible for authoring the self-servicecatalog for the researchers based on their infrastructure requirements for experimentation,development, testing, demonstration and training. The self-service catalogue is organizedthrough the creation of services. Entitlement policies then allow users to access the servicedeployments. This is illustrated in Figure A.7, which is reproduced with permission fromVMware.

From the figure, creating a new service involves the following steps:


Figure A.6: Options for the automation of self-service.

Figure A.7: Steps involved in creating a new service.

1 Navigate to the Administration tab;

2 Select Catalog Management (not shown);

3 Select Services;

4 Click New to begin creating a new service.


The new service represents a logical container for self-service content that is available,through an entitlement policy, to the users based on group access. This is illustrated inFigure A.8 (adapted from VMware). As shown in the figure, a new service requires a 1

Figure A.8: New service request.

Name and a 2 Description. Once a service is created, existing blueprints can be madeavailable through the service offering. A user can select an identification 3 Icon for theirservice.

D. Blueprint authoring: The LAT uses the vRealize Converged Blueprint Designer to createblueprints using an inventory of components [23]. The inventory of blueprint components iscreated using templates for software components such as web servers, databases and customapplications, NSX components such as load balancers, networks with embedded securityzoning or other blueprints.

The administrator uses a design canvas to select available component types to createthe desired blueprint. These blueprints vary in complexity, for example a platform as aservice (PaaS) offering would be considered a simple design of a VM from a template defininga guest operating system (OS), central processing unit (CPU), memory and disk size. Acomplex blueprint would involve network design with multiple zones and security services,common enterprise services, applications and embedded software. When the blueprint designis completed and tested, it is added to a list of the Defence Research and DevelopmentCanada (DRDC) CCDC service deployments that are published in the services catalog.

A multi-machine blueprint service is illustrated in Figure A.9, which is reproduced withVMware’s permission. The figure shows the creation of blueprints through the followingprocess:

1 The upper left corner categorizes the available types of resources that can be usedon a design canvas. Some of the resources include VMs or software components suchas web servers.

2 In step 1 above, a designer selects Machine Types (underlined in red). A list ofavailable types is provided as shown below the left red rectangle in the figure. Inthis example a vSphere VM is selected (underlined in red) and then dragged onto the


Figure A.9: Multi-machine blueprint interface.

canvas as a component of a blueprint. Software, network, anything as a service (XaaS)and other building blocks can be added as required to provide real world researchinfrastructure testbeds.

E. Entitlement authoring: As explained in Section 2, entitlements provide granularity forroles and permissions, service availability and lifecycle management. They are a requiredfunction for service delivery, all services must be entitled at some level before they areavailable for use. Administrators use them to create a set of policies that determine theservices user groups (or business groups) can deploy and how they can perform lifecyclemanagement of their services after provisioning.

The entitlements, which are developed using vRealize’s Entitlements, add governanceand additional controls to the experimentation, development, testing, demonstration andtraining environment. For all available services, they are created and managed under CatalogManagement option (Administrationtab → CatalogManagement → Entitlements) [23].

The following entitlement options are available per Business Group user or group:

• IaaS blueprints;

• PaaS/AppServices blueprints;

• XaaS services;

• Actions or custom actions (reboot, power-off, destroy, reconfigure);

• Service catalogs;


• Approval policies.

A.2.2 Steps in the wireless testing and experimentation deployment example

The steps required to deploy the wireless testbed are as follows:

1. Researcher logs into a machine in their office;

2. Researcher logs into the CCDC through the web based portal shown in Figure A.10(adaptedwith permission from VMware). The portal, which is the starting point to accessall of the services offered through the cloud, would have a CCDC uniform resourcelocator (URL) bookmark in the user’s web browser. To access the services, the userlogs in to the vRA through the self-explanatory sequence of steps numbered 1 to 3 inFigure A.10.

Figure A.10: Operational user’s login screen. This would the entry point for users to request new or manage existing service deployments.

3. A tab-based window layout, as illustrated in Figure A.11 (reproduced with permissionfrom VMware), would then displayed across the top of the screen.

Figure A.11: Operational user tab.

The operational user tabs shown in the figure support the following functions (assummarised by the tab number):

1 Review the home page;

2 Request additional services;

3 Review existing services;

4 Track user’s submitted or saved requests;


5 Check user’s inbox for approval requests, action items and reclamationrequests.

4. Through the Catalog tab shown in Figure A.12 (reproduced with permission fromVMware), the researcher could deploy an existing service catalog for the wirelessexperimentation.

Figure A.12: Operational user Catalog Tab.

5. The catalog request would retrieve, through the “All Services” option, catalog itemsthat have been made available to the user group as illustrated in Figure A.13.

Figure A.13: Operational user’s view of the services catalog. The user in this case selects the CCDC Wireless Experiment using the request button highlighted in red.

In the figure, there are 9 services available in the catalog, of which “CCDC WirelessExperimentation” is one. Each of the catalog items represents a predesigned blueprint.Each blueprint is a separate topology consisting of VMs, network security and storageservices that the user can request and manage.These services could created to provide a mechanism for organizing logical groupingsof related catalog items. For example the CCDC could create a specific servicegroup called “DRDC–CCDC Services”, which could list specific blueprints availableas research experiments or SDK deployments. Such a service grouping is also a wayto control global entitlements for the catalog items under that grouping.


6. Assume that the researcher clicks on the “CCDC Wireless Experiment Blueprint” linkfrom the list of infrastructure services shown in Figure A.13. Then the “New Request”form opens and provides the researcher with the options shown in Figure A.14. The

Figure A.14: Operational user’s new service request form.

form must be completed to determine the 1 description, 2 reasoning for the request,number of deployments and lease duration. The default entries on the request formcould be modified on the form template for the blueprint when it is launched.

7. The request could then be submitted for approval through a flexible process thatcan be set up to auto approve requests. The process could also include conditionalapprovals for users that request special resources that are not based on the defaultservice deployments. When approvals are included into the workflow, the manager ofthe scientific development and test resources would approve the request through theportal.

8. On clicking the “Requests” tab, researchers could view the current status of theirrequests as shown in Figure A.15. In this example the experiment was provisionedwith default settings, so the approval is automatic as reflected by the “Successful”status (in red oval next to 2 ).

9. With the CCDC vRealize policy set up to auto approve requests with the defaultblueprint settings selected, the vRA could perform an automated “zero touch” deploy-ment of the templated blueprint into the shared virtual infrastructure. The blueprinttemplate could contain all the data sources, datasets and network services required toperform real world experiments [19, 20]. No manual setup would be required by the


Figure A.15: User’s view of request status. The figure shows a successful request.

user, the infrastructure would be ready for experimentation. Data cross pollinationcould be prevented by the default separation enforced through micro segmentation.A workflow example of this process is illustrated in Figure A.16. In the figure, step

Figure A.16: Workflow of a successful deployment request.

1 allows the researcher to pick a service deployment. In step 2 the researchersubmits the request for auto approval because the infrastructure blueprint was selectedin the default settings. Finally, in step 3 a “zero touch” automated infrastructure


deployment is completed by vRealize.

10. When the request is completed, the blueprint is deployed and clicking the “Items” tabdisplays multiple components that include other VMs, networking, application andstorage configurations. The tab shows this successful deployment as illustrated inFigure A.17 (reproduced with permission from VMware).

Figure A.17: Items tab showing the VMs set up by a user in a successful service deployment.

In the figure, the Items tab displays all of the VMs the user has set up per servicedeployment. If a user deployed both a wireless research enclave and a developmentSDK, the VMs deployed will be listed under the root service deployment. In anexperiment that includes multiple network zones, such as the Internet, DMZ andinternal corporate networks, the VM names (as in the design template) and thenetwork zone, distinguish the roles of particular VMs in the experiment.

11. The following procedures are used to manage deployments:(a) Interacting with service deployments: The CCDC Wireless Experimentation IaaS

has now been deployed using the vRA’s self-service catalog. It is important, then,to explore the entitlements assigned to the development user to control interactionswith the deployed VMs. As explained in Section 2, entitlement policies couldcontrol ongoing management functions each user performs. Ongoing managementfunctions include the following operations:

• Connecting to the VM through platform tools like remote desktop proto-col (RDP) and secure socket shell (SSH) or connection brokers like CitrixXenDesktop;

• Power management;• Reprovisioning;• Reconfiguring of resources (e.g. to add or remove or modify CPU, memory,

storage, network);• Lease extension;• Archiving and reactivation;• Lease management;• Destroying or deprovisioning;• Any custom command or orchestrated task that the administrator has added.


(b) Reviewing Entitlements: Entitlements can be reviewed as illustrated in Fig-ure A.18 (reproduced with permission from VMware).

Figure A.18: Entitlements granted to the operational user.

The process illustrated in the figure is as follows: Click on Items tab 1 , thenClick on the Machines 2 section to see the VMs. Then select dev-xx row 3(where xx is the number assigned to a VM) to select the desired VM. Finallyclick the Actions drop-down menu 4 to see the list of all the entitlements theuser can perform on this VM. For example, the entitlements granted to theuser in Figure A.18 include day-2 self-service capabilities17 such as connectionmethods (with RDP for example), creating snapshots, destroying a deployment,or rebooting (see red outline adjacent to option 4 for a list of entitlements).

(c) Accessing VMs: The process for accessing VMs is illustrated in Figure A.19(reproduced with permission from VMware). First, the user must follow the

Figure A.19: Operational user accessing remote console access to resources.

process for reviewing entitlements illustrated in Figure A.18. Then, the user mustclick on the “Connecting to Remote Console” menu item to log in to the VM.Connecting to a VM through the SSH requires knowing the Internet protocol (IP)address of the deployed VM. The IP address can be determined through thefollowing process (see Figure A.19):

17Day-2 activities refer to features that are used later in a managed workflow.


1 Select dev-xx row, where xx is the number assigned to the VM;2 Click the Actions drop-down menu;3 Select Connect to Remote Console. With this option, a remote connect

window illustrated in Figure A.20 opens.

Figure A.20: Remote console login screen.

The user must log in by entering a lab user-id and password. If the VM is Linux,the user must, as illustrated in Figure A.21, 1 run the command ifconfig.Then the user must 2 find the inet addr value in the eth1 block of the outputof the command. This is the IP address of this VM.

Figure A.21: Remote console IP address.


Note: If this VM has a windows guest operating system, the ifconfig commandis replaced with the ipconfig command. Now the IP address can be used toconnect, using tools such as Putty, as shown in Figure A.22.

Figure A.22: Connecting to remote VM through Putty.

(d) Lease Changes: Should the researcher determine that the wireless testing environ-ment requires changes such as: increased disk space, CPU, memory, etc., changescan be made from the Entitlements interface (Figure A.18 above) to request theadditions. The steps to change the lease are illustrated in Figure A.23. The user

Figure A.23: Requesting a change to the leased period for a service deployment (Experimental testbed).

must request lease changes by taking the following steps: (1) Select the wirelessservice deployment (parent level icon, sub items are the blueprint components).(2) Click on the actions drop-down button. (3) From the list that appears select,“Change Lease”.A new window shown in Figure A.24 appears. From the figure, the expirationdate 1 and time 2 can be edited and 3 submitted.

A.2.3 Experimental testbed design examples

This capability is illustrated through three examples that follow.


Figure A.24: Submitting a change to the leased period for a service deployment.

A.2.3.1 Example 1: Complex corporate network design

This example represents the design for a typical enterprise network. The NSX edge servicesgateway (ESG) is used to separate and provide security between three networks, the Internet(or external network), DMZ and internal network. This setup is illustrated in Figure A.25 [21].

Figure A.25: Envisioned CCDC complex experimental testbed service deployment. The figure shows conceptual examples of three network deployments in CCDC, the Internet,

DMZ and internal network.

The setup in the figure uses the NSX ESG to provide NAT functionality, load balancing and L2/layer 3 (L3) VPNs. All of the network services in each zone could be assembled using


vRA’s converged blueprints. A typical network security model could then deployed andenforced within the testbed to simulate cyber security impacts from various attack vectorsoriginating from the Internet, DMZ or corporate network. Such a security model could berealised through policies explained below.

Security policies for Example 1

1. Internet or external network: The External/Internet zone security policy isenforced by a firewall rule that allows all traffic from all networks as illustrated inFigure A.26. This rule, which is shown in the figure, would be required to ensure the

Figure A.26: CCDC’s envisioned external zone security rules.

proper functioning of NSX micro-segmentation. This is a necessary default policy toprevent connectivity that is not explicitly defined in a white list.

2. DMZ network: The DMZ security policy is enforced by two firewall rules as illus-trated in Figure A.27. The first rule specifies that traffic is allowed from any networks

Figure A.27: CCDC’s envisioned DMZ zone security rules.

or the Internet, DMZ and Internal network into the DMZ Open zone. The DMZPrivate zone allows traffic from the DMZ and Internal Network zones.

3. Internal network: The Internal network security policy is enforced by two firewallrules as illustrated in Figure A.28.As shown in the figure, the first rule specifies that traffic is allowed from the DMZprivate zone and Internal network into the Internal Open zone. The Internal privatezone allows traffic from the Internal network zones only.


Figure A.28: CCDC’s envisioned internal zone security rules.

4. Default rule (last in the list): The final firewall rule in the ordered list is calledthe Default Rule. It is the last rule because it acts as a catch all rule. As illustratedin Figure A.29, the default rule blocks all previously undefined traffic among all zones.This rule represents the default micro-segmentation security policy.

Figure A.29: CCDC’s micro-segmentation default security rule.

Build up and tear down capability

An important CCDC capability would be to build up and tear down experiments as requiredby the researcher as will be discussed in Section 4. A blueprint design for the corporatenetwork in Example 1 could be provisioned with all of the network services (e.g. AD, DNS,application workloads, security policy) automatically deployed for experimentation. Thetear down capability could be carried out on demand by the user or through a concept basedon leased time. In this case, the user could manually release resources after the completionof their projects. Alternatively, the resources could be automatically released after theexpiration of leased time. When resources such as software licenses, CPU, memory andstorage are released, they could be automatically reclaimed for future re-use. The vRA andvRO components in CCDC are responsible for automatically managing the reclaiming andreallocation of those resources [8].

A.2.3.2 Example 2: Wireless design

This example provides a specialized transmission line testbed that could be used by re-searchers working on a communication framework experiment. This testbed could supportwireless experiments and sensor testing. The designed testbed could provide for one-waycommunication and micro-segmentation within the virtual infrastructure to ensure thatdevelopment and testing efforts are securely contained. Wireless technologies such as (Wi-Fi,global system for mobile communication (GSM), radio frequency identification (RFID) andBluetooth) could connect to the physical network. Uplinks from the virtual to the physicalnetwork could provide the necessary integration.


Such a wireless design could be a good example of CCDC’s virtual to physical networkintegration capability, although the current vSphere hypervisor does not offer wirelessadapter capabilities [16]. However NSX could integrate the virtual and physical networksallowing for the wireless experimentation. NSX could ensure that the wireless integration,which could takes place at the physical layer, allowing virtual traffic flow uplinks to theappropriate wireless device [34–36]. NSX Virtual networking and security capabilities couldensure that the servers can communicate as required, providing a securely isolated enclavewith demonstration capabilities. The wireless design could provide full packet capturecapabilities within the virtual infrastructure, which could host the applications used in suchdata collection.

The basic wireless topology illustrated in Figure A.30 could be offered through differentblueprints based on the testing and experimentation objectives. The wireless testbed could

Figure A.30: Traffic flow steps from West to East (Virtual to Physical integration). The figure shows physical to virtual and virtual to physical connectivity through a Cisco Nexus

3000 switch.

support different wireless technologies such as Wi-Fi, GSM, RFID and Bluetooth. As shown in the figure, different wireless hardware would be required for different technology testbed blueprints. Such hardware could connect at the physical layer and NSX designs integrate them into the physical network as shown in the figure [34–36].

An alternative design is shown in Figure A.31(a). In this case a Wi-Fi experimental blueprint simulates attacker and defender networks.

The same network blueprint could be used with GSM as shown in Figure A.31(b). In all cases, the CCDC private cloud could allow the re-use of compute, network and storage resources.


(a) WI-FI. (b) GSM.

Figure A.31: Wireless experimental testbed blueprint design.

A.2.3.3 Example 3: SDK design example

The SDK testbed provides a stand-alone self-service development environment, which couldalso be used to provide a secure demonstration capability. Such a setup is illustrated in theblueprint shown in Figure A.32. The figure features the following:

Figure A.32: SDK experimental testbed blueprint design.

1. An environment for testing new software and experimentation;

2. A development environment that could support software development, including the


SDK platform, OSs, tools and topologies required;

3. An environment capable of supporting instances such as the Automated ComputerNetwork Defence (ARMOUR) SDK, MATLAB or Exata-NS2.

Similar to other setups, the SDK design could support secure one-way communicationand micro-segmentation to ensure that all development and testing activities are securelycontained.

The vRA converged blueprints could support a mechanism called Software Components,which provides on-the-fly installation of software during the provisioning of SDK servicedeployments [7]. Such a setup is illustrated in Figure A.33. The Software Components tool

Figure A.33: Converged blueprint view of embedding on-the-fly software components [7].

could provide an agile approach to life cycling SDK software components [7]. SDK toolscould be added to the software components list. Templates could become less granularin functionality. For example, a template might be restricted to a basic OS, but addingsoftware components, such as scripts, to the OS becomes part of the blueprinting process.


Index

B

blueprint, 11, 37, 39, 43, 67, 71, 76

C

catalog, 11catalog items, 11classified, 46cloud orchestration, 13collaboration, 9, 49 fcross domain solution, 50

D

data centre, 10day-2, 74direct-attached storage, 9Distributed Execution Manager, 65

E

edge services gateway, 7entitlements, 11, 37, 68

O

old projects, 42operations and management, 42optimization, 42

P

partners, 49physical layer, 8policy-based automation, 10portability, 39private cloud, 5provisioning, 42public cloud, 5

R

re-use, 23revive, 23

S

scale out, 8scale up, 8SDK deployment, 62server virtualisation, 9service authoring, 65software defined components, 50software defined data center, 10software defined storage, 9storage area network, 9

V

vApp, 11virtual desktop infrastructure, 7

Z

zero touch, 12zero trust, 13

H

hyper converged infrastructure, 8

I

IT as a Service, 10

L

lab administration team, 16 lease, 20

M

maintenance, 42micro-segmentation, 13


DOCUMENT CONTROL DATA*Security markings for the title, authors, abstract and keywords must be entered when the document is sensitive

1. ORIGINATOR (Name and address of the organization preparing thedocument. A DRDC Centre sponsoring a contractor’s report, or atasking agency, is entered in Section 8.)

DRDC – Ottawa Research Centre3701 Carling Avenue, Ottawa ON K1A 0Z4,Canada

2a. SECURITY MARKING (Overall security marking ofthe document, including supplemental markings ifapplicable.)

CAN UNCLASSIFIED

2b. CONTROLLED GOODS

NON-CONTROLLED GOODSDMC A

3. TITLE (The document title and sub-title as indicated on the title page.)

The Cyber Capability Development Centre (CCDC) Concept: An infrastructure for cyberresearch, experimentation, testing and evaluation, demonstration and training

4. AUTHORS (Last name, followed by initials – ranks, titles, etc. not to be used. Use semi-colon as delimiter)

Dondo, M.; Risto, J.; Simmelink, D.; Worth, P.

5. DATE OF PUBLICATION (Month and year of publication ofdocument.)

May 2019

6a. NO. OF PAGES (Totalpages, including Annexes,excluding DCD, coveringand verso pages.)

104

6b. NO. OF REFS (Totalcited in document.)

44

7. DOCUMENT CATEGORY (e.g., Scientific Report, Contract Report, Scientific Letter)

Reference Document

8. SPONSORING CENTRE (The name and address of the department project or laboratory sponsoring the research anddevelopment.)

DRDC – Ottawa Research Centre3701 Carling Avenue, Ottawa ON K1A 0Z4, Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicableresearch and development project or grant number underwhich the document was written. Please specify whetherproject or grant.)

05ac

9b. CONTRACT NO. (If appropriate, the applicable contractnumber under which the document was written.)

10a. DRDC DOCUMENT NUMBER

DRDC-RDDC-2018-D07110b. OTHER DOCUMENT NO(s). (Any other numbers which may

be assigned this document either by the originator or by thesponsor.)

11a. FUTURE DISTRIBUTION WITHIN CANADA (Approval for further dissemination of the document. Security classification must alsobe considered.)

Public release

11b. FUTURE DISTRIBUTION OUTSIDE CANADA (Approval for further dissemination of the document. Security classification must alsobe considered.)

Public release

12. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Use semi-colon as a delimiter.)

testing; experimentation; validation; collaboration

13. ABSTRACT/RÉSUMÉ (When available in the document, the French version of the abstract must be included here.)

The vision for the Cyber Capability Development Centre (CCDC) is to facilitate science and tech-nology (S&T) research, experimentation, testing, evaluation, demonstration and training activitieswithin the Cyber Operations and Signals Warfare (COSW) section of the Defence Research andDevelopment Canada (DRDC)-Ottawa. The CCDC concept is driven by identified research anddevelopment (R&D) requirements solicited from the scientific community in the COSW sectionof DRDC-Ottawa [1]. It was determined, through a business case analysis, that the best wayto meet such requirements could be to construct a virtualised infrastructure based on VMwaretechnology. The resulting implementation could provide an agile and effective cyber researchinfrastructure to meet the CCDC’s original vision. This report documents the overall conceptthat addresses the DRDC’s cyber S&T research infrastructure needs and functional (business)requirements. The report’s focus is on how the CCDC’s technologies could meet researchers’needs and in turn provides detailed steps on how the researchers could use the capabilities pro-vided by the CCDC to support their S&T activities. Throughout the report, the CCDC is shownto have the potential for superior time and resource saving capabilities compared to the classi-cal experimentation approaches. The report also outlines how the current CCDC vision couldbe extended to incorporate classified level II capabilities as well as connectivity with local andinternational partners.

La vision du Centre de développement des cybercapacités (CDC) est la suivante : faciliter larecherche, l’expérimentation, la mise à l’essai, l’évaluation, la démonstration et les activités deformation en S & T au sein de la Section Cyberopérations et guerre des transmissions (COGT)de Recherche et développement pour la défense Canada (RDDC). Le concept du CDC se fondesur les besoins établis en recherche et développement (R & D) émanant de la communautéscientifique de la Section des COGT du Centre de recherches d’Ottawa de RDDC [1]. On adéterminé, au moyen d’une analyse de rentabilisation, que le meilleur moyen de répondre à cesbesoins pourrait être de bâtir une infrastructure virtualisée basée sur la technologie VMware.Une fois mise en place, elle pourrait constituer une cyberinfrastructure de recherche souple etefficace qui serait conforme à la vision initiale du CDC. Ce rapport vient documenter le conceptgénéral portant sur les besoins fonctionnels (et opérationnels) de RDDC en matière de cyberin-frastructure de recherche en S & T. Dans le présent rapport, on décrit surtout : comment faireen sorte que les technologies du CDC puissent répondre aux besoins des chercheurs et aussicomment fournir à ces derniers un plan détaillé pour pouvoir utiliser les capacités du CDC afin desoutenir leurs activités de S & T. Dans l’ensemble du rapport, on démontre que le CDC pourraitdisposer de capacités supérieures permettant d’épargner temps et ressources comparativementaux approches traditionnelles d’expérimentation. Dans le rapport, on souligne également que lavision actuelle du CDC pourrait englober les capacités classifiées de niveau II, de même que lescapacités de connectivité avec les partenaires locaux et internationaux.

the cyber capability development centre (ccdc) concept · 2019. 9. 19. · defence research and...

Documents