the chemical, biological, radiological/nuclear explosive...

The Chemical, Biological, Radiological/Nuclear Explosive (CBRNE) Consolidated Risk Assessment (CRA) Rating Tool Guide

Shaye Friesen DRDC – Centre for Security Science

Defence Research and Development Canada Reference Document DRDC-RDDC-2017-D001 January 2017

Template in use: Normal.dotm

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2017

© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2017

DRDC-RDDC-2017-D001 i

Abstract

The aim of this document is to provide a basic user’s guide of the Chemical, Biological, Radiological/Nuclear Explosive (CBRNE) Consolidated Risk Assessment (CRA) process and rating tool. Defence Research and Development Canada’s Centre for Security Science (DRDC CSS) developed the CRA rating tool to aid risk analysts in evaluating the risks posed by a broad range of CBRNE hazard scenarios. Additionally, the background to the CRA tool and scoring approach is included for reference purposes, so that end users can understand how risk ratings are generated and calculated. The CRA tool is a Microsoft Excel file and is available from DRDC CSS.

ii DRDC-RDDC-2017-D001

Résumé

L’objectif du présent document est d’offrir un guide d’utilisation de base pour l’outil de traitement et de cotation de l’évaluation consolidée des risques (ÉCR) relatifs aux agents chimiques, biologiques, radiologiques, nucléaires et explosifs (CBRNE). (RDDC CSS) a élaboré l’outil de cotation de l’ÉCR pour aider les analystes à évaluer les risques courus dans divers scénarios mettant en jeu des agents CBRNE. Le document comprend également, à titre de référence, des renseignements généraux sur l’outil ÉCR et l’approche de cotation afin que les utilisateurs finaux puissent mieux comprendre comment générer et calculer les cotes de risques. L’outil ÉCR est un fichier Microsoft Excel disponible auprès de RDDC CSS.

DRDC-RDDC-2017-D001 iii

Table of Contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Résumé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.1 Consolidated Risk Assessment (CRA) Process Overview . . . . . . . . . . 2 3 The CRA Formula and Rating Tool . . . . . . . . . . . . . . . . . . . . 3

3.1 Characteristic Scenarios . . . . . . . . . . . . . . . . . . . . . . 4 3.2 Rating Impacts . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3 Rating Relative Technical Feasibility (RTF) . . . . . . . . . . . . . . . 6 3.4 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . 7

4 Risk Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Rating Criteria . . . . . . . . . . . . . . . . . . . . . . . . . 13 Annex A

iv DRDC-RDDC-2017-D001

List of Figures

Figure 1: CRA process and methodology. . . . . . . . . . . . . . . . . . . . 2

Figure 2: Example Worksheets from CRA Tool. . . . . . . . . . . . . . . . . 3

Figure 3: Example Worksheet from CRA Tool. . . . . . . . . . . . . . . . . . 5

Figure 4: Screenshot of dropdown menu. . . . . . . . . . . . . . . . . . . . 6

Figure 5: Screenshot of the RTF ratings—note, the final score is derived by selecting the lowest sub-category rating. . . . . . . . . . . . . . . . . . . . . . 7

Figure 6: Screen shot of confidence levels. . . . . . . . . . . . . . . . . . . . 8

Figure 7: Example of Vulnerability Scores (with Error Bars) displayed within the CRA Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Figure A.1: Impact Grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Figure A.2: RTF Grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

DRDC-RDDC-2017-D001 1

1 Introduction

1.1 Aim


2 DRDC-RDDC-2017-D001

2 Background

The CBRNE CRA was originally designed in 2002 by the Canadian Security Intelligence Service (CSIS) with support from the Department of National Defence Intelligence (DND), and the Canadian Food Inspection Agency (CFIA). The methodology is based on the RAND Corporation Delphi Method—a structured process developed for technology forecasting that collects and distils knowledge from a group of experts. The methodology has evolved over time, with refinements to the criteria set, scoring methods, the number of threat scenarios assessed, and the inclusion of a process to capture the science and technology (S&T), capability and response gaps. The CRA was augmented in 2004 to include explosives threat scenarios. The CRA is a comprehensive, bottom-up process that engages experts from federal S&T, operational, law enforcement, and intelligence communities. The assessment has been conducted annually since 2002, normally through two-day workshops, where a representative set of scenarios are rated for technical feasibility, impact, vulnerability, likelihood and risk. The ratings are achieved through consensus of various stakeholders and subject matter experts (SMEs).

2.1 CRA Process Overview

The CRA process occurs in four steps (see Figure 1). In the first step, SMEs identify scenarios. In step two, SMEs evaluate the vulnerabilities by estimating the impact and relative technical feasibility (RTF) of the CBRNE hazard scenarios. In step three, the intelligence community is engaged to provide an intelligence judgement, based on current intelligence reporting, and aims to portray the history of attacks, terrorist capability and intent. In the final step, the impact and RTF combine to form a vulnerability rating, which, together with the intelligence judgement, result in a priority risk rating for each hazard scenario.

Figure 1: CRA process and methodology.

S&T - Operations - Law Enforcement - Intel - Allied

Identify ScenariosScoping, Generation,

Agreement

Determine VulnerabilitiesRelative Technical Impact

Feasibility

~1 month ~3-4 months ~2 months ~2-3 months

IntelligenceIntelligenceJudgement

Assess Likelihood

• Develop characteristic scenarios that describe CBRNE events

• Describes how a threat/hazard could exploit a target vulnerability

• List of characteristic scenarios

Assess AssesRisk

• Evaluate vulnerabilities of each scenario for Relative Technical Feasibility:• Access to Materials• Equipment• Access to Target• Technical Expertise• Access to Critical

Information

• Evaluate vulnerabilities of each scenario for Impact:• Human Losses• Dead• Injured• Magnitude of

Response• Economic Damage ($)

• Assess the likelihood of each scenario by factoring in Intelligence Judgement

• Determine the risk for each scenario (combine Vulnerability and Intelligence Judgement)

• Prioritize risks• Identify key S&T

mitigating factors

CBRNE Consolidated Risk Assessment


3 The CRA Formula and Rating Tool

The CRA tool was created to help risk analysts define scenarios and rate them across a broad range of different criteria. The tool is built in Microsoft Excel. See Figure 2 below for a screen shot of the main page, which is displayed upon opening the tool.

Figure 2: Example worksheets from CRA Tool.

Within the CRA risk tool, there are a number of worksheets at the bottom to input data. These include:

1. Results: worksheet for hazard scenarios; the ratings for impact, RTF, intelligence judgement and confidence estimates are inserted here by risk analysts.

2. Impact Grid: displays the impact scoring matrix for malicious CBRNE hazard scenarios (see Annex A, Figure A.1).

3. RTF Grid: displays the RTF rating scale for the malicious CBRNE hazard scenarios (see Annex A, Figure A.2).

4. Confidence: displays the metrics for measuring the confidence intervals associated with the CBRNE hazard scenarios.

5. Plot: displays a graphical depiction of the results of the ratings, according to Vulnerability (impact and RTF) + Risk.

6. Scoring lists: displays the tables for converting the impact and RTF scores to a common scale. Dollars ($) are used as the common unit due to the widespread practice in other areas of risk assessment [1,2]. In the CRA, the equivalency of deaths to $ is set to 1 death = $10 M.


The tables can be modified, if a different set of common measurement units is selected by the risk analyst. To do this, the measurement units need to be specified or adjusted in the tables, and the results will be translated in the drop down menus on the “Results” table (i.e., the worksheet where the scenarios and rating values are inserted). After converting all the components of the impact scores to $, resulted values are added linearly, and the logarithm (log) of the sum of the scores is taken, as shown in the equation below.

(1)

The RTF ratings follow a similar “order of magnitude” approach, to facilitate the comparison of multiple CBRNE hazard scenarios across target categories, such as people, critical infrastructure, food and consumer products, and agro system (plant and animal). The RTF tables are formulated with a baseline vignette (i.e., 1x (baseline vignette), with the remainder of the scenarios being assessed as 100x less likely than base 1, or 100x more likely than base 1, where rtfl is the associated RTF rating (5 being the maximum rating considered, and the rating scheme adopted was designed to identify the easiest hazard scenarios to execute). Thus, if Vulnerability= Impact + RTF,1 then we follow a logarithmic scale that implies the following:

(2)

If one incorporates intelligence judgement, then we define risk as:

(3)

This can be used to calculate the risk, using iso-risk contours (i.e., lines of equal risk) on an Impact vs. RTF plot, or a Vulnerability vs. Intelligence Judgement plot, as shown later in this document.

Note: The “Results” worksheet in the CRA risk tool can be unlocked to view the conversion of the impact scores in the form of VLOOKUP tables, using the scoring method described above.

3.1 Characteristic Scenarios

Each worksheet can be populated with a representative list of hazard scenarios/vignettes for CBRNE events, with blank spaces provided to allow insertion of hazard-specific information. To begin, risk analysts enter the scenario code/ID if available, and a general description of the characteristic scenario. The description should include the material/agent, target and effects or consequences, along with other relevant details. Generally speaking, a five year timeframe is considered for all events in the CRA process. There is no limit to the number of rows that can be added in the tool. However, the user is advised to ensure the “Plots” worksheet incorporates the data range where scenarios were incorporated (see Figure 3).

1 This is not meant to imply an algebraic addition; rather that vulnerability is plotted as a combination of impact and RTF.


Figure 3: Example worksheet from CRA Tool.

Once the hazard scenarios have been inputted into the tool, it is necessary to evaluate them with respect to vulnerability and likelihood.

3.2 Rating Impacts

In the tool, the impact for a hazard scenario is divided into four sub-categories, with each scored from 0 to 9:

Dead

Injured

Magnitude of Response

Economy

To evaluate the impacts, choose the rating that best describes impacts using the drop-down menu. Once all of the ratings for each sub-category are entered into the CRA tool, an overall value is returned (see Figure 4).


Figure 4: Screenshot of dropdown menu.

In order to ensure a consistent scoring method, the impact ratings are defined in terms of an “order of magnitude” of the severity associated with a loss for each sub-category. This implies a constant ratio for the increase in severity of impacts from one rating to the next. The impact grid (Figure A.1) uses a logarithmic scale, with half scores for ratio increments of the square root of 10, used to increase the resolution of the assessment.

3.3 Rating RTF

The CRA tool permits users to assess RFT. The RTF of each hazard scenario is assessed against five sub-criteria, consisting of the following scores from 0 to 9:

Material

Equipment

Access to Target

Technical Expertise

Access to Critical Information

As with the impact grid, the RTF grid (Figure A.2) follows a similar logarithmic scale. However, estimating the RTF of the hazard scenarios is slightly different, as the estimates must take into account whether mounting an attack is even technically feasible. The overall feasibility score in the CRA tool (i.e., the RTF Matrix Rating column) for a given hazard scenario is based on the concept of the highest hurdle, whereby each sub-category can be seen as a critical link on the path to a successful attack scenario. The final RTF score is determined by selecting the lowest rating across all sub-categories, thus reflecting the highest hurdle or weakest link constraint on the


feasibility of a given hazard scenario. In other words, the lower the feasibility, the higher the hurdle. This rating technique is consistent with other federal risk assessment methodology guidelines [3]. Once users are finished recording their ratings for RTF, the lowest value is entered into the RTF Matrix Rating column in the worksheet (see Figure 5).

Figure 5: Screenshot of the RTF ratings—note, the final score is derived

by selecting the lowest sub-category rating.

3.4 Confidence Intervals

The CRA tool allows users to assess the uncertainties associated with the hazard scenarios during risk assessment workshops. The degree of confidence in the ratings produced may vary, depending on the availability of data, relevant experience and existing body of knowledge. Uncertainties are estimated for the impact and RTF categories as a whole, using descriptive language ladders ranging from “very high” to “very low.” Users capture their ratings for confidence in the corresponding drop-down menus, with the results being recorded Template. The definitions for each of the confidence intervals is located in the worksheets along the bottom of the CRA tool, along with the degrees of uncertainty (e.g., ranging from 80–100% confidence) and associated values. Capturing the confidence level associated with each rating produced provides an essential piece of information that helps complete the risk assessment picture. Figure 6 presents definitions for confidence levels for each rating:


Figure 6: Screen shot of confidence levels.

Capturing confidence estimates as an essential part of the risk assessment process, allows users and stakeholders to visualize and interpret the results. For instance, very low confidence in the rating of a particular hazard scenario produces error bars in the “Plot” worksheet that is intended to convey caution in the interpretation of the hazard scenarios. Very high error bars indicate a degree of uncertainty with respect to RTF ratings, while wide error bars suggest impact uncertainty.

Note: In order to view the error bars on the “Plot” worksheet, it is necessary to unlock the columns in the “Results” worksheet.


4 Risk Plots

In the CRA tool, risk plots are produced by bringing forward the results of the ratings for impact and RTF for each hazard scenario. This information is then used to determine overall vulnerability and (when combined with intelligence judgement) risk ratings. For example, a hazard scenario is determined to have an impact score of 1.4 and a RTF Score of 2. The CRA tool automatically calculates the scores for impact and RTF in the following plot (see Figure 7).

Figure 7: Example of Vulnerability Scores (with Error Bars) displayed within the CRA Tool.

The overall risk rating for the hazard scenarios is also displayed in the “Plots” worksheet, adjacent to the vulnerability graph. It is calculated by combining the final vulnerability rating (impact + RTF) with the likelihood (i.e., intelligence judgement) rating.2 Vulnerability and intelligence judgement data are plotted in the exact same way, to show which hazard scenarios posed the highest risk. Hazard scenario risk ratings can be compared against one another in a relative fashion to identify priorities threats/hazards. The CRA decision support tool allows users to quickly visualize results, compare the distribution of the vulnerabilities and risks in a given scenario set, and establish acceptable or unacceptable risk levels.

Note: the labels for each of the hazard scenarios in the “Plot” worksheet are produced manually.

2 Note: this paper excludes a discussion of the rating factors and criteria associated with intelligence judgement, which will be documented in a separate report.


5 Conclusion

With the aid of the CRA tool, the impact and RTF ratings for multiple hazard scenarios were captured by the CBRNE Communities of Practice (CoP) during stakeholder engagement workshops. The CRA tool was an explicit part of the CRA process that allowed hazard scenarios to be captured in advance. It facilitated the capture of SME consensus ratings, and allowed for the results to be automatically tabulated and graphically presented back to key stakeholders, which extended the benefits of using software tools for the CRA process. The methodology and rating tool/techniques can be suitably modified to address non-malicious threats/hazard risk assessments. If needed, additional rating factors can be added to contribute to refining the overall approach, depending on information requirements and purpose of the assessment.


References

[1] Waller, D. (2016). Proposal to modify the Consolidated Risk Assessment (CRA), Defence Research and Development Canada, Scientific Report, DRDC-RDDC-2016-R133. p. 8.

[2] Lin, X. G. and Jarrett, R. (2009). A Practical Approach to Quantitative Risk Assessment, Division of Mathematical and Information Sciences, Canberra & Melbourne, Risk Conference, Wellington, 2009.

[3] Public Safety. All Hazards Risk Assessment Methodology Guidelines 2012–2013, (Online): http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/ll-hzrds-ssssmnt/index-eng.aspx#ahra_emp. (Access date: 01 Nov 2016).


This page intentionally left blank.


Rating Criteria Annex A

Figure A.1: Impact Grid.


Figure A.2: RTF Grid.

DOCUMENT CONTROL DATA (Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Designated)

1. ORIGINATOR (The name and address of the organization preparing the document. Organizations for whom the document was prepared, e.g., Centre sponsoring a contractor’s report, or tasking agency, are entered in Section 8.) DRDC – Centre for Security Science Defence Research and Development Canada 222 Nepean St., 11th Floor Ottawa, Ontario K1A 0K2 Canada

2a. SECURITY MARKING (Overall security marking of the document including special supplemental markings if applicable.)

UNCLASSIFIED

2b. CONTROLLED GOODS

(NON-CONTROLLED GOODS) DMC A REVIEW: GCEC DECEMBER 2013

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in

parentheses after the title.) The Chemical, Biological, Radiological/Nuclear Explosive Consolidated Risk Assessment Rating Tool Guide

4. AUTHORS (last name, followed by initials – ranks, titles, etc., not to be used) Friesen, S.

5. DATE OF PUBLICATION (Month and year of publication of document.) January 2017

6a. NO. OF PAGES (Total containing information, including Annexes, Appendices, etc.)

18

6b. NO. OF REFS (Total cited in document.)

3 7. DESCRIPTIVE NOTES (The category of the document, e.g., technical report, technical note or memorandum. If appropriate, enter the type of report,

e.g., interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.) Reference Document

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.) DRDC – Centre for Security Science Defence Research and Development Canada 222 Nepean St., 11th Floor Ottawa, Ontario K1A 0K2 Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research and development project or grant number under which the document was written. Please specify whether project or grant.)

9b. CONTRACT NO. (If appropriate, the applicable number under which the document was written.)

10a. ORIGINATOR’S DOCUMENT NUMBER (The official document number by which the document is identified by the originating activity. This number must be unique to this document.) DRDC-RDDC-2017-D001

10b. OTHER DOCUMENT NO(s). (Any other numbers which may be assigned this document either by the originator or by the sponsor.)

11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.)

Unlimited

12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement audience may be selected.)) Unlimited

13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include here abstracts in both official languages unless the text is bilingual.)


---------------------------------------------------------------------------------------------------------------

L’objectif du présent document est d’offrir un guide d’utilisation de base pour l’outil de traitement et de cotation de l’évaluation consolidée des risques (ÉCR) relatifs aux agents chimiques, biologiques, radiologiques, nucléaires et explosifs (CBRNE). (RDDC CSS) aélaboré l’outil de cotation de l’ÉCR pour aider les analystes à évaluer les risques courus dans divers scénarios mettant en jeu des agents CBRNE. Le document comprend également, à titre de référence, des renseignements généraux sur l’outil ÉCR et l’approche de cotation afin que les utilisateurs finaux puissent mieux comprendre comment générer et calculer les cotes de risques. L’outil ÉCR est un fichier Microsoft Excel disponible auprès de RDDC CSS.

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a published thesaurus, e.g., Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.) Chemical, Biological, Radiological/Nuclear Explosive (CBRNE); Risk Assessment; Methodology; Rating Tool.

the chemical, biological, radiological/nuclear explosive...

Documents