fisma cybersecurity performance metrics and scoring moses_dot... · metrics and scoring ......

FISMA Cybersecurity Performance Metrics and Scoring

DOT Cybersecurity Summit

Office of the Federal Chief Information Officer, OMB

OMB Cyber and National Security Unit, [email protected]

DOT Cybersecurity Summit ¦ FISMA Cybersecurity Performance Metrics and Scoring ¦ May 2, 2017

OMB receives agency FISMA metrics through DHS’s CyberScope System. Metrics reporting schedule per M-17-05 1. Quarterly CIO FISMA metrics (CFO Act only) 2. Monthly PIV/CAC submissions (CFO Act Only) 3. Annual IG, CIO, and SAOP Metrics (All Agencies)

Cybersecurity Metrics 2.


OMB uses FISMA metrics for nine processes and products that drive agency Performance: 1. President’s Management Council (PMC) Assessment 2. Cybersecurity Cross-Agency Priority (CAP) Reports 3. Annual FISMA Report to Congress 4. CyberStat Reviews 5. PortfolioStat Reviews 6. FedStat Reviews 7. President’s Budget – Cybersecurity Crosscut 8. Cabinet Engagements 9. Policies and Guidance

Cybersecurity Metrics in Action 3.


Agency one-pagers provide greater context for FY 2016 cybersecurity performance data • Goal of improving readability and look and

feel of report. • FY 2016 IG Metrics provide independent

assessment of FY 2016 CIO metrics. • OMB anticipates a decrease in IG scores due

scoring methodology changes. • One-pagers will also serve as one-stop

summary for future OMB, DHS and IG oversight discussions (e.g., CyberStat, etc.).

FY 2016 Annual FISMA Report to Congress – One Pagers

4. Annual FISMA Report to Congress


PMC Assessment Background

In the wake of the OPM incidents, OMB recognized the need for a cybersecurity performance assessment that provides agency Deputy Secretaries and EOP with an understand of the agency’s hygiene.

OMB developed the PMC Cybersecurity Assessment in late 2014 using agencies’ FISMA metrics data and assessment criteria from the NIST Cybersecurity Framework.

This assessment is a vehicle for driving agency performance and ensuring accountability from the Deputy Secretary on down through the organization.

OMB has matured the assessment process and products to ensure clear and effective communication to agencies, and uses the output from this process to inform its oversight and budget processes.

5.


Leveraging the NIST Cybersecurity Framework

The 23 Civilian CFO-Act Agencies receive a quarterly assessment from OMB that provides overall rating based on performance across the five NIST Cybersecurity Framework function areas: Identify – Develop the organizational understanding to manage cybersecurity risk to systems, assets, data, and capabilities.

Detect – Develop and implement the appropriate activities to identify the occurrence of a cybersecurity event.

Protect – Develop and implement the appropriate safeguards to ensure delivery of critical infrastructure services.

Respond – Develop and implement the appropriate activities to take action regarding a detected cybersecurity event.

Recover – Develop and implement the appropriate activities to maintain plans for resilience and to restore any capabilities or services that were impaired due to a cybersecurity event.

6.


Rating Standards 7.

3 2 1 0 Agency has not met foundational targets

Agency has met government-wide targets

Agency has met foundational targets

Agency has exceeded government-wide targets

EOP provides ratings to agencies for each NIST framework function area on a 0-3 scale:

In first quarter of FY 2016, only five agencies had information security programs that met or exceeded government-wide performance goals. By the end of FY 2016, 13 agencies had met these targets and all others were making significant progress toward this end as a direct result of PMC Assessment process.


Sample of Individual Agency Scorecard 8.

Justice has met government-wide targets 2 Identify 1 2 3 Hardware Asset Management (CAP) 100% Software Asset Management (CAP) 97% Unclassified systems with security ATO 87% Review of contracts with sensitive information Policy empowering incident commanders

1

Protect 1 2 3 Vulnerability Management (CAP) 100% Secure Configuration Management (CAP) 95% Unprivileged PIV logical access (CAP) 95% Privileged PIV logical access (CAP) 100% Remote access security Insider Threat Program Media destruction policy

2

Detect 1 2 3 Anti-Phishing and Malware Defense (CAP)

Anti-Phishing 5 of 7 Malware Defense 3 of 5 Other Defense 2 of 4

EINSTEIN Program Attempts to access data detected and investigated Test exfiltration attempts are caught

2

Respond 1 2 3 Incident response plan Participating in C-CAR protocol Roles and responsibilities are verified No active critical vulnerabilities > 30 days

3

Recover 1 2 3 Disaster and incident recovery plans Incident notification 100% Credit monitoring BPA Credit repair contract

3

Overall Example agency has met government-wide targets

Overall rating is the average of the component scores, rounded to the nearest whole number.

(1+2+2+3+3)/5 = 2

Legend Criteria Met Criteria Not Met Criteria NA

To achieve a particular rating in a framework area, an agency must meet all criteria within both the given level and all prior levels. Level 1 - Foundational targets Level 2 - Government-wide targets Level 3 - Exceeding government-wide targets


Baseline Metrics 9.

In addition to the metrics underlying the PMC Assessment, OMB also analyzes agency performance on measures not used to calculate agency scores. The intent is to better understand current agency performance so ambitious but realistic targets can be set for future assessments. Current Baseline Metrics include:

• HTTPS implementation (M-15-13) • Endpoints with data encrypted at rest (FIPS 140-2) • Users with significant security responsibilities who have completed role-based

security training • Time to revoke role-based credentials following the termination of

employees/contractors


Identify Hardware Asset Management (CAP)

Level 1 – 80% Level 2 – 95% (CAP Goal) Level 3 – 100%

Software Asset Management (CAP) Level 1 – 80% Level 2 – 95% (CAP Goal) Level 3 – 100%

Unclassified information systems with a security ATO

Level 1 – 80% Level 2 – 95% Level 3 – 100%

Review of contracts with sensitive information

Level 1 – Review of key contracts is in progress Level 2 – Review of key contracts is completed Level 3 – All contracts contain clauses on protection, detection, reporting of information

Policy empowering incident commanders

Level 1 – In place

Protect Vulnerability Management (CAP)


Secure Configuration Management (CAP)


Remote access security Level 1 – FIPS 140-2 validated Level 2 – 30 minute time out Level 3 – prohibit split tunneling

Unprivileged PIV logical access (CAP)

Level 2 – 85% (CAP Goal) Privileged PIV logical access (CAP)

Level 2 – 100% (CAP Goal) Insider Threat Program

Level 1 – Initial operating capability Level 2 – Full operating capability

Media destruction policy Level 1 – In place

Detect Anti-Phishing and Malware Defense (CAP)

Level 1 – 1 of 3 key indicators ≥ 90% Level 2 – All key indicators ≥ 90% (CAP Goal) Level 3 – All key indicators 100%

EINSTEIN Program Level 2 – Fully Implemented

Attempts to access large volumes of data are detected and investigated

Level 2 – Detected and investigated

Test exfiltration attempts are caught

Level 3 – Test conducted in past year and attempt was caught

Respond Incident response plan

Level 1 – Developed, tested once annually Level 2 – Tested twice annually Level 3 – No more than 180 days old

Participating in C-CAR protocol Level 1 – Participated in most recent C-CAR call

Roles and responsibilities are verified

Level 2 – Verified during incident response testing

No active critical vulnerabilities > 30 days

Level 2 – No vulnerabilities identified

Recover Disaster and incident recovery plans

Level 1 – Developed, but not tested regularly Level 2 – Tested annually Level 3 – Less than one year old

Incident notification Level 1 – Policy that establishes timeline for public or internal notifications after the detection or discovery of a compromise of PII is in place Level 2 – Metrics tracking for notifications in place Level 3 – Metrics indicate 100% compliance

Credit monitoring BPA Level 2 – In place

Credit repair contract Level 3 – In place

Appendix: EOP Cybersecurity Assessment Criteria 10.


Appendix: Sample Action Items 11.

The following items require additional information/details on how the agency is working to meet government-wide targets:

Category Criteria Questions Actions Needed

Identify CAP Goal: Hardware Asset Management 1.2, 1.4, 3.16 Detail actions to improve Hardware Asset Management capabilities or explain impediments to OMB

Identify CAP Goal: Software Asset Management 1.5, 3.17 Detail actions to improve Software Asset Management capabilities or explain impediments to OMB

Identify Review of key contracts with sensitive information 1.8 Complete review of key prioritized contracts or explain impediments to OMB

Protect CAP Goal: PIV logical access (unprivileged users) 2.4, 2.4.1 Detail actions to improve PIV usage amongst unprivileged users or explain impediments to OMB

Protect CAP Goal: PIV logical access (privileged users) 2.5, 2.5.1 Detail actions to improve PIV usage amongst privileged users or explain impediments to OMB

Protect Privileged user count has achieved target 2.5 Reduce number of privileged users where possible

Protect Insider Threat Program, per E.O. 13587 2.30 Provide a status update of program implementation in next submission

Respond Incident response plan developed and tested biannually 4.7 Increase specificity regarding the frequency of incident response plan testing

and updating

Recover Recovery plans have been developed and tested annually 5.5 Ensure that enterprise-wide incident recovery plan is in place and updated on

annual basis


Annual Assessment Timeline 12.

January 1-15 CFO Act Agencies

complete Q1 FISMA CIO assessment

April 1-15 CFO Act Agencies


July 1-15 CFO Act Agencies


October 1-31 All agencies

complete annual FISMA CIO assessment

February 15 Q1 PMC

assessments delivered to agency CIOs

May 15 Q2 PMC


August 15 Q3 PMC


December 15 Q4 PMC


EOP conducts FY Q1 PMC

assessments


assessments


assessments


assessments

March 1 Annual FISMA report to congress released (from previous FY)


IG FISMA Metrics - History

• Collaboration with OMB, DHS, CIGIE, and other stakeholders

• Historically, the OIG FISMA metrics were mostly yes/no questions that did not enable an easy determination of Effectiveness

• In 2015, a FISMA metrics subcommittee of the FAEC IT Committee was formed to develop effectiveness based measures


Maturity Model Approach

14

• Maturity model incorporates Federal requirements and maps to best practices (e.g., CMMI, CoBIT, NIST, C2M2)

• Maturity indicators map to CIO metrics, NIST 800-53 and supporting

special publications, President’s Management Council, and other governmentwide focus areas/initiatives

• FISMA Metrics Subcommittee has incorporated comments from various stakeholders including FAEC, CIGIE, and the CIO/CISO community


Effectiveness within the Maturity Model

Level 1 Ad-hoc

Level 2 Defined

Level 3 Consistently Implemented

Level 4 Managed & Measurable

Level 5 Optimized

Effectiveness

Desired Results

Operate Implement


FISMA IG and CIO Metrics

FY 2017 FISMA IG Metrics are broadly aligned with the FY 2017 FISMA CIO Metrics

Function (Section) IG Metrics CIO Metrics Identify (Risk Assessment) X X Protect (Configuration Management) X X Protect (Identification and Authentication) X X Protect (Security Training) X X Detect (ISCM) X X Respond (Incident Response) X X Recover (Contingency Planning) X X

DOT Cybersecurity Summit ¦ FISMA Cybersecurity Performance Metrics and Scoring ¦ May 2, 2017 Scoring Methodology

• Agency IGs will assess capabilities on a spectrum of potential maturity levels

• Overall maturity for each NIST Function will be recommended based on its average maturity level

• Goal is to provide a representative maturity level, but IGs can substitute a different score if they choose

• Overall agency maturity will be determined by the IG with no automatically generated recommendation

• This allows IGs to customize their assessments based on agency circumstances

FY 2017 FISMA IG Metrics scoring methodology will seek to provide a balanced assessment of agency information security capabilities

DOT Cybersecurity Summit ¦ FISMA Cybersecurity Performance Metrics and Scoring ¦ May 2, 2017 Next Steps

• Access the metrics at DHS.gov/FISMA • Hold follow-on training session in July • Work with DHS to make changes to the Cyberscope application

• For 2018, develop a review guide or companion document to the

metrics for IG use

DOT Cybersecurity Summit ¦ FISMA Cybersecurity Performance Metrics and Scoring ¦ May 2, 2017 Q&A

• When is the due date for FISMA this year? • Will the scoring methodology be different this year?

• Will specific questions be weighted differently than others? • Is Level 4 still considered to be the bar for Effectiveness? Can an

agency have an Effective program at Level 3? • Will IGs be required to provide comments in Cyberscope for all

responses not at a Level 4?

Sample of the common questions received by the FISMA Metrics Subcommittee

fisma cybersecurity performance metrics and scoring moses_dot... · metrics and scoring ......

Documents