Role Of Code Standards in Business Risk Mitigation

1

Dave Norton Dark Matter Advisory Advisor Consortium for Information & Software Quality david.norton@it-cisq.org

Finance Cyber Incidents in the UK, Up 1087% Increase Year on Year

©2020 CISQ 2

• 21%, are related to third-party failure, i.e., systems the reporting organization did not control.

• However, many of the other incidents had their origins in third-party developed software now owned by the reporting organization.

Even CEOS Are Paying The Price For Poor IT Quality

©2020 CISQ 3

British Airways’ chief executive Álex Cruz says he will not resign despite a “catastrophic” IT system failure that grounded scores of flights

Paul Pester forced to step down as CEO of TSB after the disruption caused to millions of customers by the bank’s very public failed IT upgrade

Former Equifax CEO Richard Smith says he is "deeply sorry" for the security breach in which sensitive personal information of as many as 143 million Americans was compromised

Complex Technology Stack

©2020 CISQ 4

Mul

ti-la

ngua

ge, m

ulti-

laye

r Arc

hite

ctur

e

EJB PL/SQL

Oracle

SQL Server

DB2

T/SQL

Hibernate

Spring

Struts .NET

COBOL

IMS

Messaging

Sybase

• Code style & layout • Expression complexity • Code documentation • Class or program design • Basic coding standards • Developer level

Unit Level 1

Technology Stack

Java Java

Java

Web Services • Single language/technology layer

• Intra-technology architecture • Intra-layer dependencies • Inter-program invocation • Security vulnerabilities • Development team level

Technology Level 2

Integration quality Architectural compliance Risk propagation Application security Resiliency checks Transaction integrity Function point, Effort estimation

Data access control SDK versioning Calibration across

technologies IT organization level

System Level 3

JSP ASP.NET APIs

We Want More Productivity, But at What Cost?

©2020 CISQ 5

Everyone wants faster time to market, but few want to hear about the risks

Increasing Technical Debt

©2020 CISQ 6 Software Quality Iceberg (Code Complete, Steve McConnell)

Code complexity

Maintainability

Internal Coupling

Functional Size

Redundant code Testability

External Coupling

Operating Cost Maintenance Cost

Reliability Performance

Business Value

https://forio.com/simulate/dpnorton66/tech-debt-v3/simulation/#

Simulation Of 120 Day Project

7

Example After 120 Day Project – Average Team

©2019 CISQ 8

Refactoring FTE Tech Debt Refactoring Cost

Team Size Inject Rate Rate Days Left At $240 At $1040

5 10 - 25% 10% 63.2 $15,168 $65,728

10 10 - 25% 10% 126.4 $30,336 $131,456

20 10 - 25% 10% 252.8 $60,672 $262,912

https://forio.com/simulate/dpnorton66/tech-debt-v3/simulation/#

Microservices and API’s Can Accelerate Architecture Debt and Complexity

Tactical Enterprise

Com

plex

ity

Complexity is not a constant

It is not a linear function of the enterprise

It's a nonlinear function that may level "S" or rise exponentially

One or Two Poor API’s Could Push You Over The Edge

In a nonlinear system, 90% of the complexity is a result of less than 10% of the node connections.

9

Suppliers Have To Build Quality In From The Start

©2020 CISQ 10

©2020 CISQ 11

Suppliers Have To Build Quality In From The Start

Finding The Right Balance Is Difficult, However We Can Make It Easer

12

Productivity Quality

©2020 CISQ

We Need Standards We Can Implement With DevOps

We built this city, we built this city on rock an' roll 13 ©2020 CISQ

We Need Standards We Can Implement With DevOps

We built this city, we built this city on rock an' roll 14

Let’s Learn From The Past

©2020 CISQ 15

As industries mature they automate, from robots to fly-by-wire

We Need To Start With Standards

https://www.it-cisq.org/standards/ 16

Building A Trust Relationship Based On Standards

17

RFP

©2020 CISQ

Embed Software Quality & Sizing Standards Into Request For Proposal or Quotes

18

RFP

RFP

©2020 CISQ

©2020 CISQ 19

Sample RFP CISQ has been referenced by the U.S. General Services Administration (GSA), formally citing CISQ requirements in a Information Technology (IT) statement of work from the Office of the CIO for the Office of Public Buildings. GSA is an independent agency of the U.S. government that supports general services of Federal agencies. See page 21, section 5.9 in GSA’s document, Schedule 70 Blank Purchase Agreement for IT and Development Services… “PB-ITS (Project Based IT Services) is seeking to establish code quality standards for its existing code base, as well as new development tasks. As an emerging standard, PB-ITS references the Consortium for Information Software Quality (CISQ) for guidance on how to measure, evaluate and improve software.”

Embed Software Quality & Sizing Standards Into Request For Proposal or Quotes

Agree Productivity Levels With Suppliers Based On Automated Sizing Code – Combine With Manual Sizing Of Requirement

RFP

RFP

20 ©2020 CISQ

Embed The Agreed Sizing Method and Productivity Into The Statements of Work

RFP

RFP

©2020 CISQ 21

©2020 CISQ 22

1. Contracting and Productivity 1. Productivity

The contracted is based on a bases level of productivity of 18 Function Pointers per Staff Month [1]. A staff month is defined as 22 days per calendar month, 8 hours per day, equalling 176 working hours per month. Attentively the contracted is based on a bases level of productivity of 9.5 hours per function point [1].

1. Rate

The supplier shall invoice at a rate of € 300 [2] per function point delivered to the client as measured by ISO 19515 Information technology — Object Management Group Automated Function Points (AFP), 1.0 defined in section 3.4 Exceptions to the rate and activities that will not be invoiced by function point must be agreed in advance of contract signing.

Embed The Agreed Sizing Method and Productivity Into The Statements of Work

Suppliers Should Be Ready To Develop to the Standards

RFP

RFP

©2020 CISQ 23

Hello hello

RFP

RFP

Suppliers Should Be Ready To Develop to the Standards

©2020 CISQ 24

Suppliers’ Teams Should Use Tools That Support CISQ AFP and ISO Sizing Standards

Automatic Analysis Of The Size Of The Code In Function Points

©2020 CISQ 25

Team Dashboards Should Clearly Show The Size Of Code Developed and Enhanced

©2020 CISQ 26

Teams Are Still Free To Use Agile & DevOps Story Point Sizing, Automated Function Points Counted In The Background

©2020 CISQ 27

Hello hello 28

RFP

RFP

Do Not Just Focus On Size of The Code, Verify The Quality – Automatically

Hello hello

Do Not Just Focus On Size of The Code, Verify The Quality – Automatically

29

RFP

RFP

29

Suppliers Teams Should Verify Code Quality, and Check For Vulnerabilities Against CISQ Standards

•Security: Measures weaknesses in source code representing the most exploited security weaknesses in software including the CWE/Sans Institute Top 25 Most Dangerous Security Errors and OWASP Top 10 •Reliability: Measures weaknesses in source code impacting the availability, fault tolerance, and recoverability of software •Performance Efficiency: Measures weaknesses in source code impacting response time and utilization of processor, memory, and other resources •Maintainability: Measures weaknesses in source code impacting the comprehensibility, changeability, testability, and scalability of software •Technical Debt: A measure of corrective maintenance effort due to the CISQ code quality weaknesses remaining in a software application

©2020 CISQ 30

Hello hello

End to End Trust Relationship Based On Standards

31

RFP

RFP

31

Building A Foundation Quality Standards That Fit Modern Methods and Architecture

• Automated

• Product focused vs project

• Support Event and API Architecture

• Integrated in to DevOps & DevSecOps

Toolchain

Quality Standards That Are :

©2020 CISQ 32

Focus on Culture and Behavior – Be Specific

©2020 CISQ 33

• Don’t expect everyone to like automation, some people just like doing it the hard way

• Incentivize the behavior you want for the individual and team.

• Have agreed metrics and KPI linked to automation.

• Show results

Develop The Correct Skills, But Focus More on Behaviors

©2020 CISQ 34

Engineering Value Stream Design

Dashboard Design

Standards Champion (A Hard Role)

Toolchain Integration

Stay in Control With Agile Governance – Don’t Push From The Top, Grow From The Bottom

• Communities of Practice

• Lease Train Toolchain Consistency

• Automation Best

Practice

©2020 CISQ 35

Gamify - Link Automation & Consistency to Team Autonomy

Auto

nom

y

Time of Deployments

Intra-day allowed

After hours and on weekends

Frequency of Deployments

No limits on changes per

today

Few changes per week

Change Advisory

Board CAB for

information purposes only

CAB for all changes

Freeze Periods

Only exceptional change freeze periods apply

All freeze periods apply

Continuous Integration

Environments

Quality Assurance Incident

Management Release

Management

Coding Practices

Team A

Level of Automation

Team B

Obtain Commitment From The Team and Product Owner Agreement

©2020 CISQ 37

Certify The Environment and Lock It Down, But Make Sure There Is A Process To Change It Quickly and Consistetly

©2020 CISQ 38

Set Targets Based On CISQ Measures To Reduce TCO

• Security: Security weakness and vulnerabilities • Reliability: Availability, fault tolerance, and recoverability of software • Performance Efficiency: Response time and resources utilization • Maintainability: Changeability, testability, and scalability of software • Technical Debt: Corrective maintenance effort

Build Standards Into The Contract Sample RFP CISQ has been referenced by the U.S. General Services

Administration (GSA), formally citing CISQ requirements in a Information Technology (IT) statement of work from the Office of the CIO for the Office of Public Buildings. GSA is an independent agency of the U.S. government that supports general services of Federal agencies. See page 21, section 5.9 in GSA’s document, Schedule 70 Blank Purchase Agreement for IT and Development Services… “PB-ITS (Project Based IT Services) is seeking to establish code quality standards for its existing code base, as well as new development tasks. As an emerging standard, PB-ITS references the Consortium for Information Software Quality (CISQ) for guidance on how to measure, evaluate and improve software.”

©2020 CISQ 40

Hello hello

Working With Suppliers & CISQ

Scorecard Measurement and discussion in

governance committees to help set behavior

SLAs Treat software enhancements

and maintenance as a service; track levels, penalties, credits

Recommendation email Email to vendor delivery leaders

that they should consider using CISQ guidelines for all ADM work

Acceptance criteria Measure and demand minimal

set of acceptance criteria for any new development or release

RFP Initial statement of requirements

and project definition can set the tone for quality of deliverables

SOW Definition of specific project

scope and deliverable can include definition of quality and security

Six Levels of Engaging Vendors with CISQ Standards

Focus on Outcomes

©2020 CISQ 42

Use CISQ Guidelines To Help You On Your Journey

• Using Software Measurement in SLAs: Integrating CISQ Size and Structural Quality Measures into Contractual Relationships

• Sample Acceptance Criteria with CISQ Standardized Metrics

• Contracting Best Practice - Improve Supplier Productivity Using the Automated Function Point Standard

• Contracting Best Practice - Lower Risk and Improve Outcomes with Suppliers by Using Software Structural Quality Standards

©2020 CISQ 43

Help Us Develop The Next Generation Of Digital Standards

©2020 CISQ 44