how to achieve security, reliability, and productivity in less time

Confronting the mission-critical software testing challengeEpisode 1:

How to achieve security, reliability, and productivity in less timeRod CopeCTO

Presenter

Rod CopeCTORogue Wave Softwarerod.cope@roguewave.comTwitter: @RodCope

1. A brief history of testing2. Code security3. Software reliability4. Pulling it together5. Q&A

Agenda

What is mission-critical?

A brief history of testing

The evolution of testing

1970s – 80s Debugging == testing

All I need is unit testing

How did we survive without automated testing?

DevOps is awesome!

Challenges with different methodsAdvantages Disadvantages

Debugging/printfs

• Immediate• Minimal set up

• Limited view of system• Limited tests• Doesn’t scale across

code/team sizeUnit testing • Close to code

• A form of documentation

• Limited view of system• Limited tests• Cumbersome for single

developer to set upBasic automated testing

• Consistency and repeatability

• Speed• Frees developer time

• Can be slow to run• Can be slow to update

DevOps/CI testing

• Scalable & fast• Frees developer time

• Initial set-up costs• Only effective for larger

Debugging/printfs

DevOps/CI testing

Debugging/printfs

DevOps/CI testing

Debugging/printfs

DevOps/CI testing

All have sources of risk

• Human error• Software issues• Hardware

issues

And challenges:• Demands for shorter release times• Increasing feature complexity• Requirements for standards

compliance• Increasing open source use

Poll #1What is the primary method you use to test code?• Code reviews• Unit tests• Manual tests at build time• Automated tests at build time• Automated testing using CI tools

Code security

Changing security landscape

More complex software running inside systems

Multiple sources of software being integrated

Software has to run for many years

This requires a very significant security, safety, & functional verification process

Harder to secure code

Some research

"Security is not considered important"

"Security takes too much time"

"I feel pressured to complete development"

0% 5% 10% 15% 20% 25%

Why are companies not putting more emphasis on security in

their applications?

0% 10% 20% 30% 40% 50%

Is security a priority for your company?

2015 Survey of Automakers and SuppliersPonemon Institute / Rogue Wave Software / Security

Innovation

One of the top flaws in the 2015

National Vulnerability

Database

Example: Memory buffer problems

CWE-119: Software can read or write to locations outside of the boundaries of the memory buffer

• Not checking size of input on copy• Bug allowing writing to arbitrary

locations• Out-of-bounds read• Pointers outside expected range• Untrusted pointer dereference• Uninitialized pointers• Expired pointer references• Access of memory beyond buffer end

Real vulnerability: GNU libc

CVE-2015-1472https://sourceware.org/ml/libc-alpha/2015-02/msg00119.html

• Under certain conditions wscanf can allocate too little memory for the to-be-scanned arguments and overflow the allocated buffer.

• Theoretically, any Linux machine connected to the internet, using this version, is at risk

GNU libc example: fail

GNU libc example: fix

Top four best security practices

• Numeric errors• Code injection• Improper input

validation• Memory buffer

problems

• Numeric errors• Cryptographic issues• Code injection• Memory buffer problems

• Numeric errors• Cryptographic

issues• Code injection• Resource

management errors

• Numeric errors• Resource management errors• Improper access control• Improper input validation

Clean design

Methodical process

Good tools

Careful analysis

Poll #2How much time do developers in your company spend on security (as a percentage of work time)?• 0%• 1 – 25%• 26 - 50%• 51 – 75%• 76 – 100%

Software reliability

Why is reliability important?

May 2015• Boeing 787 Dreamliner had software bug which

caused “total loss of electrical power” after 248 days

December 2015• A software error which calculates prison sentences caused more

than 3,200 US prisoners to be released 49 days early on average

January 6, 2016

• NEST ‘smart’ thermostat software update caused complete battery drain, shutting off heat during January

• Matt Rogers, NEST co-founder & VP Eng:”the bug took a few weeks to show up”

• 2.5 million smart thermostats in U.S. alone

Key industry standards

The argument for standards compliance

• Re-use the expert research of others• Complements existing testing approaches• Recognizable by customers• May already by a requirement

Significantly reduces the cost of producing reliable software

Security:

MISRA C example

a |= 256;b |= 128; c |= 064;

Sets bit 8 of variable a(256 decimal = 0100000000 binary)

Sets bit 7 of variable b(128 decimal = 0010000000 binary)

Is bit 6 set?(64 decimal = 000100000 binary)

Rule 7.1: Octal constants (other than zero) and octal escape sequences shall not be used.

• No, because in C, any constant that begins with 0 is interpreted as an octal number.

• So c is set to the wrong value!

Pulling it together

Why Agile?

Agile increasing

People over processes

Collaborate to build trust and foster change

Set expectations clearly

Test and measure

Share successes

Enable with tools

Pick the right

artifacts

Choose what to keep/throw

Lessons learned

Continuous testing• Check for security issues• Measure conformance to

standards• Examples of CI systems:

TeamCity, Jenkins• Examples of test tool:

static code analysis

Automate testing

AcceptCheck in

Check inDev 2

Check inDev 3 Release

ChangeAdjust and Track

FeedbackReview

Next Iteration

Release to

Market

TestTest

Keys to successful CITo work in a true CI environment test tools must be designed to be:

Automated

Fast(er)

Scalable

RelevantTo reduce feedback time, only changed code should be tested (including regression)

By requiring minimal resources & deploying across multiple agents

By reporting only the information that is required for the given context (example: only the diffs since the last build / build X)

Supporting the most important CI build management systems

Summary

• Identify and prevent vulnerabilities before release

Security: Clean design, methodical process, careful analysis, good tools

• MISRA, OWASP, ISO 26262Reliability: Adopt proven standards

• Jenkins, static code analysis

Automate with tools that are fast, scalable, and relevant

Follow up

Free white paper:

Fitting static code analysis into continuous integrationwww.roguewave.com/resources/white-papers/static-code-analysis-into-continuous-integration

Missed this webinar? Watch it on-demand

How to achieve security, reliability, and productivity in

less time

Watch now.

Stay tunedConfronting the mission-critical software testing

challengeFeb. 8: Static analysis works for mission-critical systems, why not yours?Compare different techniques for testing by analysis and dive into static code analysis, including the types of problems found, barriers to adoption, and fitting it into various developer environments.

Feb. 22: What if you could eliminate the hidden costs of development?Combat different types of development inefficiency by examining error-prone tasks, waiting for resources, “bug fix crowdsourcing,” and more to learn what the industry is doing about them and what you can do to get ahead.

how to achieve security, reliability, and productivity in less time

Software

report 2014-2015 · report 2014-2015 annual report of the...

sv04 - reliability solutions driving asset optimization and...

featured article achieve comprehensive reliability by ......

productivity one of the most important responsibilities of...

security convenience freedom safety time productivity...

an idc infobrief harness people productivity to achieve

using human factors to achieve high reliability in

download 7 ways to achieve operations reliability with apm...

high productivity reliability and cost...

achieve higher productivity, reduced costs, and shorter...

achieve greater safety, reliability, and energy efficiency

zero-compromise idaas: achieve both security and workforce...

indonesian e-commerce requires scalability, reliability and...

reliability productivity green choices - printronix p7000

enhanced reliability empowered productivity

best practices to increase productivity and reliability of...

achieve productivity and quality without...

mobile asset management: achieve higher productivity ... ·...

3 ways you can achieve better asset reliability

service portfolio - monition · monition · future-proofing...