SB41: Assessing Risk at the Software Level

Matt Moynahan, CEOMay 21, 2008

Agenda

The Unbounded Risk of Insecure Software

Risk Introduced from both Development and Procurement Processes

Why has Measuring Risk at the Software Level Been so Difficult

Components of a Risk Assessment Framework at the Software Level

Conclusion

The Unbounded Risk of Insecure SoftwareApplications are the “Attack Surface” Leading to the Data

3

State of Software Industry: » Over $750 Billion in off-the-

shelf, internally developed and outsourced software produced or sold each year

» This is the world’s largest manufacturing industry with no uniform standards or insight into security, risk or liability of the final product

» Historical approach of leveraging customers to “debug”or “test” software products

» Over 7,000 new vulnerabilities reported in 2007

Key Statistics (Gartner/NIST):

95% of all vulnerabilities are in software

75 % of attacks are at the application level

78 % of threats target business information

Only 10% of IT security budget is being spent on application security

Some Recent Real World Examples

4

Oct. 10th, 2007 – Commerce Bank’s website was exploited via SQL Injection and 3,000 customer records were stolen

Oct. 6th, 2007 – Hacker locks user accounts and block sales using exposed admin functions

Oct. 2nd , 2007 – Town of Pembroke School System website exposes SSN of employees due to insufficient authorization

Aug. 12th , 2007 – United Nations website was exploited via a SQL Injection attack resulting in several documents having their content altered

Oct. 23rd , 2007 – Colorado Rockies website is brought down by hackers preventing fans from purchasing world series tickets

Conflicting Trends Impacting Software Assurance Efforts

5

Economic GlobalizationEconomic Globalization Decreasing Trust in CyberspaceDecreasing Trust in Cyberspace

Increasing Need for TransparencyIncreasing Need for TransparencyLiability Issues in Supply ChainLiability Issues in Supply Chain

Agenda

The Unbounded Risk of Insecure Software

Risk Introduced from both Development and Procurement Processes

Why has Measuring Risk at the Software Level Been so Difficult

Components of a Risk Assessment Framework at the Software Level

Conclusion

Application Development and Procurement Have Become Increasingly Distributed and Complex

Developed In-house Purchased

Application Development and Procurement Have Become Increasingly Distributed and Complex

Enterprise Employees

US Dev. Center A

3rd Party Libraries

Offshore

Open Source

Developed In-house

US Dev. Center B

Company Employees

Contractors

Foreign Contractors

Foreign Sub-Contractors

Application Development and Procurement Have Become Increasingly Distributed and Complex

ISV Employees

Foreign Contractor

Outsourcer Employees

Global

ISV (COTS)

OutsourcePartner B

Purchased

OutsourcePartner A

License 3rd

Party Libraries

License 3rd

Party Libraries

Open Source

Indian Contractor

Chinese Contractor

Application Development and Procurement Have Become Increasingly Distributed and Complex

Enterprise Employees

US Dev. Center A

3rd Party Libraries

Offshore

Open Source

Developed In-house

US Dev. Center B

Company Employees

Contractors

Foreign Contractors

ISV Employees

Outsource

Outsourcer Employees

Global

ISV (COTS)

OutsourcePartner B

Purchased

OutsourcePartner A

License 3rd

Party Libraries

License 3rd

Party Libraries

Open Source

Foreign Sub-Contractors

Foreign Contractor

Indian Contractor

Chinese Contractor

The Result is “Weak” Applications of Unknown Origin and Security Quality

Source: U.S. CERT, Feb. 2008

Explosion in # of NewApplication Vulnerabilities

Impossible to Determine Who Has Tested Code for Security

Agenda

The Unbounded Risk of Insecure Software

Risk Introduced from both Development and Procurement Processes

Why has Measuring Risk at the Software Level Been so Difficult

Components of a Risk Assessment Framework at the Software Level

Conclusion

Would you purchase a hot water heater without knowing how much it would cost to operate?

Why Have Independent Software Assessments Been So Difficult to Achieve?

Would Coke and Pepsi agree to a taste test in a grocery store if they had to give you their secret formulas (e.g. source code)?

14

Other Industries Have Been Successful in Creating Demand for Security in Products

Auto Safety Software SecurityPrior Status • Limited visibility into safety of cars

• No benchmark or consistent safety measurement

• Focus on end-point (driver behavior), not source (car manufacturer)

• Result: high insurance rates, unbounded risk

• Limited visibility into safety of COTS• No benchmark or consistent security

measurement• Focus on end-point (enterprise),

not source (software vendor)• Result: high operational cost,

unbounded risk

Industry Initiative

Results• Shift to source of risk• Safety improvements (seat belts,

air bags, crumple zones, stability control) built into cars

• Objective benchmark for auto safety

• Shift to source of risk• Security built into the SDLC• Objective benchmarking and

reasonable security thresholds for application security

Market Pressure Changes Vendor Behavior

July 2001 – code red worm running on Microsoft Web server software, causes $2.6 billion in productivity lossOctober 2001 – Gartner recommends some customers to seek alternatives to Microsoft softwareJan. 15, 2002: Bill Gates sends email to all employees

“When we face a choice between adding features and resolving security issues, we need to choose security. Trustworthy Computing is the highest priority for all the work we are doing.” – Bill Gates

February 2003 - Microsoft signs pact with China and other countries to reveal Windows source code in response to backdoor security concerns2004 - Microsoft trains over 500,000 IT professionals worldwide on security best practices, including 140,000 partners2005 - Microsoft makes Secure Development Lifecycle (SDL) mandatory for all Microsoft Internet applicationsNov. 2006 - CIO Insight Study reveals that attacks on Microsoft products is having an affect on IT purchasing – 30 percent of IT executives have moved systems off of Windows to reduce their overall security risk

15

Microsoft SDL

Quantifying Risks at the Software Level

Why Is It So Difficult?– Lack of security metrics and standards– Limited access to source code across “nested” supply chain– Vendor self-certification not sufficient given current threat space– Manual code reviews too time consuming and too expensive for

broad adoptionWhat is Needed?– Ability to quantify security risk for individual applications and entire

application portfolios based on mission criticality– Complete, accurate and cost-effective assessment– Vendor motivation – positive “network effects” - from achieving

independent certification– Enterprise “sensitivity” to the state of the software industry

16

Agenda

The Unbounded Risk of Insecure Software

Risk Introduced from both Development and Procurement Processes

Why has Measuring Risk at the Software Level Been so Difficult

Components of a Risk Assessment Framework at the Software Level

Conclusion

Making Sense of App Security Testing Terminology

White Box (“Static”) Testing – Consultants; Coverity; Fortify, OunceTesting an application with full knowledge and access to all source code and architecture documents. In theory, having full application knowledge can reveal bugs and vulnerabilities more quickly than the "trial and error" method of black box testing.If performed using the application source code it can be impossible to identify flaws based on system misconfiguration that exist only in the application deployment environment.

Black Box (“Dynamic”) Testing – Cosultants; SPI Dynamics; WatchfireTesting an application without having specific knowledge to its internal, no access to the source code, and limited or no knowledge of the web application architecture. Most closely mimics how an attacker typically approaches applications. However, due to the lack of internal application knowledge, the uncovering of bugs and/or vulnerabilities can take longer and may be limited in scope.

18

Making Sense of Web App Security Testing Terminology

Gray Box (“Hybrid”) Testing – Example: Security ConsultantsTesting an application while having some knowledge of the internals of a system. It is a combination of both black and white box testing, and combines aspects of each. Allows security analysts to run multiple testing techniques against a target application (automated and manual) and enables better prioritization of efforts; higher discovery of vulnerabilities; and better approximation of advantages of an attacker.

19

Static BinaryAnalysis

DynamicAnalysis

Manual Analysis

Key Characteristics– Test properly (100 % of code base) – Test practically (prioritized, cost-effective, accurate and

automated)– Standards-based metrics that enable internal and

external comparisons– Inclusion of temporal nature of attacks– Test without infringing upon IP rights

Key Assurance Solution Components– Integrating multiple testing techniques– Automation instrumented to assist humans– Metrics that incorporate deployment environment– A dynamic service model not requiring access to source

code

Independent Application Security Assurance

20

It Is Not About Which Technique is Better but Your Approach To Securing Apps that is Most Important

Application Assurance Level

Acc

epta

ble

Ris

k

Automated Static Source & Binary Analysis

Cost & Broader CWE Coverage

Automated Dynamic Analysis

Manual Penetration Testing

Manual Review

Code Security Testing Techniques

Current Score Target Score

Low High

Customer Questionnaire

PARTNER

PARTNER

Low Medium High Very HighVery Low

Based On An Industry Standards-Based Rating SystemEquivalent approach to “Moody’s” of the Software IndustryIndependent assurance based on industry standards (CWE, CVSS, NIST)Testing for both pre-deployment and deployment phasesCan be implemented quickly without the need for any additional hardware, software or training requirementsConsistent and accommodates different deployment environmentsEasy to implement

22

Software Risk Assessment Ratings

23

75 77

50

85 82 75

95979190+ 90+ 90+

80+80+80+

70+ 70+

60+

Sample High Level Report

24

Improvement Can Be Visibly Tracked Over Time

25

Commitment from Vendor to Comply with the Remediation Roadmap becomes part of Acceptance Criteria

Summary Conclusions

1. Understand sources (internal & external) and types (vulnerabilities; security features; backdoors) of risk across entire application portfolio (law of weakest link)

2. Take a pragmatic approach: Focus on minimizing risk based on criticality of your application portfolio

3. Use a Grading system to Understand Risk Levels in Software

4. Leverage industry standards to maintain consistent benchmark risk levels of internally developed applications, 3rd party COTS suppliers and development contractors

5. Work collaboratively with the software community and reward COTS suppliers and service providers that make a commitment to improving software security

26

Abstract

SB41: Assessing Risk at the Software Level (200L) May 21, 2008: 10:45AM – 12:00PM

A critical piece of securing any organization’s infrastructure is the ability to assess the risk of the software running within that organization. In fact, more than 70 percent of security attacks are now happening at the software level. Whether those applications are developed in-house, offshore, directly purchased, or inherited through an M&A transaction, enterprises are clearly at risk. If this risk is not managed properly, organizations face the consequences of a data breach that could cost them customers, drive down their stock price, and/or devastate their brand. This session will examine approaches to third party software validation and how it can help organizations to ensure the quality of the software and applications they implement. Attendees will leave armed with strategies to diminish operational risk and maintain corporate value.