Treating Security Vulnerabilities As Software Defects

SANS What Works In AppSec 2010

Friday February 5th, 2010

1

Agenda

• Something Most Security Vendors Won’t Tell You

• The Wrong Way to Do It

• A More Excellent Way

• Strategies

• Demo

• Questions?

Something Most Security Vendors Won’t Tell You

2

Something Most Security Vendors Won’t Tell You

Finding Vulnerabilities is Easy

Fixing Vulnerabilities is Valuable

3

The Wrong Way to Do It

4

The Wrong Way to Do It

Dan: What is your application security strategy

A: We bought Scanner XYZ

Dan: Cool! Have you started using it?

A: Yes. The analyst who wanted us to buy it ran a bunch of scans when we got

the license key.

Dan: All right! Did you find anything?

A: Oh yeah! We found all sorts of scary stuff.

Dan: Well what did you do about it?

A: We sent the PDF report to the development team and told them to fix the

problems.

Dan: Were they successful?

A: I don’t know. I guess I should check in on that…

5

Why Is This Bad?

• PDFs are blobs

• Email is infinitely ignorable

• Lumps all vulnerabilities together

• No guidance for developers

• Just plain rude

6

A More Excellent Way

7

A More Excellent Way

• Treat (Application) Security Vulnerabilities as Software Defects

• Why?

– Developers have to fix the issues eventually

– Developers understand defects

– Even most “loosely-structured” development teams have defect tracking systems

8

What Makes a Good Security Defect?

9

What Makes a Good Security Defect?

• Why do I care?

• Scope

• Where is it?

• How do I fix it?

10

Why Do I Care?

11

Why Do I Care?

• Do not rely solely on the defect to communicate this

– Simply pumping defects into the defect tracking system is unlikely to be effective

• Provide context

• Provide steps to reproduce

– Automated if possible

• Transparency!

12

Scope

13

Scope

• Defects that take 5 minutes to fix take far longer to administer

– Especially with mature (elaborate) QA processes

• Maximum time: 16 hours– http://www.joelonsoftware.com/items/2007/10/26.html

• Target: 1-16 hours

– Long enough to be an actual task, short enough to be predictable

– Defects for technical vulnerabilities should be shorter

– Defects for logical vulnerabilities can be longer

14

Where Is It?

15

Where Is It?

• Providing location information removes a “barrier” to fixing

• Better location information leads to quicker fix times

• Dynamic analysis: attack surface location

– Vulnerability type, URL, possibly parameter

– (For web applications)

• Static analysis: code location

– Filename

– Line (and hopefully column)

– Include actual code if possible in case underlying codebase has changed

16

How Do I Fix It?

17

How Do I Fix It?

• Prescriptive guidance is required here

– Removes a reason not to fix

– Leads to consistency

• Does your organization have an ESAPI? Does it address this issue?

18

Why Is This Approach Better?

• Defects are structured data

• Defects are durable

• Vulnerabilities have been portioned out into tractable chunks of “work”

• We have provided prescriptive guidance

• Communicates with developers via systems they already use

19

Strategies

20

Strategies

• Group by location

• Group by type

• Group by severity

21

Grouping By Location

• By file/URL or by directory

• Pros:

– Helpful if there is one “owner” for that area of the code

– Can help to minimize requirements for QA regression testing

• Cons:

– Different vulnerability types require different fixes

– Can be hard to keep things straight

22

Grouping By Type

• By vulnerability type (XSS, SQL injection, authorization issue, etc)

• Pros:

– Similar vulnerabilities often have very similar fixes

– Economies of assembly lines – get Henry Ford on vulnerabilities

– Approach with a “punchlist” mentality

• Cons:

– There can be LOTS of vulnerabilities of a given type if bad coding idioms are in use

23

Grouping By Severity

• High, medium, low

• Pros:

– Can help you game certain metric programs

• Cons:

– Least tied to how developers work

– Different types of vulnerabilities

– Cutting across functional areas

24

Strategies (continued)

• Combine more than one

– Group by type or severity and then by location

25

What About BIG Issues?

• Serious issues can map to multiple defects

• REALLY serious issues can map to enterprise change management

initiatives

26

What About Non-Software Vulnerabilities?

• Transition to change management systems rather than defect tracking

systems

27

Demo

28

Contact

Dan Cornell

dan@denimgroup.com

(210) 572-4400

@danielcornell

Web: www.denimgroup.com

Blog: blog.denimgroup.com

Vuln Mgr: vulnerabilitymanager.denimgroup.com

29