static analysis security testing for dummies... and you

Aspect Security | 9175 Guilford Road, Suite 300 | Columbia, MD 21046 | www.aspectsecurity.com

Static Analysis Security Testing for Dummies… and You

OWASP LASCONAustin, TXOct 23, 2015

Application security that just works

©2015 Aspect Security. All Rights Reserved 2

WARNING

 50 slides incoming.

 We’ll be moving fast!

 Come ask questions the end.



ABOUT ME

Kevin FealeyPrincipal Consultant & Practice Lead,

Automation & Integration Services7 years AppSec experience, 2nd LASCON

@secfealzKey Interests:

• Process efficiency/effectiveness (DevOps, SecDev)• Open Source and Commercial Tools• Understanding your SDLC & security processes



ABOUT YOU

 Does your organization use SAST?• Commercial• Open-Source

 How do you use SAST?• All manual scans• Automated in some way

‒ CI/IDE/Other



WHY DO WE NEED TOOLS?

Development

Production

Security

• Manual security activities are bottlenecks for new features and bug fixes to get to production

• With the push for faster deployments, the bottleneck is tightening

• Business goals trump security needs• Lots of “Risk-based decisions”

• Security has to find a way to keep up (and catch up)



APPSEC TOOLBELT

RAST, WAF, Manual Code Review, Pen Testing, Threat Models, Architecture Reviews,…


STATIC APPLICATION SECURITY TESTING TOOLS



STATIC APPLICATION SECURITY TESTING TOOLS

Key benefit and drawback



QUESTION OF THE DAY:

Q: Why do SAST tools have such a bad reputation?• Vendors are not educating their customers

‒ Tool vendors are focused on selling licenses, rather than providing guidance for a successful rollout (ie. implementation services)

‒ Most tools are marketed as low-touch silver-bullets› Leads to running a tool out-of-the-box, with no vulnerability

management program, education of developers, or communication plan

• Most current security tools are complex and noisy, but continue to evolve ‒ They are still better than manual assessments for scale, if tuned and

integrated properly



AGENDA

1. Common SAST Questions1. Why does SAST scanning take so long?2. Why does SAST find so many false-positives?

2. Tailoring SAST, PMD as an example3. SAST Integration Scenarios



BACKGROUND

 Focus on statically typed & compiled languages• SAST on dynamically typed languages (Python, Ruby,

etc.) is much more difficult‒ Fewer tools available‒ Tools that exist are not as powerful

 My bias is toward Java• Similar process for other languages

 Generalizing how SAST works• Some tools may work differently



SAST AT THE HIGHEST LEVEL

Source/

Byte Code

Model

Extraction

Intermediate Representati

ons

Analysis

Results

?Proprietary

Models




Source/

Byte Code

Model

Extraction


ons

Analysis

Results

?Proprietary

Models

Analysis is performed on intermediate representations, not on source/byte code




Source/

Byte Code

Model

Extraction


ons

Analysis

Results

Parser/Tokenizer per language Common rule/analysis engine

New Language Support = Translator + Rules



HIGH LEVEL SAST PROCESS

Compilation

Scanner Model

Extraction

Pattern Matching

Control Flow

AnalysisData Flow Analysis

Translation Scan

Performed by SAST Tool

*The activities listed are not performed by all SAST tools. Some tools perform a subset of these – others perform additional activities.

find “password” in*.properties

out.write(request.

getParameter(“param”));

Database connection not

closed in ‘finally’ block

Syntactic / Semantic Checking



OTHER TYPES OF ANALYZERS

Semantic •Unsafe function invocations

Structural •Pervasive issues that impact large portions of the application, like dead code and loggers not declared as static

Configuration •Checks configuration files for reasonable timeouts, etc.



SAST WORKFLOW WITH JAVA

Compile Java Files

Compile JSPs

Perform Framew

ork Analysis

Pattern Matchi

ngAnalysis

Generate

Results

Translation Scan

*The activities listed are not performed by all SAST tools. Some tools perform a subset of these – others perform additional activities.

Compile-time vs Runtime issues

Dependency issues

Only supported

frameworks



FRAMEWORK HANDLING (APPSCAN SOURCE)


TYPES OF ANALYSIS



PATTERN MATCHING (APPSCAN SOURCE)

 Accessible through the Security Analyst UI Modify/Create rule-sets (ex. Java, SQL) Or create new scan rules



CONTROL FLOW ANALYSIS

XMLReader parser = XMLReaderFactory.createXMLReader("org.apache.xerces.parsers.SAXParser"); parser.parse(input);//Prevent XXE attacksparser.setFeature("http://apache.org/xml/features/disallow-doctype-decl",true);



DATA FLOW ANALYSIS

Source Taint

Sink

Data Flow / Trace Nodes

Tainted Data

Vulnerability?



TAINT ANALYSIS

Untrusted Source Taint

Sink

a.tainted = true

Vulnerability!

b.tainted = truec.tainted = trued.tainted = true

e.tainted = true



STRING ANALYSIS (APPROXIMATED)

Source Taint

Sink

a.taint = a;

Vulnerability!

b.taint = b.substring(0, b.length()-4)

…



LOST SINKS

Intermediate Representations (IRs) are not produced for dependencies

Lost Sink

Most SAST tools do not know what happens in externalMethodCall()Some tools allow for Lost Sink Resolution

• Is tainted data returned from this method?



EVERY PATH IS CHECKED

•SAST does not know what will happen at runtime.•How many function calls happen when bar() is invoked? bar2()?

• SAST will check both.


ANSWER SUMMARY



WHY DOES SAST SCANNING TAKE SO LONG?

1. The entire application is compiled2. The compiled code is translated to various models

1. Every line of code is translated to the appropriate IRs

3. Scanning is a multi-step process:1. Regex search finds low hanging fruit2. Semantic (context-aware) search finds use of dangerous functions3. The order of execution for every potential function is checked

(control flow analysis)4. [Nearly] Every possible path in the application is searched for

known dangerous patterns (data flow analysis)5. Findings are sorted/categorized to produce your report



WHY DOES SAST FIND SO MANY FALSE-POSITIVES?

•Every potential data flow and control flow path is checked

• SAST tools do not know which logic will be executed at runtime• Paths that will never be executed will be checked

•SAST tools do not know which sources your business trusts

• Data from a database may be populated by an administrator, a malicious user, or another well-intentioned application

•Many SAST tools are architected with post-processing in mind, whether manual or automated



INTERPROCEDURAL VS INTRAPROCEDURAL

 Interprocedural:• Whole program analysis• Tracks variables across objects/procedures/functions• Most commercial tools

 Intraprocedural:• Single procedure analysis• Most open-source tools• Most dynamically-typed languages scanners


PMD



WHAT IS PMD?

PMD is a source code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and so forth. It supports Java, JavaScript, PLSQL, Apache Velocity, XML, XSL.

Additionally it includes CPD, the copy-paste-detector. CPD finds duplicated code in Java, C, C++, C#, PHP, Ruby, Fortran, JavaScript, PLSQL, Apache Velocity, Ruby, Scala, Objective C, Matlab, Python, Go.

-https://pmd.github.io/



WHY DID I CHOOSE PMD?

PMD was not written to find security issues.

Many non-security tools that you are already using can be re-purposed or extended to provide security value.

QA Team



SHOUT OUT

 Only PMD security-focused ruleset I’ve found: https://github.com/GDSSecurity/GDS-PMD-Security-Rules



PMD RULE ASSUMPTIONS

 Assumptions:• Organization builds applications with Spring• Spring annotations are used to specify allowed HTTP

request method ‒ @RequestMapping(method=RequestMethod.POST)

• Functions lacking this annotation allow any HTTP method

• Security policy states that at least one HTTP method must be explicitly set for each entry point. Only HTTP GET and POST are allowed.


PMD: STEP 1Writing test cases



SPRING MVC CONTROLLERS

 Test Case #1: Correct Method

 Test Case #2: Incorrect Method

 Test Case #3: Missing Method



SPRING MVC CONTROLLERS

 Test Case #4: Correct Multiple Methods

 Test Case #5: Incorrect Multiple Methods


PMD: STEP 2Generate and Analyze the AST

(Reverse-Engineering)



WHAT ARE WE LOOKING FOR?

“RequestMapping” variable as a “Name” object

Child of “NormalAnnotation”

“method” variable

Type: “MemberValuePair”

Value we are looking for as a “Name” object

Child of “PrimaryPrefix”


PMD: STEP 3Write the Rule



WRITING OUR RULE



FINAL RESULT



PMD REPORT

 Not listed:CorrectRequestMethond.javaMultipleCorrectRequestMethods.java



ALTERNATIVELY – XPATH ONLY

 //NormalAnnotation  /Name[@Image="RequestMapping"]  [count(..//MemberValuePair//PrimaryPrefix/Name[@Image="RequestMethod.POST"]) < 1]

Doesn’t handle multiple methods – that was too hard.


WHERE DOES SAST FIT IN?



SAST TIPS

SAST is meaningless without vulnerability management and remediation

• Any tool is only a part of an appsec program‒ People, Process, Technology

• Do not scale until you can have vetted processes• Developers can’t remediate effectively without support

 SAST belongs in CI• Does not need to be run on every build

‒ Scan frequency based on application risk (nightly/weekly?)• Commercial SAST != developer tool

‒ Running of tool should be transparent to devs

 Developers should never see false positives• Results should always be triaged/filtered

‒ False positives cause extra work for developers‒ False positives reduce confidence in the process



MORE TIPS

Alternatives to vendor-supplied consoles• SonarQube, ThreadFix, GRC tools, etc.

 Define continuous improvement for SAST• Regular modification of filters and rules• Tool configuration is an opportunity to strengthen

secdev



ENTERPRISE ROLLOUT OF APPSCAN SOURCE: STRATEGY

Application PortfolioLess CriticalMore Critical

Cov

erag

e / A

ssur

ance

Scan

ScanScan

Full

Sca

n/R

evie

w

Remediation Guidance

Incr

ease

Cov

erag

e Reduce R

isk

• More time to review critical applications• More time to find and fix complex issues


APPENDIX



OWASP BENCHMARK

  As of 10/08/2015

• G: Open-Source SAST average (16.40%)

G



TOOLS IMPROVE!

  Results from September, 2015 – October, 2015



COMMERCIAL SAST STRENGTHS AND WEAKNESSES

  As of 10/08/2015



VERY IMPORTANT

• Remediation requires more than just tools• Effective vulnerability management program• Training• Developer support

• Never deliver false-positives to developers



COMMERCIAL SAST TOOLS TESTED

Checkmarx CxSASTCoverity Code Advisor (On-Demand and stand-alone version)HP Fortify (On-Demand and stand-alone versions)IBM AppScan SourceParasoft JtestVeracode SAST



SCANNER MODEL EXTRACTION

Intermediate Representations

• For commercial tools, usually proprietary formats

• Common formats (per vendor) regardless of language

• Some tools build models for each file (intraprocedural); others for the whole application (interprocedural)

Modified AST, NST, Framework modeling, etc.

Kevin Fealey

Would like to get additional details about AppScan Source on this from IBM. What files/formats are produced?



SENSOR INTEGRATION FRAMEWORK

• Configure your Sensors• Configure your Publishers• Build job and see your results

https://github.com/aspectsecurity/sensor-integration-framework