Continuous Application Security“We’re Pulling Out All the Stops”

Jeff Williams@planetlevel

Factories Instrument Everything

Programmable Controllers

Connectors and Adapters

Live Dashboard

Identify problems before they become PROBLEMS

The Phoenix Project

What Is Continuous AppSec?

What: The right defenses for every application are…

PresentCorrectUsed Properly

How: Portfolio and enterprise security controls are verified…

ContinuouslyAutomaticallyIn real time

Building Continuous AppSec

Dynamic Interactive JUnitManual Static

DEV CI TEST QA STAG OPSSEC

Continuous AppSec

Analytics

Your IT Organization…

3) Collect big data security analytics

1)Transform our existing tools into SENSORS2) Instrument entire software organization

Check Your Headers

https://cyh.herokuapp.com/cyh

• CheckYourHeaders– http://cyh.heroku.com/cyh

• OWASP Dependency Check– http://www.owasp.org/index.php/OWASP_Dependency_Check

• Nmap– http://nmap.org

• Sslyze– https://github.com/iSECPartners/sslyze

• OWASP ZAP– http://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

• Minion• Gauntlt

Initial Sensors

• I hate presentations that wait until the end to show me the result.

• If you hate it, please feel free to check your email or play angry birds

• If you like it, I’ll give you the details…

Results/Demo

Monkey Architecture

Hosts Dashboard

Digesters

Sensors

HostsHosts

Speak Evil!

Hear Evil!

See Evil!

Monkey Server

What’s In an AppSec Sensor

Sensor

Config

ToolLauncher

• Config is stuff like– Hostname– Target URLs– Perhaps full sitemap– Credentials– Tool options

• Recursive• Output format• Destination directory

– Etc…

Sensor Launcher and Config?

• Launcher is a small script that runs tool with specified config

Managing Sensors with Puppet

class depcheck {package { 'openjdk-7-jdk':ensure => installed, }exec { "/usr/sbin/update-alternatives --set java /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java": }exec { "/usr/sbin/update-alternatives --set javac /usr/lib/jvm/java-7-openjdk-amd64/bin/javac": }… check bashssh_authorized_key { 'monkey': ….. }… check permissions file { "/home/monkey/agentmonkey/TOOLS/dependency-check-1.1.4-release": source => "puppet:///modules/depcheck/dependency-check-1.1.4-release", recurse => true,}file { "/home/monkey/agentmonkey/SCRIPTS/depcheck.sh": source => "puppet:///modules/depcheck/depcheck.sh", … }cron { "cronDepcheck": command => "/home/monkey/agentmonkey/SCRIPTS/depcheck.sh", user => monkey, minute => 0, }}

Handling Results

rsyncRaw Sensor Data

ARMS Server (with Sensors)

Monkey Server

Standardizing the Data

Digesters

XML

JSON

Text

PDF

CSV

Monkey Format*

*Currently CSV

Digesters

RAW

DIGESTED

Python, XPath, etc…

• Timestamp – April 14, 2014 10:10 AM EDT• IPAddress – 192.168.2.234• Hostname – webgoat.internal• AppName – WebGoat • URL – http://webgoat.internal/WebGoat/attack• LOC – /filepath/Foo.java @ 123 • Tool – DependencyCheck • Category – Platform• Subcategory – Libraries• TestName – CheckCVE• TestDesc – Verify library is…• TestResult – Library has CVE-2011-124• ASVS – V6.2• CWE – CWE-2013-03• Security – 40 (0 to 100)• Coverage – OOS• Confidence – 100

Monkey Format

Initial Categories

Auth’n

Auto-complete

Auth’z

Path Traversal

Headers

Access Control

Caching

Content

CSP

Cookies

Framing

Robots

XSS

Injection

CrossJS

SQL

XSS

Platform

Libraries

Transport

Algorithms

Certificates

Heartbleed

STS

Mixed Content

Unknown

Dashboards

Monkey has aself-organizing dashboard

Sensors report their own category, subcategory, and testname

Cat: TransportSubCat: HeartBleedTestName: heartbleed

HeartBleed

Designing a HeartBleed Sensor

Experiment Style

Negative

Positive

Environment

Dev

CI

Test

QA

Staging

Security

Analysis Technique

Manual

SAST

Passive

IAST

DAST

Data Sources

Code

HTTP

Configuration

Choose based on:• Speed• Accuracy• Feedback• Scalability• Ease of Use• Cost

Data Flow

Control Flow

Platform

Connections

Sampling

Prod

Intelligence

JUnit

• Download scanner• Realize it’s written in Go• Download Go compiler• Add Sensor to Monkey (20 minutes)• Build Digester (10 minutes)• Continuous monitoring enabled in 1 hour!

• And then I realized my mistake…

Adding HeartBleed to Monkey

The Better Way to Test for HeartBleed

Sensors?

What sensors should we add next?

What’s In Your Expected Model?

ExpectedThreat Model

Abuse Cases

Policy

Standards…

Requirements

There is no security without a model

What Are You Actually Testing?

ActualPentest

Code Review

Tools

Arch Review

…

Unfortunately…

ActualExpected

Not being tested

(aka RISK)

Doesn’t need testing(aka WASTE)

Are You Secure?

Secure?

Data Protection

Minimize Sensitive

Data

Role Based Access Control

Encrypt Data in Storage and Transit

Full Disk Encryption

with TrueCryp

t

Programmati

c Encryptio

n with ESAP

I

Libraries

Presen

t and

Up-

to-date

Encryption

Correctness

with

Junit

Tests

ESAPI Used Pro

perly

TLS Everywhere with

Venafi

Logging and Intrusion Detection

Business Concern (category)

Defense Strategies (subcategory)

Actual Defenses(testname)

Sensors

Aligning Sensors with Business Concerns

Fraud Availability

Continuous Application Security!

Expected

Actual

ApplicationPortfolio

A A AA A AA A A

A A AA A AA A A

Application security dashboards

Translate “expected” into sensors

New Threats,Business Priorities

How to Get Started

Thank You!

Hit me up on twitter @planetlevel

Identification and Authentica

tion

Input Vali

dation and Enco

ding

Sessi

on Manage

ment

Sensiti

ve Data Protecti

on

Access

Control/A

uthorizati

on

Error H

andlin

g

Logging and In

trusio

n Detection

Cross

Site Request

Forge

ry (CSR

F)

Platform Sec

urity

Database Sec

urity

Code Quality

Syste

m Availab

ility - D

OS Protecti

on

Accessi

ng Exte

rnal S

ervice

s0%

10%

20%

30%

40%

50%

60%

70%

80%

90%Applications with at Least One Vulnerability in Category

Higher Risk

Lower Risk

Aspect 2013 Global AppSec Risk Report

Source File Result @PreAuthorize

TestSBMBugtrackerController.java @PreAuthorize("hasAnyRole('ROLE_BUG_CREATE','ROLE_BUG_EDIT')")

UpdateSBMBugtrackerController.java @PreAuthorize("hasRole('ROLE_BUG_EDIT')")

SelectBugtrackerController.java @PreAuthorize("hasRole('ROLE_BUG_CREATE')")

CheckAppStatusController.java MISSING

ViewConsoleEventsController.java @PreAuthorize("hasRole('ROLE_CONSOLE_VIEW')")

DeleteEngineConfigController.java @PreAuthorize("hasRole('ROLE_ENGINE_PROFILES')")

DownloadEngineController.java @PreAuthorize("hasRole('ROLE_ENGINE_DOWNLOAD')")

EngineConfigController.java @PreAuthorize("hasRole('ROLE_ENGINE_DOWNLOAD')")

ErrorController.java MISSING

InboxController.java @PreAuthorize("isAuthenticated()")

InstallationWizardController.java @PreAuthorize("isAuthenticated()")

InviteAFriendController.java @PreAuthorize("isAuthenticated()")

LoginController.java MISSING

DeleteMessageController.java @PreAuthorize("isAuthenticated()")

GetSystemMessagesController.java @PreAuthorize("isAdmin()")

Access Control Intelligence Sensor

Known Vulnerable Libraries Sensor

Libraries

SAST

Negative

CI

Run DependencyCheck during every build(and do a build once a month even if nothing changed)

• Run tests through ZAP• ZEST to check CSRF Token• Get results via ZAP REST API

CSRF Defense Sensor

HTTP

Passive

Positive

QA

A Junit Sensor?

Injection Sensors

Data Flow

IAST

Negative

Dev

Use code instrumentationtools for DFA vulnerabilities

• What would you like to gather from all your applications?

• Inventory? Architecture? Outbound connections? Lines of code? Security components?

• All possible…. and all at devops speed and portfolio scale

Architecture, Inventory, and More…

Security Intelligence Sources

HTTPTraffic

Backend Connections

Configuration Data

Libraries and Frameworks

Data Flow

Control Flow

Vulnerability Trace

Enterprise Controls Dashboard

Expected Defense Defense Present?

Defense Correct?

Applications Tested?

Training and Support

Authentication Authorization Cryptography

Validation Escaping Tokens Logging Intrusion Detection Random Numbers

Browser Security

Safe API Wrappers

Object Reference Management

Error Handling

Basic Infrastructure

DEV CI TEST QA STAG OPSSEC

Puppet

rsync

Sensor

Raw Results