gray, the new black gray-box web vulnerability testing brian chess founder / chief scientist fortify...
TRANSCRIPT
Gray, the New BlackGray-Box Web Vulnerability Testing
Brian ChessFounder / Chief Scientist
Fortify Software, an HP CompanyJune 22, 2011
Todo
• Define gray-box testing• Why black-box is insufficient• What we built• Examples• Haters club
Definitions
• Black-box testing• System-level tests• No assumptions about implementation
Definitions
• White-box testing• Examine implementation• Test components in isolation
Definitions
• Gray-box testing• System-level tests (like black-box)• Examine implementation (like white-box)
The Software Security Game
• Objective• Rules vs. Strategy• Playing Field
OBJECTIVE:Protect everything
OBJECTIVE:Exploit one vulnerability
Rules for the Defender
1. Don’t attack the attacker
Rules vs. Strategy
Rules• Don’t attack the attacker
Strategy• Emulate attacker’s techniques
Who wins?
• Technology• Expertise
Who wins?
• Time
• Technology• Expertise
Who wins?
• Technology• Expertise• Time
Changing the odds
The Defender’s Advantage
• Time
• InsideAccess
• Technology• Expertise
Prior Art
• 2005: Concolic testing: Sen, University of Illinois
• 2008: Microsoft SAGE: Godefroid, MSR
• 2008: Test Gen for Web Apps: Shay et al, U. Washington
• 2008: Accunetix: Accusensor
Access to the Software
Allows for ‘Hybrid’ analysis
Dynamic
Analysis
Black-box Approach
Static Analysis
White-box Approach
‘Hybrid’ Analysis
Dynamic Analysis
Static Analysis
Mostly Broken
Correlation Engine
The ‘Real-Time Hybrid’ Approach
Dynamic Analysis
Static Analysis
Good Results
Real-Time Analysis
Correlation Engine
Evolving to Integrated Analysis
Dynamic Analysis
Application
Real-Time Analysis
Real-time link
• Find More• Fix Faster
Find More
• Reduce false negatives• Automatic attack surface identification• Understand effects of attacks
• Detect new types of vulnerabilities• Privacy violation, Log Forging
Attack surface identification
/login.jsp
/pages/account.jsp
/pages/balance.jsp
/admin/admin.jsp
• File system• Configuration-driven• Programmatic
Understand effects of attacks
/admin/admin.jsp✗
Command Injection
sysadmin$./sh
✔
Fix Faster
• Reduce False Positives• Confirm vulnerabilities
• Provide Actionable Details• Stack trace• Line of code
• Collapse Duplicate Issues• Tie to root cause
Reduce False Positives
/admin/admin.jsp
SQLi?✔
Actionable Details
/login.jsp
Collapse Duplicate Issues
/login.jsp
/pages/account.jsp
/pages/balance.jsp
1 Cross-Site Scripting 2 3 1
JavaBB – Case Study
• Open Source Bulletin Board
• Additional Vulnerabilities• Finds18 SQL Injection results
• Root cause analysis• 18 SQL injection results have 1 root cause
Vulnerability Diagnosis
Confirmed SQL Injection
Actionable Details
Line of Code
Parameters
Stack Trace
Yazd – Case Study
• Open Source Forum
• Additional Attack Surface• Discovers hidden ‘admin’ area• 3 Additional Cross-Site Scripting results
• Root cause analysis• Collapses 34 XSS into 24 root-cause vulnerabilities
Attack surface identification
Hidden ‘admin’ area
Collapse Duplicate Issues
One More Case Study
Future
• Automated anti-anti automation
The Case Against “Hybrid”
• Hard to find attack surface with static analysis• Static/dynamic correlation doesn’t work• Doesn’t help with false positives / false negatives• Nobody will run a software monitor (cheating!)
The Case for Gray-Box Testing
• Black-box is a losing game• Find more
• Attack surface• Vulnerability diagnosis
• Fix faster• Root cause analysis• Collapse duplicates
Gray, the New BlackGray-Box Web Vulnerability Testing
Brian ChessFounder / Chief Scientist
Fortify Software, an HP CompanyJune 22, 2011