![Page 1: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/1.jpg)
Gray, the New BlackGray-Box Web Vulnerability Testing
Brian ChessFounder / Chief Scientist
Fortify Software, an HP CompanyJune 22, 2011
![Page 2: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/2.jpg)
![Page 3: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/3.jpg)
![Page 4: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/4.jpg)
Todo
• Define gray-box testing• Why black-box is insufficient• What we built• Examples• Haters club
![Page 5: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/5.jpg)
Definitions
• Black-box testing• System-level tests• No assumptions about implementation
![Page 6: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/6.jpg)
Definitions
• White-box testing• Examine implementation• Test components in isolation
![Page 7: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/7.jpg)
Definitions
• Gray-box testing• System-level tests (like black-box)• Examine implementation (like white-box)
![Page 8: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/8.jpg)
The Software Security Game
• Objective• Rules vs. Strategy• Playing Field
![Page 9: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/9.jpg)
OBJECTIVE:Protect everything
OBJECTIVE:Exploit one vulnerability
![Page 10: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/10.jpg)
Rules for the Defender
1. Don’t attack the attacker
![Page 11: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/11.jpg)
Rules vs. Strategy
Rules• Don’t attack the attacker
Strategy• Emulate attacker’s techniques
![Page 12: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/12.jpg)
Who wins?
• Technology• Expertise
![Page 13: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/13.jpg)
Who wins?
• Time
• Technology• Expertise
![Page 14: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/14.jpg)
Who wins?
• Technology• Expertise• Time
![Page 15: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/15.jpg)
Changing the odds
![Page 16: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/16.jpg)
The Defender’s Advantage
• Time
• InsideAccess
• Technology• Expertise
![Page 17: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/17.jpg)
Prior Art
• 2005: Concolic testing: Sen, University of Illinois
• 2008: Microsoft SAGE: Godefroid, MSR
• 2008: Test Gen for Web Apps: Shay et al, U. Washington
• 2008: Accunetix: Accusensor
![Page 18: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/18.jpg)
Access to the Software
Allows for ‘Hybrid’ analysis
Dynamic
Analysis
Black-box Approach
Static Analysis
White-box Approach
![Page 19: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/19.jpg)
‘Hybrid’ Analysis
Dynamic Analysis
Static Analysis
Mostly Broken
Correlation Engine
![Page 20: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/20.jpg)
![Page 21: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/21.jpg)
The ‘Real-Time Hybrid’ Approach
Dynamic Analysis
Static Analysis
Good Results
Real-Time Analysis
Correlation Engine
![Page 22: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/22.jpg)
Evolving to Integrated Analysis
Dynamic Analysis
Application
Real-Time Analysis
Real-time link
• Find More• Fix Faster
![Page 23: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/23.jpg)
Find More
• Reduce false negatives• Automatic attack surface identification• Understand effects of attacks
• Detect new types of vulnerabilities• Privacy violation, Log Forging
![Page 24: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/24.jpg)
Attack surface identification
/login.jsp
/pages/account.jsp
/pages/balance.jsp
/admin/admin.jsp
• File system• Configuration-driven• Programmatic
![Page 25: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/25.jpg)
Understand effects of attacks
/admin/admin.jsp✗
Command Injection
sysadmin$./sh
✔
![Page 26: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/26.jpg)
Fix Faster
• Reduce False Positives• Confirm vulnerabilities
• Provide Actionable Details• Stack trace• Line of code
• Collapse Duplicate Issues• Tie to root cause
![Page 27: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/27.jpg)
Reduce False Positives
/admin/admin.jsp
SQLi?✔
![Page 28: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/28.jpg)
Actionable Details
/login.jsp
![Page 29: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/29.jpg)
Collapse Duplicate Issues
/login.jsp
/pages/account.jsp
/pages/balance.jsp
1 Cross-Site Scripting 2 3 1
![Page 30: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/30.jpg)
JavaBB – Case Study
• Open Source Bulletin Board
• Additional Vulnerabilities• Finds18 SQL Injection results
• Root cause analysis• 18 SQL injection results have 1 root cause
![Page 31: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/31.jpg)
Vulnerability Diagnosis
Confirmed SQL Injection
![Page 32: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/32.jpg)
Actionable Details
Line of Code
Parameters
Stack Trace
![Page 33: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/33.jpg)
Yazd – Case Study
• Open Source Forum
• Additional Attack Surface• Discovers hidden ‘admin’ area• 3 Additional Cross-Site Scripting results
• Root cause analysis• Collapses 34 XSS into 24 root-cause vulnerabilities
![Page 34: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/34.jpg)
Attack surface identification
Hidden ‘admin’ area
![Page 35: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/35.jpg)
Collapse Duplicate Issues
![Page 36: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/36.jpg)
One More Case Study
![Page 37: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/37.jpg)
Future
• Automated anti-anti automation
![Page 38: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/38.jpg)
The Case Against “Hybrid”
• Hard to find attack surface with static analysis• Static/dynamic correlation doesn’t work• Doesn’t help with false positives / false negatives• Nobody will run a software monitor (cheating!)
![Page 39: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/39.jpg)
The Case for Gray-Box Testing
• Black-box is a losing game• Find more
• Attack surface• Vulnerability diagnosis
• Fix faster• Root cause analysis• Collapse duplicates
![Page 40: Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011](https://reader036.vdocuments.net/reader036/viewer/2022062407/56649cdb5503460f949a66d5/html5/thumbnails/40.jpg)
Gray, the New BlackGray-Box Web Vulnerability Testing
Brian ChessFounder / Chief Scientist
Fortify Software, an HP CompanyJune 22, 2011