Download - Black Box Scanner Presentation
-
8/12/2019 Black Box Scanner Presentation
1/33
Stanfor
dCompu
terSecu
rityLab
State of The Art:Automated Black Box
Web Application VulnerabilityTestingJason Bau, Elie Bursztein,Divij Gupta, John Mitchell
-
8/12/2019 Black Box Scanner Presentation
2/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Background
Web Application Vulnerability Protection High incidence vulnerabilities (XSS, SQLI, ) Required for standards compliance (e.g PCI)
XSS
-
8/12/2019 Black Box Scanner Presentation
3/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Security Tools for Apps
Vulnerability Detection Techniques: Manual vs. Automated White-Box vs. Black-Box Code review, Static analysis, Pen testing Automated Black Box Testing
Cheaper? Less intrusive to workflow?
-
8/12/2019 Black Box Scanner Presentation
4/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Scanner 1
-
8/12/2019 Black Box Scanner Presentation
5/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Scanner 2
-
8/12/2019 Black Box Scanner Presentation
6/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Goals of Study
What vulnerabilitiesare tested by scanners? How representativeare scanner tests of in-the-
wild vulnerabilities
What can user expectfrom scanner? What is hardand needs more human review?
-
8/12/2019 Black Box Scanner Presentation
7/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Non-Goals
Not a product ranking Not a benchmark of particular tools
-
8/12/2019 Black Box Scanner Presentation
8/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Take Aways
How to take advantage of scanner How (If) to combine it with human audit What to expect as improvement
-
8/12/2019 Black Box Scanner Presentation
9/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Outline
Vulnerability categories tested by scanners How prevalent are these in the wild? Common application results Custom testbed design Custom testbed results
Coverage Detection False Positives
-
8/12/2019 Black Box Scanner Presentation
10/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Local Remote
>$100K total retail price
Survey of Leading Products
-
8/12/2019 Black Box Scanner Presentation
11/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Category Example Vulnerabilities
Cross Site Scripting XSSSQL Injection SQLI
Cross Channel Scripting
(Other forms of injection)
Arbitrary File UploadRemote File Inclusion
OS command Injection
Session ManagementSession Fixation and Prediction
Authentication Bypass
Cross-Site Request Forgery CSRF
SSL/Server Config Self-Signed Cert, HTTP Trace
Info Leakage Temp file access, path traversalError message disclosure
Vuln Categories From Scanners
-
8/12/2019 Black Box Scanner Presentation
12/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Test Vectors By Category
Test Vector Percentage Distribution
-
8/12/2019 Black Box Scanner Presentation
13/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Reported Vulnerabilities"In the Wild"
Data from aggregator and validator ofNVD-reported vulnerabilities
-
8/12/2019 Black Box Scanner Presentation
14/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Scanners vs. In-the-Wild
Top 4 for both: XSS SQLI XCS Info Leak
Scanners have many more info leak vectors Easier to write?
-
8/12/2019 Black Box Scanner Presentation
15/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Good: Info leak, Session (Anecdote from re-test)
Decent: XSS/SQLIPoor: XCS, CSRF (low vector count?)
Detecting Known Vulnerabilities
Vulnerabilities forprevious versions of Drupal, phpBB2, and WordPress
-
8/12/2019 Black Box Scanner Presentation
16/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Mainly built over summer by 1 undergrad in PHP Measure Performance
o Test Duration / Network Traffic
Measure Coverageo Links coded in various technologies (Flash, SilverLight, ...)o Can scanner follow link?
Measure Vulnerability Detection Rateo XSS (Type 1, Type 2, Advanced)o SQLI (Type 1, Type 2)
o Cross Channel Scriptingo CSRF
o Session Managemento Server/Crypto Configo
Information Leako Malware
Our Custom Testbed
-
8/12/2019 Black Box Scanner Presentation
17/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Performance did not correlate well with vulnerability detection
Scanner Performance
-
8/12/2019 Black Box Scanner Presentation
18/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
% Successful Link Traversals By Technology,Averaged over all Scanners
Scanner Page Coverage
-
8/12/2019 Black Box Scanner Presentation
19/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Context?
Vulnerability Detection
-
8/12/2019 Black Box Scanner Presentation
20/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
XSS Testbed
Type 1: Textbook ReflectedVulnerability User input, http header page w/o sanitization
Type 2: Stored Vulnerability User input DB Served Page Some viewable only by different user
Advanced Novel Tags: e.g. , Novel Channels:
URL $_SERVER['PHP_SELF'] Filename error msg,
-
8/12/2019 Black Box Scanner Presentation
21/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
XSS Results
Anecdote about Type 2
-
8/12/2019 Black Box Scanner Presentation
22/33Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Type 1: User input SQLI on page generationo Basic: ; --oAdvanced: , LIKE, UNION
Type 2: Input DB SQL Queryo
Only basic caseso Unsanitized form input (username) DB, later used inSQL query
SQLI Testbed
l
-
8/12/2019 Black Box Scanner Presentation
23/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
SQLI Results
l
-
8/12/2019 Black Box Scanner Presentation
24/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Code Injection by Attacker Manipulate server or client browser Tests:
o XPATH injectiono Malicious File Uploado Direct Object Refo Cross-Frame Scriptingo Open Redirectso Server Side Includes
o Header Injectiono Flash Parameter Injecto SMTP Injection
XCS Results
CSRF R l
-
8/12/2019 Black Box Scanner Presentation
25/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Post-login formso w/o hidden random tokeno with weak [0,9] tokeno with same token each time
JSON Hijackingo No session id sent with AJAX
request for sensitive data
Anecdote: Told by one vendorCSRF not checked on purpose
CSRF Results
S i M
-
8/12/2019 Black Box Scanner Presentation
26/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Login / form errorso Login form not httpso Reg. credentials in clear
o Autocomplete pwd fieldo Weak pwds and pwd
recovery questiono Weak reg. page CAPTCHA
Cookie errorso Not HttpOnlyo Auth tokens not httpso Persistent Auth token value
MD5 (pwd)o Logout fails to clear cookieo Path restriction to '/'
Session Management
S /C t Mi C fi
-
8/12/2019 Black Box Scanner Presentation
27/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
Server Mis-Config:o HTTP Trace enabledo open_basedir not set in phpo allow_url_fopen set in php
Crypto Mis-Configo Self Signed Certo Weak SSL Cipher
Server/Crypto Mis-Config
I f L k
-
8/12/2019 Black Box Scanner Presentation
28/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
SQL error message Username existence Backup files Comment/Path Disclosure Path Traversal
Inclusion of/etc/secret.txt
Info Leak
M l P
-
8/12/2019 Black Box Scanner Presentation
29/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
JavaScript key-logger on login page Malicious graphic uploaded by user
o .jpg with appended PHPo Directly reference-able
No Scanner Detectedo Because not part of PCI compliance?
Malware Presence
F l P iti
-
8/12/2019 Black Box Scanner Presentation
30/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
TestbedTrapso alert()sas site behavior (not part of injection) Scanners avoided
o Benign (not-executed) region within tags Tripped 2 scanners (reported 1 and 13 times)
On a testbedof ~90 confirmed vulnerabilities
Some scanners with low false positive rates also had high relativedetection rates
False Positives
Ob ti
-
8/12/2019 Black Box Scanner Presentation
31/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
No individual scanner tops across all categorieso Best XSS, SQLI Bottom 3 Session Managemento Top 3 Session Management Found 0 SQLI
o Rough break along XSS/SQLI/XCS andSession/Config/Info lines
Scanners exist :o High Detection Rate, Low False Positive Rateo Low Detection Rate, High False Positive Rateo Low Detection Rate, Low False Positive Rate
Observations
C l i 1
-
8/12/2019 Black Box Scanner Presentation
32/33
Jason Bau [email protected] of the Art: Automated Black Box Web Application Vulnerability Testing
XSS, SQLI, XCS, Info Leak most common in-the-wild Black Box Scanner "effort" roughly proportional to this Can improve coverage of technologies like Flash, SL Scanners relatively adept at detecting
Historical vulnerabilities Textbook XSS and SQLI Info Leak, Session, and Server/Crypto Mis-config
Easier test vectors to write/interpret
Conclusions 1
C l i 2
-
8/12/2019 Black Box Scanner Presentation
33/33
Can stand improvement ono CSRF, Malware, XCS Low test vector count Not vendor focus?oAdvanced (novel) forms of XSS, SQLI Faster reactive process
o Stored forms of XSS, SQLI (acknowledged by a CTO) Better DB modeling
Conclusions 2