Evaluation of Penetration Testing Software

Research

Penetration testing is an evaluation of system security by simulating a malicious

attack, which, at the most fundamental level, consists of an intellectual attempting to

bypass the rules and “firewalls” that establish software security. As it is impossible to

achieve 100% security, the goal of penetration testing an unyielding and unadaptive

ruleset is to decrease the chance that a system can be compromised.

Testing is generally conducted from one of three viewpoints; white box, black

box, and gray box. Fundamentally, white box is complete knowledge of software and

access to underlying code. This includes comprehensive testing by debugging and

creating specialized test programs that evaluate all routes through the code. Although

thorough and comprehensive, white box testing is also expensive and time-consuming. In

contrast, black box testing is viewing the remote system as an unknown box which

simply performs an operation on the input to procure the output. As a result, without

knowledge of system internals, black box testing is generally less comprehensive and

thus costing less money and time. Finally, gray box testing is a mixture of white and

black where the researcher conducts testing at the black box level with code access at the

white box level for generating test cases.

Practices

In addition to those three viewpoints at which penetration testing can be

performed, there are also three large penetration testing methodologies; the Open Source

Security Testing Methodology Manual (OSSTMM)1, the Information Systems Security

Assessment Framework (ISSAF)2, and the NIST Guideline on Network Security Testing

(Special Publication 800-42)3. However, of these three, the most accepted and

comprehensive is the OSSTMM, an open and peer-reviewed methodology that, when

properly applied, accurately measures security without assumptions and anecdotal

evidence.

The OSSTMM consists of Information Security, Process Security, Internet

Technology Security, Communications Security, Wireless Security, and Physical Security

modules, each of which has specific tasks and goals that need to be completed and

verified. Practices which are especially relevant to the Drupal project include those of the

Internet Technology Module that concern automated software, exploitation vectors,

privilege control, and heavy load situations. Tasks for automated vulnerability scanners

include testing with at least two redundant tools, utilizing popular exploits and cracking

tools, and checking for both false positives and false negatives in discovered

vulnerabilities. Exploitation vectors to examine include buffer overflows in long strings,

SQL injection, brute-force password discovery, cross-site scripting (XSS), bypass of

input validation in encoded strings (unicode, etc), server-side includes, cookie

manipulation, hidden field modifications, HTTP header manipulation, and input

sanitization. Privilege control emphasizes the concept of granting resource and system

control at the lowest possible level, thus preventing a compromised daemon running as

root to infect and control the entire machine. Ensuring that a system does not reveal

valuable information under stress or become unstable during a denial-of-service attack 1http://www.isecom.org/osstmm/ 2http://www.oissg.org/issaf 3http://csrc.nist.gov/publications/nistpubs/800-42/NIST-SP800-42.pdf

(DOS) is also an important goal. These tasks and goals are summarized by figure A.

Tools Tools for penetration testing include vulnerability scanners, packet sniffers,

exploitation software, packet crafters, password crackers, and port scanners. For the

purposes of this evaluation, however, only active open-source vulnerability scanners will

be considered. This includes tools such as Nikto4, Paros5, WebScarab6, Wikto7, and

Sara8, however, tools such as Nessus9, Whisker10, Spike11, and WebInspect12 will be

4http://www.cirt.net/code/nikto.shtml 5http://www.parosproxy.org/index.shtml 6http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project 7http://www.sensepost.com/research/wikto/ 8http://www-arc.com/sara/ 9http://www.nessus.org/nessus/ 10http://www.wiretrip.net/rfp/w.asp 11http://www.immunitysec.com/resources-freesoftware.shtml

Figure A (OSSTMM v2.2 p.49 (Section C – Internet Technology Security) – ISECOM)

excluded.

Evaluations were performed by setting up a “scanner” and a “target”—a virtual

machine running 32-bit Ubuntu Gutsy (7.10) desktop edition with drupal, mysql-

server5.0, and apache2.2-common (outdated; 5.2-2ubuntu2.1, 5.0.45-1ubuntu3, and

2.2.4-3build1, respectively) from the Ubuntu repositories. All configuration was left to

the default, except for timezone, Drupal module configuration, and user setup. Timezone

and locale was set to GMT -7 with no DST. All Drupal modules were enabled without

additional configuration. All users that needed to be created were named “ubuntu”. In

addition, the default Apache “test” directory was removed and a blog post to Drupal was

made so that the default “welcome” screen would not be shown.

It is important to note, however, that the purpose of this evaluation is to highlight

the features and capabilities of each vulnerability scanner, not to actually determine

security vulnerabilities present in Drupal, the MySQL database, and the Apache

webserver. A sample post was committed so that the default “welcome” screen would not

appear. The Drupal installation is shown in figure B. In addition, false positives and false

negatives were not checked for.

12http://www.spidynamics.com/products/webinspect/

Nikto Interface: Console

Language: Perl

Last Update: November 2007

Nikto is a web server assessment tool designed to find software

misconfigurations, insecure file permissions, and outdated software. It supports SSL,

proxies, basic client authentication, and CGI scanning. Furthermore, Nikto also features

IDS evasion techniques (using libwhisker), report generation, file/folder name mutations,

among others.

Verdict: Nikto was easy to download, install, and setup. Configuration was a

breeze, and scanning was quick and painless, finishing in less than a minute. In addition

to the speed, Nikto was also comprehensive, reporting number of vulnerabilities not

detected by other scanners (fig. C).

Figure B. Drupal installation on a remote virtual

Paros Interface: GUI

Language: Java

Last Update: August 2006

Paros is a vulnerability assessment proxy that supports editing both HTTP and

HTTPS packets on the fly. It also supports recording web traffic, scanning for common

vulnerabilities, and spidering a website. In addition, Paros has plugin support and report

generation functionality. The web scanner searches for a number of different

vulnerabilities such as HTTP PUT, directory browsing, obsolete/default files, SQL

injection, Carriage Return/Line Feed injection (CRLF), server side includes, parameter

tampering, and cross-site scripting.

Figure C. Nikto scan on a Drupal webserver.

Verdict: Paros has great potential, however, the data it presents is a little

overwhelming (fig. D). Furthermore, although feature-packed, the vulnerability scanner

seems to be a

weaker than

Nikto's and could

be improved

(figure E).

WebScarab Interface: GUI

Language: Java

Last Update: May 2007

WebScarab is an HTTP and HTTPS application analysis framework. Although

having many of the same features as Paros, WebScarab does bring a number of

Figure D. Paros main view (web traffic recorder). Figure E. Paros webspider (top) and alert/scanner (bottom) interfaces. Separate images were combined.

previously unseen abilities to the table, such as SessionID analysis, fuzzing, bandwith

simulating, and the execution of user-inputted Java expressions.

Verdict: WebScarab's neat interface (fig. F) and superior features make it a must-

have for web vulnerability scanning. The only downside is that it may take some time to

master WebScarab.

Wikto Interface: Console

Language: C# .NET

Last Update: October 2007

Wikto is a web server assessment tool based on Nikto, but with additional

Figure F. WebScarab's main interface.

features. New features include a file/folder scanner, and Google SOAP API integration

when combined with WinHTTrack13 (a web server mirroring tool) and HTTprint14 (a web

server fingerprinting tool). Wikto can utilize the Google SOAP API to mirror a website

from Google's cache and analyze it, instead of directly accessing the website and

triggering an Intrusion Detection System (IDS). Wikto also can utilize a “Google-

hacking” database to search for inadvertently indexed files. Wikto also utilizes fuzzy

logic and other scanning optimizations when performing a Nikto scan.

Verdict: Although, seemingly a great tool, Wikto is essentially Nikto with a GUI,

as many of the additional “features” do not work out of the box or at all. This includes the

Google SOAP API integration, as Google no longer supports the API and has stopped

giving out API keys as of December 5th, 2006. Additional software by SensePost (Aura15)

does bypass this restriction. The Nikto database scanner (fig. G) is also much slower than

Nikto itself, despite the optimizations and improvements. Wikto's numerous

dependencies detracts from its abilities, as addition software does need to be installed for

full functionality. Furthermore, Wikto is only supports Windows, as it makes use of the

.NET runtime and does not work with Mono on Wine. Note that WinHTTrack, HTTprint,

and Aura were not installed during testing.

13http://www.httrack.com/ 14http://www.net-square.com/httprint/ 15http://www.sensepost.com/research/aura/

SARA Interface: Console/HTML

Language: Perl

Last Update: November 2007

Sara is a security analysis tool that can check for SQL injection vulnerabilities,

initiate a remote self-scan, interface with nmap and SAMBA, process HTTPS, check for

SSH server vulnerabilities, and can differentiate results depending on whether it is

running on a “trusted” or “untrusted” host. It also supports firewalled environments,

integration with the National Vulnerability Databse (NVD), 3rd party plugins, and running

in daemon mode as a webserver (fig. H). It can also be run as a console tool.

Verdict: SARA's poor on-line and included documentation made it hard to

compile and utilize; it often complained about modules and libraries that were not present

and could not be identified. SARA's reports and results were hard to access, as they only

showed up when running as in daemon mode, although they were detailed and

comprehensive. Furthermore, SARA hung when scanning in both daemon and console

mode, with Wireshark logging no network usage. Although a great tool with a number of

new and interesting features, SARA simply did not compile or run properly.

Figure G. The Nikto webscanner view of Wikto.

Summary of Findings/Recommendations

Many penetration testing tools provided the same basic functionality, however, the

quality and thoroughness of each differed. Among the top tools were Nikto and

WebScarab; not only were they quick and efficient, but they were also thorough and

comprehensive. One tool did not compile and run correctly, SARA, as poor

documentation did not enable easy dependency installation. Most tools supported both

Linux and Windows, although some only supported one or the other.

The Drupal project should utilize at least two penetration testing tools,

specifically Nikto and WebScarab to ensure quality and thoroughness. In addition, other

software beyond the scope of this document such as nmap16, Nessus, Hping17, and John

the Ripper18 should also be utilized to test for overall system security. Furthermore, the

Drupal project should also consider physical security issues such as whether an intruder

can simply enter the server room and reconfigure Drupal, or whether plaintext database

passwords are stored on the hard drive. These security evaluations should be performed 16http://www.insecure.org/ 17http://www.hping.org/ 18http://www.openwall.com/john/

Figure H. SARA daemon/webserver

according to the OSSTMM manual at least once every major release, preferably when

any core or at-risk component is severely modified. The Drupal project should also work

together with many Linux distributions to ensure that software repositories are up-to-date.