using declarative information in web application frameworks to improve security

Using Declarative Information in WebApplication Frameworks to Improve Security

Master Thesis

Advisor: Prof. Dr. Marc Rennhard

Kevin Denver

Winterthur, 14. April 2011

Abstract

This thesis is concerned with improving security in web applications by using declarative infor-mation provided in web application frameworks. The security testing tool that was researched anddeveloped in this thesis is especially useful for detecting privilege escalation vulnerabilities andmisconfigurations in access control checks in web applications with at least two distinguishable userroles in an automated fashion whereas one can be anonymous.

Depending on the overall design of the web application, adding access control checks to each ac-cessible resource can be a very tedious task and it is very likely that some checks might have goneforgotten, rendering the application vulnerable to privilege escalation attacks. By using declarativeinformation provided in source- or configuration files of a developed web application it becomespossible to detect privilege escalation vulnerabilities and misconfigurations in access control checksin an automated fashion allowing a developer to make the web application more secure.

Finding access control misconfigurations and thus possible privilege escalation vulnerabilities isbeing done in three phases: (1) The source code and configuration files of a targeted web applica-tion are being parsed by a Java command line Framework Parser specifically tailored to support aparticular used web application framework. The Framework Parser’s output is a general Meta LayerXML file containing all the found data such as a (non) complete list of available URLs for seedingthe web crawling component used in phase 2, a (non) complete list of available users and user roles,a (non) complete list of existing access constraints, etc. (2) The second phase consists of a webcrawling component which takes the Meta Layer XML file as an input. The crawling of the targetedweb application is done for each configured user respectively role in the Meta Layer XML file to findall possible transitions and enforced access restrictions. (3) The third and final phase consists of aJava Swing Visualisation Application that analyses the collected data from the crawling componentand visualises it as a directed graph whereas nodes correspond to found HTML resources and edgescorrespond to transitions between HTML resources. The analysis is being done in an automatedfashion based on the information available in the Meta Layer XML file. In addition the VisualisationApplication allows a user to interact and inspect the collected data and enables him to find eventualsecurity threats visually.

The test cases showed that the developed security testing tool is indeed capable of detecting privile-ge escalation vulnerabilities in an automated fashion in real life applications. Whereas false positivesmight be generated if the crawling component is being prematurely terminated. The number of falsepositives can be reduced by allowing the crawling component to examine the deployed web applica-tion more thoroughly.

i

Abstract

Diese Thesis beschreibt, wie Web Applikationen mittels deklarativen Information aus Konfigurati-onsdateien oder Applikations-Quellcode in ihrer Sicherheit verbessert werden können.

Die in dieser Thesis entwickelte Security Testing Applikation ist besonders für das Auffinden vonPrivilege Escalation Schwachstellen und für die Detektierung von Konfigurationsfehlern in AccessControl Beschränkungen geeignet, falls die zu testende Web Applikation mindestens zwei unter-scheidbare Benutzerrollen besitzt, wobei eine der Rollen auch ein anonymer Benutzer sein kann.

Je nach Design der Web Applikation kann es sehr mühsam für einen Entwickler sein, jede erreich-bare Ressource mit Access Control Beschränkungen auszustatten und es ist sehr wahrscheinlich,dass gewisse Beschränkungen vergessen gehen und die Web Applikation somit verwundbar fürPrivilege Escalation Schwachstellen wird. Durch die Verwendung von deklarativen Informationen,die aus Konfigurationsdateien oder aus dem Quellcode gewonnen werden können, wird es möglichPrivilege Escalation Schwachstellen oder Konfigurationsfehler in Access Control Beschränkungen au-tomatisiert aufzudecken, was es einem Entwickler erlaubt seine Web Applikation sicherer zu machen.

Die Detektierung von Privilege Escalation Schwachstellen oder Konfigurationsfehlern in Access Con-trol Beschränkungen findet in drei Phasen statt: (1) Der Quellcode und die Konfigurationsdateien derzu testenden Web Applikation werden durch eine Java Kommandozeilen-Applikation - FrameworkParser - analysiert, welche eigens für das verwendete Web Applikations-Framework zugeschnitten ist.Das Resultat des Framework Parser ist eine generische Meta Layer XML Datei, welche all die deklara-tiv gefundenen Informationen enthält wie: eine (nicht) komplette Liste aller erreichbaren URLs derApplikation, die später vom Web Crawler als Seeds verwendet werden können, eine (nicht) kompletteListe aller bekannten Benutzer und Benutzerrollen, eine (nicht) komplette Liste bereits konfigurierterAccess Control Beschränkungen etc. (2) Die zweite Phase besteht aus einem Web Crawler, der dieMeta Layer XML Datei als Input verwendet. Die zu testende Applikation wird dann vom Crawlermehrmals mit jedem konfigurierten Benutzer respektive mit jeder Benutzerrolle aus der Meta LayerXML gespidert. Dabei werden alle Übergänge zwischen den Ressourcen und die vorhandenen AccessControl Beschränkungen aufgezeichnet. (3) Die dritte und letzte Phase beinhaltet eine Java SwingVisualisation Application, welche die aufgezeichneten Daten analysiert und als gerichteter Graphdarstellt, wobei Knoten den gefundenen URLs entsprechen und Kanten den Übergängen zwischenURLs. Die Analyse der Daten findet anhand der vorhandenen Informationen aus der Meta Layer XMLDatei automatisiert statt. Zusätzlich erlaubt die Visualisation Application dem Benutzer möglichePrivilege Escalation Schwachstellen oder Konfigurationsfehler in Access Control Beschränkungenvisuell zu finden.

Eine Testreihe mit verschiedenen Web Applikationen hat gezeigt, dass der entwickelte Ansatz in die-ser Thesis durchaus funktioniert und es möglich ist Privilege Escalation Schwachstellen oder Konfi-gurationsfehler in Access Control Beschränkungen automatisiert aufzudecken. Jedoch können FalsePositives generiert werden, wenn der Web Crawler frühzeitig beendet wird. False Positives könnenreduziert werden, wenn dem Web Crawler mehr Zeit eingeräumt wird um die Web Applikation zutesten.

ii

Contact

Zürcher Hochschule für Angewandte Wissenschaften (ZHAW)c/o Institut für angewandte Informationstechnologie InITSteinberggasse 13Postfach 805CH-8401 Winterthur

Tel.: +41 (0) 58 934 75 87Fax: +41 (0) 58 935 75 87

EMail: [email protected]://www.zhaw.ch

http://www.init.zhaw.ch

Name Function E-MailKevin Denver Student [email protected]. Dr. Marc Rennhard Project Leader & Advisor [email protected]

iii

mailto:[email protected]

http://www.zhaw.ch

http://www.init.zhaw.ch



Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Java Web Application Frameworks 22.1 Java Technologies for Web Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.1.1 Java Servlet API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.1.2 JavaServer Pages Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1.3 JavaServer Pages Standard Tag Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.4 JavaServer Faces Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Securing Java Web Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2.1 Working with Realms, Users, Groups and Roles . . . . . . . . . . . . . . . . . . . . . . . . . 92.2.2 Specifying Security Roles Using Deployment Descriptor Elements . . . . . . . . . . . 112.2.3 Specifying Security Roles Using Annotations . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2.4 Specifying Security Constraints Using Deployment Descriptor Elements . . . . . . 132.2.5 Specifying Security Constraints Using Annotations . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Securing Java EE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3.1 Specifying Method Permissions Using Annotations . . . . . . . . . . . . . . . . . . . . . . 152.3.2 Specifying Method Permissions Using Deployment Descriptors . . . . . . . . . . . . . 16

2.4 Java Security Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.4.1 Java Authentication and Authorization Service (JAAS) . . . . . . . . . . . . . . . . . . . . 172.4.2 Spring Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4.3 PicketBox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.4 OWASP Enterprise Security API (ESAPI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.5 Apache Shiro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.6 jGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.4.7 Apache Commons Validator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.5 Popular Web Application Frameworks for the Java Programming Language . . . . . . . . . . 282.6 Spring Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.6.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.6.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.7 Apache Struts 1.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.7.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.7.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.8 Apache Struts 2.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.8.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.8.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.9 Apache MyFaces Tomahawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.9.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.9.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.10 Wicket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.10.1 Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

iv

Contents

2.10.2 Extendibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3 Related Work 463.1 Declarative Security Policies and Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.2 Aspect-Oriented Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4 Finding Web Application Vulnerabilities by using Declarative Information 604.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.2 Injection Flaws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.3 Broken Authentication and Session Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664.4 Insecure Direct Object References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664.5 Security Misconfiguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.6 Insecure Cryptographic Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.7 Failure to Restrict URL Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.8 Insufficient Transport Layer Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.9 Unvalidated Redirects and Forwards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5 Finding Privilege Escalation Vulnerabilities by using Declarative Information 765.1 Differences between RBAC and ABAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.2 Detecting Privilege Escalation Vulnerabilities (RBAC) . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5.2.1 Extractable Declarative Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775.2.2 Crawling the Web Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.2.3 Evaluation of the collected Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

6 Design 836.1 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.2 Meta Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6.2.1 Meta Layer XML Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.3 Framework Specific Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6.3.1 Struts 1.x Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916.4 Web Crawler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.4.1 HtmlUnit Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.4.2 Submitting a HTML Form with HtmlUnit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976.4.3 HtmlUnit JavaScript Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996.4.4 Design of the Web Spider Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.4.5 Smartly filling out HTML Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036.4.6 Spidering with different User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.4.7 Saving the Crawled Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

6.5 Working with Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066.5.1 Graph Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.5.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

6.6 Saving the Crawled Data in Neo4j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1096.7 Analysis Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

6.7.1 Comparing crawled Data with the <access-restrictions> Tag . . . . . . . . . . . 1116.7.2 Graph Differences - Clustering by User /User Roles . . . . . . . . . . . . . . . . . . . . . . 1126.7.3 Edge Analysis - Clustering by URL Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

v

Contents

6.7.4 Clustering by URL Paths and Re-Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.7.5 Showing Differences by Re-Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.7.6 Software Testing Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

6.8 Presentation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1176.8.1 Java Swing Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

6.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

7 Testing 1207.1 Web Crawler - WIVET Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1207.2 Test Environment & Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217.3 Struts Test Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.3.1 Phase 1: Parsing Struts Source-/Configuration Files . . . . . . . . . . . . . . . . . . . . . . 1237.3.2 Phase 2: Crawling the deployed Web Application . . . . . . . . . . . . . . . . . . . . . . . . 1277.3.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.4 WordPress 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1297.4.1 Phase 1: Preparing the Meta Layer XML File . . . . . . . . . . . . . . . . . . . . . . . . . . . 1307.4.2 Phase 2: Crawling the deployed Web Application . . . . . . . . . . . . . . . . . . . . . . . . 1327.4.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

7.5 PrivaSphere Secure Messaging Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1347.5.1 Phase 1: Preparing the Meta Layer XML File . . . . . . . . . . . . . . . . . . . . . . . . . . . 1347.5.2 Phase 2: Crawling the deployed Web Application . . . . . . . . . . . . . . . . . . . . . . . . 1367.5.3 Phase 3: Analysing the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

7.6 False Positives in the Analysis Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1387.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

8 Summary 1418.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1418.2 Further Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

9 User Manual & Development Guide 1439.1 User Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

9.1.1 Hard- & Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1439.1.2 Installation Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1439.1.3 Running the Meta Layer Extraction Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1449.1.4 Running the RBAC Spider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1459.1.5 Running the RBAC Visualiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1469.1.6 Video Tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

9.2 Development Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1489.2.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1489.2.2 Installing the Development Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1499.2.3 Building with Maven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1499.2.4 Free/Third-Party Libraries Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1499.2.5 Use logback for debugging purposes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

9.3 RBACVisualiser Screenshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

Appendix 160List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162List of listings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

vi

Contents

A Original Assignment 168

B Meta Layer XML Structure DTD 171

C Web Spider UML Class Diagram 172

D URL Extractors UML Class Diagram 173

E HtmlUnit JavaScript Performance (Runtime Numbers) 174

F HtmlUnit JavaScript Performance (Code) 175

vii

Contents

• Chapter 1: The first Chapter gives a brief overview over the main aspects of this thesis. This includes themotivation behind the thesis and how combining white- and black box testing techniques improve thedetection of vulnerabilities in web applications.

• Chapter 2: Investigates the current state of the art practices for developing web applications with theJava programming language. Furthermore, the available security features such as access controls, inputvalidation, etc. shall be analysed and compared between the available web application frameworks. Thisshould give a broad overview over the current practices in developing large scale web applications withcomplex flows, user management features and security restrictions and how developers are able to adddeclarative information to their web applications which is an important topic for this thesis. Only opensource frameworks are being considered because a part of this thesis might be the enhancement of sucha framework, which is obviously difficult without the actual source code.

• Chapter 3: Presents the current state of the research literature related to adding declarative informationto web application source code to improve security. The papers summarised have been found throughthe Google Scholar search engine. The available research on enhancing web application security declar-atively can be roughly divided into three categories: added security through declarative security policies,adding an aspect oriented approach to web application security and static code analysis with the aid ofmetadata.

• Chapter 4: This Chapter tries to identify possible ways to enhance the detection of security vulnerabil-ities in web applications with an automated security testing tool by incorporating declarative securityinformation. The OWASP Top Ten project has been used to identify the most common web applicationvulnerabilities and for each one an appropriate solution has been proposed to enhance the detectionof such vulnerabilities by using a white box testing approach. Each vulnerability from the OWASP TopTen project is being shortly described and for each popular web application framework such as Struts1.x, Struts 2.x and Wicket the framework specific features are shown which help a developer to prevent aparticular vulnerability.

• Chapter 5: The following Chapter finalises the main idea of this thesis, namely detecting privilege es-calation vulnerabilities and misconfigurations in role based access control checks (RBAC) in web appli-cations by using declarative information provided by web application framework specific configurationfiles. After analysing the current research in the field of enhancing web application security throughthe use of declarative information, as outlined in Chapter 3, did not reveal any suitable idea worth con-tinuing for this thesis and evaluating possibilities of finding common web application vulnerabilitiesthrough the use of declarative information in Chapter 4, the idea of doing role based access controlpolicy checks seems to be the most valuable for a developer or testing agent.

• Chapter 6: This Chapter finalises the design for detecting privilege escalation vulnerabilities by consid-ering declarative information in web application frameworks. The Chapter contains the design for theunderlying storage solution as well as UML diagrams for the Java classes which have to be developed.

• Chapter 7: The Testing Chapter is concerned with functionality testing to verify whether the securitytesting tool meets the intended specifications and functional requirements laid out in the developmentand design specifications. The goal is to verify that it is indeed possible and feasable to detect privilegeescalation vulnerabilities and misconfigurations in real life web applications by using the developedsecurity testing tool.

• Chapter 8: This Chapter recapitulates critically the findings and limitations of this thesis and proposesa list of further steps for the project.

• Chapter 9: Contains the user manual and the development guide for the developed security testingtools. The user manual leads new users through the installation process of the tools and helps to getfamiliar with the various command line arguments. The development guide is for advanced users whowant to extend the functionality of the testing tools and build them from source.

viii

1 Introduction

This first Chapter gives a brief overview over the main aspects of this thesis. This includes the mo-tivation behind this thesis and how combining white- and black box testing techniques improve thedetection of security related vulnerabilities in web applications.

1.1 Motivation

The Automated Security-Testing Framework (ASTF) which has been developed at the Institute of Ap-plied Information Technology (InIT) as part of a KTI-Project, combines different security and non-security testing tools. The main focus of ASTF lies in doing continuous and reproducible softwaretests during the lifetime of an application. ASTF relies on third party tools which perform the actualtests. Tools like nmap or Nessus are used to scan for open ports or known security holes. The detec-tion of typical web application vulnerabilities like Cross Site Scripting and SQL Injection are coveredby w3af. Yet all of these tools take an external perspective of the test object to derive test cases. Thismethod of testing is also called black box testing. While this method can uncover flaws in the ap-plication, one cannot be sure that all existent attack vectors are being tested. White box testing onthe other hand takes more of an internal perspective on the test object and uses as much additionalinformation as possible to derive test cases. Such additional information can consist of the actualsource code of the application or metadata about the internal structure of the application.One of the future goals of ASTF is to provide a mechanism to conduct white box security tests inan automated way on different kinds of web applications. The objective of this thesis is to build aworking web application security testing tool by combining white and black box testing techniquesas described in [13] and [12].

1.2 Limitations

The approach described throughout this thesis generates the most benefit in regard to detecting priv-ilege escalation vulnerabilities and misconfigurations in access control checks in web applicationsthat use non-container managed access controls and thus enforce these programmatically in thesource code. However, the designed approach is generic enough to be used with web applicationsthat use container managed access controls and is capable of visualising the enforced access con-trols. The main difference between web applications that use container managed access controlsand those who do not is that it is far more likely to find security threats and misconfigurations inweb applications that enforce their access controls programmatically because it can be assumed thatthe container is properly implemented and tested to enforce the access controls as they have beenconfigured by a developer. Using the here described approach would be a bit exaggerated for onlyvisualising the configured and properly enforced access controls in the deployment descriptor of thecontainer but nonetheless possible. The approach is tailored to suit common role based access con-trols (RBAC) policies and does not work with attribute based access controls (ABAC) policies (moreon the differences between RBAC and ABAC can be found in Chapter 5).

Chapter 1. Introduction | 1

2 Java Web Application Frameworks

The following Chapter investigates the current state of the art practices for developing web appli-cations with the Java programming language. Furthermore, the available security features such asaccess controls, input validation, etc. shall be analysed and compared between the available webapplication frameworks. This should give a broad overview over the current practices in developinglarge scale web applications with complex flows, user management features and security restrictionsand how developers are able to add declarative information to their web applications which is an im-portant topic for this thesis. Only open source frameworks are being considered because a part of thisthesis might be the enhancement of such a framework, which is obviously difficult without the actualsource code.

2.1 Java Technologies for Web Applications

This Section provides an overview of the Java technologies involved in creating Java web applications.Most of the information provided here has been taken from the Sun Developer Network (SDN)1 orthe Oracle Java EE 5 Tutorial2. The Java technologies used to create web applications are a part of theJava EE platform, in addition to many of the Java Platform, Standard Edition (Java SE) classes andpackages.

Java EE web components are either Servlets or pages created using JSP technology (JSP pages)and/or JavaServer Faces technology. Servlets are Java programming language classes that dynami-cally process requests and construct responses. JSP pages are text-based documents that execute asServlets but allow a more natural approach to creating static content. JavaServer Faces technologybuilds on Servlets and JSP technology and provides a user interface component framework for webapplications. Static HTML pages and applets are bundled with web components during applicationassembly but are not considered web components by the Java EE specification. Server-side utilityclasses can also be bundled with web components and like HTML pages, are not considered webcomponents. As shown in Figure 2.1, the web tier, like the client tier, might include a JavaBeans com-ponent to manage the user input and send that input to enterprise beans running in the business tierfor processing.

Containers are the interface between a component and the low-level platform-specific functionalitythat supports the component. Before a web, enterprise bean, or application client component canbe executed, it must be assembled into a Java EE module and deployed into its container.

The assembly process involves specifying container settings for each component in the Java EEapplication and for the Java EE application itself. Container settings customise the underlying sup-port provided by the Java EE server, including services such as security, transaction management,Java Naming and Directory Interface (JNDI) lookups, and remote connectivity. One security relatedfeature is that the Java EE security model lets one configure a web component or enterprise bean so

1http://java.sun.com/developer/technicalArticles/tools/webapps_1/ [10/11/10]2http://download.oracle.com/javaee/5/tutorial/ [10/11/10]

Chapter 2. Java Web Application Frameworks | 2

http://java.sun.com/developer/technicalArticles/tools/webapps_1/

http://download.oracle.com/javaee/5/tutorial/

Figure 2.1: Web Tier and Java EE Applications

that system resources are accessed only by authorised users.

The deployment process installs Java EE application components in the Java EE containers as illus-trated in Figure 2.2. The following types of containers exist:

• Java EE server: The runtime portion of a Java EE product. A Java EE server provides EJB andweb containers.

• Enterprise JavaBeans (EJB) container: Manages the execution of enterprise beans for Java EEapplications. Enterprise beans and their container run on the Java EE server.

• Web container: Manages the execution of JSP page and Servlet components for Java EE appli-cations. Web components and their container run on the Java EE server.

• Application client container: Manages the execution of application client components. Appli-cation clients and their container run on the client.

• Applet container: Manages the execution of applets. Consists of a web browser and Java pluginrunning on the client together.

Figure 2.3 illustrates the availability of the Java EE 5 platform APIs in each Java EE container type. Thefollowing Sections give a brief summary of some of the technologies required by the Java EE platform,and the APIs used in Java EE applications.

2.1.1 Java Servlet API

The Java Servlet API allows a developer to define HTTP-specific classes. A Servlet class extendsthe capabilities of servers that host applications that are accessed by way of a request-responseprogramming model. Although Servlets can respond to any type of request, they are commonly usedto extend the applications hosted by web servers. Thus, a software developer may use a Servlet toadd dynamic content to a web server using the Java platform. The generated content is commonly


Figure 2.2: Java EE Server and Containers

Figure 2.3: Java EE Platform APIs


HTML, but may be other data such as XML. Servlets are the Java counterpart to non-Java dynamicweb content technologies such as CGI and ASP.NET. Servlets can maintain state in session variablesacross many server transactions by using HTTP cookies or URL rewriting. The Servlet API, containedin the Java package hierarchy javax.servlet, defines the expected interactions of a web containerand a Servlet. A web container is essentially the component of a web server that interacts with theServlets. The web container is responsible for managing the lifecycle of Servlets, mapping a URLto a particular Servlet and ensuring that the URL requester has the correct access rights. Listing2.1 shows a simple Java HTTP Java Servlet which returns a valid HTML document containing thesentence Hello World! and Listing 2.2 shows the necessary deployment descriptor to deploy theServlet. The deployment descriptor defines that the HelloWorldServlet is implemented by thecom.samples.HelloWorld class and that the Servlet can be accessed through the /HelloWorld

URL.

1 public final class HelloWorld extends HttpServlet {

2 @Override

3 public void doGet(HttpServletRequest req , HttpServletResponse resp)

4 throws ServletException , IOException {

5 PrintWriter out = resp.getWriter ();

6 out.println("<html >\n" +

7 "<head ><title >Hello World </title ></head >\n" +

8 "<body >\n" +

9 "<h1 >Hello World!</h1 >\n" +

10 "</body >\n" +

11 "</html >");

12 }

13 }

Listing 2.1: Simple Java HelloWorld Servlet (Specification 2.5)

1 ...

2 <web -app>

3 <servlet >

4 <servlet -name>HelloWorldServlet </servlet -name>

5 <servlet -class >com.samples.HelloWorld </servlet -class >

6 </servlet >

7

8 <servlet -mapping >

9 <servlet -name>HelloWorldServlet </servlet -name>

10 <url -pattern >/HelloWorld </url -pattern >

11 </servlet -mapping >

12 </web -app>

Listing 2.2: Deployment Descriptor for the HelloWorld Servlet

The new Java Servlet 3.0 specification which has been approved in the Java Specification Request(JSR) 315 and can be downloaded from http://jcp.org/en/jsr/detail?id=315noticeably sim-plifies the development of Servlets by introducing a set of new Java annotations.

• @HandlesTypes - This annotation is used to declare the class types that a Servlet-

ContainerInitializer can handle.

• @HttpConstraint - This annotation is used within the ServletSecurity annotation to rep-resent the security constraints to be applied to all HTTP protocol methods for which a corre-sponding HttpMethodConstraint element does NOT occur within the ServletSecurity

annotation (see Section 2.2).


http://jcp.org/en/jsr/detail?id=315

• @HttpMethodConstraint - This annotation is used within the ServletSecurity annotationto represent security constraints on specific HTTP protocol messages (see Section 2.2).

• @MultipartConfig - Annotation that may be specified on a Servlet class, indicating that in-stances of the Servlet expect requests that conform to the multipart/form-data MIME type.

• @ServletSecurity - This annotation is used on a Servlet implementation class to specify se-curity constraints to be enforced by a Servlet container on HTTP protocol messages (see Section2.2).

• @WebFilter - Annotation used to declare a Servlet Filter.

• @WebInitParam - This annotation is used on a Servlet or Filter implementation class to specifyan initialisation parameter.

• @WebServlet - Annotation used to declare a Servlet.

As the HelloWorld Servlet from Listing 2.1 is annotated using the @WebServlet annotation as seenin Listing 2.3, it gets initialised during the startup of the web container. Note that the deploymentdescriptor (as illustrated before in Listing 2.2) is optional in this case. Basically all the configurationelements which could be set through a deployment descriptor file can now be set through the newlyprovided Java annotations, making the XML deployment descriptor obsolete. This includes initiali-sation parameters and security constraints.

1 @WebServlet(name="HelloWorldServlet", urlPatterns ={"/HelloWorld", "/Hello"},

2 initParams = {

3 @WebInitParam(name="param1", value="Hello "),

4 @WebInitParam(name="param2", value="World!")})

5 public final class HelloWorld extends HttpServlet {

6 @Override



9 PrintWriter out = resp.getWriter ();

10 out.println("<html >\n"

11 + "<head ><title >Hello World </title ></head >\n"

12 + "<body >\n"

13 + "<h1 >"

14 + getInitParameter("param1")

15 + getInitParameter("param2")

16 + "</h1 >\n"

17 + "</body >\n"

18 + "</html >");

19 }

20 }

Listing 2.3: Simple Java Servlet (Specification 3.0)

2.1.2 JavaServer Pages Technology

JavaServer Pages (JSP) technology provides a simplified, fast way to create dynamic web content.JSP technology enables rapid development of web-based applications that are server- and platform-independent. JSP technology lets the developer add snippets of Servlet code (see Section 2.1.1) di-rectly into a text-based document. Typically, a JSP page is a text-based document that contains twotypes of text:

• Static data, which can be expressed in any text-based format, such as HTML or XML.


• JSP technology elements, which determine how the page constructs dynamic content.

The packages involved in creating JSP pages are javax.el, javax.servlet.jsp, javax.servlet.jsp.el and javax.servlet.jsp.tagext. JSPs are compiled into Servlets by a JSP compiler. Thecompiler either generates a Servlet in Java code that is then compiled by the Java compiler, or it maycompile the Servlet to byte code which is directly executable. JSPs can also be interpreted on-the-fly,reducing the time taken to reload changes. Listing 2.4 shows a simple JSP page which returns a validHTML document containing the sentence Hello World!.

1 <%@ page contentType="text/html;charset=UTF -8"%>

2 <html >

3 <head ><title >Hello World </title ></head >

4 <body >

5 <h1 ><% out.println("Hello World!"); %></h1>

6 </body >

7 </html >

Listing 2.4: Simple JSP Page

2.1.3 JavaServer Pages Standard Tag Library

The JavaServer Pages Standard Tag Library (JSTL) is a component of the Java EE web application de-velopment platform. It extends the JSP specification (see Section 2.1.2) by adding a tag library of JSPtags for common tasks, such as XML data processing, conditional execution, loops and internation-alization. JSTL provides an effective way to embed logic within a JSP page without using embeddedJava code directly. The use of a standardised tag set, rather than breaking in and out of Java codeleads to more maintainable code and enables separation of concerns between the development ofthe application code and user interface (UI). Listing 2.5 shows how JSTL can be used to dynamicallyprint ten times Hello World! into a valid HTML document.

1 <%@ taglib uri="http :// java.sun.com/jstl/core" prefix="c" %>

2 <html >


4 <body >

5 <c:forEach var="i" begin="1" end="10">

6 <c:out value="Hello World ${i}!" />

7 <br />

8 </c:forEach >

9 </body >

10 </html >

Listing 2.5: Simple JSTL usage Example

2.1.4 JavaServer Faces Technology

JavaServer Faces (JSF) technology is a UI framework for building web applications. The main com-ponents of JavaServer Faces technology involve a GUI component framework, a flexible model forrendering components in various markup languages and technologies and a standard RenderKit forgenerating HTML markup. JSF is a request-driven Model-View-Control (MVC) web framework basedon component driven UI design model, using XML files called view templates or Facelets views.Requests are processed by the FacesServlet, which loads the appropriate view template, builds acomponent tree, processes events and renders the response (typically HTML) to the client. The stateof UI components (and some other objects) is saved at the end of each request (called stateSaving)


and restored upon next creation of that view. Several types of state-saving are available, includingclient-side and server-side state saving. Out of the box, JSF 1.x uses JavaServer Pages (JSP) for itsdisplay technology, but can also accommodate other technologies (such as XUL and Facelets). JSF 2uses Facelets by default for this purpose. Facelets is a more efficient, simple and yet more powerfulview description language. Listing 2.6 shows the Java class used by the JSF view in Listing 2.7 todisplay Hello World! to the client.

1 public final class HelloWorldBean {

2 private String hello = "Hello World!";

3 public String getHello () {

4 return this.hello;

5 }

6 }

Listing 2.6: Hello World Java Bean (POJO)

1 <%@ taglib uri="http :// java.sun.com/jsf/core" prefix="f" %>

2 <%@ taglib uri="http :// java.sun.com/jsf/html" prefix="h" %>

3 <html >


5 <body >

6 <f:view >

7 <h1><h:outputText value="#{ helloWorldBean.hello}" /></h1 >

8 </f:view >

9 </body >

10 </html >

Listing 2.7: Hello World JSF Page

Projects like MyFaces3 or ICEFaces4 create and maintain an open source JavaServer Faces implemen-tation, along with several component libraries containing UI widgets for building web applicationswith JSF.

2.2 Securing Java Web Applications

In Java EE, the component containers are responsible for providing application security. A containerprovides two types of security: declarative and programmatic. The following Sections discuss theseconcepts in more detail.

Using Deployment Descriptors for Declarative Security: Declarative security expresses an applica-tion component’s security requirements using deployment descriptors. A deployment descriptor is aXML document that describes the deployment settings of an application, a module, or a component.Because deployment descriptor information is declarative, it can be changed without the need tomodify the source code of the web application. At runtime, the Java EE server reads the deploymentdescriptor and acts upon the application, module, or component accordingly.

Deployment descriptors must provide certain structural information for each component if this in-formation has not been provided in annotations or is not to be defaulted. Different types of compo-nents use different formats, or schema, for their deployment descriptors. The security elements ofdeployment descriptors include the following:

3http://myfaces.apache.org/ [10/11/10]4http://www.icefaces.org/main/home/ [10/11/10]


http://myfaces.apache.org/

http://www.icefaces.org/main/home/

• Enterprise JavaBeans components use an EJB deployment descriptor that must be namedMETA-INF/ejb-jar.xml and must be contained in the EJB JAR file. The schema for enter-prise bean deployment descriptors is provided in the EJB 3.0 Specification (JSR-220), whichcan be downloaded from http://jcp.org/en/jsr/detail?id=220.

• Web Services components use a jaxrpc-mapping-info deployment descriptor defined in JSR109. This deployment descriptor provides deployment-time mapping functionality betweenJava and WSDL. In conjunction with JSR 181, JAX-WS 2.0 complements this mapping function-ality with development-time Java annotations that control mapping between Java and WSDL.The schema for web services deployment descriptors is provided in Web Services for Java EE(JSR-109), which can be downloaded from http://jcp.org/en/jsr/detail?id=109.

• Web components use a web application deployment descriptor named web.xml. The schemafor web component deployment descriptors is provided in the Java Servlet 2.5 Specifica-tion (JSR-154), Section SRV.13, which can be downloaded from http://jcp.org/en/jsr/

detail?id=154.

Using Annotations: Annotations enable a declarative style of programming and so encompass boththe declarative and programmatic security concepts. Users can specify information about securitywithin a Java class file using annotations. When the application is deployed, this information is usedby the Application Server. Not all security information can be specified using annotations, however.Some information must be specified in the application deployment descriptors.

Annotations eliminate the need for maintaining side files that must be kept up to date with changesin source files. Instead the information can be maintained in the source code.

Using Programmatic Security: Programmatic security is embedded in an application and is usedto make security decisions. Programmatic security is useful when declarative security alone isnot sufficient to express the security model of an application. The API for programmatic securityconsists of two methods of the javax.ejb.EJBContext interface and two methods of the Servletjavax.servlet.http.HttpServletRequest interface. These methods allow components tomake business logic decisions based on the security role of the caller or remote user.

2.2.1 Working with Realms, Users, Groups and Roles

Applications often need to protect resources to ensure that only authorised users have access. Autho-risation provides controlled access to protected resources. Authorisation is based on identificationand authentication. Identification is a process that enables recognition of an entity by a system andauthentication is a process that verifies the identity of a user, device, or other entity in a computersystem, usually as a prerequisite to allowing access to resources in a system.

A realm is defined on a web or application server. It contains a collection of users, which may or maynot be assigned to a group, that are controlled by the same authentication policy.

An application will often prompt a user for their username and password before allowing access toa protected resource. After the user has entered their username and password, that information ispassed to the server, which either authenticates the user and sends the protected resource or doesnot authenticate the user, in which case access to the protected resource is denied.






In some applications, authorised users are assigned to roles. In this situation, the role assigned to theuser in the application must be mapped to a group defined on the application server.

For a web application, a realm is a complete database of users and groups that identify valid users ofa web application (or a set of web applications) and are controlled by the same authentication policy.The Java EE server authentication service can govern users in multiple realms.

In case of the Apache Tomcat Application Server5, principals can be configured in the tomcat-

users.xml configuration file (or in the deployment descriptor web.xml). The following snippetshows an excerpt from the tomcat-users.xml configuration file which defines a new role calledEmployee and a user john.smith with a password secret:

<tomcat-users>

...

<role rolename="Employee"/>

...

<user username="john.smith" password="secret" roles="Employee"/>

</tomcat-users>

The deployment descriptor web.xml of the web application needs to define how users will authenti-cate themselves to the application. This example uses the FORM authentication method, other meansare BASIC, DIGEST and CLIENT-CERT as specified in Version 2.5 of the Java Servlet Specification. TheFORM based authentication requires the specification of a login page (that is automatically shown tounauthenticated users when trying to access secured resources), as well as a login error page (shownin case of a bad username/password combination). Finally, in what appears to be a redundant task,the security roles that the application uses. The following excerpt shows the necessary declarationsin the web.xml configuration file for a FORM based authentication (more detailed in Section 2.2.2):

...

<security-role>

<role-name>Employee</role-name>

</security-role>

<login-config>

<auth-method>FORM</auth-method>

<realm-name>Secured Application Area</realm-name>

<form-login-config>

<form-login-page>/login.jsp</form-login-page>

<form-error-page>/login.jsp?error</form-error-page>

</form-login-config>

</login-config>

...

The login form login.jsp is just a simple HTML form that is submitted to a special container man-aged handler called j_security_check (in case of Tomcat, this is automatically handled by a valvethat is implicitly added to the application context). This handler checks for two special request pa-rameters: j_username and j_password. The following snippet shows the login form and the neces-sary HTML input elements:

5http://tomcat.apache.org/ [07/04/11]


http://tomcat.apache.org/

...

<form method="post" action="j_security_check">

Username: <input type="text" name="j_username"/>

Password: <input type="password" name="j_password"/>

<input type="submit" value="Log In"/>

</form>

...

2.2.2 Specifying Security Roles Using Deployment Descriptor Elements

Application developers communicate how the security is to be set up for the deployed applicationdeclaratively by use of the deployment descriptor mechanism. A deployment descriptor enables anapplication’s security structure, including roles, access control and authentication requirements tobe expressed in a form external to the application. A web application is defined using a standardJava EE web.xml deployment descriptor. A deployment descriptor is a XML schema document thatconveys elements and configuration information for web applications. The deployment descriptormust indicate which version of the web application schema (2.4 or 2.5) it is using and the elementsspecified within the deployment descriptor must comply with the rules for processing that version ofthe deployment descriptor. Version 2.5 of the Java Servlet Specification, which can be downloaded athttp://jcp.org/en/jsr/detail?id=154, contains more information regarding the structure ofdeployment descriptors.

Listing 2.8 is an example of the elements in a deployment descriptor that apply specifically to declar-ing security for web applications or for resources within web applications.

1 <web -app ...>

2 <display -name >A Secure Application </display -name >

3 <servlet >

4 <servlet -name >catalog </servlet -name >

5 <servlet -class >com.webapp.CatalogServlet </servlet -class >

6

7 <security -role -ref >

8 <role -name >EMP </role -name >

9 <role -link >Employee </role -link >

10 </security -role -ref >

11 </servlet >

12

13 <security -role >

14 <role -name >Employee </role -name >

15 </security -role >

16


18 <servlet -name >catalog </servlet -name >

19 <url -pattern >/ catalog /*</url -pattern >


21

22 <security -constraint >

23 <web -resource -collection >

24 <web -resource -name >CartInfo </web -resource -name >

25 <url -pattern >/ catalog/cart/*</url -pattern >

26 <http -method >GET </http -method >

27 <http -method >POST </http -method >

28 </web -resource -collection >

29 <auth -constraint >

30 <role -name >Employee </role -name >



31 </auth -constraint >

32 <user -data -constraint >

33 <transport -guarantee >CONFIDENTIAL </transport -guarantee >

34 </user -data -constraint >

35 </security -constraint >

36

37 <login -config >

38 <auth -method >BASIC </auth -method >

39 </login -config >

40 </web -app >

Listing 2.8: Specifying Security Roles using Deployment Descriptor elements

• <security-role-ref> (Line 7)The security role reference element contains the declaration of a security role reference in theweb application’s code. The declaration consists of an optional description, the security rolename used in the code and an optional link to a security role. The security role name specifiedhere is the security role name used in the code. The value of the role-name element mustbe the String used as the parameter to the HttpServletRequest.isUserInRole(String

role) method. The container uses the mapping of security-role-ref to security-role

when determining the return value of the call. The security role link specified here containsthe value of the name of the security role that the user may be mapped into. The role-linkelement is used to link a security role reference to a defined security role. The role-link

element must contain the name of one of the security roles defined in the security-role

elements.

• <security-role> (Line 13)A security role is an abstract name for the permission to access a particular set of resources inan application. The security-role element is used with the security-role-ref elementto map roles defined in code to roles defined for the web application.

• <login-config> (Line 37)The login configuration element is used to specify the user authentication method to be usedfor access to web content, the realm in which the user will be authenticated and in the case ofform-based login, additional attributes. When specified, the user must be authenticated beforeaccess to any resource that is constrained by a security constraint is being granted. The typesof user authentication methods that are supported include basic, form-based, digest and clientcertificate.

2.2.3 Specifying Security Roles Using Annotations

In Servlets, one can use the following annotations to secure a web application: @DeclareRoles and@RunAs.

The @DeclareRoles annotation is used to define the security roles that comprise the security modelof the application. This annotation is specified on a class and it typically would be used to define rolesthat could be tested (for example, by calling isUserInRole(String)) from within the methods ofthe annotated class.

Listing 2.9 is an example of how this annotation would be used. In this example, Employee is the onlysecurity role specified, but the value of this parameter can include a list of security roles specified bythe application.


1 @DeclareRoles("Employee")

2 public final class CalculatorServlet extends HttpServlet {

3 @Override

4 public void doGet(HttpServletRequest req , HttpServletResponse resp) {

5 if(req.isUserInRole("Employee")) {

6 // ...

7 }

8 }

9 }

Listing 2.9: @DeclareRoles Annotation Example

Specifying @DeclareRoles("Employee") is equivalent to defining the following in the web.xml

deployment descriptor:

<security-role>

<role-name>Employee</role-name>

</security-role>

The syntax for declaring more than one role is shown in the following example:

@DeclareRoles({"Administrator", "Manager", "Employee"})

When a call is made to isUserInRole from the annotated Java class, the caller identity associatedwith the invocation of the class is tested for membership in the role with the same name as the argu-ment to isUserInRole. If a security-role-ref has been defined for the argument role-name,the caller is tested for membership in the role mapped to the role-name.

The @RunAs annotation defines the role of the application during execution in a Java EE container. Itcan be specified on a class, allowing developers to execute an application under a particular role. Therole must map to the user/group information in the container’s security realm. The value elementin the annotation is the name of a security role of the application during execution in a Java EEcontainer. Listing 2.10 is an example that uses the @RunAs annotation. The @RunAs annotation isequivalent to the run-as element in the deployment descriptor.

1 @RunAs("Administrator")

2 public final class CalculatorServlet extends HttpServlet {

3 @Override

4 public void doGet(HttpServletRequest req , HttpServletResponse resp) {

5 // ...

6 }

7 }

Listing 2.10: @RunAs Annotation Example

2.2.4 Specifying Security Constraints Using Deployment Descriptor Elements

Security constraints are a declarative way to define the protection of web content. A security con-straint is used to define access privileges to a collection of resources using their URL mapping. Se-curity constraints are defined in a deployment descriptor by using the following XML elements (seeListing 2.8):

• <web-resource-collection> (Line 23)A web resource collection is a list of URL patterns (the part of a URL after the host name and


port which should get constrained) and HTTP operations (the methods within the files thatmatch the URL pattern which should get constrained (for example, POST, GET) that describe aset of resources to be protected.

• <auth-constraint> (Line 29)

An authorisation constraint establishes a requirement for authentication and names the rolesauthorised to access the URL patterns and HTTP methods declared by this security constraint.If there is no authorisation constraint, the container must accept the request without requiringuser authentication. If there is an authorisation constraint but no roles are specified withinit, the container will not allow access to constrained requests under any circumstances. Thewildcard character * can be used to specify all role names defined in the deployment descriptor.

• <user-data-constraint> (Line 32)A user data constraint establishes a requirement that the constrained requests be received overa protected transport layer connection. This guarantees how the data will be transported be-tween client and server. The choices for type of transport guarantee include NONE, INTEGRALand CONFIDENTIAL. If no user data constraint applies to a request, the container must acceptthe request when received over any connection, including an unprotected one.

Security constraints only work on the original request URI and not on calls made through aRequestDispatcher (which include <jsp:include> and <jsp:forward>). Inside the appli-cation, it is assumed that the application itself has complete access to all resources and would notforward a user request unless it had decided that the requesting user also had access.

The Java Servlet Specification defines the request URI as the part of a URL after the host name andport. For example, let’s consider an e-commerce site with a browsable catalog which is accessible foranyone and a shopping cart area that’s only accessible for customers. A developer setts up the pathsfor the web application so that the pattern /cart/* is being protected but nothing else. Assumingthat the application is installed at context path /myapp, the following is true:

• http://localhost:8080/myapp/index.jsp is not protected.

• http://localhost:8080/myapp/cart/index.jsp is protected.

2.2.5 Specifying Security Constraints Using Annotations

The @ServletSecurity annotation (since version 3.0 of the Servlet Specification) provides analternative mechanism for defining access control constraints equivalent to those that could oth-erwise have been expressed declaratively via security-constraint elements in the deploymentdescriptor (see Listing 2.8). The Servlet container will enforce these constraints on the URL patternsmapped to the Servlets mapped to the annotated class. Listing 2.11 shows an example on how touse the @ServletSecurity annotation. The added constraints enforce that only a user with thecustomer role is allowed to access the CartServlet.

1 @WebServlet(name="cart", urlPatterns ={"/cart/"})

2 @ServletSecurity (( httpMethodConstraints = {

3 @HttpMethodConstraint(value = "GET", rolesAllowed = "customer"),

4 @HttpMethodConstraint(value = "POST",

5 emptyRoleSemantic = EmptyRoleSemantic.DENY)}))

6 public final class CartServlet extends HttpServlet {

7

8 @Override




11 // ...

12 }

13

14 @Override

15 public void doPost(HttpServletRequest req , HttpServletResponse resp)


17 // ...

18 }

19 }

Listing 2.11: @ServletSecurity Annotation Example

2.3 Securing Java EE Applications

Enterprise beans are the Java EE components that implement Enterprise JavaBeans (EJB) technology.Enterprise beans run in the EJB container, a runtime environment within the Application Server. Be-sides the already discussed @DeclareRoles and @RunAs annotations (see Section 2.2.3), beans canbe additionally annotated with method permission constraints as discussed in the following Sections.

2.3.1 Specifying Method Permissions Using Annotations

The method permissions for the methods of a bean class can be specified on the class, the businessmethods of the class or both. Method permissions can be specified on a method of the bean class tooverride the method permissions value specified on the entire bean class. The following annotationsare used to specify method permissions:

• @RolesAllowed(list-of-roles)

The value of the @RolesAllowed annotation is a list of security role names to be mapped tothe security roles that are permitted to execute the specified method or methods. Specifyingthis annotation on the bean class means that it applies to all applicable business methods ofthe class.

• @PermitAll

The @PermitAll annotation specifies that all security roles are permitted to execute the spec-ified method or methods. Specifying this annotation on the bean class means that it applies toall applicable business methods of the class.

• @DenyAll

The @DenyAll annotation specifies that no security roles are permitted to execute the specifiedmethod or methods.

Listing 2.12 illustrates the use of these annotations in an enterprise bean for updating a shoppingcart in an e-commerce application. Only users belonging to the CartUser role are allowed to updatetheir shopping carts through methods provided by the CartBean.

1 @Stateful ()

2 public class CartBean implements Cart {

3 private String customerName;

4 private String customerId;

5 private List <String > contents;

6


7 @RolesAllowed("CartUser")

8 public void addBook(final String title) {

9 // ...

10 }

11

12 @RolesAllowed("CartUser")

13 public void removeBook(final String title) throws BookException {

14 // ...

15 }

16

17 @DenyAll

18 public void debug() {

19 // ...

20 }

21 }

Listing 2.12: @RolesAllowed Annotation Example

2.3.2 Specifying Method Permissions Using Deployment Descriptors

Method permissions can be defined by using the method-permission element in the deploymentdescriptor as discussed below:

• Each method-permission element includes a list of one or more security roles and a list ofone or more methods. All the listed security roles are allowed to invoke all the listed methods.Each security role in the list is identified by the role-name element. Each method (or set ofmethods) is identified by the method element.

• The method permissions relation is defined as the union of all the method permissions definedin the individual method-permission elements.

• A security role or a method can appear in multiple method-permission elements.

There are three legal styles for composing the method element:

• The first style is used for referring to all of the business interface, home interface, componentinterface and web service endpoints methods of a specified bean.

<method>

<ejb-name>CartBean/ejb-name>

<method-name>*</method-name>

</method>

• The second style is used for referring to a specified method of the business interface, homeinterface, component interface or web service endpoints methods of the specified enterprisebean. If the enterprise bean contains multiple methods with the same overloaded name, theelement of this style refers to all of the methods with the overloaded name.

<method>

<ejb-name>CartBean</ejb-name>

<method-name>addBook</method-name>

</method>

• The third style is used for referring to a specified method within a set of methods with anoverloaded name. The method must be defined in the business interface, home interface,


component interface or web service endpoints methods of the specified enterprise bean. Ifthere are multiple methods with the same overloaded name, however, this style refers to all ofthe overloaded methods. All of the parameters are the fully-qualified Java types, for example,java.lang.String.

<method-permission>

<role-name>CartUser</role-name>

<method>

<ejb-name>CartBean</ejb-name>

<method-name>addBook</method-name>

<method-params>

<method-param>String</method-param>

</method-params>

</method>

</method-permission>

2.4 Java Security Frameworks

The following Sections look at various Java security frameworks which can be used to secure Javaweb or EE applications. Most of the described security frameworks offer programmatic security fea-tures with a few exceptions that can be integrated into an application container to enforce securityconstraints.

2.4.1 Java Authentication and Authorization Service (JAAS)

The Java Authentication and Authorization Service (JAAS) was introduced as an optional package(extension) to the Java 2 SDK, Standard Edition (J2SDK), v 1.3. JAAS was integrated into the J2SDK1.4. JAAS can be used for two purposes:

• for authentication of users, to reliably and securely determine who is currently executing Javacode, regardless of whether the code is running as an application, an applet, a bean or a servlet;and

• for authorisation of users to ensure they have the access control rights (permissions) requiredto do the actions performed.

For an application developer, JAAS is a standard library that provides: a representation of identity(Principal) and a set of credentials (Subject), a login service that will invoke the applicationcallbacks to ask the user things like username and password and a service that tests if a Subject wasgranted a permission by an administrator.

For the security system integrator, JAAS provides the following interfaces: to provide an identitynamespace to applications, to attach credentials to threads (Subject) and for developing loginmodules. These modules invoke callbacks to query the user, check their response and generate aSubject.

JAAS authentication is performed in a pluggable fashion. This permits applications to remain inde-pendent from underlying authentication technologies. New or updated authentication technologiescan be plugged under an application without requiring modifications to the application itself. Ap-plications enable the authentication process by instantiating a LoginContext object, which in turnreferences a Configuration to determine the authentication technology(ies) or LoginModule(s),


to be used in performing the authentication. A typical LoginModule may prompt for and verify ausername and password. Others may read and verify a voice or fingerprint sample6.

To authenticate a subject (user or service), the following steps are performed:

1. An application instantiates a LoginContext.

2. The LoginContext consults a Configuration to load all of the LoginModules configuredfor that application.

3. The application invokes the LoginContext’s login method.

4. The login method invokes all of the loaded LoginModules. Each LoginModule attempts toauthenticate the subject. Upon success, LoginModules associate relevant Principals andcredentials with a Subject object that represents the subject being authenticated.

5. The LoginContext returns the authentication status to the application.

6. If authentication succeeded, the application retrieves the Subject from the LoginContext.

2.4.2 Spring Security

The Spring Security project started in 2003 (formerly known as “Acegi Security”) and turned into asophisticated authentication and access control system and became widely adopted as the standardsolution for securing Spring Framework based applications (see [26, p.5]). Spring Security can also beused with different MVC frameworks such as Struts.

Spring Security exists to fill a gap in the universe of Java third-party libraries [...] Standardssuch as Java Authentication and Authorization Service (JAAS) or Java EE Security do offersome ways of performing some of the same authentication and authorization functions,but Spring Security is a winner because it packages up everything you need to implementa top-to-bottom application security solution in a concise and sensible way. Additionally,Spring Security appeals to many because it offers out of the box integration with manycommon enterprise authentication systems; so it’s adaptable to most situations with littleeffort (beyond configuration) on the part of the developer. [26, p.18])

Spring Security’s primary influence over an application is through a series of ServletRequestfilters. The DelegatingFilterProxy is a Servlet filter that allows Spring Security to wrap allapplication requests and ensure that they are appropriately secured (added to the deploymentdescriptor web.xml). Listing 2.13 shows an example Spring Security XML configuration file called*-security.xml, whereas the wildcard stands for the name of the web application. This style ofconfiguration, using a Spring Security-specific XML dialect is known as the security namespace style,named after the XML namespace (http://www.springframework.org/schema/security) as-sociated with the XML configuration elements. Listing 2.13 adds various constraints to the webapplication based on URL patterns such as restricting users to access resources available through thefollowing URL /secure/extreme/** if they do not belong to the ROLE_SUPERVISOR role.

1 <?xml version ="1.0" encoding ="UTF -8"?>

2 <beans ...>

3 <http use -expressions ="true">

6http://download.oracle.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html

[11/11/10]


http://www.springframework.org/schema/security

http://download.oracle.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html

4 <intercept -url pattern ="/ login.do" access =" permitAll"/>

5 <intercept -url pattern ="/*" access =" hasRole(’ROLE_USER ’)"/>

6 <intercept -url pattern ="/ secure/extreme /**"

7 access =" hasRole(’ROLE_SUPERVISOR ’)"/>

8 <intercept -url pattern ="/ secure /**"

9 access =" isAuthenticated ()" requires -channel ="https"/>

10

11 

12 <session -management invalid -session -url ="/ timeout.jsp">

13 <concurrency -control max -sessions ="1"

14 error -if-maximum -exceeded ="true" />

15 </session -management >

16 </http >

17 </beans:beans >

Listing 2.13: Sample Spring Security XML Configuration File

Spring offers the @PreAuthorize annotation for ensuring that a particular Java method can onlybe executed when the user is in the specified user group. Listing 2.14 shows an example on howto use the @PreAuthorize annotation for securing a method which a user can use to change hispassword. The security constraint enforces that users must be logged in as a valid user of the sys-tem before they are allowed to change a password. This is all that is required to ensure that valid,authenticated users can access the change password function. Spring Security will use a runtimeaspect oriented programming (AOP) pointcut to execute before advice on the method and throws anAccessDeniedException if the security constraints are not met.

1 public interface IUserService {

2 @PreAuthorize("hasRole(’ROLE_USER ’)")

3 public void changePassword(String username , String password);

4 }

Listing 2.14: @PreAuthorize Annotation Example

The SpEL (The Spring Expression Language) binding used by the Spring Security method annotationssupports more sophisticated expressions, including expressions that incorporate method parame-ters. For example, it might make sense to restrict the changePassword method from Listing 2.14further, so that a user attempting to invoke the method must satisfy two constraints:

• The user must be attempting to change their own password or

• the user is an administrator (in which case it is valid for the user to change anyone’s password,possibly through an administrative interface)

Listing 2.15 shows how the username of the principal is being checked against the username methodparameter (#username - the method parameter name prefixed with a # symbol).

1 @PreAuthorize("#username == principal.username and hasRole(’ROLE_USER ’)")

2 public void changePassword(String username , String password);

Listing 2.15: @PreAuthorize Annotation Example with SpEL Directive

Two final Spring Security-dependent annotations are @PreFilter and @PostFilter, which areused to apply security-based filtering rules to Collections or Arrays (with @PostFilter only). Thistype of functionality is referred to as security trimming or security pruning and involves using theprincipal’s security credentials at runtime to selectively remove members from a set of objects. Thisfiltering is performed using SpEL expression notation within the annotation declaration.

Listing 2.16 illustrates how a collection of categories are being filtered by the use of the @PostFilterannotation.

The filterObject is re-bound to the SpEL context for each element in the Collection. The SpELexpression must return a Boolean value. If the expression evaluates to true, the object will remain inthe Collection, while if the expression evaluates to false, the object will be removed.

1 @PostFilter("(! filterObject.customersOnly) or

2 (filterObject.customersOnly and hasRole(’ROLE_USER ’))")

3 Collection <Category > getCategories ();

Listing 2.16: @PostFilter Annotation Example

The @PreFilter annotation can be applied to a method to filter Collection elements that arepassed into a method based on the current security context. Functionally, once it has a referenceto a Collection, this annotation behaves exactly the same as the @PostFilter annotation, with acouple of exceptions:

• @PreFilter supports only Collection arguments and does not support Array arguments.

• @PreFilter takes an additional, optional attribute filterTarget, which is used to specifi-cally identify the method parameter to filter when the annotated method has more than oneargument.

Listing 2.17 shows how the @PreFilter annotation can be added to a method in order to filter thecollection passed to it.

1 @PreFilter("(! filterObject.customersOnly) or

2 (filterObject.customersOnly and hasRole(’ROLE_USER ’))")

3 public Collection <Category > filterCategories(

4 Collection <Category > categories);

Listing 2.17: @PreFilter Annotation Example

2.4.3 PicketBox

PicketBox7 (formerly JBoss Security) is a security framework for Java applications. The features avail-able are: Authentication, Authorization/Access Control, Auditing and Mapping (Principal/Roles/At-tribute). PicketBox is a simple framework with a single configuration file. PicketBox and Spring Secu-rity are very similar but PicketBox has a slightly different philosophy behind it. While Spring Securitydoes not use the features of container managed security, PicketBox on the other hand tries to integratewith containers such as JBoss Application Server seamlessly so that applications using PicketBox canhave seamless security into Java EE components such as EJB3. PicketBox is best used together withthe Seam 3 web development framework. Seam 3 will utilise PicketBox as its security foundation (notyet released). PicketBox basically enhances the container managed security features by providing ad-ditional authentication modules. Authentication of users can be done as seen in Section 2.2.1 with aform pointing to j_security_check. In order to use the features of PicketBox, a developer has todefine the appropriate PicketBox login module in the application server context (context.xml).

7http://www.jboss.org/picketbox [3/12/10]


http://www.jboss.org/picketbox

2.4.4 OWASP Enterprise Security API (ESAPI)

ESAPI (The OWASP Enterprise Security API)8 is a free, open source web application security controllibrary that makes it easier for programmers to write lower-risk applications. The ESAPI librariesare designed to make it easier for programmers to retrofit security into existing applications. TheESAPI libraries also serve as a solid foundation for new development. The API contains several inputvalidation helpers, a web application firewall (WAF), modules for preventing Clickjacking9 attacks,modules for form-based user authentication, methods for enforcing access control lists, etc. Theconfiguration is done through text files which are placed inside the WEB-INF folder of a Java webapplication.

The code snippet in Listing 2.18 shows how to use the input validation feature of the ESAPI library toprevent Code Injection Attacks.

1 try {

2 String cleanComment = ESAPI.validator ().getValidInput(

3

4 "comment", // A descriptive name of the parameter that you are

5 // validating. This value is used by any logging or

6 // error handling that is done with respect to

7 // the value passed in.

8

9 request.getParameter("comment"), // The actual user input

10 // data to validate.

11

12 "CommentRegex", // The regular expression name that maps

13 // to the actual regular expression from

14 // "ESAPI.properties ".

15 // e.g.: CommentRegex =^[\\s\\p{L}\\p{N}.]{0 ,1024}$

16

17 300, // The maximum post -canonicalized String length allowed.

18

19 false , // If allowNull is true then an input that is NULL or an

20 // empty string will be legal. If allowNull is false then

21 // NULL or an empty String will throw a ValidationException.

22

23 true); // If canonicalize is true then input will be

24 // canonicalized before validation

25 } catch (ValidationException e) {

26 // do error handling here ...

27 }

Listing 2.18: ESAPI Example on how to do Input Validation

The ESAPI web application firewall (WAF) implemented as a Servlet filter is driven by a XML policyfile that tells it what rules to enforce in the application. These rules can do a number of things, fromsimple virtual patching to complex authorisation enforcement with BeanShell scripts10.

The <authentication-rules> section allows the WAF to enforce typical J2EE authenticationrequirements. A typical J2EE application authentication pattern involves making sure a sessionvariable exists inside some base action class. If the variable does not exist, the request is consideredto be unauthenticated and is handled appropriately. Listing 2.19 shows an example which makes thewhole web application only accessible for authenticated users through the path="/.*" directive on

8http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API [15/11/10]9http://www.sectheory.com/clickjacking.htm [11/04/11]

10http://jcp.org/en/jsr/detail?id=274 [11/04/11]


http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API

http://www.sectheory.com/clickjacking.htm


line 1 with the exception of the login and index pages etc. by checking if the UserAuthKey exists inthe user’s session.

1 ...

2 <authentication -rules path ="/.*" key=" UserAuthKey">

3 <path -exception >/</path -exception >

4 <path -exception >/index.html </path -exception >

5 <path -exception >/login.jsp </path -exception >

6 <path -exception >/index.jsp </path -exception >

7 <path -exception type=" regex">/images /.*</path -exception >

8 <path -exception type=" regex">/css/.*</path -exception >

9 <path -exception type=" regex">/help /.*</path -exception >

10 </authentication -rules >

11 ...

Listing 2.19: ESAPI WAF Configuration Example on how to enforce Authorisation Constraints

2.4.5 Apache Shiro

The Apache Shiro project has been formerly known as JSecurity and Apache KI.

Apache Shiro is a powerful and easy-to-use Java security framework that performs au-thentication, authorization, cryptography and session management. With Shiro’s easy-to-understand API, you can quickly and easily secure any application - from the smallestmobile applications to the largest web and enterprise applications11.

Shiro offers quite similar features as the OWASP Enterprise Security API (see Section 2.4.4). Especiallyfor web applications Shiro offers a set of Servlet filters for securing Java web applications. These filtersare being configured in the deployment descriptor and include the following:

• AnonymousFilter (anon): A filter that allows access to a path immediately without perform-ing security checks of any kind. This filter is useful primarily in exclusionary policies.

• FormAuthenticationFilter (authc): Requires the requesting user to be authenticated forthe request to continue, and if they are not, forces the user to login by redirecting them to theconfigured loginUrl.

• BasicHttpAuthenticationFilter (authcBasic): Requires the requesting user to be authen-ticated for the request to continue, and if they are not, requires the user to login via the HTTPBasic protocol-specific challenge. Upon successful login, they are allowed to continue on tothe requested resource or URL. This implementation is a “clean room” Java implementation ofBasic HTTP Authentication specification per RFC 2617 [16].

• PermissionsAuthorizationFilter (perms): Filter that allows access if the current user hasthe permissions specified by the mapped value or denies access if the user does not have all ofthe permissions specified.

• PortFilter (port): A Filter that requires the request to be on a specific port, and if not, redi-rects to the same URL on that port.

• HttpMethodPermissionFilter (rest): A filter that translates a HTTP request’s method (e.g.GET, POST, etc.) into a corresponding action (verb) and uses that verb to construct a permis-sion that will be checked to determine access. This filter is primarily provided to support REST

11http://shiro.apache.org/ [19/11/10]


http://shiro.apache.org/

environments where the type (method) of request translates to an action being performed onone or more resources. This paradigm works well with Shiro’s concepts of using permissions foraccess control and can be leveraged to easily perform permission checks. This filter functionsas follows:

– The incoming HTTP request’s method (GET, POST, PUT, DELETE, etc.) is discovered.

– The method is translated into a more ’application friendly’ verb, such as create, edit, delete,etc.

– The verb is appended to any configured permissions for the currently matching path.

– If the current Subject isPermitted to perform the resolved action, the request is allowedto continue.

• RolesAuthorizationFilter (roles): Filter that allows access if the current user has the rolesspecified by the mapped value or denies access if the user does not have all of the roles speci-fied.

• SslFilter (ssl): The port property defaults to 443 and also additionally guarantees that therequest scheme is always HTTPS (except for port 80, which retains the HTTP scheme).

• UserFilter (user): Filter that allows access to resources if the accessor is a known user, whichis defined as having a known principal. This means that any user who is authenticated or re-membered via a remember me feature will be allowed access from this filter. If the accessor isnot a known user, then they will be redirected to the loginUrl.

The main advantage Shiro offers as compared to other existing Java security frameworks is the abilityto define ad-hoc filter chains for any matching URL path in the web application. This is being config-ured in the [url] section of the Shiro INI file. Listing 2.20 shows an excerpt from that configurationfile.

1 ...

2 [urls]

3

4 /index.html = anon

5 /user/create = anon

6 /user /** = authc

7 /admin /** = authc , roles[administrator]

8 /rest /** = authc , rest

9 /remoting/rpc /** = authc , perms[" remote:invoke "]

10 /account /** = ssl , authc

Listing 2.20: Apache Shiro Ad-Hoc Filter Chain Configuration Example

For example, line 10 from Listing 2.20 states that “Any request to the application’s path of /accountor any of it’s sub paths (/account/foo, /account/bar/baz, etc.) will trigger the ssl, authc filterchain”. Note that all path expressions are relative to the application’s context root.

2.4.6 jGuard

The following statement has been taken from the jGuard project site12:

jGuard is a library that provides EASY security (authentication and authorization) for Javaweb applications. It is built over the stable and mature JAAS framework, which is part

12http://jguard.xwiki.com/xwiki/bin/view/Main/WebHome [23/11/10]


http://jguard.xwiki.com/xwiki/bin/view/Main/WebHome

of the JAVA J2SE API. jGuard is very flexible and allows several different ways to config-ure those mechanisms for authentication and authorization, i.e., in a relational database,XML files or LDAP service.

The net.sf.jguard.filters.AccessFilter class is the main integration between the webapplication and jGuard. Access control checks are being done through the AccessFilter. TheAccessFilter is a Servlet filter and has the following duties: Configure the access control of theweb application, authenticate the users, deny unauthorised access to a URL, log off the user (the userneeds to log in again).

After the roles (principals) have been configured in the jGuardUsersPrincipals.xml

file it is necessary to define the permissions related to these principals injGuardPrincipalsPermissions.xml. These jGuard XML configuration files need to be placedinside the WEB-INF folder of the web application.

Listing 2.21 shows the jGuard jGuardPrincipalsPermissions.xml configuration file whichis needed in order to specify which user roles are allowed to access which resources of the webapplication. Frankly, the configuration seems to be tedious to maintain for larger web applications.

1 ...

2 <configuration >

3 <permissions >

4 <domain >

5 

7 <name >public </name >

8 <permission >

9 <name >LoginForm </name >

10 <class >

11 net.sf.jguard.core.authorization.permissions.URLPermission

12 </class >

13 <actions >

14 <action >/ LoginForm.do </action >

15 <action >ANY </action >

16 </actions >

17 </permission >

18 ...

19 </domain >

20

21 <domain >

22 

24 <name >secured </name >

25 <permission >

26 <name >Authenticated </name >

27 <class >

28 net.sf.jguard.core.authorization.permissions.URLPermission

29 </class >

30 <actions >

31 <action >/ Authenticated.do </action >

32 <action >ANY </action >

33 <action >you have been authenticated </action >

34 </actions >

35 </permission >

36 </domain >

37 </permissions >

38

39 <principals >

40 <principal >

41 <name >guest </name >

42 <class >net.sf.jguard.core.principals.RolePrincipal </class >

43 <permissionsRef >

44 <permissionRef name=" LoginForm"/>

45 ...

46 </permissionsRef >

47 </principal >

48

49 <principal >

50 <name >admin </name >

51 <class >

52 net.sf.jguard.core.principals.RolePrincipal

53 </class >

54 <permissionsRef >

55 <domainRef name=" secured"/>

56 </permissionsRef >

57 </principal >

58 </principals >

59 </configuration >

Listing 2.21: jGuard User Permissions Configuration Example

2.4.7 Apache Commons Validator

Commons Validator13 is an input validation component devised (originally) by David Winterfeldt asan add-on for the Struts framework. Its functionality has since been split so that Validator can serveas the basis for an independent component and is now part of Jakarta Commons. Commons Valida-tor provides two distinct sets of functionality: A configurable (typically XML) validation engine andreusable “primitive” validation methods. The Commons Validator is a basic validation frameworkthat lets a developer define validation rules for a JavaBean in a XML file. Validators as well as thevalidation definition can also be defined in the XML file. An example of a validator would be definingwhat method and class will be called to perform the validation for a required field. Validation rulescan be grouped together based on locale and a JavaBean/Form that the rules are associated with. Theframework has basic support for user defined constants which can be used in some field attributes.

The configuration of the validation rules are being done in a XML file called validation.xml whichhas to be placed inside the WEB-INF folder of a Java web application. The following example is basedon the Struts 1.x Framework. The following snippet shows an excerpt from a validation.xml con-figuration file which validates two password input fields. This might be necessary for a user regis-tration process where the user has to choose a password for his account and to make sure that theuser did not make a typing error, he has to repeat his password in a second field. The validationroutine checks if the two password fields meet the minimum length requirements and if the enteredpasswords are identical:

...

<formset>

<form name="userRegistrationForm">

...

<field property="password" depends="required,minlength,twofields">

<arg key="prompt.password"/>

13http://commons.apache.org/validator/ [3/12/10]


http://commons.apache.org/validator/

<arg name="minlength" key="${var:minlength}"/>

<arg name="twofields" key="prompt.password2"/>

<var>

<var-name>minlength</var-name>

<var-value>8</var-value>

</var>

<var>

<var-name>secondProperty</var-name>

<var-value>password2</var-value>

</var>

</field>

</form>

</formset>

Whereas the <form name="userRegistrationForm"> corresponds to the form bean definition inthe struts-config.xml configuration file as seen below:

<struts-config>

<form-beans>

<form-bean name="userRegistrationForm"

type="com.webapp.UserRegistrationActionForm"/>

</form-beans>

...

</struts-config>

Validator ships with a set of pre-defined validators, as follows:

• required - mandatory field validation.

• validwhen - validator for checking one field against another.

• minlength - validate input data is not less than a specified minimum length. Requires aminlength variable.

• maxlength - validate input data does not exceed a specified maximum length. Requires amaxlength variable.

• mask - validate format according to a regular expression. Requires a mask variable to specifythe regular expression.

• byte - validates that a field can be converted to a Byte.

• short - validates that a field can be converted to a Short.

• integer - validates that a field can be converted to an Integer.

• long - validates that a field can be converted to a Long.

• float - validates that a field can be converted to a Float.

• double - validates that a field can be converted to a Double.


• date - validates that a field can be converted to a Date. This validator uses java.text.

SimpleDateFormat to parse the date and optionally either a datePattern ordatePatternStrict variable can be used. If no pattern is specified the default short date for-mat is assumed. The difference between using the datePatternStrict and datePattern

variables is that datePatternStrict checks additionally that the input data is the samelength as the pattern specified (so for example 1/1/2011 would fail with a pattern ofMM/dd/yyyy).

• intRange - validates that an integer field is within a specified range. Requires min and max

variables to specify the range. This validator depends on the integer validator which mustalso be in the field’s depends attribute.

• longRange - validates that a long field is within a specified range. Requires min and max vari-ables to specify the range. This validator depends on the long validator which must also be inthe field’s depends attribute.

• floatRange - validates that a float field is within a specified range. Requires min and max vari-ables to specify the range. This validator depends on the float validator which must also be inthe field’s depends attribute.

• doubleRange - validates that a double field is within a specified range. Requires min and max

variables to specify the range. This validator depends on the double validator which must alsobe in the field’s depends attribute.

• creditCard - validate credit card number format.

• email - validate email address format.

• url - validates URL format. Has four optional variables (allowallschemes, allow2slashes,nofragments and schemes) which can be used to configure this validator.

– allowallschemes - specifies whether all schemes are allowed. Valid values are true orfalse (default is false). If this is set to true then the schemes variable is ignored.

– allow2slashes - specifies whether double ’/’ characters are allowed. Valid values aretrue or false (default is false).

– nofragments - specifies whether fragments are allowed. Valid values are true or false(default is false - i.e. fragments are allowed).

– schemes - use to specify a comma separated list of valid schemes. If not specified thenthe defaults are used which are http , https and ftp.

<field property="custUrl" depends="url">

<arg position="0" key="customer.url"/>

<var>

<var-name>nofragments</var-name>

<var-value>true</var-value>

</var>

<var>

<var-name>schemes</var-name>

<var-value>http,https,telnet,file</var-value>

</var>

</field>


2.5 Popular Web Application Frameworks for the Java ProgrammingLanguage

Deciding which web application framework is the most popular or widely used among applicationdevelopers is a very difficult task and can not be answered scientifically. Using services like GoogleTrends14, the Google AdWords Keyword15 or the Google Insights16 tool it is however possible to givea rough estimate about how popular a particular web framework is by comparing the number oftimes people actually typed the name of the framework into the Google search engine. Anotherestimate whether a web framework is more or less popular than another can be found by comparingthe number of available books (for instance from Amazon.com) for a particular framework. Table2.2 contains a subset of frameworks from the "Open Source Web Frameworks in Java"17 list atJava-Source.net, the roughly estimates of Google searches per month and the roughly number ofavailable books from Amazon.com. Many of the frameworks listed on Java-Source.net do not have aworking project page any more and thus considered dead. Furthermore, frameworks which have notbeen updated in the last 6 years are also considered dead.

Analysing the gathered data in Table 2.2 four frameworks clearly stand out from the rest: SpringFramework, Apache Struts 1.x or 2.x, the Google Web Toolkit18, the JavaServer Faces technology (re-spectively the popular Apache MyFaces Tomahawk implementation will be covered in the followingsecurity analysis of the frameworks) and the Apache Wicket framework. Each following web applica-tion framework Section will consist of an broad overview over the framework, the available securityfeatures and the features regarding extendibility.

2.6 Spring Framework

The following statement has been taken from the Spring Framework project site19:

Spring is a layered Java/J2EE application platform, based on code published in [18].Spring includes: a lightweight container [. . . ], a common abstraction layer for transac-tion management [. . . ], a JDBC abstraction layer [. . . ] and a flexible MVC web applicationframework [. . . ]

The Spring Framework features its own MVC framework. The Spring developers decided to writetheir own web framework as a reaction to what they perceived as the poor design of the popularApache Struts Web Application Framework20, as well as deficiencies in other available frameworks.In particular, they felt there was insufficient separation between the presentation and requesthandling layers and between the request handling layer and the model (see [18, Chap. 12]). LikeStruts, Spring MVC is a request-based framework. The framework defines strategy interfaces for allof the responsibilities which must be handled by a modern request-based framework. The goal ofeach interface is to be simple and clear so that it is easy for Spring MVC users to write their ownimplementations if they choose so. MVC paves the way for cleaner front end code. All interfaces aretightly coupled to the Java Servlet API. This coupling makes sure that the features of the Java ServletAPI remain available to developers while offering a high abstraction framework to ease working

14http://www.google.com/trends [09/11/2010]15https://adwords.google.com/select/KeywordToolExternal [09/11/2010]16http://www.google.com/insights/search/ [10/11/10]17http://java-source.net/open-source/web-frameworks [09/11/10]18http://code.google.com/webtoolkit/ [10/11/10]19http://www.springsource.org/about [10/11/10]20http://www.theserverside.com/news/1364527/Introduction-to-the-Spring-Framework [11/11/10]


http://www.google.com/trends

https://adwords.google.com/select/KeywordToolExternal

http://www.google.com/insights/search/

http://java-source.net/open-source/web-frameworks

http://code.google.com/webtoolkit/

http://www.springsource.org/about

http://www.theserverside.com/news/1364527/Introduction-to-the-Spring-Framework

Po

pu

lari

ty(#

ofG

oo

gle

sear

ches/

mo

nth

)

Co

mm

un

ity

(Las

trel

ease

dat

e)

Lea

rnin

g(#

ofb

oo

ksin

Am

azo

n.c

om

)

Ap

pFu

se~1

4’80

015/1

1/10

~1A

rib

aWeb

~720

18/1

0/10

~0C

oco

on

~260

15/0

5/08

~4E

cho

Web

Fram

ewo

rk~2

6006/0

8/09

~0G

oo

gle

Web

Too

lkit

~141

’000

28/1

0/10

~14

ICE

face

s~9

0’50

030/0

9/09

~14

Ind

uct

ion

~210

01/0

8/10

~0Ja

vaSe

rver

Face

s~5

50’0

0028/0

6/09

~22+

JBo

ssSe

am~4

0,50

001/0

8/10

~7JP

ub

lish

~110

01/0

4/10

~0jZ

eno

~22

16/0

6/09

~0O

pen

Xav

a~2

’400

28/0

7/10

~1P

ust

efix

~4’4

0012/0

1/09

~0R

estl

et~1

8’10

007/1

1/10

~1Sp

rin

gFr

amew

ork

~372

’500

20/1

0/10

~17

Stri

pes

~1’9

0016/1

2/09

~1A

pac

he

Stru

ts~1

34’8

8016/0

8/10

~20

Stre

cks

~5,4

0013/0

6/07

~0Ta

pes

try

~13’

900

01/1

1/10

~1Tu

rbin

e~7

2012/0

6/08

~0M

aku

mb

a~8

8023/1

2/09

~0A

pac

he

MyF

aces

Tom

ahaw

k~4

3’80

025/0

9/10

~6W

ebw

ork

~49’

500

21/0

7/07

~1W

icke

t~2

4’42

017/0

9/10

~3W

ingS

~009/0

4/08

~0Z

koss

~9’9

0002/1

1/10

~2

Tab

le2.

2:Po

pu

lari

tyo

fOp

enSo

urc

eJa

vaW

ebA

pp

licat

ion

Fram

ewo

rks

(Nov

emb

er20

10)


with said API. Spring also offers the functionality to exchange the underlying MVC framework whichallows the developer to use Spring as a library and frameworks such as Struts or Webworks as theunderlying MVC framework21.

Spring 2.5 introduced an annotation-based programming model for MVC controllers that uses an-notations such as @RequestMapping, @RequestParam, @ModelAttribute and so on. Controllersimplemented in this style do not have to extend specific base classes or implement specific interfaces.Controllers map relative URLs to particular Java classes implementing the desired functionality bythe developer through the use of Java annotations. As shown in Listing 2.22, the @Controller and@RequestMapping annotations allow flexible method names and signatures. @Controller and@RequestMapping form the basis for the Spring MVC implementation.

1 @Controller

2 @RequestMapping(value="/account")

3 public final class AccountController {

4 private Map <Long , Account > accounts =

5 new ConcurrentHashMap <Long , Account >();

6

7 @RequestMapping(method = RequestMethod.GET)

8 public String getCreateForm(Model model) {

9 // ...

10 return "account/createForm";

11 }

12

13 @RequestMapping(method = RequestMethod.POST)

14 public String create(Account account , BindingResult result) {

15 // ...

16 return "redirect :/ account/" + account.getId ();

17 }

18

19 @RequestMapping(value="{id}", method=RequestMethod.GET)

20 public String getView(@PathVariable Long id, Model model) {

21 // ...

22 return "account/view";

23 }

24 }

Listing 2.22: Spring MVC Controller Example

Spring provides a couple of out of the box solutions for JSP and JSTL views. Using JSP or JSTL is doneusing a normal view resolver defined in the WebApplicationContext. See Section 2.1.3 for a JSTLexample.

2.6.1 Security Features

The Spring Framework does not provide any kind of security features, however this is realised by thesub project called Spring Security (see Section 2.4.2).

2.6.2 Extendibility

Spring’s handler mapping mechanism has a notion of handler interceptors, which can be extremelyuseful when specific functionality is needed for certain requests, for example, checking for aprincipal. Interceptors located in the handler mapping must implement HandlerInterceptor

21http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/mvc.html

[11/11/10]


http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/mvc.html

interface from the org.springframework.web.portlet package. The HandlerInterceptor

is basically a workflow interface that allows for customised handler execution chains. Applicationscan register any number of existing or custom interceptors for certain groups of handlers, to addcommon pre-processing behaviour without needing to modify each handler implementation. AHandlerInterceptor is basically similar to a Servlet filter, but in contrast to the latter it allowscustom pre-processing with the option to prohibit the execution of the handler itself and custompost-processing. Servlet filters are more powerful; for example they allow for exchanging the requestand response objects that are handed down the chain. Note that a Servlet filter gets configured inweb.xml, a HandlerInterceptor in the application context.

Just like the Servlet version, the HandlerInterceptor interface defines three methods: one thatwill be called before the actual handler will be executed (preHandle), one that will be called afterthe handler is executed (postHandle) and one that is called after the complete request has finished(afterCompletion). These three methods should provide enough flexibility to do all kinds of pre-and post- processing.

The preHandle method returns a Boolean value. A developer can use this method to breakor continue the processing of the execution chain. When this method returns true, the han-dler execution chain will continue. When it returns false, the DispatcherPortlet assumesthe interceptor itself has taken care of requests (and, for example, rendered an appropriate view)and does not continue executing the other interceptors and the actual handler in the execution chain.

The postHandle method is only called on a RenderRequest. The preHandle andafterCompletion methods are called on both an ActionRequest and a RenderRequest. Ifbusiness logic is needed in these methods for just one type of request, the developer has to checkwhat kind of request it is before processing it.

Listing 2.23 shows a simple customised HandlerInterceptor which breaks the workflow chain ifthe request has been received on port 80 on the application server.

1 public final class MyInterceptor implements HandlerInterceptor {

2

3 @Override

4 public boolean preHandleAction(ActionRequest request ,

5 ActionResponse response , Object handler) {

6 if (request.getServerPort () == 80) {

7 return false;

8 }

9 return true;

10 }

11 }

Listing 2.23: Spring HandlerInterceptor Example

2.7 Apache Struts 1.x

Apache Struts is an open source web application framework for developing Java EE web applications.It uses and extends the Java Servlet API to encourage developers to adopt a MVC architecture. Strutsprovides the controller (a Servlet known as ActionServlet) and facilitates the writing of templatesfor the view or presentation layer (typically in JSP, but XML/XSLT and Velocity are also supported).The web application programmer is responsible for writing the model code and for creating a central


configuration file struts-config.xml that binds together model, view and controller. Requestsfrom the client are sent to the controller in the form of Actions defined in the configuration file;if the controller receives such a request it calls the corresponding Action class that interacts withthe application-specific model code. The model code returns an ActionForward, a string telling thecontroller what output page to send to the client. Information is passed between model and view inthe form of special JavaBeans. A powerful custom tag library allows it to read and write the contentof these beans from the presentation layer without the need for any embedded Java code. Struts iscategorised as a request-based web application framework.

Listing 2.24 shows the struts-config.xml file for configuring a form bean and the Action map-pings. If a user enters the following URL into his web browser /submitCustomerForm.do then thecom.webapp.CustomerAction Java class will be handling his requests and the CustomerForm.jspview will be displayed. Listing 2.25 illustrates a common Struts Action which contains the businesslogic.

1 ...

2 <struts -config >

3 <form -beans >

4 <form -bean name=" CustomerForm" type="com.webapp.CustomerForm "/>

5 </form -beans >

6

7 <action -mappings >

8 <action

9 path ="/ submitCustomerForm"

10 type="com.webapp.CustomerAction"

11 name=" CustomerForm"

12 scope=" request"

13 validate ="true"

14 input ="/WEB -INF/jsp/CustomerForm.jsp">

15 <forward name=" SUCCESS" path ="/WEB -INF/jsp/confirm.jsp"/>

16 <forward name=" ERROR" path ="/WEB -INF/jsp/error.jsp"/>

17 </action >

18 </action -mappings >

19 </struts -config >

Listing 2.24: Struts 1.x struts-config.xml Example

1 public class CustomerAction extends Action {

2

3 @Override

4 public ActionForward execute(ActionMapping mapping , ActionForm form ,

5 HttpServletRequest req , HttpServletResponse resp)

6 throws Exception {

7 // ...

8 ActionForward nextPage = mapping.findForward("SUCCESS");

9

10 }

Listing 2.25: Struts 1.x Action example


Apache Struts offers application managed declarative as well as programmatic access control fea-tures. Container managed security is also available and explained in Section 2.2 in more detail.


Declarative Access Control

The Struts Framework has a role based access control mechanism for securing Actions declaratively.In struts-config.xml, each Action has a roles attribute, which accepts a comma-separated listof available roles as seen in Listing 2.26 on line 6.

1 ...

2 <struts -config >

3 <action -mappings >

4 <action path ="/ AdminLogin"

5 forward ="/ admin_login.jsp"

6 roles=" administrator "/>

7 ...

8 </action -mappings >

9 ...

10 </struts -config >

Listing 2.26: Apache Struts 1.x Declarative Access Control Example

This allows for a more natural approach to access control based on the named Action classes ratherthan URL patterns. One important thing to note is that this does not secure JSPs, HTML or othernon-Action resources. If all JSPs are only accessible through the controller (which is a Struts/MVCbest practice), a recommended countermeasure is to put all JSPs in the WEB-INF folder so they arenot directly available to users. However, this requires that the container stipulates that files in theWEB-INF folder are not accessible to web users. Otherwise, JSPs can be protected by standard securityconstraints in the configuration file.

Programmatic Access Control

The Struts Framework offers several different extension points where programmatic access controlscan be implemented. Specifically, the request processor has a processRoles() method that can beextended to do custom programmatic access control checks such as calling isUserInRole() for allrequests to the application as seen in Listing 2.27 on line 5.

1 ...

2 protected boolean processRoles(HttpServletRequest req ,

3 HttpServletResponse resp , ActionMapping mapping)

4 throws IOException , ServletException {

5 if (req.isUserInRole("administrator")) {

6 // ...

7 }

8 ...

Listing 2.27: Apache Struts Programmatic Access Control Example

Additionally, Struts has tag libraries to support programmatic "content-level" access controls in JSPpages. Using the following tag, content in JSP pages can be secured according to roles:

<logic:present role="administrator"/>

<a href="admin_page.jsp">Administrator Home</a>

</logic:present>

The RequestProcessor class is where most of the per-request processing takes place. There aremany different methods that can be overridden here, but some have very direct security relevance:

• processRoles(): This method should be overridden for any Action-level access control thatmust be programmatic (for instance, if the application does not use JAAS).


• processNoCache(): This method should be overridden to tag the response as "no-cache" toavoid security vulnerabilities associated with locally caching content.

• processException(): This method should do any special exception handling that can not bedone declaratively.

• processValidate(): This method should do any special validation that can not be donedeclaratively.

Obviously, more security concerns can be addressed by extending the RequestProcessor insteadof using declarative access control mechanisms in Struts 1.x. However, it is important to note thatany request that does not go through the Action/Controller Servlet (e.g., for a static HTML page, JSP,image/multimedia file, etc.) is not intercepted by the RequestProcessor and won’t be subject tothese controls (can be fixed by writing a Servlet filter).

2.7.2 Extendibility

Struts can be extended in three ways:

1. Plugin: Plugins can be used to execute business logic at startup or shutdown of the web appli-cation.

2. RequestProcessor: A RequestProcessor can be used to execute business logic at a particularpoint during the request processing phase. For example, to check a principal or other securityconstraints.

3. ActionServlet: An ActionServlet combines the RequestProcessor and the Plugin andshould only be used in cases where neither the Plugin nor the RequestProcessor is able tofulfil the desired functionality.

Plugin

According to the Struts documentation “A plugin is a configuration wrapper for a module-specificresource or service that needs to be notified about application startup and shutdown events”. Thisfor example might be useful to clean up and close any open database connections etc. To enablesuch a plugin it needs to be added to the struts-config.xml file with a <plug-in> tag. Listing2.28 illustrates a simple plugin with a init() and destroy() method22.

1 public final class HibernatePlugIn implements PlugIn {

2

3 // This method will be called at application startup time

4 public void init(ActionServlet actionServlet , ModuleConfig config)

5 // ...

6 }

7

8 // This method will be called at application shutdown time

9 public void destroy () {

10 // ...

11 }

12 ...

13 }

Listing 2.28: Apache Struts Plugin Example

22http://onjava.com/pub/a/onjava/2004/11/10/ExtendingStruts.html [07/04/11]


http://onjava.com/pub/a/onjava/2004/11/10/ExtendingStruts.html

RequestProcessor

ActionServlet is the only Servlet in the Struts framework and is responsible for handling all of therequests. Whenever it receives a request, it first tries to find a sub-application for the current request.Once a sub-application is found, it creates a RequestProcessor object for that sub-applicationand calls its process() method by passing it HttpServletRequest and HttpServletResponse

objects.

The RequestProcessor.process() is where most of the request processing takes place. Theprocess() method is implemented using the Template Method design pattern, in which there is aseparate method for performing each step of request processing and all of those methods are calledin sequence from the process() method. For example, there are separate methods for finding theActionForm class associated with the current request, and checking if the current user has one ofthe required roles to execute action mapping.

The RequestProcessor class in the Struts distribution provides a default implementation for eachof the request-processing steps. That means a developer can override only the methods that interesthim and use default implementations for the rest of the methods. For example, by default Strutscalls request.isUserInRole() to find out if the user has one of the roles required to executethe current ActionMapping, but if a database should be queried for this, then all one has to do isoverride the processRoles() method and return true or false, based whether the user has therequired role or not.

Listing 2.29 shows a simple example of a customised RequestProcessorwith the following businesslogic added:

• The ContactImageAction should generate images instead of regular HTML pages

• Before processing every request, the RequestProcessor should check whether the user islogged in by searching for the userName attribute stored in the session. If that attribute is notfound, the user will be redirected to the login page.

1 public final class CustomRequestProcessor extends RequestProcessor {

2

3 protected boolean processPreprocess (HttpServletRequest request ,

4 HttpServletResponse response) {

5

6 HttpSession session = request.getSession(false);

7

8 if (request.getServletPath ().equals("/loginInput.do")

9 || request.getServletPath ().equals("/login.do")) {

10 return true;

11 }

12

13 if( session != null && session.getAttribute("userName") != null) {

14 return true;

15 } else {

16 try {

17 request.getRequestDispatcher("/Login.jsp")

18 .forward(request ,response);

19 } catch(Exception ex) {

20 System.out.println(e.getMessage ());

21 }

22 }

23 return false;


24 }

25

26 protected void processContent(HttpServletRequest request ,

27 HttpServletResponse response) {

28 if (request.getServletPath ().equals("/contactimage.do")) {

29 response.setContentType("image/gif");

30 return;

31 }

32 super.processContent(request , response);

33 }

34 }

Listing 2.29: Apache Struts RequestProcessor Example

ActionServlet

ActionServlet extensions are being configured in the deployment descriptor (web.xml) as seen inListing 2.30. That means the custom ActionServlet is responsible for handling all of the requeststo Struts. It might be wise to extend the default ActionServlet of the Struts framework. Listing2.30 shows how an extended ActionServlet called CustomActionServlet is being properlyconfigured in the deployment descriptor.

1 <web -app >

2 <servlet >

3 <servlet -name >action </servlet -name >

4 <servlet -class >com.webapp.CustomActionServlet </servlet -class >

5 </servlet >


7 <servlet -name >action </servlet -name >

8 <url -pattern >*.do </url -pattern >


10 </web -app >

Listing 2.30: Apache Struts ActionServlet Configuration Example

2.8 Apache Struts 2.x

The following statement has been taken from the Apache Struts project site23:

Apache Struts 2 is an elegant, extensible framework for creating enterprise-ready Javaweb applications. The framework is designed to streamline the full development cycle,from building, to deploying, to maintaining applications over time. Apache Struts 2 wasoriginally known as WebWork 2. After working independently for several years, the Web-Work and Struts communities joined forces to create Struts2. This new version of Strutsis simpler to use and closer to how Struts was always meant to be.

Struts 2 offers a new feature which allows the developer to declaratively configure the usual StrutsAction results through Java annotations. As this is a new feature, and still being field-tested, theStruts documentation is a bit sketchy. By using these annotations no additional XML configurationis needed. Listing 2.31 shows an example Struts Action, annotated with the following annotations:@Namespace and @Result. When the execute() method returns "SUCCESS" then the user will beredirected to the listPeople.action page.

23http://struts.apache.org/2.2.1/index.html [10/11/10]


http://struts.apache.org/2.2.1/index.html

1 @Namespace("/User")

2 @Results ({

3 @Result(name="SUCCESS", value="/listPeople.action",

4 type = ServletRedirectResult.class),

5 @Result(name="ERROR", value="/userError.action",

6 type = ServletRedirectResult.class)

7 })

8 public class EditUserAction extends ActionSupport {

9 @Override

10 public String execute () {

11 // ...

12 return "SUCCESS";

13 }

14 }

Listing 2.31: Struts 2 Annotation Example

As of now, not all settings which can be set and configured through the standard Struts XMLconfiguration files are implemented and supported through annotations. Listing 2.32 shows thestruts.xml configuration file needed to set up the Action mappings and Listing 2.33 shows theactual code of the Action which contains the same configuration as seen in Listing 2.31 but withoutusing Java annotations.

1 ...

2 <struts >

3 <package name="basic" extends ="struts -default">

4 <action name="index">

5 <result >/index.jsp </result >

6 </action >

7

8 <action name="User" class="com.webapp.EditUserAction" method =" execute">

9 <result name=" SUCCESS">/ listPeople.action </result >

10 <result name="ERROR">/ userError.action </result >

11 </action >

12 </package >

13 </struts >

Listing 2.32: Struts 2.x struts.xml Example

1 public final class EditUserAction extends ActionSupport {

2 @Override

3 public String execute () throws Exception {

4 // ...

5 return "SUCCESS";

6 }

7 }

Listing 2.33: Struts 2.x Action Example


The Struts 2 framework offers a Interceptor called RolesInterceptor which ensures that certainActions will only be executed if the user has the correct role. Interceptors can execute code beforeand after an Action is invoked. Most of the framework’s core functionality is implemented asInterceptors. Features like double-submit guards, type conversion, object population, validation, fileupload, page preparation, and more, are all implemented with the help of Interceptors. Each andevery Interceptor is pluggable, so the developer can decide exactly which features an Action needsto support. Interceptors can be configured on a per-Action basis. Interceptors “set the stage” for the


Action classes, doing much of the “heavy lifting” before the Action executes.

The RolesInterceptor has two parameters:

• allowedRoles a comma-separated list of roles to allow

• disallowedRoles a comma-separated list of roles to disallow

Configuration of the RolesInterceptor is done through the struts.xml file by adding a<interceptor-ref> element to the Action mapping as shown in Listing 2.34 on lines 3-6.

1 ...

2 <action name="User" class="com.webapp.EditUserAction" method =" execute">

3 <interceptor -ref name=" completeStack "/>

4 <interceptor -ref name=" roles">

5 <param name=" allowedRoles">admin ,member </param >

6 </interceptor -ref >

7 <result name=" SUCCESS">/ listPeople.action </result >

8 <result name="ERROR">/ userError.action </result >

9 </action >

Listing 2.34: Struts 2.x RolesInterceptor Configuration Example

2.8.2 Extendibility

Apache Struts 2 provides a simple plugin architecture so that developers can extend the frameworkjust by adding a JAR to the application’s classpath (in addition to whatever JARs may be required tofulfil the dependencies of the plugin itself). At load time, the framework looks for JARs containing astruts-plugin.xml file at the root of the archive. To create a plugin, all a developer needs to do isbuild a JAR and put the expected struts-plugin.xml at the root. Later on these plugin classes canbe instantiated (for example inside a Struts Action) using the @Inject annotation provided by theframework. Since a plugin contains the struts-plugin.xml file, it has the ability to:

• Define new packages with results, interceptors and/or actions

• Override framework constants

• Introduce new extension point implementation classes

Extension points allows a plugin to override a key class in the Struts framework with an alternateimplementation. For example, a plugin could provide a new class to create Action classes or maprequests to Actions. For a list of all the available extension points go to http://struts.apache.

org/2.0.14/docs/plugins.html.

2.9 Apache MyFaces Tomahawk

The following statement has been taken from the Apache MyFaces project site24:

Apache MyFaces is a project of the Apache Software Foundation and hosts several sub-projects relating to the JavaServer technology. The Apache MyFaces project provides: aJavaServer Faces implementation, several component libraries containing UI widgets forbuilding web applications with JSF (e.g. MyFaces Tomahawk, MyFaces Trinidad, MyFacesTobago), extension packages to JavaServer Faces (e.g. MyFaces Orchestra, MyFaces Ex-tensions Validator)

24http://myfaces.apache.org/ [10/11/10]


http://struts.apache.org/2.0.14/docs/plugins.html

http://struts.apache.org/2.0.14/docs/plugins.html

http://myfaces.apache.org/

A Java Server Faces example can be found in Section 2.1.4.


Java Server Faces applications respectively MyFaces applications can be secured by using containermanaged security features as seen in Section 2.2. Additionally, application managed security can beused by using appropriate libraries as seen in Section 2.4 or by implementing Servlet filters whichenforce certain security constraints. Such Servlet filters are not provided out of the box by MyFacesand have to be implemented by the developer of the web application.

Java Server Faces classes have the ability to access the user context by using the following program-matic security approach:

FacesContext context = FacesContext.getCurrentInstance();

if (context.getExternalContext().isUserInRole(’’admin’’)) {

// ...

}

It might be worthwhile to mention the ExtVal25 project which is an extension to the MyFaces project.MyFaces Extensions Validator (ExtVal) is a JSF centric validation framework. It offers powerful andeasy to use field based validation based on annotations. It has a pluggable architecture, so it ispossible to add new validation modules or support for component libraries like Trinidad or otherthird party frameworks. ExtVal is compatible with JSF 1.x and JSF 2.x. After setting up ExtVal, thebasic usage is very simple. Listing 2.35 shows how to use ExtVal in a concrete example. The @Columnannotation maps the lastName property to the LAST_NAME column in the database. It also showssome information that is derived from the table definition in the database. nullable = false

means the database will not accept an empty value in this field and length = 30 means that nomore than 30 characters can be stored in the corresponding database column. This informationcould be used for validation in the view layer (see [19, p. 321]).

1 @Pattern(value = "[A-Za-z]*")

2 @Length(minimum = 30)

3 @Column(name = "LAST_NAME", nullable = false , length = 30)

4 private String lastName;

Listing 2.35: Struts ExtVal usage example

2.9.2 Extendibility

MyFaces processes all requests in 6 phases along a defined lifecycle. With a custom lifecycle class it ispossible to extend the standard lifecycle and add additional checks and business logic. The MyFacesdocumentation does not mention any other extension points which can be used by a developer toadd additional features to the framework in a pluggable fashion. This approach is similar to theextendibility feature of Struts 1.x described in Section 2.7.

In Listing 2.36 the CustomLifecycleFactory class uses the delegating factory constructor toinstantiate the factory and get a reference of the standard LifecyleFactory from the MyFaces run-time. In the factory constructor an instance of the default lifecycle is passed to the MyLifecycle classand an instance of the custom lifecycle is being registered within the standard LifecycleFactory.

25http://myfaces.apache.org/extensions/validator/ [25/11/10]


http://myfaces.apache.org/extensions/validator/

1 public final class CustomLifecycleFactory extends LifecycleFactory {

2

3 public CustomLifecycleFactory(LifecycleFactory defaultFactory) {

4 delegate = defaultFactory;

5

6 // 1. get default lifecycle to be wrapped by our lifecycle

7 Lifecycle defaultLifecycle =

8 delegate.getLifecycle(LifecycleFactory.DEFAULT_LIFECYCLE);

9

10 // 2. register intercepting lifecycle implementation.

11 addLifecycle("MyLifecycle", new MyLifecycle(defaultLifecycle));

12 }

13 ...

14 }

Listing 2.36: MyFaces Custom Lifecycle Factory Example

Configuration of the CustomLifecycleFactory is done through an entry in thefaces-config.xml file as seen in the following code snippet:

<factory>

<lifecycle-factory>com.webapp.CustomLifecycleFactory</lifecycle-factory>

</factory>

Listing 2.37 shows how it is possible to enhance the standard MyFaces lifecycle with additionalbusiness logic and checks by overriding the appropriate methods but in the same time it is pos-sible to use the standard MyFaces lifecycle by using the private variable defaultLifecycle. AFacesContext object contains all of the per-request state information related to the processing of asingle JavaServer Faces request and the rendering of the corresponding response. It is passed to, andpotentially modified by, each phase of the request processing lifecycle.

1 public final class MyLifecycle extends Lifecycle {

2

3 private Lifecycle defaultLifecycle;

4

5 public MyLifecycle(Lifecycle defaultLifecycle) {

6 this.defaultLifecycle = defaultLifecycle;

7 }

8

9 @Override

10 public void addPhaseListener(PhaseListener listener) {

11 // ...

12 }

13

14 @Override

15 public void execute(FacesContext context) throws FacesException {

16 // ...

17 }

18

19 @Override

20 public void render(FacesContext context) throws FacesException {

21 // ...

22 }

23 ...

24 }

Listing 2.37: MyFaces Custom Lifecycle Example


2.10 Wicket

Apache Wicket, commonly referred to as Wicket, is a lightweight component-based web applicationframework for the Java programming language conceptually similar to JavaServer Faces and Tapestry.Wicket is closely patterned after stateful GUI frameworks such as Swing (event driven). Wicketapplications are trees of components, which use listener delegates to react to HTTP requests againstlinks and forms in the same way that Swing components react to mouse and keystroke events. Wicketis categorised as a component-based framework and Wicket bridges the impedance mismatchbetween the stateless HTTP and stateful server-side programming in Java.

To illustrate this new approach of an event driven web application the following example creates asimple Wicket web application with one page. This page contains a link to increment a counter anduses a label to display the number of clicks. Listing 2.38 shows the logic for the link counter whichis achieved by overriding the onClick() method of the link object. Listing 2.39 shows the HTMLlayout which is being used to display the counter to the user. Note that this is pure HTML and no JSPetc. is needed, except that the wicket:id has to be provided for displaying dynamic content (see [9]).

1 public final class HomePage extends WebPage {

2 public int counter = 0;

3

4 public HomePage(final PageParameters parameters) {

5 add(new Link("link") {

6 @Override

7 public void onClick () {

8 counter ++;

9 }

10 });

11

12 add(new Label("label", new PropertyModel(this , "counter")));

13 }

14 }

Listing 2.38: Simple Wicket Page with link Counter

1 <html >

2 <body >

3 <a href ="#" wicket:id="link">This link </a>

4 has been clicked <span wicket:id="label ">123</span > times.

5 </body >

6 </html >

Listing 2.39: Wicket Page Layout for link Counter


Wicket has been developed with security in mind, furthermore Wicket-Security26 or the Wicket-Auth-Roles27 project can be used to add access controls constraints to a Wicket web application.Wicket URLs do not expose sensitive information and all component paths are session relative.Explicit steps must be taken to share information between sessions. This means an attacker wouldhave to hijack the session and then guess the right page identifiers and version numbers, whichwould be relative to the session and the relevant component paths. Additionally, URL encryption canbe enabled with four lines of code to support highly secure web sites. Of course container managed

26http://wicketstuff.org/confluence/display/STUFFWIKI/Wicket-Security [22/11/10]27http://wicket.apache.org/learn/projects/authroles.html [24/11/10]


http://wicketstuff.org/confluence/display/STUFFWIKI/Wicket-Security

http://wicket.apache.org/learn/projects/authroles.html

security can also be applied to Wicket web applications as seen in Section 2.2.

Wicket-Security is an attempt to create an out of the box reusable authenticating and authorisationframework for Wicket28. It contains several projects which can be used stand alone or in conjunctionwith each other. Wicket-Security uses the available security mechanisms from Wicket to build uponand expand the authentication and authorisation mechanisms. In order to maximize flexibilityand customisation a new API, called Wicket Abstract Security Platform (WASP), was built on top ofWicket. WASP builds on the IAuthorisationStrategies provided by Wicket and allows individualcomponents or models to do their own authentication and/or authorisation checks. Like Wicketit uses actions to grant permissions, however where Wicket uses string based actions (render andenable) WASP uses full blown Java objects. Besides an API to implement individual authenticationand authorisation routines, Wicket-Security also contains a default implementation, called Swarm.

Wicket is only aware of the render and enable actions. They are used for making components in-visible (render), read only and clickable (enable).

• access is used to check if a component is allowed to be created.

• render is used to check if a component is allowed to be rendered.

• enable is used to check if a component is allowed to be enabled, for links this means: can theuser click on them, for components that take user input (like a textfield) this means: can theuser write to them.

• inherit is for container components and specifies that all the actions for this container areinherited by its children, meaning a developer does not need to specify permissions for eachcomponent individually.

Swarm adds the access action for component instantiation and the inherit action for permissioninheritance. Actions can imply other actions by default all actions imply the access action and theenable action implies the render action. This means if a developer specifies the enable actionwhat one really get is access, render and enable action. This also means that if permissionswithout any explicit actions is granted, one will automatically get the access action.

To make things more clear a sample standard.hive configuration file is shown in Listing 2.40.

1 grant {

2 // actually the login and HomePage are not SecureWebPage classes.

3 permission ${ComponentPermission} "com.mycompany.webapp.LoginPage",

4 "inherit , render , enable ";

5 permission ${ComponentPermission} "com.mycompany.webapp.HomePage",


7 };

8

9 grant principal com.mycompany.webapp.SpringSecurePrincipal "ROLE_USER" {

10 permission ${ComponentPermission} "com.mycompany.webapp.FirstSecurePage",


12 };

13

14 grant principal com.mycompany.webapp.SpringSecurePrincipal "ROLE_ADMIN" {

15 permission ${ComponentPermission} "com.mycompany.webapp.SecondSecurePage",


28http://wicket.apache.org/ [22/11/10]


http://wicket.apache.org/

17 permission ${ComponentPermission} "com.mycompany.webapp.FirstSecurePage:

ajaxLink",

18 "render , enable ";

19 };

Listing 2.40: Wicket-Sercurity Configuration Example

Secure pages are pages requiring an authenticated user with certain privileges. In Swarm Se-cure pages are achieved by having the secure pages either extend SecureWebPage or implementISecurePage.

Using the configuration in Listing 2.40 would grant everyone the right to see (render) the HomePage,if there are secure components on the home page one would see them too (inherit). In additionlinks to the HomePage are granted the right to be clicked (enable).

Another approach is to use Wicket-Auth-Roles for securing a Wicket web application. Wicket-Auth-Roles is included in the standard Wicket release. In order to use Auth-Roles in a Wicket web applica-tion the initial base class needs to extend AuthenticatedWebApplication as seen in the followingcode snippet:

public class ExampleWicketApplication extends AuthenticatedWebApplication {

// ...

}

From now on, web pages can be annotated with the @AuthorizeInstantiation annotation as seenin the next code snippet:

// only users with role ROLE_ADMIN are allowed

// to create instances of this page

@AuthorizeInstantiation("ROLE_ADMIN")

public class AdminAnnotationsBookmarkablePage extends WebPage {

// ...

}

When someone who does not have the role ROLE_ADMIN, Wicket will not allow the page to be fullyconstructed and throws an authorisation exception during the construction of the page. This willresult in an access denied page for the user. Wicket-Auth-Roles might not be as sophisticated asWicket-Security but it is much more intuitive to use.

2.10.2 Extendibility

The source code of the Wicket framework is freely available and thus extensible. Wicket is licensedunder the Apache License, Version 2.0 which can be downloaded from http://www.apache.org/

licenses/LICENSE-2.0. Sadly the Wicket frameworks lacks documentation of its internal archi-tecture. This circumstance makes it slightly harder to implement new features directly into the coreof the framework before knowing its internal structure and workings but nonetheless possible.

2.11 Summary

This Chapter investigated the technologies involved in creating Java web applications, availablesecurity frameworks for securing Java web or EE applications and popular Java web applicationframeworks including their security features and extendibility. No matter which web application


http://www.apache.org/licenses/LICENSE-2.0

http://www.apache.org/licenses/LICENSE-2.0

framework is being used to write a web application, container managed security as described inSection 2.2 is available and can be used to add security constraints to the URLs provided by theweb application. These constraints can either be configured through the deployment descriptorof the web application called web.xml or through annotations if simple Java Servlets (Version 3.0)are being used. Usernames, passwords and roles have to be made available to the web containerwhich enforces these constraints. Additionally to container managed security, all web applicationframeworks except for the Apache MyFaces project offer framework specific access controls. Thesecan be either configured through XML configuration files or by adding framework specific Javaannotations to the source code. Another possibility is to ignore framework specific access controlsand use a security library as seen in Section 2.4 for enforcing access controls.

Interesting to note is that basically all of the described web application frameworks do not offer anysophisticated input validation routines. Common validation such as checking for numeric valuesonly, valid phone numbers etc. is mostly available, but none enforces checks against Code InjectionAttacks although this can be solved by using security libraries such as ESAPI.

All of the available security frameworks are implemented using Servlet filters which also have tobe configured through the deployment descriptor of the web application on a per-URL basis. Thismakes them loosely coupled and the web application can be written with any technology or frame-work the developer desires. Interesting is the ESAPI project which offers a complete web applicationfirewall in form of a Servlet filter. Also very astonishing is the ability to use a scripting language (TheSpring Expression Language) inside of annotated security constraints offered by the Spring Securityproject. This allows the creation of sophisticated constraints which incorporate method parametersfrom the annotated classes.

Wicket is the first web application framework which does not use a typical MVC paradigm, instead itbehaves more like a Java stand alone application where actions respectively business logic is beingtriggered through events from the user. Especially nice is that views in Wicket consist of pure HTMLand Cascading Style Sheets (CSS) without polluting it with programming semantics like JSP does.

Table 2.4 compares the security features of the various Java web application frameworks discussed inthis Chapter. Most frameworks offer the same functionality as their counterparts with the exceptionof MyFaces. Basically all support declarative- and programmatic security features. Annotation basedsecurity features are only available in Spring and Wicket. Noteworthy is the fact that it is possibleto use Spring Security in conjunction with other frameworks, this means it is possible to use Strutstogether with Spring Security (see [30, p. 187]).


Spri

ng

Fram

ewo

rk(+

Spri

ng

Secu

rity

)

Ap

ach

eSt

ruts

1.x

Ap

ach

eSt

ruts

2.x

MyF

aces

Wic

ket

Co

nta

iner

Man

aged

Secu

rity

XX

XX

XD

ecla

rati

veSe

curi

tyX

XX

-X

Secu

rity

An

no

tati

on

sX

--

-X

Pro

gram

mat

icSe

curi

tyX

XX

XX

Secu

rity

Feat

ure

sR

BA

C,

Inp

utV

alid

atio

nR

BA

C,

Inp

utV

alid

atio

nR

BA

C,

Inp

utV

alid

atio

nR

BA

C,

Inp

utV

alid

atio

nR

BA

C,

UR

Len

cryp

tio

n,

Inp

utV

alid

atio

n

Tab

le2.

4:Ja

vaW

ebA

pp

licat

ion

Fram

ewo

rks

Secu

rity

Feat

ure

sC

om

par

iso

n


3 Related Work

The following Chapter presents the current state of the research literature related to adding declara-tive information to web application source code to improve security. The papers summarised havebeen found through the Google Scholar search engine1. The available research on enhancing web ap-plication security declaratively can be roughly divided into three categories: added security throughdeclarative security policies, adding an aspect oriented approach to web application security andstatic code analysis with the aid of metadata.

3.1 Declarative Security Policies and Constraints

Some of the declarative security features Java Standard Edition/EE offers have already been discussedin Section 2.2 and 2.3. These include container managed security and standard Java EE features.Additional declarative security features are being provided by Java security frameworks such asSpring Security described in Section 2.4 and 2.4.2.

David Scott et al. [31] introduced in their paper Abstracting Application-Level Web Security a secu-rity policy description language (SPDL) which is used to specify a set of validation constraints andtransformation rules for web applications. The proposed system consists of the SDPL which is usedto program an application level firewall (in the paper referred to as a security gateway). The securitypolicies are written in SDPL and compiled for execution on the security gateway. The security gate-way dynamically analyses and transforms HTTP request/responses to enforce the specified policies.The motivation behind this proposed system is the lack of language-level support in popular un-typed scripting languages to prevent application-level security vulnerabilities. The programmer hasto manually verify that all user input is appropriately HTML-encoded. An example of such a securitypolicy in SPDL is shown in Listing 3.1. A policy element contains a series URL and cookie elements.For each URL element a number of parameters are declared. The attributes of a parameter elementwith name = p place constraints on data passed via p:

• The maxlength and minlength attributes specify the maximum and minimum length of datapassed via p.

• Setting required to "Y" specifies that p must always contain a (non-zero length) value.

• Setting MAC to "Y" specifies that the value of p must be accompanied by a Message Authentica-tion Code (MAC) generated by the server. This prevents the user from changing the value of theparameter to arbitrary values.

• The type attribute specifies the data-type of p (either int, float, bool or string).

The method attribute determines whether the specified constraints apply to p passed as a GET-parameter or a POST-parameter. Setting method to GETandPOST means that the constraints withinthe parameter element are applicable to both GET and POST parameters with the name = p.

1http://scholar.google.com [14/04/11]

Chapter 3. Related Work | 46

http://scholar.google.com

1 <policy >

2 <URL prefix ="http :// example">

3 <parameter name="p1" maxlength ="4" type="int" required ="Y" MAC="Y"/>

4 <parameter name="p2" maxlength ="3" type=" string" method ="POST">

5 <validation >

6 format(this , "\d\d/\d\d") and

7 Int.fromString(mid(s,1,2)) <= 12 and

8 Int.fromString(mid(s,1,2)) >= 0

9 </validation >

10 </parameter >

11 </URL >

12 </policy >

Listing 3.1: Abstracting Application-Level Web Security with a Policy Description Language Example

The example in Listing 3.1 specifies constraints on parameters passed to URLs with the prefix"http://example". The first parameter element defines constraints to be applied to a parameternamed p1 (either GET or POST); the second parameter element defines constraints to be appliedto a POST parameter named p2. In some circumstances a greater degree of control is required; thisis provided by the validation element. The validation element allows complex constraints tobe encoded in a general purpose validation language. A well-formed validation expression has typeBoolean. If the validation expression of parameter p, evaluates to true then this signifies that pcontains valid data. Within validation expressions, the value of the field specified in the enclosingparameter element is referred to as this. Functions are provided to cast between different types.For example, String.fromInt(i) returns the string representation of integer i.

The main advantage of this approach is the separation of the security policy from the programmerand having a centralised security policy. The proposed approach is very similar to web applicationfirewalls such as ModSecurity2 (open source web application firewall for the Apache web server), thedifference is that the system of David Scott et al. pursues a “programmatic approach” of specifyingsecurity policies because of the SPDL compiler.

Myers [27] describes in his paper a promising technique for protecting privacy and integrity of sensi-tive data by statically checking information flow within programs that manipulate the data. He arguesthat programming language extensions to allow this static checking are too restrictive for practical useand have not been implemented. He describes a new language called JFlow, which is an extensionto the Java language that adds statically-checked information flow annotations. In JFlow, every Javavalue has a labelled type that consists of two parts: an ordinary Java type such as int and a labelthat describes the ways that the value can propagate. For example the expression int{customer:}

val; represents an integer that principal customer owns and only customer can read (the owner ofa policy is always implicitly a reader). A component expression of the form owner: reader1 , reader2, . . .denotes a policy. A label expression is a series of component expressions, separated by semicolons,such as {o1 : r1, r2;o2 : r2, r3}. The default label of an instance variable is the public label {}. JFlowoffers a few optional annotations to manage information flow and authority delegation:

• authority(p1, ..., pn ) - This clause lists principals that the method is authorised to act for.

• caller(p1, ..., pn ) - Calling code may also dynamically grant authority to a method that has acaller constraint. A method with a caller clause may be called only if the calling code possessesthe requisite static authority.

• actsFor(p1, p2) - An actsFor constraint may be used to prevent the method from being calledunless the specified acts-for relationship (p1 acts for p2 ) holds at the call site.

2http://www.modsecurity.org/ [1/12/10]


http://www.modsecurity.org/

Listing 3.2 contains a JFlow implementation of a simple password file, in which the passwords areprotected by information flow controls. Only the method for checking passwords is shown. Thismethod, check, accepts a password, a username and returns a Boolean indicating whether the stringis the right password for that user. The if statement is conditional on the elements of passwordsand on the variables user and password, whose labels are implicit parameters. Therefore, the bodyof the if statement has policy = {user; password; root:} and the variable match also musthave this label in order to allow the assignment match = true. This label prevents match frombeing returned directly as a result, since the label of the return value is the default label, {user;password}. Finally, the method declassifies match to this desired label, using its compiled-inauthority to act for root.

JFlow supports many language features that have not been previously integrated with static flowchecking, including mutable objects (which subsume function values), subclassing, dynamic typetests, dynamic access control and exceptions. JFlow is not completely a superset of Java. Certainfeatures have been omitted to make information flow control tractable. Also, JFlow does not elim-inate all possible information leaks. Certain covert channels (particularly various kinds of timingchannels) are difficult to eliminate.

1 public class passwordFile authority(root) {

2 private String [] names;

3 private String{root :}[] passwords;

4

5 public boolean check(String user , String password) {

6 where authority(root) {

7 boolean match = false;

8 try {

9 for (int i = 0; i < names.length; i++) {

10 if (names[i]. equals(user) && passwords[i]. equals(password)) {

11 match = break;

12 }

13 }

14 } catch (Exception e) {

15 // ...

16 }

17 }

18 }

19 }

Listing 3.2: JFlow Password File Example

Another project with the goal of static information flow analysis is called Jif3. Jif is a security-typedprogramming language that extends Java with support for information flow control and accesscontrol, enforced at both compile time and run time. Jif is written in Java and is built using thePolyglot extensible Java compiler framework. Static information flow control can protect the confi-dentiality and integrity of information manipulated by computing systems. The compiler tracks thecorrespondence between information the policies that restrict its use, enforcing security propertiesend-to-end within the system. After checking information flow within Jif programs, the Jif compilertranslates them to Java programs and uses an ordinary Java compiler to produce secure executableprograms. Jif extends Java by adding labels that express restrictions on how information may beused. For example, the following variable declaration declares not only that the variable x is an int,but also that the information in x is governed by a security policy:

3http://www.cs.cornell.edu/jif/ [2/12/10]


http://www.cs.cornell.edu/jif/

int {Alice→Bob} x;

In this case, the security policy says that the information in x is controlled by the principal Alice,and that Alice permits this information to be seen by the principal Bob. The policy {Alice←Bob}means that information is owned by Alice and that Alice permits it to be affected by Bob. Basedon label annotations like these, the Jif compiler analyses information flows within programs, todetermines whether they enforce the confidentiality and integrity of information. Considering thefollowing variable declaration and assignments:

int {Alice→Bob, Chuck} y;

x = y; // OK: policy on x is stronger

y = x; // BAD: policy on y is not as strong as x

The variable y is declared to be an int and Alice permits the information in y to be seen by bothBob and Chuck. The first assignment is secure because it does not make the information in y visibleto any additional principals. The second assignment is insecure because now Chuck can see theinformation that was in x and the policy on x forbids this (unless Alice or Bob happen to trustChuck). The Jif compiler makes these determinations at compile time, as part of type checking.If a Jif program type-checks, the compiler translates it into Java code that can be compiled with astandard Java compiler. The program can then be executed with a standard Java virtual machine.Although enforcement is mostly done at compile time, Jif does also allow some enforcement to takeplace at run time. Therefore, Jif programs in general require the Jif run-time library.

A higher level of abstraction for declaring access control policies, regardless of the underlying pro-gramming language, is achieved by using the Extensible Access Control Markup Language (XACML)from the OASIS standards organisation4. There are quite a few Java libraries/frameworks for parsingand enforcing XACML policy files such as: Sun’s SunXACML5 project and PicketBox XACML (formerlyknown as JBossXACML)6. XACML is a declarative access control policy language implemented inXML and a processing model, describing how to interpret the policies. XACML is based on theAttribute Based Access Control (ABAC) model that is hailed as the next generation of access controlmodels7. Instead of only using a role as the determining factor whether to grant access or not, manyattributes can be used. Roles can be used in ABAC as well - since ABAC can use multiple attributes tomake access control decisions, so ABAC can emulate RBAC perfectly while adding many additionaladvantages. This means that it is possible to add context to the access control decisions and addsfor a finer granularity, tighter controls and more flexibility for the business. With XACML, policiescan be woven around a combination of subjects (users/user-agents etc), resources (on which theaccess control is desired) and environment (IP address, Date, Time etc). Now a developer is ableto declaratively (via XML or construct policies) say things like “Allow this portion of the web sitevisible to 18 year olds when the time is between 9am and 5pm”, “You should update your own payrollinformation and can do it when you are employed and on Fridays only” etc. Enterprises have beendoing this via ACLs and other proprietary mechanisms. Now they can use a standard way.

A basic XACML architecture includes a Policy Enforcement Point (PEP) which intercepts any requestsin a normal program flow, then asks a Policy Decision Point (PDP) to make an access decision basedon the policies associated with the PDP. The PDP evaluates the XACML request created by the PEP

4http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml [2/12/10]5http://sunxacml.sourceforge.net/ [3/12/10]6http://community.jboss.org/wiki/PicketBoxXACMLJBossXACML [3/12/10]7http://blogs.kuppingercole.com/gaehtgens/2009/07/31/finally-an-open-xacml-api/ [2/12/10]


http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml

http://sunxacml.sourceforge.net/

http://community.jboss.org/wiki/PicketBoxXACMLJBossXACML

http://blogs.kuppingercole.com/gaehtgens/2009/07/31/finally-an-open-xacml-api/

and runs through the policies to make one of the four access decisions, namely:

• PERMIT - approve request

• DENY - access denied

• INDETERMINATE - error at the PDP

• NOTAPPLICABLE - some attribute missing in the request or no policy match

The main job of a PEP is to intercept requests to any resources and then create an XACML requestbased on attributes about the Subject (actor accessing the resource), Action, Attributes about theresource and environment (Time, Date, IP address etc.) and pass it to a PDP for access decision.Based on the PDP decision, the PEP can either allow access to the resource or not.

3.2 Aspect-Oriented Security

Aspect-oriented programming (AOP) is programming paradigm which entails breaking down pro-gram logic into distinct parts (so called concerns, cohesive areas of functionality). AOP isolatessecondary or supporting functions from the main program’s business logic. It aims to increasemodularity by allowing the separation of cross-cutting concerns, forming a basis for aspect-orientedsoftware development.

Logging as well as security constraints exemplifies a cross-cutting concern because a logging strategynecessarily affects every single logged part of the system. Logging thereby cross-cuts all loggedclasses and methods. The following example code taken from [20] shall illustrate the possibilities ofAOP programming. The example code has been tested using the AspectJ8 project which is an opensource AOP implementation for the Java programming language.

The MessageCommunicator class in Listing 3.3 has two methods: one to deliver a general messageand the other to deliver a message to a specific person. Listing 3.4 shows a simple class to exercisethe functionality of the MessageCommunicator class.

1 public class MessageCommunicator {

2 public void deliver(String message) {

3 System.out.println(message);

4 }

5

6 public void deliver(String person , String message) {

7 System.out.println(person + ", " + message);

8 }

9 }

Listing 3.3: Class encapsulating the Message-Delivery Functionality

1 public class Main {

2 public static void main(String [] args) {

3 MessageCommunicator messageCommunicator = new MessageCommunicator ();

4 messageCommunicator.deliver("Wanna learn AspectJ?");

5 messageCommunicator.deliver("Harry", "having fun?");

6 }

7 }

Listing 3.4: Class to exercise the Message-Delivery Functionality

8http://www.eclipse.org/aspectj/[29/11/10]


http://www.eclipse.org/aspectj/

Consider authentication functionality: before delivering a message, the class should check whetherthe user has been authenticated. Without using AOP, the code would probably look like the one inListing 3.5.

1 public class MessageCommunicator {

2 private Authenticator authenticator = new Authenticator ();

3

4 public void deliver(String message) {

5 authenticator.authenticate ();


7 }

8

9 public void deliver(String person , String message) {


11 System.out.println(person + ", " + message);

12 }

13 }

Listing 3.5: Class encapsulating the Message-Delivery Functionality (with Authentication)

In order to verify if the user is properly authenticated, every method needs to call authenticate()which leads to code tangling. Similar code would have to be present in all classes that require theauthentication functionality - leading to code scattering. By using an AOP approach the code fromListing 3.3 can be enhanced with the authentication functionality without changing a single line ofcode by adding an aspect to the system.

The Authenticator class asks for credentials (username and password) when the authenticate()method is called for the first time in a thread. Upon successful authentication, it stores the user ina thread local so it does not ask for credentials in the same thread again. Upon failure, it throws aruntime exception. Not shown for brevity’s sake.

1 public aspect SecurityAspect {

2 private Authenticator authenticator = new Authenticator ();

3

4 pointcut secureAccess (): execution (* MessageCommunicator.deliver (..));

5

6 before () : secureAccess () {

7 System.out.println("Checking and authenticating user");


9 }

10 }

Listing 3.6: Authentication Aspect

Compiling the classes along with the aspect is done by running the ajc command. The followingsnippet illustrates the usage and the output of the freshly compiled program:

# ajc -source 5 MessageCommunicator.java Main.java SecurityAspect.aj

# java Main

Checking and authenticating user

Wanna learn AspectJ?

Checking and authenticating user

Harry, having fun?

The SecurityAspect.aj file declares the SecurityAspect aspect. An aspect is a unit of modu-larisation in AOP, much like a class is a unit of modularisation in OOP. The declaration of an aspect


is similar to a class declaration. A pointcut selects interesting points of execution in a system,called join points. The aspect in Listing 3.6 defines a pointcut secureAccess() that selects exe-cution of all the methods named deliver() in the MessageCommunicator class. The * indicatesthat the pointcut matches any return type and the .. inside parentheses after deliver specifiesthat it matches regardless of the number of arguments or their types. In this example, the point-cut selects execution of both overloaded versions of deliver() in the MessageCommunicator class.

An advice defines the code to execute upon reaching join points selected by the associated pointcut.Listing 3.6 defines a piece of advice to execute before reaching the join points selected by thesecureAccess() pointcut. The before() part indicates that the advice should run prior to theexecution of the advised join point - in this case, prior to executing any MessageCommunicator.

deliver() method. In the advice, the user gets authenticated. With the aspect now present in thesystem, each time MessageCommunicator.deliver() is executed, the advice code performs theauthentication logic before the method.

AspectJ offers the following pointcuts:

• execution(void Point.setX(int)) - when a particular method body executes

• call(void Point.setX(int)) - when a method is called

• handler(ArrayOutOfBoundsException) - when an exception handler executes

• this(SomeType) - when the object currently executing (i.e. this) is of type SomeType

• target(SomeType) - when the target object is of type SomeType

• within(MyClass) - when the executing code belongs to class MyClass

• cflow(call(void Test.main())) - when the join point is in the control flow of a call to aTest’s no-argument main method

Pointcuts compose through the operations or ("||") and ("&&") and not ("!"). It is possible to usewildcards:

1. execution(* *(..))

2. call(* set(..))

(1) means the execution of any method regardless of return or parameter types and (2) the call to anymethod named set regardless of return or parameter types - in case of overloading there may bemore than one such set method; this pointcut picks out calls to all of them.

Additionally elements can be selected based on their types. For example:

1. execution(int *())

2. call(* setY(long))

3. call(* Point.setY(int))

4. call(*.new(int, int))


(1) means the execution of any method with no parameters that returns an int, (2) the call to anysetY method that takes a long as an argument, regardless of return type or declaring type, (3) thecall to any of Point’s setY methods that take an int as an argument, regardless of return type and (4)the call to any classes constructor, so long as it takes exactly two ints as arguments.

Even composition is possible. For example:

1. target(Point) && call(int *())

2. call(* *(..)) && (within(Line) || within(Point))

3. within(*) && execution(*.new(int))

4. !this(Point) && call(int *(..))

(1) means any call to an int method with no arguments on an instance of Point, regardless of itsname, (2) any call to any method where the call is made from the code in Point’s or Line’s typedeclaration, (3) the execution of any constructor taking exactly one int argument, regardless ofwhere the call is made from, and (4) any method call to an int method when the executing object isany type except Point.

Last but not least, methods and constructors can be selected based on their modifiers and on nega-tions of modifiers. For example:

1. call(public * *(..))

2. execution(!static * *(..))

3. execution(public !static * *(..))

which means (1) any call to a public method, (2) any execution of a non-static method and (3) anyexecution of a public, non-static method.

Bart De Win et al. [36] state that within the area of AOP ongoing research addresses a declarativedescription of security properties for application software. Such security specific language wouldbe especially valuable to define the binding between application and security mechanisms. Thecore challenge is to create the right security abstractions. This is obviously not evident. The securityabstractions will remain an evolving challenge for the foreseeable future. De Win et al. [36] haveinvestigated how well AOP can deal with the separation of security concerns from an application. Inorder to construct a more generic solution, they have suggested to abstract relevant pointcuts out ofthe aspect implementation.

Dima Alhadidi et al. [1] investigated how well AspectJ can be used for security hardening of Javaprograms. AspectJ offers three kinds of advices to describe where aspects are to be applied: Before,after or around. All of them are important from a security hardening point of view:

• A before-advice can be used to inject some security code as authentication before a criticaloperation.

• An after-advice is meant to inject some security code after some locations in the program. Forinstance, one can add some integrity checking code.


• An around-advice can be used to change a non-secure method by a more secure one at partic-ular points in the program.

They came to the conclusion that AspectJ is very well suited for security purposes but the AspectJimplementation and all other AOP frameworks lack some additional features respectively pointcutswhich would increase the places where security checks could be done, for example a loop- anddataflow pointcut etc.

An example of a multilevel security system which prevents unauthorised personnel from accessinginformation at higher classification than their authorisation and preventing personnel from declas-sifying information is proposed in the paper AspectJ for Multilevel Security of Roshan Ramachandranet al. [33]. In a case study involving a simple multi threaded payroll system where administrators canupdate and track specific information about employees, Ramachandran showed that it is possibleto enforce complex security constraints through a security monitor implemented as an aspect. Theinformation in the payroll system includes sensitive details like personal details, salary details andso forth. The working scenario is that different administrators of varying clearances access thepayroll system to access different classes of information. For example, an administrator with a lowclearance cannot see the salary details of a senior manager with a higher security classification. Also,an administrator with a higher clearance cannot leak the salary details of a higher ranking officialsuch that a user with low clearance can access its details. Thus, to preserve the confidentiality of theinformation, an access control policy was implemented in the application.

Amine Boukhtouta et al. go a step further in their paper called A Practical Framework for The DataflowPointcut in AspectJ [6]. The paper presents the design and the implementation of the dataflow

pointcut in AspectJ compiler ajc 1.5.0 (the current version is 1.6.10). From the abstract:

Some security concerns are sensitive to flow of information in a program execution. Thedataflow pointcut has been proposed by Masuhara and Kawauchi in order to easily im-plement such security concerns in aspect-oriented programming languages. The point-cut identifies join points based on the origins of values. The dataflow pointcut candetect and fix a lot of vulnerabilities that result from not validating input effectively, e.g.,web application vulnerabilities, process injection, log forging and path injection.

AOP has not been initially engineered with security in mind which resulted in many shortcomings inthis technology. There are some contributions that suggest new pointcuts for security purposes suchas the one of Masuhara and Kawauchi [23]. They have presented the design of the dataflow pointcutwith a web application example, and its prototype implementation. It has been implemented asan extension to Aspect SandBox (ASB) [24], which is a modelling framework for AOP mechanismsbased on the pointcut-advice model. Listing 3.7 shows a tracing aspect written in AspectJ wherethe pointcut traced picks out any call to any method. AspectJ has a comprehensive and expressivepointcut specification language that allows to specify particular points in the control flow of theprogram where advices are to be applied.

1 public aspect ExtTracingAspect {

2 private int callDepth;

3 pointcut traced () : !within(ExtTracingAspect);

4

5 before () : traced () {

6 print(thisJoinPoint);

7 callDepth ++;

8 }


9

10 after() : traced () {

11 callDepth --;

12 }

13

14 private void print(Object message) {

15 for(int i = 0; i < callDepth; i++) {

16 System.out.print(" ");

17 }

18


20 }

21 }

Listing 3.7: AspectJ Tracing Aspect

Adding the trace aspect to a simple Struts 1.x web application would generate an output such as seenin Listing 3.8

1 staticinitialization(com.webapp.HelloWorld.<clinit >)

2 preinitialization(com.webapp.HelloWorld ())

3 initialization(com.webapp.HelloWorld ())

4 execution(com.webapp.HelloWorld ())

5 execution(ActionForward com.webapp.HelloWorld.execute(ActionMapping ,

ActionForm ,

6 HttpServletRequest , HttpServletResponse))

7 get(PrintStream java.lang.System.out)

8 call(void java.io.PrintStream.println(String))

9 call(ActionForward org.apache.struts.action.ActionMapping.findForward(

String))

Listing 3.8: AspectJ Tracing output for a simple Struts 1.x Application

Masuhara and Kawauchi [23] have defined a dataflow pointcut for security purposes but thispointcut has not been implemented yet. The pointcut identifies join points based on the origins ofvalues. Cross-site scripting (XSS) problem in web applications is an example presented by them toclarify the need for such a pointcut. Local variables set and get pointcuts [2] have been introducedto protect the privacy and integrity of sensitive data. They allow to track the values of local variablesinside a method. Many security hardening practices require the injection of code around a set of joinpoints or possible execution paths. Examples of such cases would be the injection of security libraryinitialisation/deinitialisation, privilege change and logging.

A different pointcut has been proposed by Boukhtouta et al. [5] in their paper Capturing ReturnedComputable Values in AspectJ for Security Hardening. They have fully designed and implementedreturns and returned pointcuts as extensions to ajc-1.5.0 compiler of AspectJ. These pair ofpointcuts can capture the returned computable values of methods. Such pointcuts can be appropri-ate primitives for integrating data concerns in Java code. The returned pointcut is classified as akindred pointcut whereas the returns pointcut is classified as a context matching pointcut. Thesenewly implemented pointcuts are being used to define a data integrity service for security hardeningof distributed programs. Data integrity refers to validity of data, which can be compromised througha malicious or an accidental altering. These pointcuts returns and returned together with thecall pointcut compromise the main building block of the data integrity service. To clarify the idea,consider the following model. We consider a standard server which implements a set of remotemethods that can be invoked at any time by clients. Once a client invokes a method implementedin the server, the server runs the method. Before returning the value to the client, it creates ahash for it, which is recorded in a hash repository implemented in the data integrity service. The


server then sends the value to the client. After receiving the value, the client creates a hash for thereceived value and sends the hash to the data integrity service, which acknowledges the hash. If theacknowledgement is positive, the client carries on the execution of its program. Otherwise it throwsan exception.

Martin et al. [21] uses static source code analysis to detect common web application vulnerabilitiessuch as SQL Injection attacks and application errors by proposing a program query language (PQL).PQL allows programmers to write queries which express whether a program conforms to certaindesign rules. An important class of design rules deal with sequences of events associated with aset of related objects. A query looks like a code excerpt corresponding to the shortest amount ofcode that would violate a design rule. The programmer may also specify actions to perform when amatch is found, such as recording relevant information or even correcting an erroneous executionon the fly. The system automatically generates from the query a pair of complementary checkers: astatic checker that finds all potential matches in an application and a dynamic checker that traps allmatches precisely as they occur and can initiate user specified logging or recovery actions upon amatch. A simple PQL query for detecting SQL Injection vulnerabilities is shown in Listing 3.9.

1 query simpleSQLInjection ()

2 uses

3 object HttpServletRequest r;

4 object Connection c;

5 object String p;

6 matches { p = r.getParameter(_); }

7 replaces c.execute(p)

8 with Util.CheckedSQL(c, p);

Listing 3.9: PQL Query for finding SQL Injection Vulnerabilities

PQL attaches user specified actions to subquery matches; this capability puts PQL in the class ofaspect-oriented programming languages. The PQL system may be considered as an aspect orientedsystem that defines its aspects with respect to the dynamic history of sets of objects. An extension ofAspectJ to include dataflow pointcuts [23] has been proposed to represent a statement that receives avalue from a specific source; PQL can represent these with a two-statement query and permits muchmore complex concepts of data flow. Walker and Veggers [35] introduce the concept of declarativeevent patterns, in which regular expressions of traditional pointcuts are used to specify when adviceshould run. Allan et al. [3] extend this further by permitting PQL-like free variables in the patterns.The primary focus in the AspectJ extensions is in permitting a developer to specify applicationdevelopment concerns very finely. As a result, they devote a great deal of work to ensuring propertiessuch as guarantees that memory allocated by the matching machinery will eventually be availablefor collection. PQL, with its genesis focusing on detecting application errors, pays less attentionto this. PQL differs from these systems in that its matching machinery can recognise nonregularlanguages and in exploiting advanced pointer analysis to prove points irrelevant to eventual matches.

Kung et al. [7] designed access control aspects for web applications. In particular, they choosethe MVC-based Struts framework as the architectural style of the target application. Two aspectsuites in AspectJ have been developed to enforce fine-grained access controls in a modular andnon-invasive manner. While both suites exhibit good software engineering properties, the first suitecannot guarantee the required conditional execution and the second suite achieves it at the cost ofcode duplication. They defined three abstract aspects that capture the common code structure forenforcing fine-grained access controls, namely Authentication, Precheck and Postfilter. Theauthentication aspect is responsible for verifying that the requesting user has passed the identitycheck or must be redirected to a login page. The Precheck aspects ensure that designated user


actions get executed only when a given constraint is satisfied. The Postfilter aspect will filter outunauthorised data records from a user query result, and, if necessary, mask out sensitive data fields.

Listing 3.10 shows an aspect declaration which would be called before every execution of the Struts1.x execute(...) method. This would allow a developer to do additional security checks based onthe HTTP request. Every user request is dispatched to an Action class by the controller accordingto the Action mapping defined in the configuration file, struts-config.xml. These actions areresponsible for serving user requests or passing them to the designated business tier componentsand for returning the correct view element that the controller should forward to after finishing theuser request. This view forwarding is also based on the mapping information specified in the config-uration file. In other words, the action classes play the role of gateway between the presentation tierand the business and the data tiers. Moreover, all action classes must inherit from the class Actionand implement an execute(...) method. Thus makes it the right join point to weave in advicecode for its arguments and exposes the right amount of information to do security checks.

1 public aspect RequestAspect {

2 public pointcut getHTTPRequest(ActionMapping mapping , ActionForm form ,

HttpServletRequest request , HttpServletResponse response) : execution

(* *. execute(ActionMapping , ActionForm , HttpServletRequest ,

HttpServletResponse)) && args(mapping , form , request , response);

3

4 before(ActionMapping mapping , ActionForm form , HttpServletRequest request ,

HttpServletResponse response): getHTTPRequest(mapping , form , request ,

response) {

5 System.out.println("Request received: " + request);

6 }

7 }

Listing 3.10: Aspect for catching all Struts 1.x execute Methods

A similiar research paper to Kung’s et al. [7] is the paper from Andrew Prunicki et al. [28]. Theydescribed and designed Aclamate: An AOSD Security Framework for Access Control. Aclamate is aframework providing authorisation and access control in a non-invasive fashion. Aclamate is wellsuited to provide security for new or pre-existing applications. Upon successful authentication, asecurity context is created and attached to that session (e.g., login session). The framework usesfiltering mechanisms to control access to secured resources (URLs or objects). Aclamate provides thefollowing key features:

• URL and object security: Both URLs and objects can be declared secure.

• Returned object security: Objects that have been declared as completely secured will not bereturned as values.

• Role based access control: Role based access control is supported using the Aclamate securityconstraint mechanism.

• Data-level access control: Data-level access control is supported using the Aclamate securityconstraint mechanism.

• Pluggable architecture: Major features of the framework are customisable using plugins.

• Major UI framework support: Support for Apache Struts is included. Other frameworks will besupported via pluggable architecture.

• JSP taglib: JSP tags are included to allow secure portions of a page.


The framework is driven by security policy files containing descriptions of how to secure resources.Constraints for role based access control (RBAC) and expression based access control (i.e., for data-level access control or other dynamic filtering) are included with the framework. New constrainttypes can be created using the plug-in support. An example of such resource constraints are shownin Listing 3.11.

1 {

2 role -constraint {

3 role=" instructor ";

4 }

5 AND

6 expression -constraint {

7 expression ="this.classNumber GT ’CS -5’";

8 }

9 AND

10 expression -constraint {

11 expression ="this.school IN

12 edu.iit.aclamate.sample.SchoolFactory.

13 getInstance ().lookupSchoolsByType(’ENG ’)";

14 }

15 }

16 OR

17 role -constraint {

18 role="admin ";

19 }

Listing 3.11: Aclamate RBC Configuration Example execute Methods

The example illustrates a security constraint that is satisfied when the security context possesses theinstructor role and the object being secured is from an engineering school with a class numberstarting with CS-5 or the security context simply possesses the admin role.

The Aclamate framework allows adding security to an application without requiring changesthroughout an application to enforce security checks. While changes to the user interfaceare necessary in order to improve the user experience, these changes are less invasive thanwould be required using traditional methods. Aclamate was available on Sourceforge athttp://sourceforge.net/projects/aclamate with an example application illustrating itsusage. Sadly, the project does not exist any more on Sourceforge.

AspectJ is not the only available aspect oriented project for the Java programming language. Al-ternatives include JBossAOP9 and SpringAOP10. Kung et al. [7] although mentioned in their paperthat they had difficulties porting the AspectJ implementation of their access control aspects to theJBossAOP platform. The Spring Framework offers its own AOP implementation which is a bit dif-ferent than the common AOP projects like JBossAOP and AspectJ. The Spring AOP does not needto control the class loader hierarchy and is thus suitable for use in a Servlet container or applica-tion server and there is no need for a special compilation process. The aim of Spring AOP is not toprovide the most complete AOP implementation (although Spring AOP is quite capable); it is ratherto provide a close integration between AOP implementation and Spring IoC (Inversion of ControlContainer) to help solve common problems in enterprise applications. In Spring AOP, aspects arenormal Java classes which are being annotated with the @Aspect and @Pointcut("execution(*

com.xyz.someapp.service.*.*(..))") annotations.

9http://www.jboss.org/jbossaop [30/11/10]10http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/aop.html

[30/11/10]


http://sourceforge.net/projects/aclamate

http://www.jboss.org/jbossaop

http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/aop.html

3.3 Summary

This Chapter tried to give an overview over the current research in the field of declarative securityfor web applications. The current research has to be divided into three main categories: (1) staticand dynamic checking of source code for detecting vulnerabilities and application design issues; (2)using aspect oriented programming to add additional security checks to existing web applications ina non invasive way; (3) annotating Java source code for data flow analysis and detection of tainted,non validated user input.

Annotating Java source code for data flow analysis might be a sensible approach to detect subtleweb application vulnerabilities, yet it is a rather difficult task when faced with the need to add theseannotations to an already deployed and productive web application.

Enhancements to the AspectJ project would significantly improve the ability to do more fine grainedsecurity checks and data flow analysis, sadly the enhancements described in several research papershave not yet been implemented by the AspectJ community. Nonetheless, the non invasiveness ofAspectJ is a huge advantage when an existing web application needs to be enhanced with additionalsecurity checks or any other additional functionality. Some security frameworks have been proposedand been successfully implemented using an AOP approach, but most of these frameworks are eithernot available any more or have been abandoned years ago such as Aclamate. Bart De Win states[37] (2002) that the generalisation of aspects is currently still difficult. The limitations of the aspectlanguage sometimes makes it very hard or even impossible to generalise certain cases. As a result, acompletely reusable security framework can hardly be developed yet.


4 Finding Web Application Vulnerabilities by usingDeclarative Information

This Chapter tries to identify possible ways to enhance the detection of security vulnerabilities inweb applications with an automated security testing tool by incorporating declarative security infor-mation. The OWASP Top Ten project1 has been used to identify the most common web applicationvulnerabilities and for each one an appropriate solution has been proposed to enhance the detectionof such vulnerabilities by using a white box testing approach. Each vulnerability from the OWASP TopTen project is being shortly described and for each popular web application framework such as Struts1.x, Struts 2.x, Spring and Wicket the framework specific features are shown which help a developerto prevent a particular vulnerability.

4.1 Overview

The goal of this thesis is to write an automated security testing tool which is able to detect webapplication vulnerabilities more reliably by incorporating declarative security information than theirsole black box testing counterparts. Commonly available security testing tools only take an externalperspective from the target application, which is also called black box testing. White box testing onthe other hand takes more of an internal perspective on the targeted application and uses as muchadditional information as possible to derive test cases. Such additional information can consist ofthe actual source code of the application or metadata about the internal structure of the applicationas seen in Chapter 2 and 3.

Figure 4.1 illustrates the basic architecture of such a combined white- and black box security testingapplication. The automated testing tool has to be configured (1) by an developer or testing agentfor the specific web application. It might be possible, depending on the specific test case scenario,that the testing agent has to provide additional metadata in form of a machine readable format suchas a XML file to the testing tool. What the contents of such a XML file might be will be elaboratedin the following Sections and Chapters. The automated security testing tool needs access to theactual source code of the web application (2) in order to extract further data which helps it to craftmeaningful test cases. After all the information has been collected the actual process of findingmisconfigurations and application vulnerabilities begins (3). A human readable report for the testingagent should be created.

The following Sections are based on the OWASP Top Ten critical web application security risks whichhave just recently (2010) been updated to include the following risks:

1http://www.owasp.org/index.php/Top_10_2010 [07/04/11]

Chapter 4. Finding Web Application Vulnerabilities by using Declarative Information | 60

http://www.owasp.org/index.php/Top_10_2010

Figure 4.1: Architecture of a general combined White- and Black Box Security Testing Tool

A1 Injection A6 Security MisconfigurationA2 Cross-Site Scripting (XSS) A7 Insecure Cryptographic StorageA3 Broken Authentication and Session A8 Failure to Restrict URL Access

Management A9 Insufficient Transport Layer ProtectionA4 Insecure Direct Object References A10 Unvalidated Redirects and ForwardsA5 Cross-Site Request Forgery (CSRF)

The points A2 and A5 have been incorporated into Section 4.2 - Injection Flaws.

4.2 Injection Flaws

Injection flaws occur when an application sends untrusted data to an interpreter. Injection flaws arevery prevalent, particularly in legacy code, often found in SQL queries, LDAP queries, XPath queries,OS commands, program arguments, etc. Injection flaws are easy to discover when examining code,but more difficult to find via testing. Scanners and fuzzers can help attackers find them. Injection canresult in data loss or corruption, lack of accountability or denial of access. Injection can sometimeslead to complete host takeover.

Injection flaws can be averted by proper validating and filtering received user input. As seen inSection 2.4 there are quite a few security frameworks available which offer functions for validatinguser input. However, most of the Java web application frameworks offer additional means forvalidating user input but these offer a more high level approach to filtering. They are not particularlyconcerned about security, they rather provide means for checking if the received user input is a


number, telephone number, credit card etc. Libraries such as Common Validator (see Section 2.4.7)and ESAPI (see Section 2.4.4) should be used to orderly protect a web application from injection flaws.

The Web Application Attack and Audit Framework (w3af)2, which is a sole black box security testingtool, has the ability to detect persistent and non-persistent XSS (Cross-Site Scripting) and blind andnone-blind SQLi (SQL Injection) flaws (among others such as CSRF flaws etc.). w3af’s detection rou-tine for detecting SQLi injection flaws is rather simple. Blind SQLi are being found by inserting SQLcommands into GET and/or POST parameters such as BENCHMARK(5000000,ENCODE(’MSG’,’by5 seconds’)) which will delay server responses if the expression can be injected. If the serverresponse takes longer than usual, then this will be a strong indication that the web application mightsuffer from a SQL injection vulnerability. None-blind SQLi flaws are being detected by parsing serverresponses for databse error messages such as “You have an error in your SQL syntax;”. Adifferent approach has been suggested and realised by Denver [12], his approach combines black-and white box testing techniques for enhancing the detection rate of SQLi and blind-SQLi flaws byparsing database query log files.

The following enumeration shows framework specific features which help a developer to filter userprovided input to prevent Injection Attacks. Depending on the used web application frameworkthese validation rules can be configured declaratively and might help an automated security testingtool to create more sophisticated test cases for evaluating if a given GET or POST parameter might bevulnerable to Injection Attacks:

• Apache Struts 1.xUses the Apache Commons Validator framework as described in Section 2.4.7. Validation rulesare being configured through a XML file called validation.xmlwhich has to be placed insidethe WEB-INF folder of a deployed Java web application.

• Apache Struts 2.xValidation in Struts 2.x can either be done through annotations or through a XML configurationfile. Struts 2.x relies on a validation framework provided by XWork3 which enables the Java webapplication to execute input validation rules before actual Actions are being executed. ThisSection only provides the bare minimum to get started and focuses on the Struts 2.x extensionof the XWork validators. XWork is a command-pattern framework that is used to power Struts2 as well as other applications. XWork provides an Inversion of Control container (IoC), apowerful expression language, data type conversion, validation and pluggable configuration.

Listing 4.1 shows how a Struts Action class can be annotated to enforce user input valida-tion constraints. In this example the parameter bar is a field parameter from a standardHTML form and has to be between 6 and 10 characters long. The necessary annotations are@Validation and then additionally the specific annotations based on the kind of validationthe framework should enforce upon the parameter such as @RequiredFieldValidator and@IntRangeFieldValidator.

1 @Validation

2 public class SimpleAnnotationAction extends ActionSupport {

3 private int bar;

4

2http://w3af.sourceforge.net/ [13/12/10]3http://www.opensymphony.com/xwork/ [13/12/10]


http://w3af.sourceforge.net/

http://www.opensymphony.com/xwork/

5 @RequiredFieldValidator(type = ValidatorType.FIELD ,

6 message = "You must enter a value for bar.")

7 @IntRangeFieldValidator(type = ValidatorType.FIELD ,

8 min = "6", max = "10",

9 message = "bar must be between ${min} and ${max},

10 current value is ${bar}.")

11 public void setBar(final int b) {

12 bar = b;

13 }

14 ...

15 }

Listing 4.1: Struts 2.x XWorks Validation Example with Annotations

The other possibility is to use a XML configuration file instead of Java annotations. Listing 4.2shows the same example from Listing 4.1 but configured through a XML file.

1 <validators >

2 <field name="bar">

3 <field -validator type=" required">

4 <message >You must enter a value for bar.</message >

5 </field -validator >

6 <field -validator type="int">

7 <param name="min">6</param >

8 <param name="max">10</param >

9 <message >

10 bar must be between ${min} and ${max},

11 current value is ${bar}.

12 </message >

13 </field -validator >

14 </field >

15 </validators >

Listing 4.2: Struts 2.x XWorks Validation Example with XML

Validation rules can be specified:

– Per Action class: in a file named <ActionName>-validation.xml

– Per Action alias: in a file named <ActionName>-<alias>-validation.xml

– Inheritance hierarchy and interfaces implemented by Action class: XWork searches upthe inheritance tree of the Action to find default validations for parent classes of the Actionand interfaces implemented.

• SpringSpring 3 introduces several enhancements to its validation support. Most notably, the JSR-303Bean Validation API4 is now fully supported. JSR-303 defines a metadata model and API forJavaBean validation. The default metadata source is annotations, with the ability to overrideand extend the metadata through the use of XML validation descriptors.

Listing 4.3 shows a simple Java class with JSR-303 validation annotations. The name parameterhas to fulfil the following requirements before it is accepted by the validation routine/frame-work: (1) The data must match the regular expression which states that only the charactersa-z, A-Z are allowed; (2) the length of the name must be between 3 and 30 characters; (3) thename parameter is not allowed to be null.

4http://jcp.org/en/jsr/detail?id=303 [13/12/10]



1 public final class User {

2 @Pattern(regexp="[a-zA -Z]*")

3 @Size(min=3, max =30)

4 @NotNull

5 private String name;

6 ...

7 }

Listing 4.3: Spring 3.x Validation Example with Annotations

The validation of the class can then be done inside Spring controller classes by injecting aValidator object. Listing 4.4 shows how the User class from Listing 4.3 is being validatedafter the user submitted the corresponding HTML form.

1 @Controller

2 public final class UserController {

3 @Autowired

4 private Validator validator;

5

6 @RequestMapping(value = "/userForm", method = RequestMethod.POST)

7 public String post(User user , BindingResult result) {

8 validator.validate(user , result);

9 if (result.hasErrors ()) { return "form"; }

10 // ...

11 }

12 ...

13 }

Listing 4.4: Spring 3.x Validation Example in a Controller Class

Yet easier to use is the @Valid annotation which is not a Spring specific construct but ratherpart of the JSR-303 Bean Validation API. This obsoletes the injection of a Validator instance.All a developer has to do is rewrite the post() method from Listing 4.4 to include the @Validannotation as shown in the following code snippet:

@RequestMapping(value = "/userForm", method = RequestMethod.POST)

public String post(@Valid User user, BindingResult result) {

if (result.hasErrors()) { return "form"; }

// ...

}

An alternative to annotations is writing custom class validators by implementing theorg.springframework.validation.Validator interface and checking parameters re-spectively values programmatically.

• MyFacesValidation in MyFaces can be achieved by using the ExtVal project as described in Section 2.9and Listing 2.35. ExtVal provides several Java annotations for input validation purposes such as@Pattern, @Length, etc.

• WicketWicket validation is done programmatically during the creation of a form. The code in Listing4.5 adds a simple text field called age to a HTML form inside a Wicket page. The age field isrequired and the provided data by the user has to be greater than 18. The NumberValidatorclass provides several factory methods and ready-made validators for validating numbers. Ithas methods for longs and doubles, but they work equally well for integers and floats, respec-tively. Here is an example:


add(new TextField("age").add(NumberValidator.minimum(18)));

add(new TextField("handicap").add(NumberValidator.range(0, 3.5)));

add(new TextField("duration").add(NumberValidator.POSITIVE));

The StringValidator class can be used to check strings, which also specifies several factorymethods. Here are some examples of its use:

add(new TextField("userid").add(StringValidator.lengthBetween(8,12)));

add(new TextField("comment").add(StringValidator.maximumLength(4000)));

There are several more validators available in the Wicket framework such as a validator tocheck user input against regular expressions etc. More information about validators can befound on the Wicket project page5.

1 public class FormInput extends WicketPage {

2 private class InputForm extends Form {

3 public InputForm(String name) {

4 super(name , new CompoundPropertyModel(new FormInputModel ()));

5 form.add(new TextField("age")

6 .setRequired(true)

7 .setLabel(new Model("age"))

8 .add(NumberValidator.minimum (18)));

9 ...

10 }

11 ...

12 }

13 ...

14 }

Listing 4.5: Wicket Validation Example

With the above mentioned input validation rules, that can be added declaratively to a web applica-tion by either providing a XML configuration file or by annotating the Java source code the followingapproaches could be used by a combined white- and black-box security testing tool to enhance thedetection of parameters which might be vulnerable to Injection Attacks:

• Validation Completeness Check: The XML configuration files and Java annotations could beused to check if every GET or POST parameter, found through a sole black box web spideringcomponent or by statically analysing the Java source code of the application, is being referred toin a configured validation rule. If a GET or POST parameter has been discovered that is not beingtackled by a configured validation rule inside a XML configuration file or a Java annotation, thismight indicate that the found parameter suffers from a possible code injection vulnerability.This idea might generate false positives because not all found web application parameters suchas session IDs have automatically to be validated and even maliciously tampering with themmight not lead to a exploitation of the targeted web application.

• SQLi Detection: The following suggestion is not a real testing scenario which uses declara-tive information but rather another interesting idea for finding injection vulnerabilities by us-ing an AOP approach (see Section 3.2). By using AOP it would be possible to add aspects be-fore typical JDBC database functions are being executed such as boolean execute(String

query), ResultSet executeQuery(String query) and int executeUpdate(String

query) (all defined in java.sql.Statement.* of the Java 6.0 API). These aspects would then

5http://wicket.apache.org/apidocs/1.4/org/apache/wicket/validation/validator/package-

summary.html [07/04/11]


http://wicket.apache.org/apidocs/1.4/org/apache/wicket/validation/validator/package-summary.html

http://wicket.apache.org/apidocs/1.4/org/apache/wicket/validation/validator/package-summary.html

be able to examine the queries issued to the database tier. In combination with a black box test-ing tool which tries to insert maliciously crafted SQL statements into GET and/or POST parame-ters, unvalidated input parameters could be detected with much more precision than commonblack box security testing tools. This approach is basically the same as the one proposed byDenver [12] but without the need of parsing database query log files. This aspect oriented ap-proach would provide the same benefits and thoroughness as Denver’s proposal, the only pre-requisite is that the existing web application would have to be recompiled with AspectJ to in-corporate the newly proposed aspects which analyse the issued SQL statements to the databasetier. Additionally, this approach could be enhanced to detect stored XSS flaws as well. Using thisapproach for detecting reflected XSS vulnerabilities makes no sense because sole black box se-curity testing tools can detect them equally well without a deeper understanding of the innerworkings of the targeted web application.

• Detection of Unvalidated Redirects and Forwards: See Section 4.9.

• Detection of OS Command Execution: By adding an aspect to the Process

java.lang.Runtime.exec(String) method it would be possible to verify and check ifarguments passed to the exec(String) method can be injected by a malicious user such asnc -l -p 3333 -v -e /bin/bash which opens a remote shell on the target host. Injectingmalicious OS commands could be automated by using a spidering component similar todetecting SQLi vulnerabilities.

• Detection of Dangerous File Access: By adding an aspect to the constructor of thejava.io.File class it becomes possible for an aspect to verify if the file path can besupplied by a malicious user such as /etc/passwd on Linux operating systems. Other meth-ods worth monitoring include the following: constructor of the java.io.FileReader

class, constructor of the java.io.FileInputStream class, constructor of thejava.io.BufferedInputStream class, constructor of the java.io.RandomAccessFile

class through an automated spidering component similar to detecting SQLi vulnerabilities.

4.3 Broken Authentication and Session Management

Developers frequently build custom authentication and session management schemes, but buildingthese correctly is hard. As a result, these custom schemes frequently have flaws in areas such aslogout, password management, timeouts, remember me, secret question, account update, etc.Finding such flaws can sometimes be difficult, as each implementation is unique. Such flaws mayallow some or even all accounts to be attacked. Once successful, the attacker can do anything thevictim could do and thus privileged accounts are frequently targeted.

Except for some simple automated tasks such as checking if the session IDs are chosen randomlyenough to prevent an attacker from guessing session IDs and checking whether the session IDs arebeing reinitialised after a user authenticated himself successfully to the application for preventingSession Fixation attacks, detecting other session management vulnerabilities in an automated waybecomes infeasible without doing a manual code review and is thus being ignored in this thesis.

4.4 Insecure Direct Object References

Applications frequently use the actual name or key of an object when generating web pages. Appli-cations do not always verify if the user is authorised to access the targeted object. This results in aninsecure direct object reference flaw. Testers can easily manipulate parameter values to detect such


flaws and code analysis quickly reveals whether authorisation is properly verified. Such flaws cancompromise all the data that can be referenced by the parameter. Unless the name space is sparse, itis easy for an attacker to access all available data of that type.

The OWASP ESAPI security library as described in Section 2.4.4 provides a class called Access

ReferenceMap. This class is especially designed for preventing insecure direct object referencesattacks. A simple example can be seen in Listing 4.6. The AccessReferenceMap interface is usedto map from a set of internal direct object references to a set of indirect references that are safe todisclose publicly. This can be used to help protect database keys, file names and other types of directobject references. As a rule, developers should not expose their direct object references as it enablesattackers to attempt to manipulate them.

Indirect references are handled as Strings, to facilitate their use in HTML. Implementations cangenerate simple integers or more complicated random character strings as indirect references.Implementations should probably add a constructor that takes a list of direct references.

Note that in addition to defeating all forms of parameter tampering attacks, there is a side bene-fit of the AccessReferenceMap. Using random strings as indirect object references, as opposedto simple integers makes it impossible for an attacker to guess valid identifiers. So if per-userAccessReferenceMaps are used, then request forgery (CSRF) attacks will also be prevented6.

1 ...

2 Set fileSet = new HashSet ();

3 fileSet.addAll (...); // add direct references (e.g. File objects)

4 AccessReferenceMap map = new AccessReferenceMap(fileSet);

5 // store the map somewhere safe - like the session!

6 String indRef = map.getIndirectReference(file1);

7 String href = "http :// www.aspectsecurity.com/esapi?file=" + indRef );

8 ...

9 // if the indirect reference doesn ’t exist , it’s likely an attack

10 // getDirectReference throws an AccessControlException

11 // you should handle as appropriate

12 String indref = request.getParameter("file");

13 File file = (File)map.getDirectReference(indref);

Listing 4.6: ESAPI Example on how to use the AccessReferenceMap Class

Whether direct references are being used by a web application can easily be determined with a soleblack box security testing application which is able to analyse GET and POST parameters for repeatedpatterns in URLs such as page1.php?userid=42&fileid=5, page1.php?userid=43&fileid=6,etc.

None of the popular web application frameworks offer a feature for preventing insecure direct objectreferences except for Wicket. Wicket is secure by design to these kind of attacks because of its uniqueMVC model which does not expose sensitive information in URLs and all component paths are ses-sion relative (see Section 2.10). Because there exists no declarative information in web applicationframeworks that would prevent insecure direct object references and who could be extracted by asecurity testing framework these kind of vulnerabilities are being ignored in this thesis.

6http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/

AccessReferenceMap.html [13/12/10]


http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/AccessReferenceMap.html

http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/AccessReferenceMap.html

4.5 Security Misconfiguration

Security misconfiguration can happen at any level of an application stack, including the platform,web server, application server, framework and custom code. Developers and network administratorsneed to work together to ensure that the entire stack is configured properly. Automated scannersare useful for detecting missing patches, misconfigurations, use of default accounts, unnecessaryservices, etc. Such flaws frequently give attackers unauthorised access to some system data orfunctionality. Occasionally, such flaws result in a complete system compromise.

Tools like Nessus7 or OpenVAS8 do a pretty good job on detecting security misconfigurations and thusthese kind of security vulnerabilities are being ignored in this thesis. The Open Vulnerability Assess-ment System (OpenVAS) is a framework of several services and tools offering a comprehensive andpowerful vulnerability scanning and vulnerability management solution. The Nessus vulnerabilityscanner features high-speed discovery, configuration auditing, asset profiling, sensitive data discov-ery and vulnerability analysis of a companies security posture.

4.6 Insecure Cryptographic Storage

The most common flaw in this area is simply not encrypting data that deserves encryption. Whenencryption is employed, unsafe key generation and storage, not rotating keys and weak algorithmusage is common. Use of weak or unsalted hashes to protect passwords is also common. Externalattackers have difficulty detecting such flaws due to limited access. They usually must exploitsomething else first to gain the needed access. Failure frequently compromises all data that shouldhave been encrypted. Typically this information includes sensitive data such as health records,credentials, personal data, credit cards, etc.

Whether data should or should not be encrypted inside the data store is a decision which can notbe made by an automated security testing tool and thus this point from the OWASP Top Ten is beingignored in this thesis.

4.7 Failure to Restrict URL Access

Web applications are not always protecting page requests properly. Sometimes, URL protection ismanaged via configuration and the system is misconfigured. Sometimes, developers must includethe proper code checks, and they forget. Detecting such flaws is easy. The hardest part is identifyingwhich pages respectively URLs exist to attack. An attacker, who is an authorised system user, simplychanges the URL to a privileged page. If there is a misconfiguration the access is granted to theattacker even if he does not provide the necessary user role to access that specific page respectivelyURL.

By using declarative information provided by the Java web application frameworks the disadvantageof not knowing the privileged resources respectively URLs can be overcome by parsing specificconfiguration files. The following Sections elaborate which configuration files contain the necessaryinformation for each of the popular Java web application frameworks:

7http://www.nessus.org/nessus/intro.php [13/12/10]8http://www.openvas.org/ [13/12/10]


http://www.nessus.org/nessus/intro.php

http://www.openvas.org/

• Application ContainerThe deployment descriptor as seen in Section 2.2.2 and Listing 2.8 contains valuable infor-mation about security constraints which are being enforced by the container and are beingconfigured based on URLs. Additionally, the deployment descriptor might contain informationabout existing users and roles. So a white box security testing tool would know at least whatkind of users and roles are existent and would be able to extract a list of restricted URL paths.This is only true if the web application uses container managed security features and theconstraints are thus being configured in the deployment descriptor.

If the web application rather uses Java annotations than a deployment descriptor then the sameinformation can be extracted by parsing the source code of the Java web application. The an-notations used to configure access constraints equal to the ones configured in the deploymentdescriptor are called @ServletSecurity as seen in Section 2.2.5 and @DeclareRoles as seenin Section 2.2.3. This is only true if the web application uses container managed security fea-tures and the constraints are thus being configured through Java annotations inside the sourcecode of the Java web application.

• Struts 1.xAs seen in Section 2.7 the Struts Web Application Framework is being configured through a XMLfile called struts-config.xml. This particular configuration file contains Action mappingswhich have to be configured by a developer in order to use the web application framework at all.These mappings map relative URLs to Java classes implementing the desired functionality ofthe developer. By parsing the struts-config.xml configuration file a complete list of all thedeclaratively configured URLs a targeted web application serves to its users can be extracted.The struts-config.xml configuration file might even contain declarative information aboutexisting user roles and privileges based on the roles element as seen in Section 2.7.1.

• Struts 2.xSimilar to Struts 1.x the Struts 2.x Web Application Framework provides the same informationthrough a XML file called struts.xml as seen in Section 2.8 and Listing 2.32. Additionally tothe XML configuration file, Struts 2.x offers Java annotations to configure the same informationinside the source code of the web application. By parsing the source code of the targeted webapplication the following annotation is relevant @Result as seen in Listing 2.31.

• SpringSpring similar to Struts 1.x and Struts 2.x provides declarative information about Action map-pings which maps relative URLs to particular Java classes implementing the desired function-ality by the developer through the use of Java annotations. The specific annotations are called@RequestMapping as seen in Section 2.6 and can be extracted by a security testing tool togather a complete list of all the URLs a targeted web applications serves to its users.

• WicketIn case of the Wicket framework (see Section 2.10) nothing like declarative Action mappingsexist for mapping relative URLs to Java classes as it is the case in Struts 1.x or Struts 2.x basedweb applications. In Wicket, links are being programmatically created and added to the viewthrough specific code instructions in the Java class files. The following snippet shows two wayshow a common link is being created in the Wicket framework:

// Method 1

String url = urlFor(ConfirmRegistrationPage.class,

new PageParameters("id=" + registrationId));


// Method 2

add(new BookmarkablePageLink("link", ConfirmRegistrationPage.class));

In order to extract a complete list of possible URLs a Wicket application serves for its usersit would be necessary to scan the source code of the targeted web application and look forurlFor(...) and new *link(...) commands.

Besides the already mentioned web application frameworks, libraries used to secure Java web ap-plications also provide configuration files containing declaratively added information about existingURLs a particular web application serves or rather URLs that have to be protected by the said securitylibrary as seen in Listing 2.2. Usually these configuration files do not contain a complete list of all theavailable URLs a particular web application serves but rather only the ones worth protecting from thepublic like an administrator interface.

• Spring SecurityAs seen in Section 2.4.2 the Spring Security library is being configured through a XML file called*-security.xml and might contain a non complete list of URLs that have to be protected aswell as information about existing users and roles as Listing 2.13 shows.

• ESAPI WAFThe ESAPI web application firewall as seen in Section 2.4.4 is being configured through a XMLpolicy file shown in Listing 2.19 and might contain a non complete list of URLs that are beingserved by the targeted web application and have to be protected by the web application firewall.

• Apache ShiroSimilar to the ESAPI WAF the Apache Shiro project (see Section 2.4.5) might provide a con-figuration file containing relative URLs that are being served to the users of the targeted webapplication but have to be protected by the Apache Shiro library as shown in the INI config-uration file in Listing 2.20. Eventually, if the developer uses the roles filter chain then theconfiguration file also contains information about existing users and roles.

• jGuardThe example configuration file of jGuard shown in Listing 2.21 also contains a non completelist of relative URLs with declaratively added information about which roles are being allowedfor a particular resources in the web application.

After a white box security testing tool parsed the desired configuration files of a Java web applicationframework or a library used to secure a Java web application the following information is availableto the testing tool: a (non) complete list of the URLs the targeted web application serves and a (non)complete list of existing users and roles the web application knows about. With the mentioned whitebox information extracted from the configuration files, the following approaches become feasible todetect URL access restriction vulnerabilities or access control misconfigurations more precisely thana sole black box security testing tool:

• Enhancing Spidering Coverage: Spidering or crawling a web application is usually the firststep a black box security testing tool performs prior to launching routines for finding vulner-abilities in the targeted web application. A web crawler is one type of bot or software agent.In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits theseURLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit.Black box security testing tools usually want only one URL to start their spidering process, e.g.:


http://www.example.com. How many pages and resources are then discovered by the spi-der solely depends on how well the spider is implemented and how well the spider componentinteracts with the targeted web application. This includes filling out forms and interpretingdynamic scripting languages such as JavaScript. The coverage (how many different pages thespider is able to find) can be increased by providing the spidering routine with a larger amountof seeds to start with. These seeds can be provided by parsing the previously mentioned con-figuration files or source code of the targeted web application. This largely reduces the issuethat a black box security testing application has, namely identifying pages respectively URLs toattack for finding unprotected resources.

• RBAC Policy Checks: Given the previously mentioned configuration files not only contain a(non) complete list of URLs but also information about which users and roles are allowed toaccess a particular resource (or a collection of URLs) it would be possible for a combined white-and black-box security testing tool to conduct role based access control policy checks. Givensuch a security testing tool would be able to impersonate each existing and configured userrole known to the web application, a map of the web application could be created by usingan automated spidering component recording a list of all the URLs a particular user role hasbeen granted access to and the allowed transitions between HTML pages. This would allowa developer or testing agent to easily spot URLs respectively pages which should have beenprotected by an access control mechanism but are nonetheless accessible to user roles withlower access rights due to a missing access control check or due to a misconfiguration.

4.8 Insufficient Transport Layer Protection

Applications frequently do not protect network traffic. They may use SSL/TLS during authentication,but not elsewhere, exposing data and session IDs to interception. Expired or improperly configuredcertificates may also be used. Detecting basic flaws is easy by observing the site’s network traffic.More subtle flaws require inspecting the design of the application and the server configuration.Such flaws expose individual user data and can lead to account theft. If an administrator accountwas compromised, the entire site could be exposed. Poor SSL setup can also facilitate phishing andMan-In-The-Middle (MITM) attacks.

Configuration files with transport layer constraints include the following:

• Application ContainerThe deployment descriptor might contain an optional <user-data-constraint> element.A user data constraint establishes a requirement that the constrained requests be receivedover a protected transport layer connection. This guarantees how the data will be transportedbetween client and server. The choices for type of transport guarantee include NONE, INTEGRALand CONFIDENTIAL. If no user data constraint applies to a request, the container must acceptthe request when received over any connection, including an unprotected one. Listing 2.8shows an example deployment descriptor with a user data constraint that protects all URLswith the prefix /catalog/cart/* through SSL/TLS connections.

If the web application rather uses Java annotations than a deployment descriptor then the sameinformation can be extracted by parsing the source code of the Java web application. The an-notations used to configure access constraints equal to the ones configured in the deploymentdescriptor are called @ServletSecurity and @HttpMethodConstraint as seen in Section2.2.5 and Listing 2.11. This is only true if the web application uses container managed secu-


rity features and the constraints are thus being configured through Java annotations inside thesource code of the Java web application.

• Struts 1.xStruts 1.x does not offer a way to add declarative transport constraints in any form to an existingweb application. The only way to add non container managed transport layer constraints is towrite a custom RequestProcessor as described in Section 2.7.1.

• Struts 2.xFor the Apache Struts 2.x Framework (see Section 2.8) there exists a project called struts2-ssl-plugin9 which goal it is to provide a plugin to enforce HTTPS constraints through Java anno-tations. The following code snippet shows how to use the plugins @Secured annotation toenforce HTTPS connections for a particular Struts Action:

@Secured

public class IndexAction extends ActionSupport {

plubic String execute() {

// ...

return SUCCESS;

}

}

• SpringSpring based applications which use the Spring Security project contain a configuration file asshown in Listing 2.13. By adding requires-channel="https" to an URL constraint, SpringSecurity enforces that the data shared between the server and the user is being protectedthrough SSL/TLS.

• WicketThe Apache Wicket framework (see Section 2.10) also offers a Java annotation called@RequireHttps which can be used by a developer to annotate page classes. The frameworkwill then enforce that these pages can only be accessed through the HTTPS protocol.

• ESAPI WAFThe ESAPI WAF XML configuration file offers a <enforce-https> element for specifying a listof URLs which can only be accessed through HTTPS, otherwise the WAF will redirect standardHTTP traffic to HTTPS. More on ESAPI can be found in Section 2.4.4. The following excerptshows how to completely make a web application accessible over HTTPS only with the excep-tion of the index.html page and the images folder:

<url-rules>

<enforce-https path="/.*">

<path-exception>/index.html</path-exception>

<path-exception type="regex">/images/.*</path-exception>

</enforce-https>

</url-rules>

• Apache ShiroThe Apache Shiro configuration file as seen in Section 2.4.5 provides a SslFilter chain whichenforces HTTPS connections for a specific set of URLs.

9http://code.google.com/p/struts2-ssl-plugin/ [11/12/10]


http://code.google.com/p/struts2-ssl-plugin/

After a white box security testing tool parsed the desired configuration files of a Java web applicationframework or a library used to secure a Java web application, the following information is available tothe testing tool: a (non) complete list of the URLs the targeted web application serves and a completelist of URLs that are being protected over SSL/TLS. With the mentioned white box information ex-tracted from the configuration files, the following approaches become feasible for detecting missingtransport layer constraints:

• Transport Layer Policy Checks: Checking whether all or some specific pages are properly se-cured through SSL/TLS is easily done by spidering the targeted web application and comparingthe pages which are only accessible through SSL/TLS and those who are not with a pre-definedlist of URLs that should be protected by SSL/TLS. This would allow a developer/administratorto quickly spot misconfigurations in the way resources are being protected by the HTTPS pro-tocol. Such a pre-defined URL list can either be provided by an administrator/developer or canbe collected automatically from various configuration files as seen in Section 4.7. Whether spi-dering the targeted web application in order to find unprotected resources is really necessaryor if it is sufficient to only parse the configuration files with transport layer constraints has tobe evaluated later if necessary.

4.9 Unvalidated Redirects and Forwards

Applications frequently redirect users to other pages or use internal forwards in a similar manner.Sometimes the target page is specified in an unvalidated parameter, allowing attackers to choosethe destination page. Detecting unchecked redirects is easy by finding POST or GET parameterscontaining a full URL or parameters where one can set a full URL. Unchecked forwards are harder tofind since they target internal pages. Such redirects may attempt to install malware or trick victimsinto disclosing passwords or other sensitive information. Unsafe forwards may allow a malicioususer to bypass access controls.

Unvalidated redirects and forwards can easily be detected with sole black box security testingtools by analysing GET and POST parameters and looking for full URLs in parameters such ashttp://www.example.com/redirect.php?url=http://www.example.com/page2.php.

Additionally to using a sole black box security testing tool as mentioned above, an AOP approachcan also be used for detecting unvalidated redirects and forwards. Given the web application usesthe void sendRedirect(String) method to programmatically issue redirects, an aspect couldbe used to check whether the parameter passed to the sendRedirect(String) function can besupplied by a malicious user. The sendRedirect(String) method is specified in the Servlet APIin javax.servlet.http.HttpServletResponse. Again, the only prerequisite is that the exist-ing web application needs to be recompiled with AspectJ to incorporate the newly proposed aspectswhich analyse the passed parameters to the sendRedirect(String) method.

4.10 Summary

This Chapter tried to identify possible ways to enhance the detection of security vulnerabilities inweb applications with an automated security testing tool by incorporating declarative security infor-mation. The OWASP Top Ten project has been used to identify the most common web applicationvulnerabilities and for each one an appropriate solution has been proposed to enhance the detectionof such vulnerabilities by using declarative information.


The following common web application vulnerabilities have been covered in this Chapter: InjectionFlaws, Broken Authentication and Session Management, Insecure Direct Object References, SecurityMisconfigurations, Insecure Cryptographic Storage, Failure to Restrict URL Access, Insufficient Trans-port Layer Protection and Unvalidated Redirects and Forwards. Not all of these vulnerabilities canbe enhanced by incorporating declarative security information in an automated white box securitytesting tool such as Broken Authentication and Session Management, Security Misconfigurations,Unvalidated Redirects and Forwards, Insecure Direct Object References and Insecure CryptographicStorage.

Detecting Broken Authentication and Session Management vulnerabilities can only be spotted by amanual code review and there are no declarative security information in the configuration or sourcecode files of a targeted web application which could be used by an automated security testing tool.

Detecting Security Misconfigurations would be feasible if the approach would be narrowed down to afew specific cases where declarative security information is available in the targeted web applicationsuch as detecting weak cryptographic algorithms in SSL certificates used by the application containerbut there are already very good tools available to do these kind of checks (see Nessus, OpenVAS andopenssl).

Detecting Insecure Cryptographic Storage issues is another case where there is really no availabledeclarative information available in the configuration- or source code files of a targeted web applica-tion and can only be resolved by a manual code review and thus is being ignored in this thesis.

Detecting Unvalidated Redirects and Forwards issues can be rather easily spotted by an automatedblack box only security testing tool that spiders the targeted web application and can not really beenhanced by using white box information sources and thus is being ignored in this thesis.

Detecting Insecure Direct Object References issues is also a candidate for an automated black boxonly security testing tool and the common Java web application frameworks do not offer declarativeinformation regarding direct object references. A black box security testing tool could easily detectinsecure direct object references by analysing URLs for repeated patterns between multiple URLssuch as a repeated user id page.php?userid=5, page.php?userid=6 found through spidering thetargeted web application.

On the other hand issues such as Injection Flaws, Insufficient Transport Layer Protection and Failureto Restrict URL Access qualify for using white box information sources to enhance the precision anddetection of security threats in an automated fashion in Java web applications.

Injection Flaws can be detected more precisely by incorporating web application framework specificvalidation configuration files such as configuration files used by the Apache Commons Validatorproject or by scanning the Java source code of the web application for framework specific validationJava annotations. The automated security testing tool would then be able to find GET or POST

parameters by spidering the targeted web application which have not been mentioned in one ofthe frameworks validation routines which would suggest an Injection Flaw. This is certainly a verypromising approach because Injection Flaws are still the most prevalent vulnerabilities found in webapplications. The WhiteHat Website Security Statistic Report (Fall 2010) estimates that Cross-SiteScripting (71%) and Information Leakage (70%) remain by far the most prevalent vulnerabilitiesoccurring in 7 out of 10 websites10. The biggest challenge of this approach is the statical code

10http://www.whitehatsec.com/home/resource/stats.html [10/01/11]


http://www.whitehatsec.com/home/resource/stats.html

analysis for detecting unvalidated GET and POST parameters. The problem is that the frameworksdo not offer one way to achieve object population and validation checks but rather several and itbecomes difficult to distinguish between Java variables which fulfil a particular purpose in the codeor are being provided by a user through a HTML form. Most web application frameworks such asStruts 1 & 2 and Spring automatically call Java setter methods in the Java classes associated with theHTML forms to populate the Java objects with the values entered by a user in the displayed HTMLinput elements. A static code scanning routine would have to know what to look for in the sourcecode of the application depending on the used method. This rules out the detection of parametervalidation code done through third party libraries such as ESAPI and the like and the approachcan only be successful if the provided validation means of the framework are being used by thedeveloper. If this approach is at all achievable in a feasible and tangible way is hard to tell butwould nevertheless provide a testing agent or developer with relevant and interesting informationabout possible Injection Flaws. Even by using a crawling component to identify all available HTMLforms respectively GET and POST parameters, the code or the configuration files would have to bechecked whether the found parameters are being validated or not. This would simplify the problemonly if all the validation rules and constraints have been configured in a single configuration file,otherwise the statical code analysis would still be needed to make educated decisions whether aparameter is being properly validated or not. Spring is certainly the most complicated frameworkin respect of the many ways parameter validation can be achieved and Struts 1.x is certainly one ofthe easier ones. Limiting the implementation to Struts 1.x only might not be a good idea becauseit would significantly reduce the usefulness of the security testing tools because newer Java webapplications should not be developed using the relatively old Struts 1.x web application frame-work. In order to implement this approach the web application would have to meet several criteriasuch as used libraries and methodologies in parameter validation in order to guarantee useful results.

Transport layer constraints can be configured in various ways such as in the deployment descriptorof an application container, through web application framework specific configuration files orframework specific Java annotations and thus qualifies for the use in an automated white-boxsecurity testing tool to detect Insufficient Transport Layer Protection issues. The security testing toolwould be able to spider the targeted web application and record non SSL/TLS secured resourceswith SSL/TLS protected resources which would help a developer to spot misconfigurations and evendiscrepancies in configured transport layer constraints and actually deployed constraints.

Failure to Restrict URL Access issues can be detected through the use of white box information sourcescoming from specific web application framework configuration files or the actual source code of thetargeted web application in combination with a black box spidering component. This would allowa security testing tool do discover missing access control checks and misconfigurations in the wayaccess constraints are being enforced throughout the Java web application. By spidering the targetedweb application by impersonating user roles known to the web application a map of all resourcesa particular role has been granted access can be created. This map allows a developer or testingagent to easily spot misconfigurations in enforced access constraints and even discrepancies betweenintended access constraints and the actually enforced constraints in the deployed web application.


5 Finding Privilege Escalation Vulnerabilities byusing Declarative Information

The following Chapter finalises the main idea of this thesis, namely detecting privilege escalationvulnerabilities and misconfigurations in role based access control checks (RBAC) in web applicationsby using declarative information provided by web application framework specific configuration files.After analysing the current research in the field of enhancing web application security through theuse of declarative information, as outlined in Chapter 3, did not reveal any suitable idea worth con-tinuing for this thesis and evaluating possibilities of finding common web application vulnerabilitiesthrough the use of declarative information in Chapter 4, the idea of doing role based access controlpolicy checks seems to be the most valuable for a developer or testing agent (see Section 4.7). FindingInjection Flaws through the use of declarative information as described in Section 4.2 would also bevery beneficial for a developer or testing agent but this topic, especially the detection of SQLi vulner-abilities by using an automated as well as a combined white- and black-box security testing tool, hasalready been discussed and implemented by Denver [12].

5.1 Differences between RBAC and ABAC

Before the actual idea of this thesis is outlined in Section 5.2 it might be necessary to make adistinction between role based access controls (RBAC) and attribute based access controls (ABAC)because the proposed approach is currently only suited for web applications enforcing role basedaccess constraints.

The main difference between RBAC and ABAC is that ABAC also considers the object respectivelydata on which a subject wants to perform any kind of operation. Typical web application frameworksdo not offer the ability to configure ABAC policies, they are instead providing RBAC policies based onURL constraints such as only allowing a user to access certain URLs based on his role (see Sections2.8 for an example).

There are quite a few Java libraries/frameworks for parsing and enforcing XACML policy files suchas: Sun’s SunXACML project and PicketBox XACML. XACML is a declarative access control policylanguage implemented in XML and a processing model, describing how to interpret the policies.Instead of only using a role as the determining factor whether to grant access or not, many attributescan be used. Roles can be used in ABAC as well - since ABAC can use multiple attributes to makeaccess control decisions, so ABAC can emulate RBAC perfectly while adding many additionaladvantages. This means that it is possible to add context to the access control decisions and adds fora finer granularity, tighter controls and more flexibility for the business. With XACML, policies canbe woven around a combination of subjects (users/user-agents etc), resources (on which the accesscontrol is desired) and environment (IP address, Date, Time etc). For more information about thistopic see Section 3.1.

Web applications which do not use a XACML based approach usually enforce ABAC policies pro-grammatically. Another way of enforcing ABAC policies, without using the XACML specification,

Chapter 5. Finding Privilege Escalation Vulnerabilities by using Declarative Information | 76

can be achieved by implementing a custom AccessController provided by the ESAPI project (seeSection 2.4.4). The AccessController interface defines a set of methods that can be used in a widevariety of applications to enforce access controls. In most applications, access control checks mustbe performed in multiple different locations across the various application layers.

Finding privilege escalation vulnerabilities in web applications using ABAC policies is not a trivialtask, especially when the policies are being programmatically enforced because of the wide variety ofpossible ways to implement such checks. An alternative way of doing completeness checks for ABACpolicies involves a great deal of configuration effort for a testing agent. Because statical code analysisis infeasible, due to the complexity of programmatically enforced ABAC checks, a lot of additionalwhite box information has to be provided by a testing agent such as when a particular resource isbeing protected based on a time constraint etc. and is thus being ignored for this thesis.

5.2 Detecting Privilege Escalation Vulnerabilities (RBAC)

Privilege escalation is the act of exploiting a configuration oversight to gain elevated access toresources that are normally protected from a user. The result is that a user with more privileges thanintended by the application developer can perform unauthorised actions. This corresponds with oneof the OWASP Top Ten points Failure to Restrict URL Access described in Section 4.7.

The idea of this approach is to find misconfigurations in the way users respectively user roles havebeen granted access to specific parts of the web application. This boils down to a completeness checkand allows a developer or testing agent to easily spot misconfigurations if given a list (or graphicalrepresentation such as a directed graph) of all the URLs and page transitions which are accessibleto a specific user role (e.g.: GUEST, MANAGER, etc.). Any mismatch between declared access controlrules and actual reachability is flagged as a security vulnerability. The testing tool would be able todetermine if some parts of the web application can not be reached at all or if there are ways to reach aspecific page that requires high access privileges via any path that allows access with lower privileges.

As described in Section 4.7 the hardest part for a malicious user is identifying which pages respectivelyURLs exist to attack. An attacker, who is an authorised system user, simply changes the URL to aprivileged page. If there is a misconfiguration the access is granted to the attacker even if he doesnot provide the necessary user role to access that specific page respectively URL. Additionally, tobroaden the approach for a combined white- and black box security testing approach the developedtool should be aware of all the available users respectively roles the targeted web application knowsabout in order to check all available URLs with each available user role for privilege escalation attacks.

5.2.1 Extractable Declarative Information

The necessary information is being declaratively added by a developer during development to webapplication framework specific configuration files, deployment descriptors or even to the source codeof the web application through Java annotations. The following Sections describe for each popularJava web application framework: (1) which configuration or source code files contain a (non) com-plete list of available URLs and (2) which configuration or source code files contain a (non) completelist of user roles. Some of the configuration files outlined here have already been discussed in Section4.7 but are nonetheless repeated for the sake of completeness.


Application Container

The deployment descriptor as seen in Section 2.2.2 and Listing 2.8 contains valuable informationabout security constraints which are being enforced by the container and are being configuredbased on URLs. These configured URL patterns give a first overview over the overall structure of thetargeted web application. Important to note is that these configured URL patterns in the deploymentdescriptor represent constraints and might not contain a complete list of all available URLs theweb application serves to its users. Additionally, the mentioned security constraints might containinformation about which user roles are allowed access to configured URLs. Again, these are onlysecurity constraints and might not contain a complete list of all the available users.

If the web application rather uses Java annotations than a deployment descriptor then the sameinformation can be extracted by parsing the source code of the Java web application. The annotationsused to configure access constraints equal to the ones configured in the deployment descriptor arecalled @ServletSecurity as seen in Section 2.2.5 and @DeclareRoles as seen in Section 2.2.3.

All the above mentioned information sources only apply if the web application uses container man-aged security features and the constraints are thus being configured either through Java annotationsor through the deployment descriptor.

In case the web application is being deployed inside an Apache Tomcat Web Application Server thenan additional configuration file called $CATALINA_BASE/conf/tomcat-users.xml might containa (non) complete list of existing users and user roles. The tomcat-users.xml configuration file isbeing referred to as a UserDatabaseRealm in Tomcat’s documentation1. UserDatabaseRealm is animplementation of the Tomcat Realm interface that uses a JNDI resource to store user information.By default, the JNDI resource is backed by a XML file. It is not designed for large-scale production use.At startup time, the UserDatabaseRealm loads information about all users and their correspondingroles, from an XML document. The users, their passwords and their roles may all be editing dynam-ically, typically via JMX. Changes may be saved and will be reflected in the XML file. The followingsnippet shows an excerpt from a tomcat-users.xml configuration file containing three differentusers:

<tomcat-users>

<user name="tomcat" password="tomcat" roles="tomcat" />

<user name="role1" password="tomcat" roles="role1" />

<user name="both" password="tomcat" roles="tomcat,role1" />

</tomcat-users>

Struts 1.x

As seen in Section 2.7 the Struts Web Application Framework is being configured through a XML filecalled struts-config.xml. This particular configuration file contains Action mappings which haveto be configured by a developer in order to use the web application framework at all. These mappingsmap relative URLs to Java classes implementing the desired functionality of the developer. By parsingthe struts-config.xml configuration file a complete list of all the declaratively configured URLs atargeted web application serves to its users can be extracted. The struts-config.xml configura-tion file might even contain declarative information about existing user roles and privileges based onthe roles element as seen in Section 2.7.1.

1http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#UserDatabaseRealm [08/04/11]


http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#UserDatabaseRealm

Struts 2.x

Similar to Struts 1.x, the Struts 2.x Web Application Framework provides the same informationthrough a XML file called struts.xml as seen in Section 2.8 and Listing 2.32. Additionally to theXML configuration file, Struts 2.x offers Java annotations to configure the same information insidethe source code of the web application. By parsing the source code of the targeted web applicationthe following annotation is relevant @Result as seen in Listing 2.31.

Spring

Spring, similar to Struts 1.x and Struts 2.x, provides declarative information about Action mappingswhich maps relative URLs to particular Java classes implementing the desired functionality by the de-veloper through the use of Java annotations. The specific annotations are called @RequestMapping

as seen in Section 2.6 and can be extracted by a security testing tool to gather a complete list of all theURLs a targeted web applications serves to its users.

5.2.2 Crawling the Web Application

Now that all declaratively added data is available to the security testing tool such as an (non)complete list of all available URLs and a (non) complete list of all users and user roles a devel-oper or testing agent might have to manually complete the extracted information in order toreceive a more precise test result. This is not at all necessary but guarantees that the targeted webapplication is more thoroughly tested by checking all the available URLs with each available user role.

An automated web crawling component is now able to create a map of all the resources a particularuser is being granted access to by the web application respectively which access controls are beingenforced by the web application for a particular user on a particular resource. In order to get acomplete picture of the targeted web application, the spidering process has to be done n times,depending on how many different user roles are defined in the web application. This of courseimplies that the spidering component is capable of acquiring the various found user roles, forexample through a HTML based authentication form, and might have to be configured manuallyby a developer or testing agent if the information could not be extracted from various source orconfiguration files mentioned in Section 5.2.1. Additionally, the behaviour of the web spideringcomponent has to be slightly modified in contrast to standard spiders which are most commonlyused by black box security testing tools. The main difference between a sole black box security testingspider as seen in w3af is that discovered URLs are only tested once for common vulnerabilities suchas Injection Flaws but in the here described approach it is very important to consider the used pathand the enforced access control restrictions along this path to finally retrieve a discovered URL. Itis utterly important to make this approach more thorough because it might be possible that thereare ways to reach a specific page which requires high access privileges through a specific path thatallows access to the same page with lower privileges and to make this approach more thorough allthe discovered paths and resources have to be tested by the spidering component with all availableuser roles.

Lets consider the following programmatically enforced RBAC policy: A user u with role r is allowedto access page A, B and C through the transition seen in Definition 5.1.

Au r−→ B

u r−→C (5.1)

The provided white box information also contains a user b with role s which is programmaticallyallowed to access page C through the transition seen in Definition 5.2.


Abs−→C (5.2)

The goal of the spidering component is to find deviations for the programatically enforced RBACpolicies by trying to visit the pages A, B , C as seen in Definition 5.3.

Abs−→ B B

bs−→C Bbs−→ A C

bs−→ B (Cbs−→ A) (5.3)

The transition Cbs−→ A which is the inverted version of the allowed transition A

bs−→ C might or mightnot pose a security threat to the web application.

The drawback of this approach is that it is only as precise and thorough as the spidering coverage.The coverage of the spidering component depends on how well the spider is implemented and howwell the spider component interacts with the targeted web application. This includes filling outforms and interpreting dynamic scripting languages such as JavaScript. If the spidering componentis not able to detect certain parts of the web application because it has to follow a specific work flowsuch as checking out from an e-commerce site where specific actions have to be made in a certainorder to reach a specific part of the web application, then these parts respectively resources are notbeing tested for privilege escalation vulnerabilities. So it is important to make sure that the spideringcomponent reaches a high coverage and one way to achieve this is by using declarative informationfound in the configuration files of the web application.

There exist two ways to achieve a higher spidering coverage by using declarative information foundin configuration files of the web application framework:

• Seeding the Web Spidering Component: The spidering component can be seeded with a (non)complete list of all the available URLs known to the web application by parsing the mentionedconfiguration files in Section 5.2.1. These seeds reduce the issue that common black box se-curity testing tools have, namely finding valid URLs to attack and they as well increases thecoverage of the spidering component because the component is aware of valid and existingURLs and is able to query them.

• Providing Test-Data: The idea is to provide the spidering component with a list of valid datawhich has to be entered into common HTML forms around the web application and is be-ing accepted by the business logic of the targeted web application. This would guarantee thatthe automated spidering component is not blocked out of certain parts of the web applicationbecause of business logic constraints such as checking out of an e-commerce site where thespidering component would have to provide a valid credit card number in order to reach thecheckout page. These kinds of information can not be automatically extracted from source orconfiguration files but rather have to be provided by a developer or testing agent in some kindof machine readable format such as a XML file.

With these gathered information it is already possible to create a virtual model of the web applica-tion in form of a weighted directed graph whereas the weight correlates to the user role used to reacha particular resource. Tung et al. [34] already discussed this idea in their paper called A Novel Ap-proach to Automatic Test Case Generation for Web Applications. They identified that a virtual modelof a web application (a directed graph) has several dependencies such as data- and control depen-dencies. Data dependencies for web applications hold when a statement in a page defines variables,which are then used as a statement in another page. Based on the shared scope of variables of datadependencies, these can be classified into application-, session- or request-scope. Access controlconstraints, whether the user is in possession of the proper user role to access a given page, falls into


the category of control dependencies. Furthermore, this model can be very helpful for the spideringprocess for identifying and removing cycles by using graph algorithms such as Tarjan’s Algorithm [32].Additionally, it can be used to ensure that all available nodes and edges are being visited by the spiderby using graph algorithms such as Breadth-first- or Depth-first.

5.2.3 Evaluation of the collected Data

After the spidering of the targeted web application is complete all the necessary information is avail-able to analyse the collected data and spot privilege escalation vulnerabilities and misconfigurationsin access control checks. There exist three possible ways to detect privilege escalation vulnerabilitiesand misconfigurations in access control checks from the collected data:

• Comparing configured Access Controls with actual deployed Access Controls: Given the ac-cess control rules have been configured by a developer through the use of source code annota-tions or configuration files as described in Section 5.2.1 and Section 2.2.2 it becomes possibleto detect discrepancies between configured access controls and actual access control checks inthe deployed web application in an automated fashion. If no access control checks have beenconfigured in the source code of the application or in the deployment descriptor it might benecessary for a developer to provide these informations manually in order to use this detectionmethod in an automated fashion. Lets consider a configured access control constraint whichspecifies that only the user with an ADMINISTRATOR role is allowed to access URLs match-ing the following pattern: /admin/*. If the spidering component, while using a user withlower privileges, has been granted access to the following resource /admin/vulnerable.do

clearly qualifies as a discrepancy between configured and actual deployed access restrictionsand would be flagged as a misconfiguration in access control checks or even a privilege escala-tion vulnerability.

• Outlier Detection: An outlier detection algorithm such as OutRank [25] which is a stochasticgraph-based outlier detection framework for detecting outliers in data using random walkscan be used. At the heart of the algorithm lies a Markov chain model that is built upon a graph,which assigns an outlier score to each object. By transforming the adjacency matrix of thegraph into transition probabilities, they model the problem as a Markov chain process and findthe dominant eigenvector of the transition probability matrix. The values in the eigenvectorare then used to determine the outlierness of each object.

Using such an algorithm would help finding privilege escalation vulnerabilities in an auto-mated fashion. Consider an administration panel in a web application which should only beaccessible to the user with an ADMINISTRATOR role. Due to a missing access control checka single page inside the administration panel is accessible to a user with lower privileges.By using the above mentioned OutRank algorithm it should become feasible to identify thatparticular resource with a missing access control check as an outlier in the administrationpanel.

If and how good this outlier detection works with real life data would have to be evaluated first.

• Visually: The collected data could be visually rehashed as a directed graph whereas nodes cor-respond to found URLs and edges represent transitions between URLs. By using clusteringalgorithms and re-colouring the collected data from the spidering component as described in[8] and [22] it would become possible for a developer or testing agent to quickly spot outliersvisually.


Depending on the declaratively information available to a combined white- and black-box securitytesting tool capable of detecting privilege escalation vulnerabilities and missing access control checksthe approach can be used in a more automated fashion and this qualifies for integration into an au-tomated security testing framework or continuous integration/testing tool.

5.3 Summary

This Chapter described the core approach of this thesis, namely detecting privilege escalation vul-nerabilities or misconfigurations in access control checks through the use of declarative informationin web application source- or configuration files.

The approach consists of three phases: (1) the extraction of declarative information from the deploy-ment descriptor and the web application framework’s source- or configuration files. (2) Spideringthe deployed web application with an automated spidering component capable of acquiring all theuser roles known to the web application. (3) Analysing the collected data for privilege escalationvulnerabilities or misconfigurations in access control checks.

Phase 1: Extracting Declarative Information: The first phase extracts declarative information outof existing source- or configuration files of the targeted web application. The source code has to beavailable to the proposed security testing tool. This phase is tailored to the used web applicationframework used by the developer such as Struts 1.x, Struts 2.x, Spring, Wicket, etc. The following datais being extracted, if available: a (non) complete list of available URLs for seeding the web spideringcomponent used in phase 2, a (non) complete list of available users and user roles, a (non) completelist of existing access constraints. A developer or testing agent might have to complete the collecteddata if he wants to have better test results but at least he has to provide the necessary information onhow the web spidering component can acquire the existing user roles (e.g.: through a HTML basedauthentication form).

Phase 2: Crawling the Web Application: The spidering phase consists of an automated web crawlingcomponent which spiders the deployed and targeted web application. The spidering phase is donemultiple times depending on how many different users have been found and configured in phase 1.The spider tries to create a complete map of the existing access controls in place and thus followseach available path and tries to access all found resources with each available user role.

Phase 3: Evaluation of the collected Data: Phase 3 analyses the collected data from phase 2 andis able to spot privilege escalation vulnerabilities or misconfigurations in access control checks inan automated fashion either by comparing configured access constraints with the actual deployedaccess constraints or by using an outlier detection algorithm. Additionally, if not enough informationcould be extracted from the web application’s source- or configuration files, the collected datais being rehashed as a directed graph and by using clustering algorithms and re-colouring of thecollected data the developer or testing agent is able to visually spot outliers indicating privilegeescalation vulnerabilities or misconfigurations in access control checks.


6 Design

This Chapter finalises the design for the chosen approach, namely detecting privilege escalation vul-nerabilities by using declarative information as described in Chapter 5. The design for the mentionedapproach ought to be extensible and enough generic to be used with various web application frame-works such as Struts, Spring, etc. Additionally, this Chapter contains the design for the underlyingstorage solution as well as UML diagrams for the Java classes which have to be developed.

6.1 Framework

The basic architecture of the proposed security testing framework can be seen in Figure 4.1. Theautomated testing tool has to be configured (1) by an developer or testing agent for the specific webapplication. The automated security testing tool needs access to the actual source code of the webapplication (2) in order to extract further data which helps it to craft meaningful test cases. After allthe information has been collected the actual process of finding misconfigurations and applicationvulnerabilities begins (3). A human readable report for the testing agent should be created. The to bedeveloped security testing framework consists basically of four layers as seen in Figure 6.1. The layersare being described shortly (from bottom to top) in the following listing:

• Meta Layer: The Meta Layer gathers the declarative information provided by a particular webapplication framework or developer and transforms the gathered data into a general purposerepresentation. This makes sure that the security testing framework stays generic enough andcould be extended by providing a new parser for a particular web application framework andits configuration files.

• Web Crawler: The crawler uses the collected data from the Meta Layer XML file and performsa spidering of the targeted web application. Based on the gathered data, the crawler can per-form its task more from a black- or white box perspective depending on how much declarativeinformation could be extracted from the web application’s configuration files or on how muchadditional information has been provided by a developer or testing agent.

• Analysis: The Analysis Layer evaluates the collected data from the crawler and the Meta Layerand tries to spot misconfigurations in the way the targeted web application enforces RBAC poli-cies. Some of the analysis can be done in an automated fashion and some will have to be donemanually and visually by the developer or testing agent.

• Presentation: The Presentation Layer summarises the findings visually or by creating a hu-manly readable report.

The framework will in the end consist of three separate applications: (1) A command line toolfor extracting declarative information out of the source- and configuration files of a targeted webapplication by using the Meta Layer XML file format. (2) A web crawler component which will usethe previously created Meta Layer XML file to map the deployed web application and its accessrestrictions by performing several crawling attempts with varying principals configured in the MetaLayer XML file. The spidering will be more black- or white-box based depending on how muchdeclarative information could be extracted. This might include test data for filling out HTML forms

Chapter 6. Design | 83

Figure 6.1: Framework Layers

and usernames and passwords for various principals. (3) A graphical user interface for visually rep-resenting access restrictions and page transitions to the testing agent. The graphical user interfacewill combine the Analysis Layer as well as the Presentation Layer and offers the testing agent severalview modes to easily spot possible privilege escalation vulnerabilities by presenting the collecteddata as a directed graph. These view modes include clustering of the mapped web application basedon URL paths, clustering based on principals, re-colouring of single nodes based on their attributesand highlighting differences between two users by creating a diff based on their access restrictions.

These three tools will provide a developer or testing agent with a framework for mapping accessrestrictions of a targeted web application in an automated fashion. By introducing the MetaLayer XML file this approach can be extended and enhanced for any number of web applicationframeworks and programming languages. This thesis focuses on web applications written in theJava programming language but can be easily extended for applications written in Ruby or PHP.Depending on how much information the developer is willing to provide in the Meta Layer XML file,the identification of possible security threats and misconfigurations can be done more automatedand by reusing a previously set up Meta Layer XML file it is possible to do regression tests onthe targeted web application. This is especially interesting for integrating the proposed securitytesting tool into a continuous integration tool such as Jenkins1 (formerly known as Hudson) or anautomated security testing framework such as ASTF. Even if the developer or testing agent did notprovide any information for the Analysis Layer he will be able to visually and manually verify theimplemented access restrictions in the targeted web application by using the graphical user interface.

The novelty of the described approach lies in the mapping and visual representation of enforcedaccess restrictions by web applications in an automated fashion with the possibility to find privilegeescalation vulnerabilities by using declarative information found in source- or configuration files.

The four layers are being explained in more detail in Sections 6.2, 6.4, 6.7 and 6.8.

6.2 Meta Layer

The Meta Layer decouples web application framework specific parsing logic, which is needed for ex-tracting declaratively extracted information, from the rest of the security testing framework. This

1http://jenkins-ci.org/ [19/03/11]


http://jenkins-ci.org/

decoupling keeps the security testing framework generic and extendible. The Meta Layer providesother layers (as seen in Figure 6.1) with data, such as a (non) complete list of available URLs for seed-ing the web crawling component described in Section 6.4, a (non) complete list of available users anduser roles, a (non) complete list of existing access constraints, etc., gathered from source- or config-uration files of a targeted Java web application. The gathered data is transformed by the Meta Layerinto a general purpose format which is independent of the underlying web application frameworkthe information originated from. Being more specific, the Meta Layer needs to provide interfaces forcollecting information out of web application frameworks and classes for storing respectively repre-senting the collected data. A general web application framework parser might consist of two com-ponents: a module for statically parsing the Java source code and a module for parsing frameworkspecific XML configuration files. One way to achieve the decoupling from framework specific logicand the actual security testing tool is to define a general purpose XML structure which can be writtenby the framework specific parsers. Using a pre-defined XML structure to provide the Meta Layer withthe collected data has several advantages as opposed to automatically loading the collected data intoa set of pre-defined database tables: 1) A testing agent would be able to manually edit the generatedXML file to make adjustments to the collected data if needed; 2) the data could be easily passed on tofuture security testing tools.

6.2.1 Meta Layer XML Structure

The following Section describes the layout of the previously mentioned general purpose XMLstructure which is used to present collected web application framework specific information to theMeta Layer. The XML file starts with a <web-meta> tag and the following tags such as <info> areenclosed within. The XML Document Type Definition (DTD) for the Meta Layer XML file can befound in Appendix B. Note: not all proposed XML tags in the Meta Layer XML file are needed fordetecting privilege escalation vulnerabilities but are rather being added for completeness becausethey are easily extractable and might be useful for future tools.

The minimum required fields in order for the approach to work are the: <users>, <roles> and<login> tag.

<info> Tag

The <info> tag contains some general information about the created Meta Layer XML file such ascreation date and a version string of the tool which created the XML file. Additionally there exists thepossibility for the developer or testing agent to add some notes. Listing 6.1 shows an example for the<info> tag.

1 <info >

2 <created >Fri Jan 28 15:30:16 2011</ created >

3 <timestamp >1296225011 </ timestamp >

4 <createdBy >Spring2FrameworkParser v1.0</createdBy >

5 <notes >

6 ...

7 </notes >

8 </info >

Listing 6.1: Meta Layer XML Structure: <info> Tag


<classes> Tag

The <classes> tag contains a list of all the found Java classes in the source directory of the web ap-plication. A class is added to the XML structure by appending a <class> tag. The id attribute is laterbeing used in the XML structure to make references to a particular class. The id uniquely identifiesa particular class. The mentioned id attribute is being automatically generated by incrementing aninteger starting by 1. Listing 6.2 shows an example of the <classes> tag.

1 <classes >

2 <class id="1" name="com.app.LoginController" />

3 <class id="2" name="com.app.WelcomeController" />

4 ...

5 </classes >

Listing 6.2: Meta Layer XML Structure: <classes> Tag

<users> Tag

The <users> tag is being used to define users which have access to the targeted web application.A <user> tag can have an arbitrary number of <attribute> tags to add more values to a userobject. Adding more attributes to a user object has to be done by the developer or testing agentand is not extractable from declarative information provided in source- or configuration files of thetargeted web application. The id attribute is used to uniquely identify a user and the id is being usedthroughout the XML structure to make references to a particular user as seen in Listing 6.4. Listing6.3 shows an example of the <users> tag.

The optional <disable-form-submit> tag can be added to a <user> or <role> tag to instrumentthe spidering component to disable HTML form submits during the discovering of new URLsfor a certain URL path. This is usually a good idea when the spidering component accesses theadministration panel of a targeted web application, otherwise the spidering component might deleteusers or change passwords when trying to find new URLs by automatically filling out HTML forms asdescribed in Section 6.4.5.

1 <users >

2 <user id="1" name=" Alice">

3 <attribute name=" username" value="alice11435 -6" />

4 <attribute name=" password" value=" wairaehoov7A" />

5 <attribute name="email" value=" [email protected]" />

6 ...

7 <disable -form -submit pattern ="/ administrator /*" />

8 </user >

9 <user id="2" name="Bob">

10 <attribute name=" username" value="bob11447 -9" />

11 <attribute name=" password" value=" UYa9wung7ooy" />

12 <attribute name="email" value=" [email protected]" />

13 ...

14 </user >

15 ...

16 </users >

Listing 6.3: Meta Layer XML Structure: <users> Tag


<roles> Tag

The <roles> tag contains a list of all the found user roles in the configuration files or source code ofthe web application. The priority tag has to be set by the developer or testing agent, labeling theroles based on their abilities, meaning the role with the highest authority starts with a 1 and all otherroles with less abilities are classified lower. This means that the administrator role usually is the rolewith the highest clearance and should be labeled with a priority attribute 1. This information islater being used by the web crawler component to identify parts of the web application which userswith lesser privileges should not be allowed access to. The child node <user-id-ref> is used tomap defined users to available user roles. Users are being defined through the <users> tag as seenin Listing 6.3. Listing 6.4 shows an example of the <roles> tag.

The optional <disable-form-submit> tag is described in Section 6.2.1. If the<disable-form-submit> tag is being added to a <role> tag instead of a <user> tag thenthe directive is being passed on to all users belonging to this particular role.

1 <roles >

2 <role id="1" name=" ADMINISTRATOR" priority ="1">

3 <user -id-ref >1</user -id-ref >

4 <disable -form -submit pattern ="/ administrator /*" />

5 </role >

6 <role id="2" name=" MANAGER" priority ="2">



9 ...

10 </role >

11 ...

12 </roles >

Listing 6.4: Meta Layer XML Structure: <roles> Tag

<access-restrictions> Tag

The <access-restrictions> is an optional tag used to specify URL patterns that are being re-stricted and should only be accessible for a particular role or a group of roles similar to the constraintsfound in a deployment descriptor. The information has to be manually provided by a testing agent ordeveloper if it could not be extracted from declarative information found in source- or configurationfiles. If the tag is present in the Meta Layer XML file the Analysis Layer can spot discrepancies betweenactual deployed access restrictions and configured access restrictions automated and more eas-ily (as described in Section 5.2.3). Listing 6.5 shows an example of the <access-restrictions> tag.

1 <access -restrictions >

2 <url pattern ="/ login.do" access =" permitAll" />

3 <url pattern ="/ country /*">

4 <role -id-ref >1</role -id-ref >


6 </url >

7 <url pattern ="/ admin/*">


9 </url >

10 </access -restrictions >

Listing 6.5: Meta Layer XML Structure: <access-restrictions> Tag


<login> Tag

The <login> tag has to be completely specified by the developer or testing agent and can not beextracted from declarative information. The tag is being used to describe how users are being au-thenticated by the targeted web application and is essential for the spidering process to acquire theconfigured user roles. The spidering component uses the provided information to automatically spi-der the web application with various users and roles. Listing 6.6 shows an example on how to con-figure a form based authentication scheme through the <form-based-login> tag for the followingHTML form:

<form id="userLoginForm" method="POST">

<input type="text" id="inputLoginUsername" />

<input type="password" id="inputLoginPassword" />

<input type="hidden" id="x_1" />

<input type="submit" value="Sign in" />

</form>

The <form-url> tag (Listing 6.6, Line 3) points to the URL which contains the HTML login form.The <form-name> tag contains the name or the id of the form which specifies the input elementsfor the username and password. The <form-name> tag is used to identify the form which is neededfor the login process of the web application and is important if the HTML page contains multipleHTML forms. This might be the case if the page contains a login form for users who already have anaccount and on the same page exists a form for user registration. The <data> tag contains the datathat should be filled into the form. The data is being represented in the JavaScript Object Notation(JSON)2 format. The data can be configured in the following ways:

• Static: {"<HTML Element Name or Id>":"<Value>", ...}

This is the standard way of configuring data which should be filled into HTML elements. Thekey is the name or id of the HTML element and the second is the actual data that should befilled in: {"inputLoginUsername":"[email protected]", ...}.

• Dynamic: {"<HTML Element Name or Id>":"##<User Attribute Name>##", ...}

This is the dynamic way of configuring the data which should be filled into HTML elements.The key is the name or id of the HTML element and the second is the name of a previously con-figured user attribute through the <user> tag as seen in Listing 6.3. This allows the developeror testing agent to use the users email as username instead of his username by configuring thedata in the following way: {"inputLoginUsername":"##email##", ...}.

The <submit-element> is used to specify the element that a user would have to click in order tosubmit the form. This does not have to be necessarily a HTML submit element but could also bean image or any other kind of HTML element. The <submit-element> can either contain an id ora name of a HTML element or the value of a HTML element provided in the HTML source throughthe value="..." attribute. If no <submit-element> is being specified, the login module tries tofind the correct item to click by searching the HTML source code through an XPath Query for HTMLelements that can be used to submit a form.

1 <login >

2 <form -based -login >

3 <form -url >/login.do </form -url >

4 <form -name >userLoginForm </form -name >

2http://www.json.org/ [29/01/11]


http://www.json.org/

5 <data >{" inputLoginUsername ":"## username ##",

6 "inputLoginPassword ":"## password ##", "x_1 ":"5"} </data >

7 <submit -element >Sign in </submit -element >

8 </form -based -login >

9 </login >

Listing 6.6: Meta Layer XML Structure: <login> Tag

<urls> Tag

The <urls> tag is used to provide a list of available URLs which have been extracted out ofconfiguration files or the source code of the web application. These URLs can then be used bythe spidering component as seeds in order to increase the coverage. The id parameter is beingautomatically generated by incrementing an integer starting at 1. Optionally, an URL can have anadditional <class-id-ref> parameter. This additional parameter can be used to make a referenceto the implementing class of that particular URL. Especially the Spring and Struts web applicationframeworks provide these kind of information in their Action mapping settings. Listing 6.7 shows anexample of the <urls> tag.

1 <urls >

2 <url id="1" path ="/ login.do">

3 <class -id-ref >1</class -id -ref >

4 </url >

5 <url id="2" path ="/ welcome.do" />

6 <url id="3" path ="/ editUser.do" />

7 ...

8 </urls >

Listing 6.7: Meta Layer XML Structure: <urls> Tag

<views> Tag

The <views> tag is optional and is used to list all the available files that contain the actual HTML,JSP etc. view generating code of a web application. Additionally it is possible to add two optionalattributes called <class-id-ref> and <url-id-ref>. <class-id-ref> is used to connect a viewfile with the Java class who uses that particular view for displaying information to the user. The<url-id-ref> tag is used to make a connection between a view and the actual URL that returnsthat particular view to the users browser. Listing 6.8 shows an example of the <views> tag.

1 <views >

2 <view path ="/ login.jsp">


4 <url -id-ref >1</url -id-ref >

5 </view >

6 <view path ="/ welcome.do" />

7 <view path ="/ editUser.do" />

8 ...

9 </views >

Listing 6.8: Meta Layer XML Structure: <views> Tag

<form-data> Tag

The <form-data> tag is also an optional tag but can be used by the developer or testing agent toprovide the spidering component with valuable test data that guarantees page transitions and thus


increases the spidering coverage immensely as described in Section 5.2.2. Without such data thespidering component would fill randomly generated numbers, strings or email addresses into HTMLforms (see Section 6.4.5). The data has to be provided in the exactly same way as described in the<logins> tag and shown in Listing 6.6. Listing 6.9 shows an example of the <form-data> tag.

1 <form -data >

2 <form >

3 <form -url >/ selectCountry.do </form -url >

4 <form -name >selectCountry </form -name >

5 <data >{" ctry ":" United Kingdom "}</data >

6 <submit -element >Select </submit -element >

7 </form >

8 <form >

9 <form -url >/ selectLanguage.do </form -url >

10 <form -name >selectLanguage </form -name >

11 <data >{" lang ":" German "}</data >

12 <submit -element >Select </submit -element >

13 </form >

14 ...

15 </form -data >

Listing 6.9: Meta Layer XML Structure: <form-data> Tag

Meta Layer API

Figure 6.2 shows a simplified UML diagram of the Meta Layer XML API. The particular classes forstoring and setting the actual extracted information from the web applications source and configura-tion files has been omitted in Figure 6.2 for simplicity reasons. The following code snippet shows theusage of the Meta Layer API for writing a new Meta Layer XML file with the name metalayer.xml

containing a <class> and <url> element:

MetaLayerWriter writer = MetaLayerFactory.getWriter();

Class class = writer.createClass();

class.setName("org.apache.struts.app.validator.CityStateZip");

Url url = writer.createUrl();

url.setPath("/exercise/welcome.do");

try {

writer.saveTo(new File("metalayer.xml"));

} catch (MetaLayerWriterException e) {

e.printStackTrace();

}

Reading a Meta Layer XML file is as easy as writing a Meta Layer file. The following code snippetillustrates the process:

MetaLayerReader reader = MetaLayerFactory.getReader();

try {

MetaLayerConfig config = reader.parse(new File("metalayer.xml"));

} catch (MetaLayerReaderException e) {

e.printStackTrace();

}


Figure 6.2: MetaLayer API

6.3 Framework Specific Parsers

With the Meta Layer XML structure in place it is easy to write parsers for specific web applicationframeworks that collect all the available declarative information and provide it to the actual securitytesting tool. This approach makes the actual security testing tool generic and it is easy to extendthe approach to different Java or even non Java web application frameworks as long as the parsersproduce a well formed Meta Layer XML file. Depending on the particular web application frameworksuch a framework specific parser might consist of two components: a module that is capable ofstatically analysing the source code respectively the Java classes of the web application and a modulethat is able to parse framework specific XML configuration files. Such a framework parser then writesa valid and well formed Meta Layer XML file containing all the found declarative information. Thedeveloper or testing agent is then able to add additional information to the Meta Layer XML file suchas test data (configured through the <form-data> tag as seen in Section 6.2.1) etc. before startingthe actual tests by providing the Meta Layer XML file to web spidering component of the securitytesting tool.

Figure 6.3 shows a simplified UML diagram of a framework specific parser capable of parsing webapplication configuration files and deployment descriptors. The concept is that every parser im-plements the FileListParser interface with the parse(List<File> fileList) method andthen receives a list of all the files contained in the root- and sub-directories of the user providedpath to the web application source and configuration files. The parser is then able to pick thefiles out of the provided list that it was designed to parse and add the extracted information to theMetaLayerConfig class which generates the Meta Layer XML file after all the classes implementingthe FileListParser interface have been run.

6.3.1 Struts 1.x Example

The following Section describes a framework specific parser, implemented as a Java command lineapplication, on the example of parsing source and configuration files of a Struts 1.x web application.Two command line parameters are going to be needed, namely: (1) path to the root directory of theStruts 1.x web application that contains all the source and configuration files and optionally (2) thepath to the output file e.g. /home/userX/metaLayer.xml. Internally the framework specific parser


Figure 6.3: Framework Specific Parser UML

consists of several modules capable of parsing WEB-INF/web.xml configurations and interpretingparticular Struts 1.x source and configuration files such as WEB-INF/struts-config.xml.

For Struts 1.x the following parser modules exist:

• Deployment Descriptor Configuration Parser: This is actually not Struts 1.x specific be-cause the web.xml deployment descriptor file is written in a standardised format as de-fined in the JSR-000053 Java Servlet 2.3 and JavaServer Pages 1.2 Specifications3. From thedeployment descriptor useful information such as configured user roles can be extractedand added to the Meta Layer <roles> tag. Role information can usually be found in the<security-constraint> tag embedded in the deployment descriptor as seen in Section2.2.2. An anonymous user called ANONYMOUS is always and automatically added to the MetaLayer XML file when parsing a web application.

• Struts Configuration Parser: Struts 1.x configuration files include all struts-config.xmlfiles that can be found in sub directories of the user provided root directory of the web ap-plication. A single Struts 1.x Java web application can have multiple struts-config.xml

configuration files for each sub-directory respectively sub URL. This means a Struts 1.x ap-plication which is reachable through the following URL: http://www.example.com/myApp/has at least one struts-config.xml configuration file in the WEB-INF folder. Additionally itis common for a developer to split the Struts 1.x configuration file up into smaller chunks foreasy maintainability and readability. Parts of the web application which for example can bereached by a user through the following URL: http://www.example.com/myApp/catalogwould then be configured through the struts-config.xml configuration file located in theWEB-INF/catalog/ directory. A framework parsing application for Struts 1.x web applicationshas to consider this fact and collect the necessary information according to the configured sub-paths. Struts 1.x configuration files contain the following data which can be extracted (as beingdescribed in Section 2.7):

– Struts Actions: Struts Actions contain relative URLs, the class which is responsible forhandling them and the name of the JSP file that contains the view respectively the HTML

3http://jcp.org/aboutJava/communityprocess/first/jsr053/index.html [28/02/11]


http://jcp.org/aboutJava/communityprocess/first/jsr053/index.html

code used for displaying the page to the user. Relative URLs mean that the user config-ures the URL relative to the executing context of the Struts 1.x application, e.g. a con-figured Action in the struts-config.xml file located in the WEB-INF/catalog foldermight contain an action such as <action path="show.do" ...>. In order to actuallyexecute the configured Action through typing an URL into a browser would be the fol-lowing: http://www.example.com/myApp/catalog/show.do. These found URLs arebeing added to the <urls> section of the Meta Layer configuration as mentioned in Sec-tion 6.2.1.

– Struts Forwards: The same rules apply for Struts Forwards as for Struts Actions. StrutsForward configurations also contain relative URLs which can be extracted and used bythe web crawler as seeds.

– Views: Struts Actions and Struts Forward configuration elements might contain a param-eter indicating which JSP file should be loaded when the user requests a particular URLthrough his browser. This information is not being used by the security testing frameworkbut might be handy for future tools. The extracted views are being added to the <views>tag in the Meta Layer XML file as described in Section 6.2.1.

• Source Code Parser: The source code parser module is responsible for generating a list of all thefound Struts 1.x classes with their particular package names. This information is not being usedby the proposed security testing framework but might come in handy for future tools buildingup on the Meta Layer XML file.

– Class List Generator: The class list is being generated by recursively visiting all sub foldersof the initial user provided root directory of the Struts 1.x Java web application.

– Package Name Extractor: The package names are being extracted from each found*.java plain source code file by using the parser provided by the Checkstyle project.Checkstyle uses a sophisticated parser to transform the Java code into an abstract syntaxtree (AST). An abstract syntax tree is a tree representation of the abstract syntactic struc-ture of source code written in a programming language. Each node of the tree denotes aconstruct occurring in the source code. The syntax is “abstract” in the sense that it doesnot represent every detail that appears in the real syntax. For instance, grouping parenthe-ses are implicit in the tree structure and a syntactic construct such as an if-condition-thenexpression may be denoted by a single node with two branches.

The found information is then appended to the <classes> tag in the Meta Layer XML file asseen in Section 6.2.1.

Meta Layer XML File for a Struts 1.x Application

Listing 6.10 shows a shortened version of the generated Meta Layer XML file from parsing a Struts1.x application without any additional added information from a developer or testing agent. Thescanned application is the example Struts 1.x application struts-1.3.10-apps.zip which can bedownloaded from the Struts 1.x project site.

The generated Meta Layer XML file contains <info>, <classes>, <users>, <urls> and <views>

tags populated with the information extracted from the various source and configuration files foundin the root- and sub-directories of the provided Java Struts 1.x web application folder. As mentionedbefore, the <classes> tag is not being used by the security framework and is merely there forcompleteness. An anonymous user is being added per default to the <users> tag and a testing agentwould have to enhance the Meta Layer XML file with further information about existing users. The<urls> tag contains a list of all the found Struts Actions and Struts Forwards configured in the found


struts-config.xml files. This list is then added as starting seeds to the web spidering componentas described in Section 5.2.2 and Section 6.4.

1 <?xml version ="1.0" ?>

2 <web -meta >

3 <info >

4 <created >Tue Mar 1 13:2:21 2011</ created >


6 <createdBy >Struts 1.x Parser </createdBy >

7 <notes ></notes >

8 </info >

9 <classes >

10 <class id="1"

11 name="org.apache.struts.app.dispatch.ActionDispatcherExample "/>

12 <class id="2"

13 name="org.apache.struts.app.dispatch.DispatchExampleAction "/>

14 <class id="3"

15 name="org.apache.struts.app.dispatch.EventActionDispatcherExample "/>

16 <class id="4"

17 name="org.apache.struts.app.dispatch.EventDispatchActionExample "/>

18 <class id="5"

19 name="org.apache.struts.app.dispatch.LookupDispatchExampleAction "/>

20 <class id="6"

21 name="org.apache.struts.app.dispatch.MappingDispatchExampleAction "/>

22 <class id="7"

23 name="org.apache.struts.app.examples.CustomActionForm "/>

24 <class id="8"

25 name="org.apache.struts.app.examples.CustomActionForward "/>

26 <class id="9"

27 name="org.apache.struts.app.examples.CustomActionMapping "/>

28 <class id="10"

29 name="org.apache.struts.app.examples.CustomFormBean "/>

30 <class id="11"

31 name="org.apache.struts.app.exercise.HtmlSettersAction "/>

32 <class id="12"

33 name="org.apache.struts.app.exercise.ImageAction "/>

34 <class id="13"

35 name="org.apache.struts.app.exercise.SuccessAction "/>

36 <class id="14"

37 name="org.apache.struts.app.exercise.TestBean"/>

38 <class id="15"

39 name="org.apache.struts.app.upload.UploadAction "/>

40 <class id="16"

41 name="org.apache.struts.app.upload.UploadForm "/>

42 <class id="17"

43 name="org.apache.struts.app.validator.CityStateZip "/>

44 <class id="18"

45 name="org.apache.struts.app.validator.EditTypeAction "/>

46 <class id="19"

47 name="org.apache.struts.app.validator.LocaleAction "/>

48 <class id="20"

49 name="org.apache.struts.app.validator.MultiRegistrationAction "/>

50 <class id="21"

51 name="org.apache.struts.app.validator.RegistrationAction "/>

52 <class id="22"

53 name="org.apache.struts.app.validator.RegistrationForm "/>

54 <class id="23"

55 name="org.apache.struts.app.validator.ShowFileAction "/>

56 <class id="24"

57 name="org.apache.struts.app.validator.TypeAction "/>


58 <class id="25"

59 name="org.apache.struts.app.validator.TypeForm"/>

60 <class id="26"

61 name="org.apache.struts.actions.ForwardAction "/>

62 </classes >

63 <users >

64 <user id="1" name=" ANONYMOUS" />

65 </users >

66 <roles ></roles >

67 <access -restrictions ></access -restrictions >

68 <login ></login >

69 <urls >

70 <url id="1" path ="/ dispatch/welcome.do" />

71 <url id="2" path ="/ dispatch/dispatch.do" />

72 <url id="3" path ="/ dispatch/dispatch -submit.do" />

73 <url id="4" path ="/ dispatch/dispatch -noparam.do" />

74 <url id="5" path ="/ dispatch/dispatchLocale.do" />

75 <url id="34" path ="/ exercise/welcome.do" />

76 <url id="35" path ="/ exercise/bean -write -2.do" />

77 <url id="36" path ="/ exercise/html -cancel.do" />

78 <url id="37" path ="/ exercise/html -cancel -true.do" />

79 <url id="38" path ="/ exercise/html -cancel -false.do" />

80 <url id="64" path ="/ welcome.do" />

81 <url id="65" path ="/ upload/upload.do" />

82 <url id="66" path ="/ upload/upload -submit.do" />

83 <url id="67" path ="/ validator/viewBundleExamples.do" />

84 <url id="68" path ="/ validator/validateBundleExamples.do" />

85 <url id="69" path ="/ validator/viewJsBundleExamples.do" />

86 <url id="70" path ="/ validator/validateJsBundleExamples.do" />

87 <url id="71" path ="/ validator/showStrutsConfigBundles.do" />

88 </urls >

89 <views >

90 <view path ="/ dispatch/index.jsp">

91 <url -id-ref >1</url -id -ref >

92 </view >

93 <view path ="/ dispatch/dispatch.jsp">


95 </view >




99 </view >




103 </view >

104 </views >

105 <form -data ></form -data >

106 </web -meta >

Listing 6.10: Meta Layer XML File generated from a Struts 1.x Sample Application

6.4 Web Crawler

This Section describes the open source HtmlUnit4 project and the benefits it provides for spideringweb applications automatically. Additionally, the design and performance of the web spider compo-nent, which has been developed as part of a previous thesis [10] and builds upon HtmlUnit, are being

4http://htmlunit.sourceforge.net/ [02/02/11]


http://htmlunit.sourceforge.net/

discussed and visualised.

6.4.1 HtmlUnit Overview

HtmlUnit is a “GUI-Less browser for Java programs” and is released under the Apache LicenseVersion 2. It models HTML documents and provides an API that allows developers to invoke pages,fill out forms, click links, etc... just like one does in a “normal” browser. The JavaScript support(which is constantly being improved) is fairly good and is able to work even with quite complex AJAXlibraries, simulating either the Mozilla Firefox or Microsoft Internet Explorer web browser. HtmlUnitis typically used for testing purposes although HtmlUnit is not a generic unit testing framework suchas JUnit for Java. It is especially a way to simulate a browser for testing purposes and is intended tobe used within another testing framework such as JUnit or TestNG5.

HtmlUnit uses the Mozilla Rhino engine6 to provide JavaScript support. Rhino is an open source im-plementation of JavaScript written entirely in Java. Rhino is an implementation of the core JavaScriptlanguage only and does not contain objects or methods for manipulating HTML documents. Themethods for manipulating HTML documents is instead provided by HtmlUnit. Rhino contains (asof version 1.6): All the features of JavaScript 1.7, allows direct scripting of Java, a JavaScript shellfor executing JavaScript scripts, a JavaScript compiler to transform JavaScript source files into Javaclass files and a JavaScript debugger for scripts executed with Rhino. The JavaScript language itself isstandardised by Standard ECMA-262 ECMAScript7: A general purpose, cross-platform programminglanguage. Rhino 1.3 and greater conform to Edition 3 of the Standard.

JavaScript support is essential for web spider components which want to crawl today’s web ap-plications thoroughly. There are no absolute trends about the use of JavaScript in common webapplications. Some users have scripting turned off, some browsers do not support scripting. How-ever W3C’s browser statistics show that 95% of all browsers on the Internet have JavaScript supportenabled (as of January 2008)8. It is safe to say that there are more web applications being built usingsome kind of JavaScript functionality than there are applications completely avoiding the use ofJavaScript.

The goal of web spider components especially embedded in automated web application securityrelated applications is to find and unveil as many HTML pages containing attack vectors as possible.Web spiders without JavaScript support such as the webSpider plugin of w3af would return anempty HTML document in case of stumbling upon the HTML page shown in Listing 6.12. Thisresults in a reduced accuracy because pages of the targeted web application which rely on extendedJavaScript functionality are never being checked for security related issues in an automated scan.

By using the HtmlUnit API and its JavaScript support it is relatively easy to write a multi threadedweb spider module which is able to cope with modern web applications using extended JavaScriptfunctionality for their web applications. The next Section shows how HtmlUnit is being used in asimple scenario: to retrieve a HTML page with embedded JavaScript, fill out some HTML text inputfields and in order to actually submit the form by using the HtmlUnit API to click on the submitbutton.

5Taken from http://htmlunit.sourceforge.net/[05/04/10]6http://www.mozilla.org/rhino/ [05/04/10]7http://www.ecma-international.org/publications/standards/Ecma-262.htm [05.04.10]8http://www.w3schools.com/browsers/browsers_stats.asp [11/06/10]


http://htmlunit.sourceforge.net/

http://www.mozilla.org/rhino/

http://www.ecma-international.org/publications/standards/Ecma-262.htm

http://www.w3schools.com/browsers/browsers_stats.asp

6.4.2 Submitting a HTML Form with HtmlUnit

The API of HtmlUnit is very accommodating for filling out and submitting HTML forms. Listing 6.11shows a sample Java method which retrieves a HTML page (the source code of the retrieved HTMLpage is shown in Listing 6.12) through a proxy and populates the different HTML text input fields.The received HTML page contains a login form which has been dynamically created by a JavaScriptroutine (see Listing 6.12, lines 5-15).

HtmlUnit (as of version 2.8) is able to simulate the following browsers: Microsoft Internet Explorerversions 6, 7 and 8, Netscape and Mozilla Firefox versions 2 and 3. Selecting a specific browserversion changes the HTTP user agent header which is being sent by the HtmlUnit API. Additionallysome specific JavaScript instructions are being interpreted differently based on the selected browserversion9. The specific browser version is being set upon creation of the HtmlUnit WebClient object(see Listing 6.11, Line 4).

Retrieving a HTML page is done by calling the WebClient.getPage("http://www....");

method as seen in Listing 6.11 on line 13. The HtmlUnit library retrieves the HTML page lo-cated at the given location and silently loads any additional resources such as Cascading StyleSheets (CSS) and JavaScript source files embedded with the <script type="text/javascript"

src="remote.js"></script> directive from the remote host.

HtmlUnit now uses the Mozilla Rhino engine to interpret all the downloaded JavaScript instructionsand changes the HTML document structure accordingly (if necessary). The mentioned getPage()

method returns a Java HTML page object which now contains the finished rendered HTML pagewhich now can be used for further processing.

The HTML page object can now be used to manipulate the contents of the retrieved HTML page. Togive an impression of what functionality HtmlUnit is offering through its API, the following list con-tains some of the more interesting methods which are being offered by the HTML page object (as ofHtmlUnit version 2.8). For a complete documentation of the API visit HtmlUnit’s project homepage:

• List<HtmlAnchor> getAnchors()

Returns a list of all <a href="">...</a> anchors contained in a received HTML page.

• List<HtmlElement> getElementByName(String name)

Returns the HTML element with the specified name.

• HtmlElement getFocusedElement()

Returns the element with the focus or null if no element has the focus.

• List<HtmlForm> getForms()

Returns a list of all the forms in a received HTML page.

• List<FrameWindow> getFrames()

Returns a list containing all the frames (from <frame> and <iframe> tags) in a received HTMLpage.

Some of these functions return a HtmlElement object which is an abstract class provided by Htm-lUnit. This abstract class is being used by other HtmlUnit classes to model HTML elements such as

9http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/BrowserVersion.html

[11.06.10]


http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/BrowserVersion.html

text- or password input fields, radio buttons, checkbox buttons etc. Again a short list of interestingmethods which all HtmlElement objects have in common:

• <P extends Page> P click()

Simulates clicking on this element, returning the page in the window that has the focus afterthe element has been clicked.

• <P extends Page> P dblClick()

Simulates double-clicking on this element, returning the page in the window that has the focusafter the element has been clicked.

• void focus()

Sets the focus on this element.

• Page mouseOver()

Simulates moving the mouse over this element, returning the page that this element’s windowcontains after the mouse move.

• void setAttribute(String attributeName, String attributeValue)

Sets the value of the attribute specified by name.

The example method shown in Listing 6.11 manipulates a HTML text- and a HTML pass-word input element on lines 20-29 by calling the setText() method (which is equivalent tosetAttribute("value", "myString")) and finally submits the filled out form by “clicking” onthe submit button on line 32 by calling the click() method.

The example in Listing 6.11 uses the names of the HTML elements in order to retrieve them fromthe previously retrieved HTML page. This is just one of several ways to retrieve and modify HTMLelements embedded in the HTML page object. Other methods include: retrieving HTML elementsby their id attribute, iterating through a list of all available HTML elements in the HTML page or byusing the XML Path Language (XPath). XPath is a query language for selecting nodes from an XMLdocument. The XPath language is based on a tree representation of a XML document and providesthe ability to navigate around the tree, selecting nodes by a variety of criteria. In popular use (thoughnot in the official specification), an XPath expression is often referred to simply as an XPath.

HtmlUnit’s HtmlPage object allows the developer to retrieve HTML elements embedded in the HTMLpage by issuing XPath queries such as the following code snippet which retrieves all HTML text- andpassword input elements embedded in a HTML page object.

String xPathQuery = "//input[@type=’text’] | //input[@type=’password’]";

List<HtmlElement> nodeList = htmlPage.getByXPath(xPathQuery);

These XPath queries are extensively used in the multi threaded web spider component written forthis thesis (see Section 6.4.4). The XPath queries are being used to extract new URLs pointing topages on the targeted web application which have not been visited in an ongoing web spider run.

1 public void submittingForm () throws Exception {

2 // Creates a new browser object using a proxy server

3 // and simulating Mozilla Firefox version 3

4 final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3 ,

5 "http :// myproxyserver", 8080);

6


7 // Set proxy username and password

8 final DefaultCredentialsProvider credentialsProvider =

9 (DefaultCredentialsProvider) webClient.getCredentialsProvider ();

10 credentialsProvider.addProxyCredentials("proxyUsername", "

myProxyPassword123");

11

12 // Get the first page

13 final HtmlPage page1 = webClient

14 .getPage("http :// www.example.com/login.php");

15

16 // Get the form that we are dealing with and within that form ,

17 // find the submit button and the field that we want to change.

18 final HtmlForm form = page1.getFormByName("loginForm");

19

20 final HtmlSubmitInput button =

21 form.getInputByName("loginButton");

22 final HtmlTextInput textFieldUsername =

23 form.getInputByName("username");

24 final HtmlPasswordInput textFieldPassword =

25 form.getInputByName("password");

26

27 // Change the value of the text fields

28 textFieldUsername.setText("john");

29 textFieldPassword.setText("gaephah6MueD");

30

31 // Now submit the form by clicking the button and get back the second page

32 final HtmlPage page2 = button.click();

33 }

Listing 6.11: Short HtmlUnit Example for submitting a HTML Login Form through a Proxy Server

1 <html>

2 <head><title >Login Form</title></head>

3 <body>

4 <form action="checkLogin.php" name="loginForm">

5 <script type="text/javascript">

6 function writeInputElement(inputType , inputName) {

7 document.write(’<input type="’

8 + inputType

9 + ’" name="’

10 + inputName + ’" />’);

11 }

12 writeInputElement("text", "username");

13 writeInputElement("password", "password");

14 writeInputElement("submit", "loginButton");

15 </script >

16 </form>

17 </body>

18 </html>

Listing 6.12: Dynamically created HTML Login Form with embedded JavaScript Code

6.4.3 HtmlUnit JavaScript Performance

The only drawback of HtmlUnit is its JavaScript execution speed. HtmlUnit is noticeably slower inprocessing downloaded HTML documents when its JavaScript support is enabled than other state ofthe art browsers like Mozilla Firefox and its competitors. In order to measure the performance of theHtmlUnit library a test case from the Mozilla Dromaeo JavaScript Performance Test Suite has been


used10. The test case has been slightly modified for testing the execution speed of the HtmlUnit API.

The selected test case involves modifying the HTML Document Object Model (DOM) by creating andappending several hundred new HTML elements to a retrieved page. The following functions arebeing used by the test case to measure the execution speed of HTML DOM modifications:

• createElement(tagName)

This method returns an Element object. The tagName parameter is of type String. This methodcan raise a DOMException object.

• createTextNode(data)

This method returns a Text object. The data parameter is of type String.

• cloneNode(deep)

This method returns a Node object. The deep parameter is of type Boolean.

• document.body.appendChild(newChild)

This method returns a Node object. The newChild parameter is a Node object. This methodcan raise a DOMException object.

• document.body.insertBefore(newChild, refChild)

This method returns a Node object. The newChild parameter is a Node object. The refChildparameter is a Node object. This method can raise a DOMException object.

• document.body.innerHTML

This variable can be used to modify the rendered content of a HTML document after it has beenfully loaded inside a web browser.

Figure 6.4 shows the relative runtimes in milliseconds (logarithmic scale) between Mozilla Firefox3.6.3, Chromium 6.0.431.0 and HtmlUnit 2.7. In comparison HtmlUnit is almost 4x slower thanMozilla Firefox 3.6.3 and even 19x slower than Chromium 6.0.431.0 in doing HTML DOM modifi-cations. Whereas the creation of new HTML elements is the most time consuming task for HtmlUnit.The test case has been run five times for every browser and the runtimes in Figure 6.4 are the arith-metic means of these times. See Appendix E for a detailed breakdown of the measured runtimes andAppendix F for the code used to measure the JavaScript performance.

6.4.4 Design of the Web Spider Component

The following Sections describe the overall design of the web spider component which has beendeveloped as part of this thesis. The web spider uses the HtmlUnit library described in Sections 6.4.1,6.4.2 and 6.4.3 to use its JavaScript interpreter and the ability to simulate state of the art web browsers.

The main goal throughout the design and implementation of the web spider was to build it:

• Scalable: As seen in Section 6.4.3 the HtmlUnit library is not the fastest when it comes to pro-cessing JavaScript instructions especially with HTML DOM modifications. In order to increasethe crawling speed without disabling JavaScript support the web spider has to be developed ina multi threaded fashion.

• Easily extensible: The web spider needs to be easily extensible if more functionality should beadded to the code base without any major hassle.

10https://wiki.mozilla.org/Dromaeo [11/06/10]


https://wiki.mozilla.org/Dromaeo

Figure 6.4: JavaScript Execution Time (Logarithmic Scale) in various Browsers for HTML DOM Modi-fications

• JavaScript aware: The main advantage over other web spiders is the JavaScript support of theHtmlUnit library. The full potential of the HtmlUnit API regarding JavaScript support should beused.

• Comprehensive: The web spider should be crawling a web application as thoroughly as possi-ble.

• Configurable: The user should be able to alter the behaviour of the web spider through severalparameters which he can set prior to starting the web spider. The configurable parameters arebeing described in Section 6.4.4 and Section 9.1.4.

Appendix C and D contain a simplified UML 2 class diagram of the developed web spider componentand its classes for extracting new URLs.

Configuration Settings

In order to alter the behaviour of the web spider the user can set various settings through commandline parameters. These settings mostly influence how long the web spider will be crawling througha web application before terminating the crawling process and passing the results to other modules.Additionally, a user is able to tell the web spider which resources it either should follow or ignore.

• Setting a maximum crawling level: This setting allows the user to tell the web spider how“deep” into the web application it should crawl. The first page the web spider retrieves fromthe web application is on level 0. Any links embedded in that first page lead to other pageswhich are located on level 1 and so on. So by setting a maximum crawling level n , the webspider will stop crawling any further if it reached level n .

• Setting a follow regular expression: If this setting is being used then the web spider will onlyfollow links matching the given regular expression and will ignore all other links not matchingthe provided expression.

• Setting an ignore regular expression: If this setting is being used then the web spider will crawlevery link it finds but will ignore those that match the provided expression.


• Setting a time limit: The user is able to set a maximum time limit for the spidering process. Theweb spider will stop spidering when the time limit has been exceeded. Of course the spideringprocess will be terminated before the time limit has been exceeded if either the web spider is notable to find any more resources on the targeted web application or the disk space is becominglow.

• Staying on the same domain: Usually the web spider follows any found link embedded in areceived HTML page even if the link leads to a completely different domain than the initialaddress of the targeted web application. For example a web application reachable under thefollowing address http://web1.example.com has an embedded link to a completely differ-ent domain such as http://www.example2.org. This setting tells the web spider if domainsdiffering from the initial domain of the web application should be ignored or not.

• Setting an ignore HTML element regular expression: The user is able to provide a regularexpression matching HTML element name or id tags. The spidering component will ignoreHTML elements whose name or id matches the provided regular expression when trying toautomatically fill out HTML forms. This might be useful when spidering an administrationpanel of a targeted web application, otherwise the spidering component might delete users orchange passwords.

Program Flow

Figure 6.5 shows an UML 2 activity diagram with a birds eye view on the implemented program logicof the multi threaded, JavaScript aware web spider. The steps following the Start Spidering node arebeing executed in multiple threads thus speeding up the whole spidering process. A received HTMLpage is being processed by executing different link extraction routines. These extraction routinesscan the retrieved HTML code for links pointing to previously undiscovered HTML pages of the webapplication. The currently implemented extraction routines include the following:

• A HREF link extraction: This routine simply extracts the src attribute from common HTML<a href="...">...</a> links.

• Frame extraction: This routine scans the retrieved HTML code for HTML <frame

src="..."/> and <iframe src="..."/> tags and extracts the src attribute.

• Mouse event extraction: Scans the retrieved HTML code for onmouseover and onmouseout

tags and invokes the event with HtmlUnit’s JavaScript support. If the event triggers a page redi-rect to a previously undiscovered HTML page, then the page will be added to the web spiderqueue for further processing.

• OnClick extraction: This routine scans the retrieved HTML code for onclick tags and invokesthe event with HtmlUnit’s JavaScript support. If the event triggers a page redirect to a previouslyundiscovered HTML page, then the page will be added to the web spider queue for furtherprocessing.

• OnChange extraction: This routine scans the retrieved HTML code for onchange tags andinvokes the event with HtmlUnit’s JavaScript support. If the event triggers a page redirect to apreviously undiscovered HTML page, then the page will be added to the web spider queue forfurther processing.

• HTML comment extraction: Scans the retrieved HTML code for URLs between HTML com-ments with a sophisticated regular expression. The following example shows a common HTMLcomment and a embedded URL which will be detected by this routine. The identified URLsinside HTML comments are being added to the web spider queue for further processing.


Figure 6.5: Simplified UML 2 Activity Diagram showing the Main-Flow of the Web Spider



• Submitting HTML forms: If the retrieved HTML page contains one or multiple HTML forms,this routine tries to fill out the existing input fields, drop down boxes, radio buttons etc. tosuccessfully submit the form. If the submitted form triggers a page redirect to a previouslyundiscovered HTML page, then the page will be added to the web spider queue for furtherprocessing. For more details on how forms are being submitted and how input fields are beingpopulated see Section 6.4.5.

6.4.5 Smartly filling out HTML Forms

Common HTML forms are widely used in today’s web applications for passing information fromthe user to the web application. A form can contain input elements like text fields, checkboxes,radio-buttons, submit buttons and more. A form can also contain select lists, textarea, fieldset,legend and label elements. Of course there are other technologies such as Adobe Flash, Java Applets,JavaFX, Microsoft Silverlight etc. that can be used to create expressive, feature-rich web applicationuser interfaces. These technologies should not be discussed here further. The web spider plugindeveloped for this project thesis is able to detect HTML forms embedded in retrieved HTML pages.The discovered forms are being analysed and the web spider tries to populate the form elementssuch as text fields, radio-buttons, checkboxes etc. with reasonable data to circumvent any businesslogic restrictions in a non invasive way.

The main difficulty in filling out HTML forms in an automated manner is to circumvent any re-strictions the business logic of the web application might impose on a particular HTML form. Lets

say a user has to create a new user profile in an e-commerce application. The user has to enter hisname, address and phone number in order to use the e-commerce application. Most probably theuser will be shown a HTML form with the necessary input fields such as a text field for entering hisname, address, phone number and e-mail address and a drop down list for entering his birthdayand country. The business logic of the web application probably checks the input fields and helpsthe user in filling out the necessary data. The business logic makes sure that the text field used forentering the users phone number only contains numeric values such as numbers ranging from 0 to9 before accepting the data. The main goal is to guess such restrictions based on various indicatorslike the name of the HTML element and others if no form data is being provided through the MetaLayer XML file as described in Section 6.2.1.

The routine for smartly filling out HTML forms is being implemented as a helper library. How busi-ness logic restrictions are being guessed and what the differences between those two mentionedmodes are will be explained in the following sub Sections 6.4.5.

Non Invasive

Non invasive means that no hidden HTML elements are being changed and no existing values exceptfor text fields are being overwritten. This reflects the behaviour of a normal and friendly user usingthe web application. The following indicators are being used to guess the imposed business logic onHTML elements embedded in a retrieved HTML form:

• Buzzwords: If the name or the id of a HTML element matches a predefined buzzwordsuch as [number, phone, telephone, mobile, zip, postal, day, month, year,

hour, minute, second] then the input field will be populated with a unique random nu-meric value. Other buzzwords such as [mail, email] are being used to identify email ad-dress fields. Unique and random e-mail addresses will be created in the following format[a-zA-Z0-9]@[a-zA-Z0-9].com if the name or id of the HTML element matches the e-mailbuzzword.

• Existing value detection: If the name or the id of a HTML element does not match any prede-fined buzzword the value of the element is being analysed. It might be possible that the inputfield has been pre filled out with an example by the web application developer. Let’s say theuser has to enter his e-mail address and the according HTML element is filled out with an ex-ample such as [email protected]. The routine detects the @ by using regular expressionsand generates a random unique e-mail address. If the value attribute matches the following[0-9]* regular expression a unique random numeric value is being generated. If the valueattribute matches the following [a-zA-Z0-9]* regular expression a unique random alphanu-meric value is being generated. If none of the predefined regular expressions match, a uniquerandom alphanumeric value is being generated per default.

• Max length detection: After a unique random value has been created whatsoever the HTMLelement is being checked for length restrictions. HTML input fields can be limited to a specificsize with the maxlength attribute. If such an attribute exists and the generated value is longerthan the maximum allowed size, the random value is being reduced to not violate the maximumlength restriction.

• Meta Layer: If a Meta Layer XML file has been provided by a developer or testing agent as de-scribed in Section 6.2.1 the spidering component will use the pre-configured data enclosed inthe <form-data> tag to populate a found HTML form. The <form-data> tag is being de-scribed in Section 6.2.1.


Figure 6.6: Simplified UML 2 Activity Diagram for the Non-Invasive-Smartly-Filling-Out HTML FormModule

Figure 6.6 shows an UML 2 activity diagram of the main program flow of the non invasive mode.

6.4.6 Spidering with different User Roles

For the described approach to detect privilege escalation vulnerabilities it is essential for the spider-ing component to spider the targeted web application with different user roles configured through<roles> and <users> tags in the Meta Layer XML file as seen in Section 6.2.1. The HTML pages andresources that can be accessed with the currently used user credentials during the spidering processare being stored as a directed graph in a graph optimised database as later described in Section 6.5.1.Spidering is done several times with each of the configured user roles and it is then the task of theAnalysis Layer to detect any misconfigurations and eventual privilege escalation vulnerabilities aslater described in Section 6.7. The web spider component will be using the provided information inthe Meta Layer XML file to gain access to the various user roles as described in Section 6.2.1 throughthe use of the information configured in the <login> tag.

The spidering is done in three phases whereas phase three is optional:

1. The deployed web application is being spidered with each configured user in the Meta LayerXML file. The users are being sorted on the basis of their Meta Layer XML file configuredpriority tag. Ideally the user with the highest configured priority is an administrator accountwho has access to the complete web application. It is essential that the first user that is beingused to spider the targeted and deployed web application is the one who can access the mostresources in order to increase the thoroughness of the approach. Following spidering phaseshave to rely on the previously discovered resources and try to access them with principals hav-ing lower privileges. If the spidering is done in the following order it ensures that most if not allresources can be verified whether they are accessible by principals with lower privileges. Thefirst spidering phase starts at the command line configured target URL and additionally usesthe configured URLs in the Meta Layer XML file as seeds.


2. The second spidering phase completes the first spidering phase by checking all the resourcesfor accessibility that have been discovered by other users in the first spidering phase but notby the currently used principal. This further enhances the thoroughness and completeness ofthe approach and ensures that all resources are being checked with every configured user. Ac-cessing such resources is not simply done by trying to access the resources directly, rather byfollowing the hole chain from the starting URL to the actual resource which is being requestedby the current principal. This might be necessary if the resource can only be accessed by follow-ing a certain flow of actions that are being enforced by the business logic of the web applicationsuch as a shopping cart checkout system where a user first has to provide his home addressbefore he can enter his payment options and finish the checkout process.

3. The third and last spidering phase is optional and can be enabled by specifying a command lineargument before the spidering process starts. The third phase considers all previously foundresources and tries to access them directly and thus ignoring any kind of underlying businesslogic or flow that might prevent the access. This can be seen as a flavor of Forced Browsing asdescribed here: http://www.owasp.org/index.php/Forced_browsing.

Whether a certain resource is accessible or not is determined based on two factors: (1) A resource isconsidered not accessible by a user with certain privileges if the web server response to a sent HTTP(S)request contains an unusual HTTP status code other than 200 OK in spidering phases 2 and 3 andonly if the user did not provide any regular expression through a command line parameter. HTTPstatus codes indicating an access denied error might include one of the following: 204 No Content,301 Moved Permanently, 302 Moved Temporarily, 400 Bad Request, 401 Unauthorized,etc. A complete list of all available HTTP/1.1 status codes can be found in RFC 2616[15]. (2) The userhas the possibility to specify a regular expression through a command line argument that matchesa given string such as access denied on a custom made error HTML page by the developer. The spi-dering component checks the URL, title and body content of every retrieved response against theprovided regular expression and if it matches the resource is being considered as inaccessible by thecurrently used principal.

6.4.7 Saving the Crawled Data

Figure 6.7 shows the Entity Relationship Model (ERM) used for saving discovered URLs during the spi-dering process. Table 6.2 describes the meaning of the particular fields in the UrlQueue and HttpLogtable. The data contained in these tables are being transferred to a graph optimised database for thelater analysis of the results as described in Section 6.7. The UrlQueue table is mainly used by the spi-dering module as a synchronised queue to keep track of discovered URLs and URLs which have beendiscovered but have not been processed yet. Additionally there exists a HttpLog table which simplylogs all sent HTTP requests for statistic reasons and regardless if the retrieved content are binary files,CSS style sheets etc.

6.5 Working with Graphs

Before the actual Analysis Layer is being designed and described in Section 6.7 it is important to storethe gathered data from the spidering process appropriately and make it accessible to the AnalysisLayer. One way to achieve this is to store the spidered data as a graph on which graph algorithms canbe used to make statements about possible misconfigurations and eventual security vulnerabilities.Graphs can naturally be stored in various ways, for example in a relational database. A more conve-nient way is to use graph optimised databases which usually offer a sophisticated API for working andquerying the data stored in the graph. One such graph optimised database is Neo4j.


http://www.owasp.org/index.php/Forced_browsing

Fiel

dU

rlQ

ueu

e-

Des

crip

tio

nid

Pri

mar

yK

eyurlqueue_id

Th

eid

oft

he

pre

vio

usUrlQueue

entr

yu

sed

tod

isco

ver

this

UR

Llevel

On

wh

ich

leve

lres

pec

tive

lyh

owd

eep

into

the

wep

app

licat

ion

the

UR

Lw

asfo

un

duser_id

Co

rres

po

nd

sto

theuser

id

con

figu

red

inth

eM

eta

Laye

rX

ML

file

url

Th

ed

isco

vere

dU

RL

referer

Th

eU

RL

oft

he

pre

vio

usUrlQueue

entr

yu

sed

tod

isco

ver

this

UR

Ldiscovered_at

Tim

esta

mp

mar

kin

gth

ed

ata

and

tim

eth

isU

RL

has

bee

nd

isco

vere

dfed

true

orfalse

dep

end

ing

on

wh

eth

erth

esp

ider

ing

mo

du

lep

arse

dth

isre

sou

rce

for

furt

her

UR

Lsmissing

true

orfalse

dep

end

ing

on

wh

eth

erth

eU

RL

has

bee

nad

ded

asa

seed

or

ifth

esp

ider

ing

mo

du

led

isco

vere

dit

method

Th

eu

sed

HT

TP

met

ho

dto

retr

ieve

the

HT

ML

reso

urc

e,ei

therPOST

orGET

statuscode

Th

eH

TT

Pre

spo

nse

fro

mth

ese

rver

,e.g

.:200

,404

etc.

access_denied

Wh

eth

erth

ere

ceiv

edH

TT

Pp

age

corr

elat

esto

acu

sto

mm

ade

acce

ssd

enie

dp

age

pre

vio

usl

yco

nfi

gure

db

yth

eu

ser

thro

ugh

aco

mm

and

line

argu

men

t.web_request

Seri

aliz

edJa

vacom.gargoylesoftware.htmlunit.WebRequest

ob

ject

web_response

Seri

aliz

edJa

vacom.gargoylesoftware.htmlunit.WebResponse

ob

ject

Fiel

dH

ttp

Lo

g-

Des

crip

tio

nid

Pri

mar

yK

eycharset

Th

eu

sedcharset

for

the

HT

TP

resp

on

se,e

.g.:UTF-8

method

Th

eu

sed

HT

TP

met

ho

dto

retr

ieve

the

HT

ML

reso

urc

e,ei

therPOST

orGET

url

Th

eU

RL

oft

he

reso

urc

ereferer

Th

eU

RL

oft

he

pre

vio

us

reso

urc

econtenttype

Th

eco

nte

ntt

ype

the

UR

Lp

oin

tsto

,e.g

.:text/html

,image/png

etc.

statuscode

Th

eH

TT

Pre

spo

nse

fro

mth

ese

rver

,e.g

.:200

,404

etc.

statusmessage

Th

em

essa

gese

nta

lon

gth

estatuscode

,e.g

.:OK

,Not

Found

etc.

sent_at

Dat

ean

dti

me

tim

eth

eH

TT

Pre

qu

esth

asb

een

sen

tuser_id

Co

rres

po

nd

sto

theuser

id

con

figu

red

inth

eM

eta

Laye

rX

ML

file

web_request

Seri

aliz

edJa

vacom.gargoylesoftware.htmlunit.WebRequest

ob

ject

web_response

Seri

aliz

edJa

vacom.gargoylesoftware.htmlunit.WebResponse

ob

ject

Fiel

dH

ttp

Lo

gPar

amet

er-

Des

crip

tio

nid

Co

rres

po

nd

sw

ith

theid

inth

eHttpLog

tab

lename

Nam

eo

fth

eat

trib

ute

sen

twit

haPOST

req

ues

tvalue

Th

eac

tual

dat

ase

ntw

ith

thePOST

req

ues

t

Tab

le6.

2:D

escr

ipti

on

oft

heUrlQueue

En

tity

Rel

atio

nsh

ipM

od

el


Figure 6.7: UrlQueue Database Table

6.5.1 Graph Databases

Graph databases are a type of data store which treat the relationship between things as equally impor-tant to the things themselves. Examples of data sets that are natural fits for graph databases: Friendlinks on a social network, “People who bought this also bought...” recommendation engines as seenon Amazon and the world wide web. In most types of databases, the records stored in the databaseare nodes and edges (relationships) are derived from a field on a node. In a SQL database, for exam-ple, you might have a table called “people” that includes a field friend_id. The field friend_id is areference to another record in the people table. The weakness with reference fields becomes apparentas soon as one wants to do many-to-many relationships or store data about the relationship.A person can have many friends; and one might want to track the date the friendship link was created,or whether the two people are married. The solution to this in a SQL database is a join table. In thepeople/friends example, the join table might be called friendships. But this method has someweaknesses. One is that it can greatly increase the number of tables in the database and might makeit hard to tell apart standard tables (nodes) from join tables (edges) - which makes it more difficult fornew developers to comprehend the database architecture. Another problem is that object orienteddatabases, which work quite well for mapping node (model) tables, generally have a much hardertime mapping edges. But the biggest weakness is that queries against relationship data - be it in jointable or a reference link - are extremely unwieldy. In a SQL database it typically leads to recursivejoins, which tend to lead to long, incomprehensible SQL statements and unpredictable performance.A graph database is designed to represent this type of information, so it models the data more nat-urally. It is also designed to query it such as walking the data in a convenient and performant manner.

Neo4j11 is such a graph databases which has been under development for 10 years and in produc-tion for over 7 years. Neo4j is released under a dual free software/commercial license model (whichbasically means that it is open source under the AGPLv3 but if someone is interested in using it withclosed source software, then he will have to buy a commercial license). It is a mature and robust graphdatabase that provides:

• an intuitive graph-oriented model for data representation. Instead of static and rigid tables,rows and columns, the developer can work with a flexible graph network consisting of nodes,relationships and properties.

11http://neo4j.org/ [23/01/11]


http://neo4j.org/

• a disk-based, native storage manager completely optimised for storing graph structures formaximum performance and scalability.

• massive scalability. Neo4j can handle graphs of several billion nodes/relationships/propertieson a single machine and can be shared to scale out across multiple machines.

• a powerful traversal framework for high-speed traversals in the node space.

• can be deployed as a full server or a very slim database with a small footprint (~500KB Java JARarchive).

• a simple and convenient object-oriented API.

In addition, Neo4j includes the usual database features: ACID transactions, durable persistence,concurrency control, transaction recovery, high availability, etc.

Of course there exist several open source and freely available graph databases (for example the Web-Graph Framework [4]) but in comparison Neo4j seemed the most mature and is greatly supported bygraph algorithm libraries such as Gremlin12 (see Section 6.5.2).

6.5.2 Architecture

The following Section shortly outlines the architecture used to work and process graph structures inthe proposed security testing tool. The architecture makes heavy use of open source libraries whichare being described in the following listing:

• Gremlin: Gremlin is a domain specific programming language for graphs 13. It can be used forgraph query, analysis and manipulation.

• Pipes: Pipes is a data flow framework using process graphs. A process graph is composed ofPipe vertices connected by communication edges. A Pipe implements a simple computationalstep that can be composed with other Pipe objects to create a larger computation. Larger com-putations form a process graph (i.e. Pipeline). Such graphs allow for the splitting, merging, andin general, the transformation of data from input to output 14.

• Blueprints: Blueprints is a collection of interfaces, implementations, ouplementations and testsuites for the property graph data model. Blueprints is analogous to the JDBC, but for graphdatabases. As such, it provides a common set of interfaces to allow developers to plug-and-play their graph database backend. Moreover, software written atop Blueprints works over allBlueprints-enabled graph databases 15.

• JUNG Framework: see Section 6.8.

6.6 Saving the Crawled Data in Neo4j

The crawled data is being stored in two places: a relational database and a graph optimised database.The relational database is exclusively being used during the spidering of the targeted web applica-tion. The database acts as a synchronised queue for the spidering component and feeds new threads

12https://github.com/tinkerpop/gremlin/wiki/ [26/01/11]13https://github.com/tinkerpop/gremlin/wiki/ [26/01/11]14https://github.com/tinkerpop/pipes/wiki/ [27/01/11]15https://github.com/tinkerpop/blueprints/wiki/ [27/01/11]


https://github.com/tinkerpop/gremlin/wiki/

https://github.com/tinkerpop/gremlin/wiki/

https://github.com/tinkerpop/pipes/wiki/

https://github.com/tinkerpop/blueprints/wiki/

with not yet processed URLs which have been found in HTML pages through the spidering processdescribed in Section 6.4.7.

How the data is being stored in the graph optimised database is shown in Figure 6.8 which shows theroot node of the graph and two so called reference nodes. Reference nodes are being suggested by theNeo4j Wiki and development guide because having reference nodes for each node type makes lateradding of new nodes and querying much simpler and faster. The edge relationships are being calledPrincipals and Pages for accessing the reference nodes from the root and Principal and Page

for accessing particular pages and user nodes from the reference nodes. The following code snippetshows how the pages reference node can be accessed from Neo4j:

Node pagesReferenceNode = null;

pagesReferenceNode = graphDb.getReferenceNode()

.getSingleRelationship(RelationshipTypes.PAGES,

Direction.OUTGOING).getStartNode();

Pages are linked through Transition relationships, indicating that two particular pages are linkedin the sense that Pagen+1 can be accessed through a link embedded in Pagen . Access restrictionsof a particular user are being stored in the graph by using Has_Access relationships connectingparticular page nodes with a user node for each stored page a user could successfully access.

Additionally to the shown relationships between nodes, nodes can have an arbitrary number of at-tributes. These additional information might be later used to visualise certain aspects of the collecteddata differently. Page nodes represent actual discovered HTML pages respectively discovered URLsduring the spidering process. Along side the actual node the following additional data is being storedin the graph database:

• URL: The actual URL used to retrieve the resource.

• Method: Whether it was a HTTP GET or POST request.

• SSL: Whether the connection was secured using SSL.

Principal nodes represent the configured users from the Meta Layer XML file and principal nodes arebeing stored in the graph with the following additional attributes:

• Name: The name of the principal.

• Id: The id corresponding to the id set in the Meta Layer XML file (see Section 6.2.1).

• Attributes: Arbitrary number of attributes corresponding to the attributes configured in theMeta Layer XML file as described in Section 6.2.1.

Neo4j has no indexing features built-in since the graph structure of the stored data eliminates manyof the needs for indexes existing when using other underlying data models. Simple key based lookupof nodes is however something that is not easily done through the graph structure, instead one has tomanage a lookup index manually. IndexService is a means of providing those indexing capabilitiesfor a Neo4j graph and integrate it as tightly as possible. With an IndexService a developer canassociate any number of key-value pairs to any node and do fast lookups given such key-value pairs.

The Index Service is being used in the security testing framework to find stored page nodes basedon their URL. This enhances the speed of inserting and linking new page nodes greatly. Adding andstoring a value into the IndexService is very easy and is shown in the following code snippet:


Figure 6.8: Storing the crawled data in a graph optimised Database

// Adding to the Index Service

Index<Node> idx = graphDb.index().forNodes("idxPages");

idx.add(pageNode, "url", pageNode.getUrl());

// Retrieving from the Index Service

IndexHits<Node> pageNodeHits = idx.get("url", "http://www.example.com");

pageNode = pageNodeHits.getSingle();

6.7 Analysis Layer

The Analysis Layer is now finally able to make statements about misconfigurations and eventual priv-ilege escalation vulnerabilities by interpreting the collected data from the spidering process that hasbeen stored in the relational- and graph database. The following Sections describe possible ideas formaking sense of the collected data and ways to point out possible security issues.

6.7.1 Comparing crawled Data with the <access-restrictions> Tag

The Meta Layer XML file contains a so called <access-restrictions> tag as being described inSection 6.2.1. If a developer or testing agent added <access-restrictions> data to the Meta LayerXML file it is relatively easy for the Analysis Layer to compare the crawled results against the provideddata and report any mismatches between intended restrictions and actual restrictions found in thedeployed web application. This is done by comparing the URL patterns configured in the Meta LayerXML file and their user role restrictions with the crawled data of the existing users and their rolemembership. The following scenarios could occur:

• A user has access to a resource which is forbidden in the <access-restrictions> tag: Thisscenario clearly states that there is a difference between the developers intentions and the ac-tual deployed web application and is a strong indication that there might exist either an errorin the Meta Layer XML file or that a particular resource can be accessed by a user with a dif-ferent user role than actually intended. This event can be flagged as a possible vulnerability,misconfiguration or security threat.

• A user has no access to an allowed resource in the <access-restrictions> tag: If this eventoccurs it can mean several things: either the Meta Layer XML is outdated and does not corre-spond any more with a newer version of the deployed web application or the user has been


updated to be in a different role than previously configured in the Meta Layer XML file. Thisdoes not necessarily have to be a security threat and would have to be sorted out manuallyby a developer or testing agent. So this event can be flagged as a warning that needs manualinspection.

• A resource has been found which is not mentioned in the <access-restrictions> tag: Thefollowing event also indicates that there is a misconfiguration between the Meta Layer XML fileand the actual deployed web application. The resource might be accessible by all users andhas just been forgotten to be added to the Meta Layer XML file by the developer. Nevertheless,this event should also be flagged as a warning that needs manual inspection by a developer ortesting agent.

In summary this is a really simple and straightforward approach for detecting misconfigurations, pos-sible security threats in a targeted web application given the developer or testing agent keeps the MetaLayer XML file up to date and adds the necessary access restrictions to the <access-restrictions>tag. Shortcomings are clearly the need for specifying these restrictions once for the hole web appli-cation. Depending on how many users, roles and resources the application has, this can be a tedioustask if it could not be extracted from source- or configuration files.

6.7.2 Graph Differences - Clustering by User / User Roles

Clustering algorithms can be used to partition a set of records in our case HTML resources and groupthem in a meaningful order to make it easier for a user to visually detect patterns and eventual mis-configurations and even possible privilege escalation vulnerabilities. Clustering is being described byRamakrishnan as follows:

The goal of a clustering algorithm is to partition a set of records into groups such thatrecords within a group are similar to each other and records that belong to two differentgroups are dissimilar. Each such group is called a cluster and each record belongs to ex-actly one cluster although there are clustering algorithms that allow overlapping clusters,where a record could belong to several clusters [29].

The proposed clustering attribute on how HTML resources are being partitioned will be the principalaccessibility in this analysis method. In comparison with the previously proposed analysis methodin Section 6.7.1 the user has to manually and visually examine the data and clusters to find potentialsecurity threats whereby he can be visually supported by the Presentation Layer as described inSection 6.8.

The following proposed clustering algorithm will consider each found HTML resource and partitionthem based on which user with a specific user role is able to access a particular resource successfully.Considering the web crawler completed the spidering process and checked the available resourcesR1, R2, R3, ..., Rn against three configured users with distinctive user roles U1, U2, U3 the following Clus-ters C might exist:

• A single cluster Cn for each configured user Un containing all the HTML resources Rn that areonly accessible to user Un , e.g.: C1 = {R2, R3, R4, ...}

• A single cluster C0 for each HTML resource that can not be accessed by any configured user,this also includes those resources for which the web server responded with a 404 Not Found

or similar status code.


• Multiple clusters containing those resources that could be accessed by multiple users. Theremight for example exist a cluster C12 containing all the URLs that users with id’s 1 and 2 couldaccess successfully. Furthermore, clusters C123 and C1234 containing the URLs that users withid’s 1, 2, 3 and 1, 2, 3, 4 could access without receiving an access denied exception.

The maximal number of clusters to be expected is dependent on how many users n have been con-figured in the Meta Layer XML file. The maximal number of clusters can be calculated by using theformula in Equation 6.1.

1+n∑

k=1

n !

(n −k )! ·k !(6.1)

The described formula in Equation 6.1 is standard and basic permutation theory [17] plus oneadditional cluster for the resources that could not be accessed by any user. A medium-sizedweb application with n = 4 users all having different user roles would result in a maximum of16 clusters by using the formula in Equation 6.1. This is even visually inconvenient for a user toanalyse and therefore it makes sense to only compare two users at a time resulting in a maximumof 4 clusters and provide the user with an artful interface in the Presentation Layer as described in 6.8.

By clustering the HTML resources according to their accessibility it is relatively easy for a developeror testing agent to see to what kind of resources a particular user has access to and investigate onthose resources that interest him further. Clustering the resources is certainly more convenient thanlooking at the complete graph consisting of all the resources and transitions between the HTMLpages without any kind of colouring or ordering. Clustering based on accessibility is not the onlyattribute available for clustering, it is possible to cluster a web application graph based on URL pathsas described in Section 6.7.3.

Figure 6.9 shows a possible visual representation of the user clustering algorithm. In the example twousers are being compared and thus 4 clusters are being created: a cluster containing the nodes onlyUser 1 can access, a cluster containing all the nodes only User 2 can access, a cluster with the nodesboth users can access and a fourth cluster containing all the nodes neither User 1 and 2 can access.

6.7.3 Edge Analysis - Clustering by URL Paths

Clustering based on the enclosed path of an URL is a convenient way to separate different areas orrather function domains of the web application. The only drawback of this clustering algorithm isthat it only works if the targeted application uses a URL scheme to separate different areas of the webapplication like /admin/* for an administration area. Other web applications might use an URLscheme such as /index.php?s=<NAVIGATION> for its navigation and clustering on URL paths isthen pointless because there are no paths enclosed in the URL to cluster on.

Lets consider a simple e-commerce application accessible through the URLhttp://www.example.com/shop/ and with the following URL naming scheme:

• /shop/*: This path contains a welcome page for customers and tells them something about thisparticular e-commerce application.

• /shop/products/*: All pages in this path contain descriptions about the available products cus-tomers can buy.

• /shop/products/checkout/*: Contains all the necessary pages for a customer to complete hisorder and pages for verifying his payment options.


Figure 6.9: Clustering by Users / Roles

• /shop/help/*: The following path consists of all the available help pages such as FAQ’s andshipment details.

• /shop/admin/*: The administration area where only users with administration rights have ac-cess to modify prices of products and similar activities.

Clustering on these URL paths would result in a set of 5 clusters namely:

C1 := {x | x ε All URLs in path /shop/*},C2 := {x | x ε All URLs in path /shop/products/*},. . . ,C5 := {x | x ε All URLs in path /shop/admin/*}

The maximum number of expected clusters generated with this approach greatly depends on howthe targeted web application is structured in regard to used URL paths. Clustering based on URLpaths is a convenient way to analyse from which resources transitions can be made into specificfunction domains of a web application. Especially interesting for a developer or testing agent wouldbe the cluster containing all the /admin/* resources and he would then be able to analyse fromwhich other clusters transitions can be made into the administration area. To even enhance thevisual experience it is possible to combine the ideas presented in 6.7.2 and 6.7.3 by clustering andcolouring graph vertices respectively HTML resources as described in Section 6.7.4.

Figure 6.10 shows a possible visual representation of the URL path clustering algorithm. For each URLpath found in the collected data a new cluster is being created as being illustrated through a circle ofnodes with an uniform colour. So it is especially easy to spot connections respectively transitionsbetween function domains of the web application such as edges between a globally visible part andthe administration domain.


Figure 6.10: Clustering by URL Paths

6.7.4 Clustering by URL Paths and Re-Colouring

This approach combines the ideas described in 6.7.2 and 6.7.3 by using a cluster algorithm to clusterthe vertices first and then additionally colourise them based on their accessibility attributes. By com-bining both approaches a user has the advantages of seeing both clustering algorithms combinedin one illustration, given it is possible to cluster based on URL paths and there exists more thanone user role in the targeted web application. This allows the developer or testing agent to quicklycheck if a given URL path cluster such as the administration area is homogeneous which would bethe case if all the contained vertices in that particular cluster have the same colour respectively areall accessible by the same user only. If the cluster is not uniform respectively the cluster containsvertices with a different colour than most other nodes this indicates that another user role can gainaccess to that particular HTML resource. In case of the administration area this case would indicatethat there might exist a possible security threat because a user with a different user role can gainaccess to the administration area. This is an efficient and convenient way for a testing agent tovisually and manually detect outliers rather quickly.

Figure 6.11 shows an illustration of the combined visualisations of Figures 6.9 and 6.10. The clusteringis done on URL paths and the nodes are being coloured based on user accessibility. Nodes having twocolours or a gradient are accessible by both users. This allows a developer to easily spot outliers in theadmin/* domain because the colours of the vertices are not uniform and contain vertices which arecoloured with two colours which means that another user has access to these particular resources.Whether this is a security issue or not would have to be verified manually if no access restrictions asdescribed in Section 6.7.1 have previously been configured.

6.7.5 Showing Differences by Re-Colouring

This view mode offers the user or testing agent a convenient way to quickly analyse the differencesbetween two graphs in regard to found resources between two users. The algorithm used is similar tothe diff command on Unix operating systems for comparing the differences between two files. Thegraph algorithm for showing differences between two users USER1 and USER2 works as follows:

• If a vertex is present in the graph of USER1 and USER2 then the vertex will be coloured grey.

• If a vertex is present in the graph of USER1 but not in the graph of USER2 then the vertex will becoloured red.


Figure 6.11: Clustering by URL Paths and Re-Colouring

Figure 6.12: Clustering by URL Paths and showing differences

• If a vertex is not present in the graph of USER1 but is present in the graph of USER2 then thevertex will be coloured green.

Figure 6.12 shows a sample illustration of the above described algorithm. This can be used to quicklyfind URLs respectively resources that are specifically tailored to a user e.g.: a user preference dialogaccessible through the following URL: /user/showPreferences.do?userid=42. Which turns outto be a valid method for detecting Insecure Direct Object References as described in Section 4.4.

6.7.6 Software Testing Lifecycle

Important for the Software Development Lifecycle (SDLC) which also contains a Software TestingLifecycle is to use the described and implemented security testing tool in regression tests.

Regression testing identifies when code modifications cause previously-working func-tionality to regress, or fail, ultimately allowing you to catch regression errors as soon asthey are introduced. Most organizations verify critical functionality once and then as-sume it continues to work unless they intentionally modify it. However, even routine and


minor code changes can have unexpected side effects that might break previously veri-fied functionality. 16

The proposed security testing tool should allow a developer or testing agent to mark and generateaccess restriction rules of marked vertices as foreseen in the Meta Layer XML file as described inSection 6.2.1. This allows a developer to do regression tests over time by using previously configuredaccess restrictions and enables him to identify misbehaviour of previously working code in the webapplications access restrictions constraints. This is especially interesting for integrating the proposedsecurity testing tool into a continuous integration tool such as Jenkins or an automated securitytesting framework such as ASTF to do continuous testing. More about the benefits of continuoustesting can be found in Duvall’s et al. book called Continuous Integration: Improving Software Qualityand Reducing Risk [14].

In order to make the security testing tool integrateable into a continuous integration tool it needsthe ability to be run in an automated fashion and preferably produces a humanly or machinereadable report for further analysis. This is the responsibility of the Analysis Layer and can besolved by generating a XML file from the issues found by comparing the user provided accessrestrictions in the Meta Layer XML file as described in Section 6.7.1 - Comparing crawled Datawith the <access-restrictions> Tag. This is the only data that can be extracted by the AnalysisLayer in an automated fashion without the need of a user to manually and visually analysing the data.

Listing 6.13 shows a proposed XML layout for such an issue report as described above which simplycontains all the found issues. The elements are really self explanatory and such a XML report mightcontain <error>, <warning> and <information> elements, each with a description, title and anURL pointing to the resource that caused the issue.

1 <?xml version =’1.0’ encoding=’UTF -8’?>

2 <rbacvisualiser >

3 <info >

4 <created >Sat Mar 26 13:32:53 2011</ created >


6 </info >

7 <error name=" Access Violation"

8 url="http :// localhost :8080/ administrator/v1.do" username =" EMPLOYEE">

9 EMPLOYEE has access but should have been denied!

10 </error >


12 url="http :// localhost :8080/ administrator/v1.do" username =" MANAGER">

13 MANAGER has access but should have been denied!

14 </error >


16 url="http :// localhost :8080/ administrator/v1.do" username ="CLERK">

17 CLERK has access but should have been denied!

18 </error >

19 </rbacvisualiser >

Listing 6.13: Sample XML Report created from found Issues

6.8 Presentation Layer

This Section is concerned with how the gathered data from the Meta- (Section 6.2), Crawling- (Section6.4) and most importantly from the Analysis-Layer (Section 6.7) are being visually represented to the

16http://www.wrox.com/WileyCDA/Section/id-291252.html - Adam Kolawa, Co-Founder of Parasoft [17/03/11]


http://www.wrox.com/WileyCDA/Section/id-291252.html

Figure 6.13: GUI Design Prototype

user for verification and analysis. A good resource for ideas on how to present security related data isRaffael Marty’s book Applied Security Visualization [22] as well as [8]. He suggests to use Visual Trans-formation techniques [22, p.157] to represent the data visually to the user and different sise, shapeand colours to visually highlight attributes of the collected data. Additionally aggregation of the dataplays a crucial role in reducing the complexity of the collected data and makes it possible to identifypatterns. These steps have already been discussed in Sections 6.7.2, 6.7.3 and 6.7.4 where clusteringalgorithms are being used to aggregate the data and different colours to highlight different user roles.Because the collected data of the Web Crawling Layer is in its purest form already a directed graphit makes sense to represent the data as a directed graph to the user with the mentioned aggregationfunctions at his fingertips.

6.8.1 Java Swing Application

The obvious choice for writing such a visual analysis tool is a Java Swing application because all ofthe other tools are written in Java and already written code for accessing the crawled data can beshared between applications. Figure 6.13 shows a design prototype of a tool capable of presentingthe collected data to a developer or testing agent.

The GUI design is split up in three main parts: (1) a common menu bar with menu items such asFile, Help etc, (2) a panel displaying the collected data from the web crawler as a directed graph and(3) a tabbed pane where various information can be dynamically displayed to the user such as a listof all the identified issues by comparing the collected data with the configurations made in the MetaLayer XML file or information about particular vertices and user accessibility.Visualising graphs is not a trivial task and a lot of available libraries fall short when the graphs gettoo complex and too dense. Either the tools crash with out of memory exceptions because they try tostore the hole graph in memory or they become sluggish and unresponsive.After evaluating several graph libraries (including libraries without Java Bindings) such as NodeBox17,

17http://nodebox.net/ [17/03/11]


http://nodebox.net/

GraphViz18, Protovis19 and Jung20 it became clear that the JUNG framework is very well suited andenough performant to display even complex graph structures. JUNG stands for “Java Universal Net-work/Graph Framework” and is an open source software library that provides a common and exten-sible language for the modeling, analysis and visualisation of data that can be represented as a graphor network. It is written in Java, which allows JUNG-based applications to make use of the extensivebuilt-in capabilities of the Java API, as well as those of other existing third-party Java libraries. TheJUNG architecture is designed to support a variety of representations of entities and their relations,such as directed and undirected graphs, multi-modal graphs, graphs with parallel edges and hyper-graphs. It provides a mechanism for annotating graphs, entities and relations with metadata. Thisfacilitates the creation of analytic tools for complex data sets that can examine the relations betweenentities as well as the metadata attached to each entity and relation. JUNG also provides a visualisa-tion framework that makes it easy to construct tools for the interactive exploration of network data.Users can use one of the layout algorithms provided or use the framework to create their own customlayouts. In addition, filtering mechanisms are provided which allow users to focus their attention, ortheir algorithms, on specific portions of the graph.

6.9 Summary

This Chapter outlined the ideas and design of a four piece security testing tool able to analyse web ap-plications and their access restrictions consisting of a Meta-, Crawling-, Analysis- and Presentation-Layer. The described approach is able to spot privilege escalation vulnerabilities by analysing thecollected data in a more or less automated fashion depending on how much additional white boxinformation has been provided by a developer or testing agent or how much declarative informationcould be extracted from source- or configuration files. A Meta Layer XML file has been describedwhich decouples the security testing tools from a concrete programming language or web applica-tion framework. The Meta Layer XML file is being used to provide higher layers with the collectedwhite box information. The Meta Layer XML file is being created by command line tools especiallytailored for a specific web application framework and which are capable of parsing these particularconfiguration- and/or source code files. Hence the security testing tool is not limited to Java webapplications only but can be used for web applications written in PHP, Ruby or any other program-ming language. The crawling component is responsible for creating a as much as detailed map of thetargeted web application and its implemented access restrictions as possible. The Analysis Layer isresponsible for finding privilege escalation vulnerabilities by considering the collected data and de-veloper configured access restrictions which is being done in an automated fashion. Other analysisfeatures include visual representation of the collected data for manual verification by a user. Severalcluster- and colour-algorithms as well as combinations thereof have been proposed to help a userto spot outliers in the implemented access restrictions more quickly and visually. The user is able tocreate new access restriction rules which are being added to the used Meta Layer XML file from therepresentation of the data through the Presentation Layer. This loop allows the testing agent to doregression tests of a deployed web application whenever modifications to the application have beenmade. The Presentation Layer consists of a Java Swing application using the JUNG framework fordisplaying and transforming the collected data from the spidering process as a directed graph.

18http://www.graphviz.org/ [17/03/11]19http://vis.stanford.edu/protovis/ [17/03/11]20http://jung.sourceforge.net/ [17/03/11]


http://www.graphviz.org/

http://vis.stanford.edu/protovis/

http://jung.sourceforge.net/

7 Testing

The Testing Chapter is concerned with functionality testing to verify whether the security testing toolmeets the intended specifications and functional requirements laid out in the development and de-sign specifications. The goal is to verify that it is indeed possible and feasible to detect privilegeescalation vulnerabilities and misconfigurations in real life web applications by using the developedsecurity testing tool.

7.1 Web Crawler - WIVET Results

WIVET1 is a benchmarking framework that aims to statistically analyse web link extractors. In general,web application vulnerability scanners fall into this category. These vulnerability scanners, given aURL, try to extract as many input vectors as they possibly can to increase the coverage of the attacksurface. WIVET provides a good sum of input vectors to any extractor and presents the results. TheWIVET project is released under the GNU General Public License Version 2.

Crawler High Scores

This Section compares the high scores scored by other commercial or free web spiders whether theyhave JavaScript support or not. The high scores have been taken from the WIVET homepage (seehttp://code.google.com/p/wivet/wiki/CurrentResults. Only w3af has been verified withSVN revision 3438. Other scores could not be verified because the mentioned products in the highscore miss any versioning information.

• w3af - 50%Open Source web application attack and audit framework.Version: 1.1 (from SVN server) Revision: 3438. See http://w3af.sourceforge.net/

• Acunetix - 94%Commercially available web application vulnerability scanner. Acunetix has pioneered the webapplication security scanning technology: Its engineers have focused on web security as early as1997 and developed an engineering lead in web site analysis and vulnerability detection. Seehttp://www.acunetix.com

• Cenzic Hailstorm - 88%Commercially available web vulnerability scanner. Protect your Web applications by using Cen-zic’s desktop, black box scanning solution. See http://www.cenzic.com/

• HP Webinspect - 94%Commercially available automated web application security testing and assessment tool. Getinnovative assessment technology for web services and web application security and auto-mate web application security testing and assessment. See https://www.fortify.com/

products/web_inspect.html

1http://code.google.com/p/wivet/ [24/03/11]

Chapter 7. Testing | 120

http://code.google.com/p/wivet/wiki/CurrentResults

http://w3af.sourceforge.net/

http://www.acunetix.com

http://www.cenzic.com/

https://www.fortify.com/products/web_inspect.html

https://www.fortify.com/products/web_inspect.html

http://code.google.com/p/wivet/

• IBM AppScan - 83% Commercially available desktop solution to automate web application se-curity testing. Rational AppScan Standard Edition significantly reduces costs associated withmanual vulnerability testing and helps to protect against the threat of cyber-attack by au-tomating security analysis to detect exploitable vulnerabilities. See http://www-01.ibm.com/software/awdtools/appscan/.

• MavitunaSecurity Netsparker - 92% Commercially available web application vulnerabilityscanner. Netsparker can crawl, attack and identify vulnerabilities in all custom web applicationsregardless of the platform and the technology they are built on, just like an actual attacker. It canidentify web application vulnerabilities like SQL Injection, Cross-site Scripting (XSS), RemoteCode Execution and many more. See http://www.mavitunasecurity.com/netsparker/.

As one can easily see, w3af performed the worst of the tested web spiders although one has tonote that w3af is the only free software project amongst the tested products. For a list of othercommercially or freely available web application security tools see [11].

Because of the JavaScript support provided by the web spider component, the security testing tool isable to collect considerably more pages from the WIVET benchmark framework than w3af. The webcrawling component of the developed security testing tool achieved 87%. The achieved results of thecommercially available products have not been verified during this thesis.

The developed crawling component missed the following test cases from the WIVET benchmark:

• 3_45589.php: multi-page form with a single path to final destination

• 9_26dd2e.php: unattached JS function document.location

• 16_2f41a.php: 302 redirection link in response body

• 17_2da76.php: XHR with a busy mode page 2

• 18_1a2f3.php: heavy JS library standard form creation

• 19_1f52a.php: link attached to a SWF simple button onclick event

• 19_2e3a2.php: link attached to a SWF simple button parameterised onclick event

Except test cases 19_1f52a.php and 19_2e3a2.php which would require some kind of Adobe Flashsupport it might be possible to enhance the spidering component to detect the missing test cases ina future release.

7.2 Test Environment & Application

The test environment consists of a tailored Struts 1.3.10 web application containing various test casesto test and demonstrate the abilities and limitations of the proposed security testing tool for findingmisconfigurations and privilege escalation vulnerabilities in access control checks in a targeted webapplication.

The sample web application consists of the following four user roles: ADMINISTRATOR, MANAGER,EMPLOYEE and CLERK. Each user role has his own URL path consisting of the following:/administrator/*, /manager/*, /employee/* and /clerk/*. Each path in turn consists of 10HTML pages. Figure 7.1 shows the overall layout of the tailored web application. Clearly visible are


http://www-01.ibm.com/software/awdtools/appscan/

http://www-01.ibm.com/software/awdtools/appscan/

http://www.mavitunasecurity.com/netsparker/

Figure 7.1: Struts 1.3.10 Test Web Application - URL Layout

the 4 function domains containing the 10 HTML resources. Depending on which principal is beingsupplied in the form based authentication, reachable under /login.do, the user will be redirectedto the appropriate path for his user role.

Access restrictions are managed programmatically and not through features provided by the con-tainer. The tailored Struts application implements a custom Command for checking user privilegesprior to displaying content to the user. In Struts application prior to 1.3.x this would have been acustom RequestProcessor as described in Section 2.7.1. Struts 1.3.x changed the behaviour of theRequestProcessor to a configurable chain of commands. Rather than subclassing a monolithicobject, it is possible not just to replace commands, it is also possible to insert or remove commands,if needed, to extend or streamline the request processing gauntlet. The implemented customcommand class checks if a user has previously been authenticated prior to displaying any requestedcontent, if that is not the case the user will be redirected to the login page at /login.do. If the userhas been previously authenticated the user’s request will be forwarded.

Additional access restrictions are being managed programmatically inside each JSP view beforedisplaying any HTML content. The access restrictions are being verified in every single JSP pageby checking the principals stored in the user’s session. The user object in the session has beenset previously through the login form action. Listing 7.1 shows a particular JSP page which is onlyaccessible with the administrator role and access will be denied otherwise. Lines 3 and 4 in Listing7.1 do the actual verification of the principal stored in the session.

Table 7.1 shows the various access rights for each user role. The table has to be read from the topdown and whether a particular user role has access to a particular HTML resource (Pages 1 - 10) isindicated by a Y if the access is being granted and a N if it is not. For example: a user incorporatingthe MANAGER role is able to access all of the HTML resources of users incorporating the followingroles: MANAGER, EMPLOYEE and CLERK.


1 ...

2 <%

3 User user = (User) session.getAttribute ("USER");

4 if (user.role.equalsIgnoreCase(User.ADMINISTRATOR)) { %>

5 <jsp:include page ="/jsp/administrator/content.jsp" />

6 <% } else { %>

7 <jsp:include page ="/ accessdenied.jsp" />

8 <% } %>

Listing 7.1: Checking Access Restrictions in JSP Pages

7.3 Struts Test Application

For testing purposes the sample application described in Section 7.2 contains an additional 11thHTML resource reachable through the following URL: /administrator/v1.do. This JSP page isdeliberately missing any access restrictions checks explained in Listing 7.1. This ultimately leads to aprivilege escalation vulnerability where any user that has been properly authenticated with any kindof existing user role can access that particular page in the ADMINISTRATOR restricted area. The fol-lowing Sections describe how the proposed and implemented security testing tool is being handledby a common testing agent and shows how that particular vulnerability is being detected by the tool.

7.3.1 Phase 1: Parsing Struts Source-/Configuration Files

The first step involves creating the Meta Layer XML file as described in Section 6.2 by parsing the webapplication’s source- and configuration files. This is done by running the following command in acommand shell:

# java -jar struts1parser.jar -o metalayer.xml <ROOT DIR OF WEB APPLICATION>

The automatically generated metalayer.xml file is shown in Listing 7.2. This file can now beenhanced by the developer to increase the precision of the approach. The changes made to themetalayer.xml file can be seen in Listing 7.3. The creation of the metalayer.xml file only takesa couple of seconds and contains the found classes, URLs, JSP and a standard user called ANONYMOUS.


2 <web -meta >

3 <info >

4 <created >Tue Mar 22 15:19:16 2011</ created >


6 <createdBy >Struts 1.x Parser </createdBy >

7 <notes ></notes >

8 </info >

9 <classes >

10 <class id="1" name="com.wastf.rbacstruts1.CheckUserCommand "></class >

11 <class id="2" name="com.wastf.rbacstruts1.Constants"></class >

12 <class id="3" name="com.wastf.rbacstruts1.LogOffAction"></class >

13 <class id="4" name="com.wastf.rbacstruts1.LoginAction"></class >

14 <class id="5" name="com.wastf.rbacstruts1.LoginForm"></class >

15 <class id="6" name="com.wastf.rbacstruts1.SuccessAction"></class >

16 <class id="7" name="com.wastf.rbacstruts1.User"></class >

17 </classes >

18 <users >

19 <user id="1" name=" ANONYMOUS"></user >


20 </users >

21 <roles ></roles >


23 <login ></login >

24 <urls >

25 <url id="1" path ="/ administrator/page1.do"></url >










35 <url id="11" path ="/ manager/page1.do"></url >










45 <url id="21" path ="/ clerk/page1.do"></url >










55 <url id="31" path ="/ employee/page1.do"></url >










65 <url id="41" path ="/ accessdenied.do"></url >

66 <url id="42" path ="/ login.do"></url >

67 <url id="43" path ="/ logoff.do"></url >

68 </urls >

69 <views >

70 <view path ="/jsp/administrator /1. jsp">



73 </view >

74 <view path ="/jsp/administrator /2. jsp">



77 </view >

78

79 ...

80


81 <view path ="/jsp/manager /1.jsp">



84 </view >

85 <view path ="/jsp/manager /2.jsp">



88 </view >

89 ...

90 </views >


92 </web -meta >

Listing 7.2: Initial metalayer.xml File for the Test Application

A testing agent now enhances the Meta Layer XML file to his needs and adds the programaticallychecked user roles, users and eventually desired respectively intended access restrictions. Thechanges made to the originally created file can be seen in Listing 7.3.

1 <users >


3 + <user id="2" name=" ADMINISTRATOR">

4 + <attribute name=" username" value =" adminUser" />

5 + <attribute name=" password" value =" password" />

6 + </user >

7 + <user id="3" name=" MANAGER">

8 + <attribute name=" username" value =" managerUser" />


10 + </user >

11 + <user id="4" name=" EMPLOYEE">

12 + <attribute name=" username" value =" employeeUser" />


14 + </user >

15 + <user id="5" name="CLERK">

16 + <attribute name=" username" value =" clerkUser" />


18 + </user >

19 </users >

20 - <roles ></roles >

21 - <access -restrictions ></access -restrictions >

22 - <login ></login >

23 + <roles >

24 + <role id="1" name=" ADMINISTRATOR" priority ="1">

25 + <user -id -ref >2</user -id -ref >

26 + </role >

27 + <role id="2" name=" MANAGER" priority ="2">


29 + </role >

30 + <role id="3" name=" EMPLOYEE" priority ="3">


32 + </role >

33 + <role id="4" name="CLERK" priority ="4">


35 + </role >

36 + </roles >

37 + <access -restrictions >

38 + <url pattern ="/ login.do" access =" permitAll "/>

39 + <url pattern ="/ accessdenied.do" access =" permitAll "/>

40 + <url pattern ="/ administrator /*">

41 + <role -id -ref >1</role -id -ref >


42 + </url >

43 + <url pattern ="/ manager /*">



46 + </url >

47 + <url pattern ="/ employee /*">




51 + </url >

52 + <url pattern ="/ clerk /*">





57 + </url >

58 + </access -restrictions >

59 + <login >

60 + <form -based -login >

61 + <form -url >/ login.jsp </form -url >

62 + <form -name >loginForm </form -name >

63 + <data >

64 + {" username ":"## username ##", "password ":"## password ##"}

65 + </data >

66 + <submit -element >Sign in </submit -element >

67 + </form -based -login >

68 + </login >

69 <urls >


Listing 7.3: Additional Information added to the metalayer.xml File (displayed as a diff -up

FILE1 FILE2 output)

In this case the testing agent added several pieces of information to the Meta Layer file including thefollowing:

• <roles>: The available roles in the web application. Because these roles are being program-matically verified in the source code of the web application these different roles do not show upin a standardised configuration file such as a deployment descriptor and thus have to be addedby the testing agent (Listing 7.3, Lines 24-37). The <roles> tag is being described in Section6.2.1.

• <users>: The available users incorporating the previously defined roles. Again, because theusers are being programmatically verified and are not being declared using a deployment de-scriptor these have to be added manually by a testing agent (Listing 7.3, Lines 4-19). Ideally,the testing agent configures a distinct user for each defined role. The <users> tag is beingdescribed in Section 6.2.1.

• <access-restrictions>: Access restrictions help the Analysis Layer to find vulnerabilitiesand misconfigurations in an automated fashion. In this case the testing agent defined the var-ious privilege restrictions based on the available URL paths of the web application (Listing 7.3,Lines 38-59). The information provided here correlates with Table 7.1 and Section 7.2. The<access-restrictions> tag is being described in Section 6.2.1.

• <login>: Because the web application can not be spidered before a user is not properly au-thenticated the testing agent defined the HTML form used by the web application to authenti-cate users (Listing 7.3, Lines 60-67). In this case the additional attributes defined in the <user>


elements are being used to populate the input fields of the HTML login form. The <login> tagis being described in Section 6.2.1.

With this enhanced Meta Layer XML file in Listing 7.3 the next step is to use the web crawling com-ponent and map the deployed web application’s access restrictions as they have actually been im-plemented in the application. Important to note is that this first step has to be done only once andthe enhanced Meta Layer XML file can be kept as long as the web application does not significantlychange.

7.3.2 Phase 2: Crawling the deployed Web Application

With the prepared Meta Layer XML file the crawling of the deployed web application can be started.The crawling component uses all the available users to map all the existing resource transitions andprivilege restrictions. The crawling component has to be additionally configured over the commandline interface by the testing agent.

# java -jar rbacspider.jar \

--access-denied-page "access denied" \

--forceful-browsing \

--ignore-regex "logoff.do" \

metalayer.xml \

http://localhost:8080/

The −−access-denied-page property contains a regular expression used by the web crawler toidentify if a user has been denied access to a requested resource. This is especially important ifthe web application uses custom designed error pages as it is the case in the sample Struts 1.3.10application described previously in Section 7.2. If the web application would send standard failingHTTP status codes when access to a particular resource is not being granted then this parametercould be omitted.

The−−forceful-browsing flag adds another loop to the spidering process as described in Section6.4.6 in order to request every single found URL without following the previously collected transitions.

Spidering the test application with different user roles as described in Section 6.4.6 took roughly 3hours. This is because the test application shows the applications site map2 on every HTML resourceand the spidering component tries to reach each page from all available angles respectively paths.This is intentional and should give an idea how performant the spidering and the visualisation ofthe data is if there are a lot of page transitions present. After the spidering is complete the followingarchive 2011-03-24-localhost.rbacspider has been generated which then can be opened inthe Presentation Layer for further analysis.

7.3.3 Phase 3: Analysing the Results

The final step is to analyse the collected data from the spidering phase in the graphical user interfaceprovided. Starting the application is done by double clicking the rbacvisualiser.jar file or byentering the following command into a terminal:

2A site map is a list of pages of a web site accessible to crawlers or users. It can be either a document in any form used as aplanning tool for web design or a web page that lists the pages on a web site, typically organised in hierarchical fashion.This helps visitors and search engine bots find pages on the site. - http://en.wikipedia.org/wiki/Site_map[25/03/11]


http://en.wikipedia.org/wiki/Site_map

Figure 7.2: Initial view of the collected Data from the Struts Test Application

# java -jar rbacvisualiser.jar

Figure 7.2 shows the initial view of the visualisation tool presenting the collected data after the useropened the previously generated archive. The default view is based on the URL path clusteringalgorithm described in Section 6.7.3. Of course, this view is not all that informative in the beginningbut interesting for a testing agent or developer is the Problems tab.

The Problems tab contains the findings based on the user provided <access-restrictions> in-formation and the actual findings of the spidering component based on the Analysis Layer describedin Section 6.7.1. Figure 7.3 shows the Problems tab and the particular HTML resource vulnerable toa privilege escalation vulnerability as has been intended (see Section 7.3). The Problems tab is thefirst source a testing agent consults in finding problems in the implementation of the targeted webapplication. Afterwards he is able to analyse the data further visually by hiding edges or vertices,clustering the data based on users or by re-colouring the data. Additionally he is able to generate editor add new rules to the <access-restrictions> section of the Meta Layer XML file which thencan be used in future scans of the web application.

If the testing agent would not have added the additional information in the<access-restrictions> section of the Meta Layer XML file he would still have been able todetect the privilege escalation vulnerability in the targeted web application by re-colouring thecollected data based on users as described in Section 6.7.4. Figure 7.4 shows the collected dataafter it has been re-coloured by using the ADMINISTRATOR and CLERK user. For the sake of clarityall of the main edges have been hidden and now one can clearly see that the one vertex in the/administrator/* cluster is coloured differently than the rest and this indicates that that particu-lar vertex, namely /administrator/v1.do, is accessible by both the ADMINISTRATOR and CLERK

user which is our intended privilege escalation vulnerability.

This test case has shown that the Presentation Layer is enough performant to handle even web ap-plications with a lot of vertices and transitions respectively edges between them and that the designand idea of a security testing framework for detecting misconfigurations and vulnerabilities in non-


Figure 7.3: Automatically detected Access Restriction Violations in the Struts Test Application

Figure 7.4: Re-Colouring of the Data based on Users

container managed access restriction checks in an automated fashion actually works.

7.4 WordPress 3.1

The following test case tries to show that the developed security testing framework is also capableof scanning rather large real life web applications successfully. Finding misconfigurations or priv-ilege escalation vulnerabilities in a real life community tested web application is rather unlikelybut it might be interesting to see whether the security testing framework reports false positives or not.

This test case involves the freely available blogging software WordPress3. WordPress started as ablogging system, but has evolved to be used as full content management system and so much morethrough the thousands of plugins, widgets and themes.

WordPress has been chosen for this test case to show that the developed approach in this thesiscan be used with real life web applications and that the approach is not limited to web applicationswritten in Java because WordPress is entirely written in PHP.

The test setup involves WordPress 3.1, PHP Version 5.3.3-1ubuntu9.3, Apache/2.2.16 (Ubuntu) webserver and a MySQL Ver 14.14 Distrib 5.1.49 database. Test data is being provided by importingthe XML file from http://svn.automattic.com/wpcom-themes/demo/test-data.2011-01-

17.xml.

3http://wordpress.org/ [31/03/11]


http://svn.automattic.com/wpcom-themes/demo/test-data.2011-01-17.xml

http://svn.automattic.com/wpcom-themes/demo/test-data.2011-01-17.xml

http://wordpress.org/

Administrator Manager Employee Clerk1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Administrator 1 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N2 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N3 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N4 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N5 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N6 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N7 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N8 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N9 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N

10 Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N N N N NManager 1 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N

2 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N3 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N4 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N5 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N6 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N7 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N8 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N9 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N

10 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N NEmployee 1 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N

2 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N3 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N4 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N5 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N6 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N7 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N8 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N9 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N

10 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N NClerk 1 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

2 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y3 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y4 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y5 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y6 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y7 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y8 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y9 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

10 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Table 7.1: Access Rights of the Sample Struts 1.3.10 Web Application

A vulnerable PHP page has been added to the /wp-admin/* folder which holds the administrationbackend and is only accessible to a user with administrator privileges. This vulnerable PHP pagecalled /wp-admin/vulnerable.php is missing any access control checks and is accessible to allavailable users. This is to demonstrate that the described security testing tool is indeed capableof detecting privilege escalation vulnerabilities in non Java applications properly and that the toolworks reliably.

7.4.1 Phase 1: Preparing the Meta Layer XML File

Listing 7.4 shows the used Meta Layer XML file. The difference between this Meta Layer XML file andthe one previously used during the Struts 1.3.10 test application described in Section 7.3.1 is that sev-eral parts are missing such as: the <classes>, <urls> and <views> tag elements. The <classes>and <views> tags are merely optional and do not affect the thoroughness of the approach asdescribed in Section 6.2.1, the missing <urls> tag on the other hand does. The approach does stillwork even if no URLs are being used to seed the crawling component as described in Section 6.2.1 butit is very likely that not all URLs are properly being found and tested by the web crawling component.Nonetheless, it is possible to conduct a security analysis of the targeted web application and it is pos-sible to detect missing access control checks in the web application as shown later on in Section 7.4.3.


2 <web -meta >

3 <info >

4 <created >Tue Apr 12 11:28:43 2011</ created >


6 <createdBy >Manual </createdBy >


7 <notes ></notes >

8 </info >

9 <classes ></classes >

10 <users >


12 <user id="2" name=" ADMINISTRATOR">

13 <attribute name=" username" value ="admin" />

14 <attribute name=" password" value ="admin" />

15 </user >

16 </users >

17 <roles >

18 <role id="1" name=" ADMINISTRATOR" priority ="1">

19 <user -id-ref >2</user -id -ref >

20 <disable -form -submit pattern ="/ wordpress/wp-admin /*" />

21 </role >

22 <role id="2" name=" ANONYMOUS" priority ="2">


24 </role >

25 </roles >

26 <access -restrictions >

27 <url pattern ="/ wordpress/wp -admin/*">

28 <role -id-ref >1</role -id -ref >

29 </url >

30 </access -restrictions >

31 <login >


33 <form -url >/ wordpress/wp -login.php </form -url >

34 <form -name >loginform </form -name >

35 <data >

36 {" user_login ":"## username ##", "user_pass ":"## password ##"}

37 </data >

38 <submit -element >Log In </submit -element >


40 </login >

41 <urls ></urls >

42 <views ></views >


44 </web -meta >

Listing 7.4: metalayer.xml File for a Standard WordPress 3.1 Installation

As intended by the developers of WordPress, the Meta Layer XML file contains an<access-restrictions> element stating that only the administrator user should have accessto the /wp-admin/* folder. This should later on detect the vulnerable.php file that has beenadded to the WordPress administration folder in the Analysis Layer.

The <login> tag describes the HTML form used by WordPress to authenticate users and whichgives them access to the administration panel of WordPress. The name of the form and the usedHTML input elements have to be manually extracted by a testing agent by looking at the HTMLsource code of the /wordpress/wp-login.php file. Listing 7.5 shows the HTML source code of the/wordpress/wp-login.php file with the embedded HTML form.

1 ...

2 <form name=" loginform" id=" loginform"

3 action ="http :// localhost/wordpress/wp -login.php" method ="post">

4 <p>

5 <label >Username <br />

6 <input type="text" name="log" id=" user_login" class=" input"

7 value ="" size ="20" tabindex ="10" /></label >


8 </p>

9 <p>

10 <label >Password <br />

11 <input type=" password" name="pwd" id=" user_pass" class=" input"

12 value ="" size ="20" tabindex ="20" /></label >

13 </p>

14 <p class =" forgetmenot"><label ><input name=" rememberme" type=" checkbox"

15 id=" rememberme" value=" forever" tabindex ="90" /> Remember Me </label ></p>

16 <p class =" submit">

17 <input type=" submit" name="wp -submit" id="wp-submit"

18 class="button -primary" value ="Log In" tabindex ="100" />

19 <input type=" hidden" name=" redirect_to"

20 value="http :// localhost/wordpress/wp-admin/" />

21 <input type=" hidden" name=" testcookie" value ="1" />

22 </p>

23 </form >

24 ...

Listing 7.5: /wordpress/wp-login.php HTML Source Code containing the HTML basedAuthentication Form of WordPress


With the prepared Meta Layer XML file (see Listing 7.4) the crawling of the deployed web applicationcan be started. The crawling component uses all the available users to map all the existing resourcetransitions and privilege restrictions. The crawling component has to be additionally configured overthe command line interface by the testing agent. The following commands have been used to crawlthe installed WordPress application:

# java -jar rbacspider.jar \

--ignore-regex ".*logout.*" \

--access-denied-page ".*Lost your password?.*" \

--ignore-htmlelements-regex ".*search.*" \

--forceful-browsing \

--max-threads 10 \

metalayer.xml http://localhost/wordpress/

The −−ignore-regex provided regular expression prevents the spidering component to uninten-tionally click on the sign out button and thus terminating the users session.

The −−access-denied-page regular expression matches the /wordpress/wp-login.php

file which is shown to a user with lower privileges when trying to access resources in the/wordpress/wp-admin folder. This is default WordPress behaviour and by providing this reg-ular expression the spidering component is aware when a particular user has been denied access toa requested resource.

The −−ignore-htmlelements-regex provided regular expression prevents the spidering com-ponent to fill out any search fields which are provided by WordPress to search for blog entries.This prevents the spidering component to enter an endless loop which would only be terminatedby exceeding the −−max-level spidering constraint which defaults to 25. The endless loop iscaused because whenever the spidering component automatically fills out such a search form on aWordPress page as described in Section 6.4.5 the returned results are interpreted as a new resourcerespectively path in the targeted web application. The therein found URLs are then again crawled


for possible new paths and URLs. The particular problem in this test case is that such a search fieldis present on every WordPress page and thus leads to an endless amount of transitions betweenpages. To prevent this from happening a −−max-level constraint could have been lowered fromthe default value of 25 to a smaller number to leave such a loop more quickly or by adding a−−ignore-htmlelements-regex regular expression to ignore such search fields altogether.

The−−forceful-browsing flag adds another loop to the spidering process as described in Section6.4.6 in order to request every single found URL without following the previously collected transitions.

The −−max-threads 10 flag tells the spidering component to use 10 spidering threads instead of5 which is the default value. Depending on the available random access memory this value can evenbe increased further and thus speeding up the spidering process. Nonetheless, the default value of 5threads is a good mix between performance and speed.

Spidering the installed WordPress web application with two different user roles as described in Sec-tion 6.4.6 has been manually terminated after each user spidered the application for roughly an hour.After the spidering is complete the following archive 2011-04-01-localhost.rbacspider hasbeen generated which then can be opened in the Presentation Layer for further analysis.




Figure 7.5 shows the clustered and re-coloured graph of the collected data as described in Section6.7.4. The Problems tab contains the findings based on the user provided <access-restrictions>

information and the actual findings of the spidering component based on the Analysis Layerdescribed in Section 6.7.1 and Section 7.4.1. Figure 7.5 additionally shows the Problems tab andthe particular HTML resource /wp-admin/vulnerable.php that has been added with missingaccess control checks. The detection of the intentionally added privilege escalation vulnerabilityhas been successful and the spidering component as well as the Analysis Layer correctly found thevulnerability. By clustering and re-arranging the collected data the issue is also clearly visible on theright. The violation is clearly visible in the cluster on the right which should only be accessible bythe administrator user (coloured red) but one node is also accessible by a user with fewer privileges(gradient coloured).

This test case showed that the designed and developed approach for finding possible privilege escala-tion vulnerabilities works even with large and real life applications visually by inspecting the collecteddata as well as automated given the <access-restrictions> rules exist in the Meta Layer XML file.However, one issue noticed during the execution of the test case showed that the Analysis Layer mightgenerate false positives in certain circumstances. These circumstances are explained in more detailin Section 7.6.


Figure 7.5: Automatically detected Access Restriction Violations in the WordPress Test Application

7.5 PrivaSphere Secure Messaging Service

PrivaSphere’s Secure Messaging Service4 provides a reliable and secure email communication overthe Internet. It provides advanced features such as digitally stamped receipts (certified) and can beused for sending offers, delivery of client statements (financial, personal) and delivery of medicaldata. The Secure Messaging Service provides innovative secure and authenticated Internet messagingtechnologies and service to corporations and individual users.

The following test case has been chosen in order to verify that the developed approach reliably workswith a rather large real life and productive web application. The following four users respectively userroles have been set up and have been available during the spidering and analysing of the enforcedaccess control constraints in the Secure Messaging Service web application: a normal user, a first-levelhelpdesk user, a signature service administrator and a super user. Figure 7.6 shows the email inboxfor a standard user after logging in.

7.5.1 Phase 1: Preparing the Meta Layer XML File

In the first phase the Meta Layer XML is being set up and configured. Because the source code ofthe application has not been available, the Meta Layer XML has been set up manually and eventualprivilege escalation vulnerabilities and misconfigurations in enforced access control constraintswill have to be identified visually by a testing agent. The approach works nonetheless even if nodeclarative information has been extracted from the source- and configuration files of the targeted

4https://www.privasphere.com/ [12/04/11]


https://www.privasphere.com/

Figure 7.6: Standard User Secure Messaging Service Email Inbox

web application but the results might not be as precise and thorough when using available declara-tive information and eventual vulnerabilities have to be visually and manually verified by a developer.

Listing 7.7 shows the prepared Meta Layer XML file containing the four mentioned user roles and theHTML based authentication form which is being used by the spidering component to acquire thedifferent user roles.

Listing 7.6 shows the simplified HTML source code of the HTML based authentication form fromwhich the <login> tag has been configured in the Meta Layer XML file in Listing 7.7.

1 <form id=" loginForm" name=" loginForm" method ="post" action ="../ login.">

2 <input type=" hidden" name=" routeTo" value ="index" />

3 <input name="login" id=" login" type="text" />

4 <input name=" password" type=" password" />

5 <input type="image" name=" submit" value =" submit" src ="...." /></p>

6 </form >

Listing 7.6: Secure Messaging Service HTML based Authentication Form


2 <web -meta >

3 <info >

4 <created >Tue Apr 12 11:28:43 2011</ created >


6 <createdBy >Manual </createdBy >

7 <notes ></notes >

8 </info >

9 <classes ></classes >

10 <users >


12 <user id="2" name=" NORMAL_USER">

13 <attribute name=" username" value ="M8R [email protected]" />

14 <attribute name=" password" value =" si9eiBahz1" />

15 </user >

16 <user id="3" name=" HELPDESK_USER">

17 <attribute name=" username" value ="M8R -6 [email protected]" />

18 <attribute name=" password" value =" Ahbeineip5" />

19 </user >

20 <user id="4" name=" SIGNATURE_USER">



22 <attribute name=" password" value =" Amaesa3eu5" />

23 </user >

24 <user id="5" name=" SUPER_USER">


26 <attribute name=" password" value =" shoow4Chae" />

27 </user >

28 </users >

29 <roles >

30 <role id="1" name=" SUPER" priority ="1">


32 <disable -form -submit pattern =".*" />

33 </role >

34 <role id="1" name=" SIGNATURE" priority ="2">


36 </role >

37 <role id="1" name=" HELPDESK" priority ="3">


39 </role >

40 <role id="1" name=" NORMAL" priority ="4">


42 </role >

43 <role id="1" name=" ANONYMOUS" priority ="4">


45 </role >

46 </roles >


48 <login >


50 <form -url >/hp/index.php </form -url >

51 <form -name >loginForm </form -name >

52 <data >

53 {"login ":"## username ##", "password ":"## password ##"}

54 </data >

55 <submit -element >submit </submit -element >


57 </login >

58 <urls ></urls >

59 <views ></views >


61 </web -meta >

Listing 7.7: metalayer.xml File for the Secure Messaging Service


With the prepared Meta Layer XML file (see Listing 7.7) the crawling of the deployed web applicationcan be started. The crawling component uses all the available users to map all the existing resourcetransitions and privilege restrictions. The crawling component has to be additionally configuredover the command line interface by the testing agent. The command line arguments used to run thecrawling component can be seen in Listing 7.8.

1 # java -jar rbacspider.jar \

2 --ignore -regex ".* logoff.d*|.*hp .*|.* delete .*|.* flush .*|.* reload .*|.* remove

.*|.* close .*|.* forgot .*|.* reg1 .*|.*a?clientCert .*|.* index .*" \

3 --ignore -htmlelements -regex ".* search .*|.* logoff .*|.* save .*|.* update .*|.*

remove .*|.* delete .*|.* flush .*" \

4 --sessionId -regex "jsessionid" \


5 metalayer.xml \

6 https ://www -dev.privasphere.com\

Listing 7.8: Crawling Command Line Arguments for the Secure Messaging Service

The −−ignore-regex provided regular expression prevents the spidering component to uninten-tionally click on the logoff button and thus terminating the users session. Additionally, commandssuch as deleting email messages and removing contacts from the address book are being disabled.

The −−ignore-htmlelements-regex provided regular expression prevents the spidering compo-nent to fill out any search fields which are provided by the Secure Messaging Service as described inSection 7.6. Additionally, HTML elements used to update, remove and flush user accounts are beingignored which is especially important when the spidering component uses high privileged usersto map enforced access restrictions. An easier solution to this problem would be to disable formsubmits based on URL patterns as described in Section 6.2.1. This is not possible with the overallstructure of the Secure Messaging Service because the administration panel is not separated througha distinctive URL path such as /administrator/*.

The −−sessionId-regex provided regular expression is necessary in case of the Secure MessagingService because the application does not use cookies to store session IDs, they are being appended tothe URL instead. Common and valid URLs in the Secure Messaging Service look like the following:

/adminHlpDsk_en.jsp;jsessionid=93951EA9E450BFCA6321BBBF8C1A66BB?...

Especially for spidering phases 2 and 3 as described in Section 6.4.6 it becomes necessary to alterthe discovered URLs by removing and replacing session IDs in order to make them comparable andinterchangeable between different users and roles. For example when an URL discovered by User

1 is being checked if it is accessible for User 2 then the embedded session ID of User 1 has tobe replaced with the current session ID of User 2, otherwise the access is automatically deniedbecause the session of User 1 might already have timed out.

The −−access-denied-page regular expression is missing in case of the Secure Messaging Servicebecause no customised error messages are being shown to a user when the access to a particularresource has been denied by the application. The Secure Messaging Service sends 302 Moved

Temporarily HTTP status codes instead. As described in Section 6.4.6 HTTP status codes otherthan 200 OK are being considered as blocked requests in the spidering phases 2 and 3 if no regularexpression has been given through the −−access-denied-page command line argument.

Due to a bug in the Secure Messaging Service the resources available to the super user role couldnot be correctly and fully mapped. If the HTML page which is being received after a successfullogin is being re-requested by the spidering component then the user’s session will be immediatelyterminated. Doing the same with the other user roles (a normal user, a first-level helpdesk user and asignature service administrator) worked just fine.

Spidering the Secure Messaging Service web application with four different user roles has been man-ually terminated after each user spidered the application for roughly an hour. After the spidering iscomplete the following archive 2011-04-13-www-dev.privasphere.com.rbacspider has beengenerated which then can be opened in the Presentation Layer for further analysis.





Figure 7.7 shows the initial view of the collected data from the spidering component describedin Section 7.5.2. As mentioned before, the Secure Messaging Service does not use URL paths toseparate function domains of the web application such as making all resources belonging to theadministration panel available trough URLs such as /administrator/*. Therefore most off thediscovered resources belong to one large cluster as seen in Figure 7.7. Yet, it is still possible to analysethe data manually and spot misconfigurations in access controls and eventual privilege escalationvulnerabilities visually (because there have no <access-restriction> rules been configured inthe Meta Layer XML file as seen in Listing 7.7) by clustering and comparing different user roles asdescribed in Section 6.7.2.

Figure 7.8 shows the same data collected from the spidering of the Secure Messaging Service butclustered on the signature service administrator and a normal user. The four clusters being: (1)the resources that are only accessible to the signature service administrator (coloured red), (2)the resources that are only accessible to a normal user (coloured blue), (3) the resources that areaccessible by both users (gradient coloured) and (4) the 4 resources that are neither accessible bythe signature service administrator nor a normal user (coloured pink). The developer or testingagent is now able to spot privilege escalation vulnerabilities or misconfigurations in enforced accesscontrols by checking whether the cluster containing the resources accessible to both user rolescontain URLs which should have been protected. The developer additionally has the possibility toadd <access-restriction> rules based on the crawled data in order to automate the process offinding security issues in future scans of the Secure Messaging Service.

This test case proved that the approach works with a large real life and productive web applicationin a feasible and tangible way. Although few declarative information have been provided to the spi-dering component it is still possible to get an overview over the enforced access restrictions and spoteventual security issues in the targeted web application despite the fact that the eventual issues haveto be visually detected by a developer or testing agent. Additionally, a developer or testing agent hasto be aware of the fact that the spidering component did not find all available resources and thus thevisualised resources might not be complete.

7.6 False Positives in the Analysis Layer

During the execution of the test case described in Section 7.4 it became clear that the Analysis Layermight generate false positives depending on how well the spidering component is able to access andprocess the discovered URLs and on the termination criteria provided to the spidering componentthrough command line arguments as described in Section 7.4.2 and Section 9.1.4 by a developer ortesting agent.

The false positives noticed during the execution of the test case described in Section 7.4 have beenin the form of: User X should have access but has not! and arouse from how the spi-dering component is either manually terminated by the user or how the termination criteria have


Figure 7.7: Analysing the collected Data from the Secure Messaging Service (Clustered by URL Paths)

Figure 7.8: Analysing the collected Data from the Secure Messaging Service (Clustered by Users)


been set through command line arguments. Considering the developer or testing agent added an<access-restrictions> rule to the Meta Layer XML file such as the following:

<access-restrictions>

<url pattern="/wordpress/wp-admin/*">

<role-id-ref>1</role-id-ref>

</url>

</access-restrictions>

If the spidering component is prematurely terminated either manually by the user using the interac-tively available command x followed by an ENTER to immediately terminate the currently runningspidering process (as described Section 9.1.4) or by reaching a termination criteria such as limitingthe spidering process to 10 minutes by providing the −−max-time 10 command line argument,false positives might occur because the spidering component might not have reached any URLsmatching the above <access-restrictions> rule.

This issue matches the case A user has no access to an allowed resource in the<access-restrictions> tag as described in Section 6.7.1 and leads to the generation of awarning that would not have occurred if the spidering process terminated properly or if the spideringcomponent had enough time to examine all the discovered URLs.

A way to minimise the generation of these kind of messages is either to terminate the spidering com-ponent manually by entering c followed by an ENTER which instructs the spidering component toprocess the already discovered URLs but ignore newly found URLs (as described in Section 9.1.4) orby increasing the termination criteria that the spidering component has more time to discover all theavailable URLs.

7.7 Summary

The test cases described in this Chapter showed that the proposed security testing tool is indeedcapable of detecting privilege escalation vulnerabilities in an automated way in real life applicationsif the appropriate <access-restrictions> rules have been configured by the testing agent in theMeta Layer XML file. Additionally, the test cases showed that the Presentation Layer is a very niftytool for a testing agent to visually identify issues in the collected data from the spidering process andlets him add and edit <access-restrictions> rules which then can be used in future scans of atargeted web application.

Two issues have been discovered during the execution of the test cases that a testing agent needsto be aware of, namely the creation of false positives if the spidering component is prematurelyterminated and possible spidering loops through provided search functionality in the targetedweb application but which usually are being resolved when the spidering component reaches themaximum allowed crawling level which defaults to 25 but nonetheless can take several hours toreach depending on the overall structure of the targeted web application.

In general, the more <access-restrictions> rules are being configured by the testing agent themore issues can be found in an automated fashion by the Analysis Layer when the actual collecteddata is being compared with the <access-restrictions> rules as described in Section 6.7.1.


8 Summary

This Chapter recapitulates critically the findings and limitations of this thesis and proposes a list offurther steps for the project.

8.1 Conclusion

This thesis is concerned with improving security in web applications by using declarative informa-tion provided in web application frameworks. The security testing tool that was researched anddeveloped in this thesis is especially useful for detecting privilege escalation vulnerabilities andmisconfigurations in access control checks in web applications with at least two distinguishableuser roles in an automated fashion whereas one can be anonymous. This thesis has shown that it isindeed possible to detect misconfigurations and possible privilege escalation vulnerabilities in anautomated fashion. The approach has been designed to be extensible and enough generic to use itwith different web application frameworks and even programming languages.

Depending on the overall design of the web application, adding access control checks to eachaccessible resource can be a very tedious task and it is very likely that some checks might have goneforgotten, rendering the application vulnerable to privilege escalation attacks. By using declarativeinformation provided in source- or configuration files of a developed web application it becomespossible to detect privilege escalation vulnerabilities and misconfigurations in access control checksin an automated fashion allowing a developer to make the web application more secure. The devel-oped security testing tool has been designed to be integrateable into a continuous build environmentand fulfills the requirements for doing automated regression tests for a deployed web application.

Finding misconfigurations in access control checks and thus possible privilege escalation vulner-abilities is being done in three phases: (1) The source code and configuration files of a targetedweb application are being parsed by a Java command line Framework Parser specifically tailoredto support a particular used web application framework such as Struts 1.x, 2.x, Spring etc. TheFramework Parser’s output is a general Meta Layer XML file containing all the found data such asa (non) complete list of available URLs for seeding the web crawling component used in phase 2,a (non) complete list of available users and user roles, a (non) complete list of existing access con-straints. A developer or testing agent then has the opportunity to enhance the generated Meta LayerXML file with additional information such as adding more information about users and roles, userpasswords, configuring how users have to authenticate themselves to the targeted web applicatione.g.: HTML form based authentication and URL based access restrictions as they have been intendedduring the development of the web application. Such a particular access restriction rule might statethat only users with an Administrator role should have access to HTML resources accessiblethrough URLs beginning with /administrator/*. (2) The second phase consists of a web crawlingcomponent which takes the Meta Layer XML file as an input. The spidering of the targeted webapplication is done for each configured user respectively role configured in the Meta Layer XMLfile to find all possible transitions and enforced access restrictions. The output of the spideringcomponent is a ZIP archive containing all the collected data. (3) The third and final phase consistsof a Java Swing Visualisation Application which visualises and analyses the collected data from the

Chapter 8. Summary | 141

spidering component as a directed graph whereas nodes correspond to found HTML resources andedges correspond to transitions between HTML resources. The analysis is done in an automatedfashion if the testing agent provided access restrictions rules which can be compared with the actualcollected data from the spidering. If no information has been provided regarding access restrictionsthe user has the possibility to spot privilege escalation vulnerabilities or misconfigurations visuallyand manually by various means provided by the Visualisation Application such as clustering thecollected data based on URLs, clustering based on users or by re-colouring nodes according to theiraccessibility.

The test cases showed that the developed security testing tool is indeed capable of detecting priv-ilege escalation vulnerabilities in an automated fashion in real life applications if the appropriate<access-restrictions> rules have been configured by the testing agent in the Meta Layer XMLfile or have been extracted from source- or configuration files. Additionally, the test cases showed thatthe Presentation Layer is a very handy tool for a testing agent to manually and visually identify issuesin the collected data from the spidering process and lets him add and edit <access-restrictions>rules which then can be used in future scans of a targeted web application.

Two issues have been discovered during the execution of the test cases that a testing agent needsto be aware of, namely the creation of false positives if the spidering component is prematurelyterminated and possible spidering loops through provided search functionality in the targetedweb application but which are usually being resolved when the spidering component reaches themaximum allowed crawling level which defaults to 25 but nonetheless can take several hours toreach depending on the overall structure of the targeted web application.

In general, the more <access-restrictions> rules are being configured by the testing agent themore issues can be found in an automated fashion by the Analysis Layer when the actual collecteddata is being compared with the <access-restrictions> rules as described in Section 6.7.1 andthe generated report of the automatically found issues can be used in a software testing lifecycle asdescribed in Section 6.7.6.

8.2 Further Steps

Another interesting approach that has been looked at during this thesis is the detection of misconfig-urations and eventual privilege escalation vulnerabilities in attribute based access control checks. Itmight be possible to enhance the here developed approach further to incorporate XACML files (seeSection 3.1).

Additionally, the developed Visualisation Application could certainly be enhanced with moretechniques to visualise the collected data from the spidering process. Besides the two previouslymentioned books for inspiration [8] and [22], there are certainly more clever ways to help a user findoutliers from the collected data regarding missing or wrongly enforced access control checks.

As described in Section 7.1 the spidering component could be further enhanced that the componentscores more than the currently 87% by looking into the 7 test cases that it missed.

Chapter 8. Summary | 142

9 User Manual & Development Guide

This Chapter contains the user manual and the development guide for the developed security testingtools. The user manual leads new users through the installation process of the tools and helps to getfamiliar with the various command line arguments. The development guide is for advanced userswho want to extend the functionality of the testing tools and build them from source.

9.1 User Manual

The user manual is targeted at users who want to install and run the security testing tools on theirmachines. The manual outlines the software and hardware requirements, installation proceduresand how to start the applications.

9.1.1 Hard- & Software Requirements

The three developed tools are purely written in Java and therefore compatible with various operatingsystems such as Microsoft Windows, Linux and Mac OS as long as these provide a Java Virtual Ma-chine (JVM) respectively the Java Runtime Edition (JRE). The security testing tools have been testedand noted to work with the following Java version:

java version "1.6.0_24"

Java(TM) SE Runtime Environment (build 1.6.0_24-b07)

Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)

The security testing tools are packaged for distribution in a single JAR archive so that they contain allof the free/open source third-party libraries necessary to run them successfully. Building from sourceand testing has primarily been done on a Linux Ubuntu 10.10 machine. The hardware requirementsinclude the following:

• Java(TM) SE Runtime Environment >= 1.6.0 (32 or 64 bit)

• >= 2 GB Random Access Memory (RAM)

9.1.2 Installation Instructions

The following Section outlines the necessary steps for installing and running the described testingtools which are enclosed on the attached CD-ROM:

Prerequisites: As mentioned before a working Sun JRE>= 1.6.0 is necessary in order to run the toolscorrectly. Check the manual of your operating system on how to install the Sun JRE.

Installation: The installation is fairly simple. Insert the enclosed CD-ROM and double click on theinstall.sh executable shell script in the CD-ROM’s root directory and follow the on screen instruc-tions. Alternatively open a terminal and enter the following commands:

Chapter 9. User Manual & Development Guide | 143

(a) Welcome Screen (b) Readme Screen (c) License Screen

Figure 9.1: Installer

# cd /media/<CD-ROM DRIVE>

# java -jar Binaries/Installer.jar

If you have to install the tools on a remote machine (e.g. accessed through a SSH client) where nographical user interface is available to you, you can alternatively install them in a text-only mode byadding the -console parameter to the installer like shown in the following snippet:

# java -jar Installer.jar -console

Figure 9.1 shows the first three steps of the installer. The installer has been created using the freeIzPack project. IzPack is a one-stop solution for packaging, distributing and deploying applications.It is fully cross-platform and generates a single installer. As such, it is an alternative to native solutionssuch as platform-specific installers and package managers. IzPack-generated installers only requirea Java virtual machine to run1.

9.1.3 Running the Meta Layer Extraction Tool

Using a framework specific parser to generate a Meta Layer XML file is pretty straightforward and canbe run by issuing the following command inside a terminal:

# jar struts1parser.jar

java -jar struts1parser.jar [options...] <DIRECTORY>

-o FILE : write the Meta Layer XML structure to this file

Whereas the <DIRECTORY> is the top level directory of a web application’s source and configurationfiles. The output of such a framework specific parser is a Meta Layer XML file as described in Section6.2. Listing 9.1 shows a typical output shown when the Meta Layer XML file is being created from aprovided web application’s source and configuration files.

1 [+] Found 95 files

2 [+] Creating PicoContainer ...

3 [+] Wiring PicoContainer ...

4 [+] Found Java class: ../ rbacstruts1/CheckUserCommand.java

5 [+] Found Java class: ../ rbacstruts1/Constants.java

6 [+] Found Java class: ../ rbacstruts1/LogOffAction.java

7 [+] Found Java class: ../ rbacstruts1/LoginAction.java

8 [+] Found Java class: ../ rbacstruts1/LoginForm.java

1http://izpack.org/ [13/03/11]


http://izpack.org/

9 [+] Found Java class: ../ rbacstruts1/SuccessAction.java

10 [+] Found Java class: ../ rbacstruts1/User.java

11 [+] Found Web XML Config: ../ webapp/WEB -INF/web.xml

12 [+] Found Struts Config: ../ webapp/WEB -INF/struts -config.xml

13 [+] Writing XML file to: metalayer.xml

14 [+] Stopping PicoContainer ...

Listing 9.1: Typical output from a Framework specific Parser while generating the Meta Layer XMLFile

9.1.4 Running the RBAC Spider

Listing 9.2 contains a list of all the options available to the user for tweaking the spidering processdone by the RBAC spidering tool to retrieve and process HTML pages and transitions.

1 # java -jar rbacspider.jar

2 java -jar rbacspider.jar [options ...] FILE TARGET

3

4 --access -denied -page VAL : Matches the content of a received HTML

5 resource against the following

6 expression and marks it as an access

7 denied if a match has been found

8 --check -ssl : Whether SSL certificates should

9 be checked

10 --css -enabled : Whether CSS parsing should be

11 enabled or not

12 --debug : Starts the application in debug mode

13 --export -graph FILE : Exports the spidered data in the

14 GraphML format

15 --follow -regex VAL : Only follow the URLS matching

16 this expression

17 --forceful -browsing : Whether forceful browsing should

18 be enabled or not

19 --ignore -htmlelements -regex VAL : Html elements to ignore when spidering

20 --ignore -regex VAL : The spider will ignore URLS matching

21 the following expression

22 --javascript -enabled : Whether JavaScript parsing should

23 be enabled or not

24 --max -spidering -level N : How many levels the spider should examine

25 --max -spidering -time N : Stops the spidering after x minutes

26 --max -threads N : Number of max spidering threads

27 --only -forward : Whether the spider should

28 stay on the same domain

29 --proxy -host VAL : Proxy Host

30 --proxy -password VAL : Proxy Password

31 --proxy -port N : Proxy Port

32 --proxy -username VAL : Proxy Username

33 --sessionId -regex VAL : Regular expression used to find

34 session identifiers in the URL

Listing 9.2: Command Line Options for the RBACSpider

The available command line options are pretty self explanatory and reasonable default values havebeen chosen so that a user in most cases and for most standard web application does not have toprovide any arguments to the RBAC Spider.

Additionally, the user has the possibility to interactively manipulate the crawler during a spideringprocess. This might be necessary for example when the user did not specify a termination criteria


through command line arguments such as −−max-spidering-level, −−max-spidering-timeprior to launching the process but wants to stop the procedure now. The following options are avail-able to the user:

• Stop Spider threads by entering ’x’: If the user enters x followed by pressing ENTERthe current spidering process will be immediately terminated and the next step in the spideringprocedure is being started as described in Section 6.4.6. Forcefully stopping the spidering pro-cedure with this feature is not encouraged and leads to false positives because this prevents thetesting framework to map and access all the discovered URLs with each user role. This missinginformation leads to false positives in the Analysis Layer when the <access-restrictions>rules are being compared with the collected data as described in Section 6.7.1 and Section7.6. The most likely false positive error message is User X should have access but has

not! and is indeed because the information is simply missing due to premature terminationof the spidering process.

• Disable discovering new URLs by entering ’c’: This feature is similar to stoppingthe spidering process as described above but reduces the number of false positives. The dif-ference is that the spidering process is not terminated immediately but rather the already dis-covered URLs are being properly processed as described in Section 6.4.6 and solely newly dis-covered URLs are being ignored and not saved in the spidering queue. This generally leads tofewer missing information and thus reduces the creation of false positives in the Analysis Layeras described in Section 6.7.1 and Section 7.6.

• Show Spider statistics by entering ’s’: If the user enters s followed by pressingENTER, a short statistic is displayed about how man threads are currently running and howmuch more URLs have to be processed by the spidering component before the next phase isbeing initiated as described in Section 6.4.6. A typical output might look like the following:Running threads [10] Remaining URLs [200].

9.1.5 Running the RBAC Visualiser

The RBAC Visualiser can be started by issuing the command java -jar rbacvisualiser.jar orby double clicking on the JAR archive. Additionally the RBAC Visualiser provides one command lineargument which is essential for integrating the developed security testing tool into an automatedsecurity testing framework such as ASTF. The available command line options are shown in Listing9.3.

1 # java -jar rbacvisualiser.jar

2 java -jar rbacvisualiser.jar [--export <OUTPUTFILE > <ARCHIVE >]

3

4 --export FILE : Exports the found problems as a XML file

Listing 9.3: Command Line Options for the RBACVisualiser

If the −−export flag is being provided the RBAC Visualiser does not show a graphical user interfaceto the user, instead it behaves like a common command line only application and analyses the previ-ously collected data from the spidering process and generates a XML report containing found issueswhich normally are being displayed to the user as seen in Figure 7.3. The following snippet showshow such a XML file is being created:

# java -jar rbacvisualiser.jar \

--export out.xml 2011-03-24-localhost.rbacspider


Listing 9.4 shows an example of such a created XML report containing the found issues by comparingthe <access-restrictions> information from the Meta Layer XML file with the actual collecteddata from the spidering process.

1 <?xml version =’1.0’ encoding=’UTF -8’?>

2 <rbacvisualiser >

3 <info >

4 <created >Sat Mar 26 13:32:53 2011</ created >


6 </info >


8 url="http :// localhost :8080/ administrator/v1.do" username =" EMPLOYEE">

9 EMPLOYEE has access but should have been denied!

10 </error >


12 url="http :// localhost :8080/ administrator/v1.do" username =" MANAGER">

13 MANAGER has access but should have been denied!

14 </error >


16 url="http :// localhost :8080/ administrator/v1.do" username ="CLERK">

17 CLERK has access but should have been denied!

18 </error >

19 </rbacvisualiser >

Listing 9.4: Sample XML Report created from the found Vulnerabilities

Selecting and Editing Graph

The RBAC Visualiser provides mouse and keyboard shortcuts for modifying the visualised graph.Most important is the Graph menu in the menu bar as seen in Figure 9.2 (a). A user is able to clusterthe graph differently, apply colours, filter vertices and show or hide vertex lables.

The visualiser provides two mouse modes, namely Transforming and Picking. The mouse modescan be changed by right clicking the yellow graph panel as seen in Figure 9.2 (b) and Figure 9.6. TheTransforming mode is used to move the hole graph on the screen by holding the left mouse buttonpressed and moving the mouse.

The Picking mode provides the ability to select single or multiple vertices by left clicking them. If thegraph has previously been clustered, the user is able to select all vertices belonging to a particularcluster by selecting one vertex from the cluster by simultaneously pressing CTRL as seen in Figure9.2 (c). Double clicking a particular vertex opens an additional tab in the lower tab bar containinginformation about that particular vertex such as: which user had access to that resource, how manyin or out transitions can be made etc.

Additionally, the user is able to zoom in or out by using the mouse wheel no matter what mouse modehas been set.

Exporting the Graph

For interoperability reasons the RBAC Visualiser offers the ability to export the collected data fromthe spidering process as a GraphML file. GraphML is a comprehensive and easy-to-use file format forgraphs. It consists of a language core to describe the structural properties of a graph and a flexible ex-


(a) Graph Options (b) Changing Mouse Mode (c) Selecting Vertices

Figure 9.2: RBAC Visualiser

tension mechanism to add application-specific data2. This is very convenient for a user who wants tovisualise the data in another tool such as Gephi. Gephi is an interactive visualisation and explorationplatform for all kinds of networks and complex systems, dynamical and hierarchical graphs. Gephihas at one point been considered as a basis for the RBAC Visualiser but its code base is not looselycoupled enough to allow other tools use its visualisation capabilities. Of course by using a differentgraph visualisation software the user looses the unique features of the RBAC Visualiser for displayingdiscrepancies in the access control checks.

9.1.6 Video Tutorials

The enclosed CD-ROM contains various tutorial videos in the OGG video format. These videosshow how to configure the command line tools and how to start them. The videos are located inthe root directory of the CD-ROM in a folder labelled Videos. The videos have been recorded usinggtk-recordmydesktop version v0.3.8.1 on a Ubuntu 10.10 Linux operating system.

9.2 Development Guide

The development guide is targeted at users who want to extend the functionality of the tools andbuild them from source on their own machines. The following Sections contain detailed instructionson how to build the security testing tools and what kind of tools and libraries have been used duringthe initial development phase.

9.2.1 Prerequisites

The following Section outlines the prerequisites needed for building the tools from source.

Sun Java 1.6 JDK: Development and testing has been done on a Ubuntu 10.10 Linux operating sys-tem with the Java(TM) SE Runtime Environment (build 1.6.0_24-b07) and Java(TM) SE DevelopmentKit (1.6.0_20) 32 and 64 bit. Check the manual of your Linux operating system on how to install theSun JRE and JDK.

Maven 2: Apache Maven is a software project management and comprehension tool. Based on theconcept of a project object model (POM), Maven can manage a project’s build, reporting and doc-umentation from a central piece of information. If you are using the Ubuntu operating system youcan install Maven by simply typing: sudo apt-get install maven into a terminal. See the Maven

2http://graphml.graphdrawing.org/ [27/03/11]


http://graphml.graphdrawing.org/

documentation on how to install Maven for your specific operating system(http://maven.apache.org/download.html#Installation).

9.2.2 Installing the Development Environment

Insert the enclosed CD-ROM and double click on the install.sh executable shell script in the CD-ROM’s root directory and follow the on screen instructions of the installer. Alternatively open a ter-minal and enter the following commands:

# cd /media/<CD-ROM DRIVE>

# java -jar Binaries/Installer.jar

Make sure that you select the Sources option in the package selection panel in order to install thesource files needed for extending and building the tools from source. If you have to install the sourceson a remote machine (e.g. accessed through a SSH client) where no graphical user interface is avail-able to you, you can alternatively install them in a text-only mode by adding the -console parameterto the installer like shown in the following snippet:

# java -jar Installer.jar -console

After successfully installing the Java source files, several Java libraries needed for building the securitytools have to be added to the local Maven repository on the development machine. This is necessarybecause these specific libraries are not part of an official and online available Maven repository. Opena terminal and enter the following commands in order to add the necessary libraries to the localMaven repository:

# cd <Installation Directory>/src/

# ./install-missing-maven-deps.sh

The install-missing-maven-deps.sh script has to be executed only once after installing thesources. See Section 9.2.4 for a complete list of used free/third party libraries that have been used.

9.2.3 Building with Maven

Building the security testing tools from source is fairly easy and is taken care of by Maven. Enter thefollowing commands into a terminal for building the projects from the previously installed sourcefiles:

# cd <Installation Directory>/src/

# mvn clean package source:jar -Dmaven.test.skip

Alternatively, if you want to skip the JUnit tests which are being automatically executed by Mavenafter the build process you can provide the -Dmaven.test.skip parameter to the mvn command.For more information about specific Maven commands visit http://maven.apache.org/.

9.2.4 Free/Third-Party Libraries Used

The security testing tools rely heavily on several free and third party libraries that provide the neededfunctionality. Table 9.2 provides a list of all the libraries used and their specific licensing scheme.


http://maven.apache.org/download.html#Installation

http://maven.apache.org/

9.2.5 Use logback for debugging purposes

The developed tools use the free SLF4J project to provide extensive logging capabilities for its users.The Simple Logging Facade for Java (SLF4J) serves as a simple facade or as an abstraction for variouslogging frameworks, e.g. java.util.logging, log4j and logback, allowing the end user to plugin the desired logging framework at deployment time. By making modifications to the logback.xmlfile inside the packaged jar files, the logging details can be increased in order to create a more verboseoutput for debugging and program error identification purposes.

9.3 RBACVisualiser Screenshots

Figures 9.3 to 9.16 show the developed visualisation tool and the most interesting features. Someadditional features such as filtering based on users and other non clustering view modes have inten-tionally been left out but can be enabled by clicking on the Graph menu after opening a rbacpsiderarchive file.

• Figure 9.3: Shows the initial view after opening the collected data from a previous spideringwith the web crawler component. The collected data visualised in these screenshots are comingfrom the test case described in Section 7.4. The data is clustered based on the URL paths of thefound URLs as described in Section 6.7.3 and is the default view mode after opening a project.At the bottom the URL List Tab is clearly visible which contains all the discovered URLs in thetargeted application and the HTTP responses received from the web server during the spideringprocess.

• Figure 9.4: At the bottom the Problems Tab contains the issues found during the analysis of thecollected data as described in Section 6.7.1. Severe issues are being marked with a red exclama-tion mark when an <access-restriction> rule has been clearly violated. Other non severeissues are being marked with a yellow exclamation mark and might involve false positives andHTTP error codes received from the web server. False positives might occur when the spideringprocess has been terminated early by the user and the crawling component could not accessall the discovered URLs with all available users which leads to an incomplete mapping of thetargeted web application.

• Figure 9.5: At the bottom the Access Restrictions Tab contains the configured<access-restriction> rules from the Meta Layer XML file used during the spideringprocess as described in Sections 6.2.1 and 6.7.1. Figure 9.10 shows how rules can be added oredited.

• Figure 9.6: The visualisation tool has two mouse modes used to select and edit the displayedgraph as described in Section 9.1.5. The visualiser provides two mouse modes, namely Trans-forming and Picking. The mouse modes can be changed by right clicking the yellow graphpanel.

• Figure 9.7: Shows the context menu of the URL List Tab which can be accessed by selectingone or multiple entries from the URL List table and clicking on the right mouse button. Theuser then has the possibility to highlight the corresponding nodes in the graph which belong tothe selected entries or inspect the HTTP response and request sent by crawling component toquery the selected URLs as seen in Figure 9.8.

• Figure 9.8: Shows the HTTP request and response dialog from a selected URL List entry as seenin Figure 9.8. The dialog shows the raw request and response received from the web serverwhen the crawling component tried to access the selected URL with a particular user.


• Figure 9.9: Shows the context menu of the Problems Tab which can be accessed by selectingone or multiple entries from the Problems table and clicking on the right mouse button. Theuser then has the ability to highlight the corresponding nodes in the graph which belong to theselected issues.

• Figure 9.10: Shows the context menu of the Access Restrictions Tab which can be accessed byselecting one or multiple entries from the Access Restrictions table and clicking on the rightmouse button. The user then has the ability to highlight the corresponding nodes in the graphwhich belong to the selected <access-restriction> rule configured in the Meta Layer XMLfile. The user then furthermore has the ability to edit or add new <access-restriction>

rules as seen in Figure 9.11.

• Figure 9.11: The <access-restriction> dialog shown in this Figure allows a user to edit oradd new <access-restriction> rules to the previously loaded project and Meta Layer XMLfile. Changes made to the <access-restriction> rules stay in memory and are not beingsaved until the user clicks on File→ Export Meta Layer XML File... and saves the changes to anew Meta Layer XML file.

• Figure 9.12: This Figure shows the Graph → Layouts menu which allows the user to clus-ter and re-arrange the displayed graph according to his preferences. These view modes areespecially important when the user wants to manually or rather visually inspect the col-lected data from the spidering process for any issues that have not yet been covered with an<access-restriction> rule. The various view modes are being shown in Figures 9.13 to9.16.

• Figure 9.13: The Cluster By User view mode clusters the collected data based on two users asdescribed in Section 6.7.2. The here shown dialog lets the user choose on which two users hewants the graph to be clustered on. The re-arranged and clustered graph is shown in Figure9.14. This feature is only available if there have two or more users been configured in the MetaLayer XML file.

• Figure 9.14: Shows the re-arranged, clustered and re-coloured graph after the user has selectedtwo users from the dialog shown in Figure 9.13. The exact details on the clusering algorithm isdescribed in Section 6.7.2.

• Figure 9.15: The following view mode is based on the algorithm described in Section 6.7.4which clusters the graph based on URL paths and re-colours the graph based on two selectedusers similar to the dialog shown in Figure 9.13.

• Figure 9.16: Shows the same view mode as in Figure 9.15 but zoomed in on the found issue inthe Problems Tab by comparing the actual data with the configured <access-restriction>

rules in the Meta Layer XML file. The violation is clearly visible in the cluster on the right whichshould only be accessible by the administrator user (coloured red) but one node is also acces-sible by a user with fewer privileges (gradient coloured).


Figure 9.3: RBACVisualiser: Initial view with URL List Tab

Figure 9.4: RBACVisualiser: Initial view with Problems Tab


Figure 9.5: RBACVisualiser: Initial view with Access Restrictions Tab

Figure 9.6: RBACVisualiser: Mouse Modes


Figure 9.7: RBACVisualiser: URL List Context Menu

Figure 9.8: RBACVisualiser: URL List HTTP Request and Response Window


Figure 9.9: RBACVisualiser: Problems Context Menu

Figure 9.10: RBACVisualiser: Access Restrictions Context Menu


Figure 9.11: RBACVisualiser: Add/Edit Access Restrictions Window

Figure 9.12: RBACVisualiser: Graph Menu


Figure 9.13: RBACVisualiser: Cluster by User... Menu

Figure 9.14: RBACVisualiser: Cluster by User... View


Figure 9.15: RBACVisualiser: Re-Coloured by User... View

Figure 9.16: RBACVisualiser: Re-Coloured by User... View (Zoomed In)


Mav

enA

rtifi

catI

DVe

rsio

nU

RL

Lic

ense

:D

escr

ipti

on

args

4j2.

0.12

htt

ps://

args

4j.d

ev.ja

va.n

et/

MIT

args

4jis

asm

allJ

ava

clas

sli

bra

ryth

atm

akes

itea

syto

par

seco

mm

and

lin

eo

pti

on

s/ar

gum

ents

inyo

ur

CU

Iap

pli

cati

on

.p

ico

con

tain

er2.

11h

ttp

://p

ico

con

tain

er.o

rg/

BSD

styl

eP

ico

Co

nta

iner

isa

tin

yem

bed

dab

leco

nta

iner

for

Co

nst

ruct

or

Dep

end

ency

Inje

ctio

n(C

DI)

Inve

r-si

on

ofC

on

tro

l(Io

C)

Java

com

po

nen

ts.

neo

4j1.

2h

ttp

://n

eo4j

.org/

Ap

ach

eLi

cen

se,v

3N

eo4j

isa

grap

hd

atab

ase,

afu

llytr

ansa

ctio

nal

dat

abas

eth

atst

ore

sd

ata

stru

ctu

red

asgr

aph

s.gr

emlin

0.7

htt

ps://

gith

ub.

com/t

inke

rpo

p/g

rem

linB

SDst

yle

Gre

mli

nis

ad

om

ain

spec

ific

lan

guag

efo

rtr

aver

s-in

gp

rop

erty

grap

hs.

blu

epri

nts

0.5-

SNA

PSH

OT

htt

ps://

gith

ub.

com/t

inke

rpo

p/b

luep

rin

tsB

SDst

yle

Blu

epri

nts

isa

set

of

inte

rfac

esan

dim

ple

men

ta-

tio

ns

oft

he

pro

per

tygr

aph

dat

am

od

el.

jun

g-ap

i2.

0.1

htt

p://j

un

g.so

urc

efo

rge.

net/

BSD

styl

eJU

NG

isa

Java

-bas

edo

pen

-so

urc

eso

ftw

are

lib

rary

des

ign

edto

sup

po

rtth

em

od

elin

g,an

alys

is,

and

visu

aliz

atio

no

fd

ata

that

can

be

rep

rese

nte

das

grap

hs.

jun

it4.

8.1

htt

p://w

ww

.jun

it.o

rg/

CP

Lv1

JUn

itte

stin

gfr

amew

ork

com

mo

ns-

logg

ing

1.1.

1h

ttp

://c

om

mo

ns.

apac

he.

org/l

ogg

ing/

Ap

ach

eLi

cen

se,v

2Lo

ggin

gFr

amew

ork

slf4

j-ap

i1.

6.0

htt

p://w

ww

.slf

4j.o

rg/

MIT

Logg

ing

Fram

ewo

rklo

gbac

k-co

re0.

9.21

htt

p://l

ogb

ack.

qo

s.ch/

LGP

Lv2

.1Lo

ggin

gFr

amew

ork

com

mo

ns-

net

2.0

htt

p://c

om

mo

ns.

apac

he.

org/

Ap

ach

eLi

cen

se,v

2C

olle

ctio

no

fnet

wo

rku

tili

ties

and

pro

toco

lim

ple

-m

enta

tio

ns.

com

mo

ns-

valid

ato

r1.

3.1

htt

p://c

om

mo

ns.

apac

he.

org/

Ap

ach

eLi

cen

se,v

2Fr

amew

ork

tod

efin

eva

lid

ato

rsan

dva

lid

atio

nru

les

ina

XM

Lfi

le.

h2

1.2.

127

htt

p://w

ww

.h2d

atab

ase.

com/

EP

L1.

0Ja

vaSQ

Ld

atab

ase

htm

lun

it2.

8h

ttp

://h

tmlu

nit

.so

urc

efo

rge.

net/

Ap

ach

eLi

cen

se,v

2H

tmlU

nit

isa

GU

I-Le

ssb

row

ser

for

Java

pro

gram

s.js

on

-sim

ple

1.1

htt

p://c

od

e.go

ogl

e.co

m/p/j

son

-sim

ple/

Ap

ach

eLi

cen

se,v

2JS

ON

.sim

ple

isa

sim

ple

Java

too

lkit

for

JSO

N.Y

ou

can

use

JSO

N.s

imp

leto

enco

de

or

dec

od

eJS

ON

text

.u

tils

1.07

.00

htt

p://o

ster

mill

er.o

rgG

PL

Lib

rari

esfo

rco

mm

on

pro

gram

min

gta

sks

such

asC

SVfi

les

and

Bas

e64

enco

din

g.o

ro2.

0.8

htt

p://j

akar

ta.a

pac

he.

org/o

ro/

Ap

ach

eLi

cen

se,v

2T

he

Jaka

rta-

OR

OJa

vacl

asse

sar

ea

set

of

text

-p

roce

ssin

gJa

vacl

asse

sth

atp

rovi

de

Per

l5co

mp

at-

ible

regu

lar

exp

ress

ion

s.

Tab

le9.

2:U

sed

Free/T

hir

dPa

rty

Lib

rari

es


List of Figures

2.1 Web Tier and Java EE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Java EE Server and Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Java EE Platform APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

4.1 Architecture of a general combined White- and Black Box Security Testing Tool . . . . . . . 61

6.1 Framework Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846.2 MetaLayer API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916.3 Framework Specific Parser UML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926.4 JavaScript Execution Time (Logarithmic Scale) in various Browsers for HTML DOM

Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016.5 Simplified UML 2 Activity Diagram showing the Main-Flow of the Web Spider . . . . . . . . 1036.6 Simplified UML 2 Activity Diagram for the Non-Invasive-Smartly-Filling-Out HTML

Form Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.7 UrlQueue Database Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.8 Storing the crawled data in a graph optimised Database . . . . . . . . . . . . . . . . . . . . . . . . 1116.9 Clustering by Users / Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1146.10 Clustering by URL Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.11 Clustering by URL Paths and Re-Colouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1166.12 Clustering by URL Paths and showing differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1166.13 GUI Design Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

7.1 Struts 1.3.10 Test Web Application - URL Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1227.2 Initial view of the collected Data from the Struts Test Application . . . . . . . . . . . . . . . . . 1287.3 Automatically detected Access Restriction Violations in the Struts Test Application . . . . 1297.4 Re-Colouring of the Data based on Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1297.5 Automatically detected Access Restriction Violations in the WordPress Test Application . 1347.6 Standard User Secure Messaging Service Email Inbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 1357.7 Analysing the collected Data from the Secure Messaging Service (Clustered by URL Paths)1397.8 Analysing the collected Data from the Secure Messaging Service (Clustered by Users) . . . 139

9.1 Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1449.2 RBAC Visualiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1489.3 RBACVisualiser: Initial view with URL List Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1529.4 RBACVisualiser: Initial view with Problems Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1529.5 RBACVisualiser: Initial view with Access Restrictions Tab . . . . . . . . . . . . . . . . . . . . . . . . 1539.6 RBACVisualiser: Mouse Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1539.7 RBACVisualiser: URL List Context Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1549.8 RBACVisualiser: URL List HTTP Request and Response Window . . . . . . . . . . . . . . . . . . 1549.9 RBACVisualiser: Problems Context Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1559.10 RBACVisualiser: Access Restrictions Context Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1559.11 RBACVisualiser: Add/Edit Access Restrictions Window . . . . . . . . . . . . . . . . . . . . . . . . . 1569.12 RBACVisualiser: Graph Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

List of Figures | 160

9.13 RBACVisualiser: Cluster by User... Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.14 RBACVisualiser: Cluster by User... View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.15 RBACVisualiser: Re-Coloured by User... View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1589.16 RBACVisualiser: Re-Coloured by User... View (Zoomed In) . . . . . . . . . . . . . . . . . . . . . . . 158

List of Figures | 161

List of Tables

2.2 Popularity of Open Source Java Web Application Frameworks (November 2010) . . . . . . . 292.4 Java Web Application Frameworks Security Features Comparison . . . . . . . . . . . . . . . . . 45

6.2 Description of the UrlQueue Entity Relationship Model . . . . . . . . . . . . . . . . . . . . . . . . 107

7.1 Access Rights of the Sample Struts 1.3.10 Web Application . . . . . . . . . . . . . . . . . . . . . . 130

9.2 Used Free/Third Party Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

List of Tables | 162

Listings

2.1 Simple Java HelloWorld Servlet (Specification 2.5) . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Deployment Descriptor for the HelloWorld Servlet . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Simple Java Servlet (Specification 3.0) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Simple JSP Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5 Simple JSTL usage Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.6 Hello World Java Bean (POJO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.7 Hello World JSF Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.8 Specifying Security Roles using Deployment Descriptor elements . . . . . . . . . . . . . . . . . 112.9 @DeclareRoles Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.10 @RunAs Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.11 @ServletSecurity Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.12 @RolesAllowed Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.13 Sample Spring Security XML Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.14 @PreAuthorize Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.15 @PreAuthorize Annotation Example with SpEL Directive . . . . . . . . . . . . . . . . . . . . . . 192.16 @PostFilter Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.17 @PreFilter Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.18 ESAPI Example on how to do Input Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.19 ESAPI WAF Configuration Example on how to enforce Authorisation Constraints . . . . . . 222.20 Apache Shiro Ad-Hoc Filter Chain Configuration Example . . . . . . . . . . . . . . . . . . . . . . 232.21 jGuard User Permissions Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.22 Spring MVC Controller Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.23 Spring HandlerInterceptor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.24 Struts 1.x struts-config.xml Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.25 Struts 1.x Action example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.26 Apache Struts 1.x Declarative Access Control Example . . . . . . . . . . . . . . . . . . . . . . . . . 332.27 Apache Struts Programmatic Access Control Example . . . . . . . . . . . . . . . . . . . . . . . . . 332.28 Apache Struts Plugin Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.29 Apache Struts RequestProcessor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.30 Apache Struts ActionServlet Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . 362.31 Struts 2 Annotation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.32 Struts 2.x struts.xml Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.33 Struts 2.x Action Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.34 Struts 2.x RolesInterceptor Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . 382.35 Struts ExtVal usage example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.36 MyFaces Custom Lifecycle Factory Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.37 MyFaces Custom Lifecycle Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.38 Simple Wicket Page with link Counter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.39 Wicket Page Layout for link Counter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.40 Wicket-Sercurity Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.1 Abstracting Application-Level Web Security with a Policy Description Language Example 47

Listings | 163

3.2 JFlow Password File Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483.3 Class encapsulating the Message-Delivery Functionality . . . . . . . . . . . . . . . . . . . . . . . 503.4 Class to exercise the Message-Delivery Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . 503.5 Class encapsulating the Message-Delivery Functionality (with Authentication) . . . . . . . 513.6 Authentication Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.7 AspectJ Tracing Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.8 AspectJ Tracing output for a simple Struts 1.x Application . . . . . . . . . . . . . . . . . . . . . . . 553.9 PQL Query for finding SQL Injection Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.10 Aspect for catching all Struts 1.x execute Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.11 Aclamate RBC Configuration Example execute Methods . . . . . . . . . . . . . . . . . . . . . . . 58

4.1 Struts 2.x XWorks Validation Example with Annotations . . . . . . . . . . . . . . . . . . . . . . . . 624.2 Struts 2.x XWorks Validation Example with XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.3 Spring 3.x Validation Example with Annotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.4 Spring 3.x Validation Example in a Controller Class . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.5 Wicket Validation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.6 ESAPI Example on how to use the AccessReferenceMap Class . . . . . . . . . . . . . . . . . . 67

6.1 Meta Layer XML Structure: <info> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.2 Meta Layer XML Structure: <classes> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.3 Meta Layer XML Structure: <users> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.4 Meta Layer XML Structure: <roles> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.5 Meta Layer XML Structure: <access-restrictions> Tag . . . . . . . . . . . . . . . . . . . . . 876.6 Meta Layer XML Structure: <login> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.7 Meta Layer XML Structure: <urls> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896.8 Meta Layer XML Structure: <views> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896.9 Meta Layer XML Structure: <form-data> Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 906.10 Meta Layer XML File generated from a Struts 1.x Sample Application . . . . . . . . . . . . . . 946.11 Short HtmlUnit Example for submitting a HTML Login Form through a Proxy Server . . . 986.12 Dynamically created HTML Login Form with embedded JavaScript Code . . . . . . . . . . . 996.13 Sample XML Report created from found Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

7.1 Checking Access Restrictions in JSP Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237.2 Initial metalayer.xml File for the Test Application . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237.3 Additional Information added to the metalayer.xml File (displayed as a diff -up

FILE1 FILE2 output) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1257.4 metalayer.xml File for a Standard WordPress 3.1 Installation . . . . . . . . . . . . . . . . . . . 1307.5 /wordpress/wp-login.php HTML Source Code containing the HTML based Au-

thentication Form of WordPress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1317.6 Secure Messaging Service HTML based Authentication Form . . . . . . . . . . . . . . . . . . . . . 1357.7 metalayer.xml File for the Secure Messaging Service . . . . . . . . . . . . . . . . . . . . . . . . . 1357.8 Crawling Command Line Arguments for the Secure Messaging Service . . . . . . . . . . . . . . 136

9.1 Typical output from a Framework specific Parser while generating the Meta Layer XMLFile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

9.2 Command Line Options for the RBACSpider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1459.3 Command Line Options for the RBACVisualiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1469.4 Sample XML Report created from the found Vulnerabilities . . . . . . . . . . . . . . . . . . . . . 147

Listings | 164

Bibliography

[1] D. Alhadidi, N. Belblidia, and M. Debbabi. Aspectj assessment from a security perspective. InWorkshop on Practice and Theory of IT Security, PTITS’2006, Montreal, Quebec, Canada, May17-19 2006.

[2] Dima AlHadidi, Nadia Belblidia, and Mourad Debbabi. Security crosscutting concerns and as-pectj. In Proceedings of the 2006 International Conference on Privacy, Security and Trust: Bridgethe Gap Between PST Technologies and Business Services, PST ’06, pages 45:1–45:1, New York, NY,USA, 2006. ACM.

[3] Chris Allan, Pavel Avgustinov, Aske Simon Christensen, Laurie Hendren, Sascha Kuzins, OndrejLhoták, Oege de Moor, Damien Sereni, Ganesh Sittampalam, and Julian Tibble. Adding tracematching with free variables to aspectj. In Proceedings of the 20th annual ACM SIGPLAN confer-ence on Object-oriented programming, systems, languages, and applications, OOPSLA ’05, pages345–364, New York, NY, USA, 2005. ACM.

[4] P. Boldi and S. Vigna. The webgraph framework i: compression techniques. In Proceedings of the13th international conference on World Wide Web, WWW ’04, pages 595–602, New York, NY, USA,2004. ACM.

[5] A. Boukhtouta, D. Alhadidi, and M. Debbabi. Capturing returned computable values in aspectjfor security hardening. Inf. Sec. J.: A Global Perspective, 18:87–98, January 2009.

[6] A. Boukhtouta, D. Alhadidi, and M. Debbabi. A practical framework for the dataflow pointcutin aspectj. In Availability, Reliability and Security, 2009. ARES ’09. International Conference on,pages 835 –840, 2009.

[7] Kung Chen and Chih-Mao Huang. On designing access control aspects for web applications. InAOSD 05 Workshop on Software-engineering Properties of Languages and Aspect Tehchnologies(SPLAT 05), Chicago, 2005.

[8] Greg Conti. Security Data Visualization. No Starch Press, San Francisco, CA, USA, 2007.

[9] Martijn Dashorst and Eelco Hillenius. Wicket in Action. Manning Publications Co., Greenwich,CT, USA, 2008.

[10] Kevin Denver. Development and Integration of ASTF-Plugins in the field of Web ApplicationVulnerabilities. MSE Project Thesis 1, February 2009.

[11] Kevin Denver. Evaluation geeigneter Plugins im Bereich Web Application Vulnerabilities für dieIntegration ins ASTF. MSE Seminary Thesis 1, November 2009.

[12] Kevin Denver. Development of a combined white and black box security testing tool for detect-ing injection flaws in web applications. MSE Project Thesis 2, August 2010.

[13] Kevin Denver. Preliminary Study of white box security testing for the integration into the ASTFframework. MSE Seminary Thesis 2, February 2010.

Bibliography | 165

[14] Paul Duvall, Stephen M. Matyas, and Andrew Glover. Continuous Integration: Improving Soft-ware Quality and Reducing Risk (The Addison-Wesley Signature Series). Addison-Wesley Profes-sional, 2007.

[15] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. HypertextTransfer Protocol – HTTP/1.1. RFC 2616 (Draft Standard), June 1999. Updated by RFC 2817.

[16] J. Franks, P. Hallam-Baker, J. Hostetler, S. Lawrence, P. Leach, A. Luotonen, and L. Stewart. HTTPAuthentication: Basic and Digest Access Authentication. RFC 2617 (Draft Standard), June 1999.

[17] Marshall Hall, Jr. Combinatorial theory (2nd ed.). John Wiley & Sons, Inc., New York, NY, USA,1998.

[18] Rod Johnson. Expert One-on-One J2EE Design & Development. Wrox Press Ltd., Birmingham,UK, UK, 2002.

[19] Bart Kummel. Apache MyFaces 1.2 Web Application Development. Packt Publishing, 1st edition,2010.

[20] Ramnivas Laddad. AspectJ in Action: Enterprise AOP with Spring Applications. Manning Publi-cations Co., Greenwich, CT, USA, 2nd edition, 2009.

[21] Michael Martin, Benjamin Livshits, and Monica S. Lam. Finding application errors and securityflaws using pql: a program query language, 2005.

[22] Raffael Marty. Applied Security Visualization. Addison-Wesley Professional, 1 edition, 2008.

[23] Hidehiko Masuhara and Kazunori Kawauchi. Dataflow pointcut in aspect-oriented program-ming. In In APLAS’03 - the First Asian Symposium on Programming Languages and Systems,pages 105–121, 2003.

[24] Hidehiko Masuhara and Gregor Kiczales. Modeling crosscutting in aspect-oriented mecha-nisms. pages 2–28. Springer-Verlag, 2003.

[25] H. D. K. Moonesinghe and Pang-Ning Tan. Outrank: a graph-based outlier detection frameworkusing random walk. International Journal on Artificial Intelligence Tools, 17(1):19–36, 2008.

[26] Peter Mularien. Spring Security 3. Packt Publishing, 2010.

[27] Andrew C. Myers. Jflow: Practical mostly-static information flow control. In In Proc. 26th ACMSymp. on Principles of Programming Languages (POPL, pages 228–241, 1999.

[28] Andrew Prunicki and Tzilla Elrad. Aclamate: An aosd security framework for access control.In Proceedings of the 2nd IEEE International Symposium on Dependable, Autonomic and SecureComputing, DASC ’06, pages 293–300, Washington, DC, USA, 2006. IEEE Computer Society.

[29] Raghu Ramakrishnan and Johannes Gehrke. Database Management Systems. McGraw-Hill, Inc.,New York, NY, USA, 3 edition, 2003.

[30] Ian Roughley. Practical Apache Struts 2 Web 2.0 Projects (Practical Projects). Apress, Berkely, CA,USA, 2007.

[31] David Scott and Cb Pz. Abstracting application-level web security. In In World Wide Web, pages396–407, 2002.

Bibliography | 166

[32] Robert Endre Tarjan. Depth-first search and linear graph algorithms. SIAM J. Comput., 1(2):146–160, 1972.

[33] Te Kura Tatau, Roshan Ramachandran, Supervisors David, J. Pearce, and Ian S. Welch. Aspectjfor multilevel security, 2006.

[34] Yuan-Hsin Tung, Shian-Shyong Tseng, Tsung-Ju Lee, and Jui-Feng Weng. A novel approach toautomatic test case generation for web applications. Quality Software, International Conferenceon, 0:399–404, 2010.

[35] Robert J. Walker and Kevin Viggers. Implementing protocols via declarative event patterns. InProceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of soft-ware engineering, SIGSOFT ’04/FSE-12, pages 159–169, New York, NY, USA, 2004. ACM.

[36] Bart De Win, Wouter Joosen, and Frank Piessens. Developing secure applications throughaspect-oriented programming. In Aspect-Oriented Software Development, pages 633–650.Addison-Wesley, 2002.

[37] Bart De Win, Bart Vanhaute, and Bart De Decker. How aspect-oriented programming can helpto build secure software. Informatica (Slovenia), 26(2), 2002.

Bibliography | 167

InIT Institut für angewandte Informationstechnologie

28.10.2010, Marc Rennhard (rema) 1/3

Master Thesis

Student: Kevin Denver Tutor: Prof. Dr. Marc Rennhard

Partner company: PrivaSphere AG Advisor: Prof. Dr. Hans-Peter Hutter

Expert: Dr. Tobias Christen

Start: 8. November 2010 Credits: 27 ECTS (810 h)

End: 15. April 2011

Presentation: TBA

Using Declarative Information in Web Application Frameworks to Improve Security

Introduction and Motivation

Modern Model-View-Controller (MVC) frameworks such as Struts1, Spring

2, or Java Server

Faces3 significantly help designing and developing Java web applications. In contrast to pure

servlet- and/or JSP-based applications, they allow the developers to specify some parts of the

application outside the actual code using XML configuration files or annotations. Examples

are the action mappings to specify (and easily adapt) the workflow in an application or input

validation configurations.

In a seminary thesis [1], it was analyzed how well security testing tools could use such confi-

gurations or annotations to improve their results with respect to detecting security problems.

One result of that analysis was that it may well be possible to use this information to perform

completeness checks (e.g. with respect to input validation) or to supplement the web crawler

component of a web application security testing tool with additional information.

In addition, Ralf Hauser proposed a while ago [2] that extending these web application

frameworks with additional declarative information may help a lot to both make web applica-

tion more secure already during development and enable a special kind of “white-box” testing

tool to make more precise statements and therefore improve testing quality in case the declar-

ative information has not been fully reflected into the code base. An example would be to

provide additional information with respect to access control (e.g. role-based), which de-

scribes what rights are necessary to access certain parts of the application. Assuming this in-

1 http://struts.apache.org

2 http://www.springsource.org

3 http://java.sun.com/javaee/javaserverfaces/overview.html


formation has been provided by the developer, it can be used in several ways (note that this

list is by no means meant to be complete and it’s one essential task during the thesis to define

and analyze different options):

The framework could be extended such that it directly makes use of this access control

information to enforce the corresponding rules. Ideally, the developer would no longer

have to deal with access control by programming the corresponding logic, but only has to

specify the correct rules in a declarative way. This framework extension/plugin would

then deliver all necessary functions to enforce the declarations at runtime.

A testing tool that interacts with the running web application uses the access control in-

formation to check whether access control is handled correctly by the web application. To

do so, the testing tool would get credentials with different privileges and attempt to access

different areas of the web application. Any mismatch between declared access control

rules and actual reachability is flagged as a security vulnerability. Such a testing tool

could not only be limited to role-based access control, but could also take into account

mechanisms such as mandatory access control or information labeling policies.

Using graph theory and analyzing the access control information together with further

information in the configuration files (e.g. allowed transitions between pages), a testing

tool could analyze if some parts of the application cannot be reached at all or if there are

ways to reach a page that requires high access privileges via any path that allows access

with lower privileges.

The goal of this master thesis is to build on this basis and explore the possibilities of using

available and/or new declarative information in web application frameworks to improve secu-

rity.

Task Description

Due to the open nature of this master thesis, we won’t provide a detailed step-by-step task

description as each project phase will significantly depend on the results of the previous phas-

es. We therefore only describe the main phases we expect at the current time, which are the

following:

In the first phase, you should study available web application frameworks (focus on Java-

based ones), the Java language, and related work in the relevant areas. The outcome of

this phase should provide you with the following information:

The current state of the art in the field of web application frameworks4 and Java.

The capabilities of the different frameworks, especially with respect to declarative

statements a developer can include.

How these frameworks can be extended.

What other work has recently been done in the field and published in academic pa-

pers. You should also have a look at approaches in the context of model driven se-

curity5

In the second phase, you should think about using the information already provided by

web application frameworks and how this can be supplemented with additional informa-

4 This should also answer the question whether there exist potential successor concepts to MVC as we

want to make sure the planned improvements in this thesis build on the current (and future) state of the

art.

5 Some first pointers can be found here: http://www.torsten-lodderstedt.de


tion from both the related work analysis and novel ideas by yourself. Your previous work

during the seminary thesis [1] and the ideas with respect to access control should provide

starting points, but you should certainly try to be elaborate and derive further ideas. With

the different proposed ideas, do the following:

Evaluate them with respect to their potential. Ideally, a good idea should be easy to

be used by skilled developers, should be applicable to a wide variety of frameworks

and significantly contributes to improve security.

For each proposal, state precisely how you would use the corresponding informa-

tion to improves security. As stated above, this could mean extending the frame-

work such that it uses the additional information or develop (or extend) possibly

multiple testing tools that can make good usage of this information.

In the third phase, the best-suited ideas derived during the second phase should be imple-

mented. The actual implementation (e.g. framework extension, testing tool development

or both) depends on the actual approaches selected during the second phase. In either

case, consider the following:

The developed software should be of good quality and both usable and extensible

by others.

In general, it is preferable if your selected proposals can be used with many current

web application frameworks. However, it is likely that only one or a few of them

will actually specifically be supported as part of your thesis. In this case, you’ll de-

cide together with the company partner and tutor about what frameworks will be

supported as part of your thesis.

To validate your proposals, test the developed software with different web applica-

tions. If possible, PrivaSphere’s Secure Messaging application should be among

them.

If possible, design the testing tools such that they can easily be integrated into

ASTF.

Deliverables

The deliverables of this thesis are the following:

Developed software

Written report in English, which describes the methodology and the results of your thesis

Oral presentation (40 minutes plus questions)

References

[1] Kevin Denver. Preliminary study of white box security testing for the integration into

the ASTF framework. ZHAW MSE Seminary Thesis, February 2010.

[2] Marc Rennhard and Ralf Hauser. Automated Software Security-Testing for Online-

Applications. KTI project proposal, March 2006.

Appendix B - Meta Layer XML Structure DTD<!ELEMENT accessrestrictions ( url+ ) >

<!ELEMENT attribute EMPTY > <!ATTLIST attribute name NMTOKEN #REQUIRED > <!ATTLIST attribute value CDATA #REQUIRED >

<!ELEMENT class EMPTY > <!ATTLIST class id NMTOKEN #REQUIRED > <!ATTLIST class name NMTOKEN #REQUIRED >

<!ELEMENT classidref ( #PCDATA ) >

<!ELEMENT classes ( class+ ) > <!ELEMENT created ( #PCDATA ) > <!ELEMENT createdBy ( #PCDATA ) > <!ELEMENT data ( #PCDATA ) >

<!ELEMENT disableformsubmit EMPTY > <!ATTLIST disableformsubmit pattern CDATA #REQUIRED >

<!ELEMENT form ( formurl, formname, data, submitelement ) > <!ELEMENT formbasedlogin ( formurl, formname, data, submitelement ) > <!ELEMENT formdata ( form ) > <!ELEMENT formname ( #PCDATA ) > <!ELEMENT formurl ( #PCDATA ) >

<!ELEMENT info ( created, timestamp, createdBy, notes ) >

<!ELEMENT login ( formbasedlogin ) >

<!ELEMENT notes EMPTY >

<!ELEMENT role ( useridref+, disableformsubmit? ) > <!ATTLIST role id NMTOKEN #REQUIRED > <!ATTLIST role name NMTOKEN #REQUIRED > <!ATTLIST role priority NMTOKEN #REQUIRED >

<!ELEMENT roleidref ( #PCDATA ) >

<!ELEMENT roles ( role+ ) >

<!ELEMENT submitelement ( #PCDATA ) >

<!ELEMENT timestamp ( #PCDATA ) >

<!ELEMENT url ( roleidref*, classidref? ) > <!ATTLIST url access NMTOKEN #IMPLIED > <!ATTLIST url id NMTOKEN #IMPLIED > <!ATTLIST url path CDATA #IMPLIED > <!ATTLIST url pattern CDATA #IMPLIED >

<!ELEMENT urlidref ( #PCDATA ) >

<!ELEMENT urls ( url+ ) >

<!ELEMENT user ( attribute?, disableformsubmit? ) > <!ATTLIST user id NMTOKEN #REQUIRED > <!ATTLIST user name NMTOKEN #REQUIRED >

<!ELEMENT useridref ( #PCDATA ) >

<!ELEMENT users ( user+ ) >

<!ELEMENT view ( classidref?, urlidref? ) > <!ATTLIST view path CDATA #REQUIRED > <!ELEMENT views ( view+ ) >

<!ELEMENT webmeta ( info, classes?, users?, roles?, accessrestrictions?, login?, urls?, views?, formdata? ) >

Forc

efu

lBro

win

gW

eb

Sp

iderT

hre

ad

Imp

l

Forc

efu

lBro

wsi

ng

Web

Sp

iderT

hre

ad

Man

ag

er

<<

create

>>

Forc

efu

lBro

wsi

ng

Web

Sp

iderT

hre

ad

Man

ag

er(

dep

s :

Th

read

Man

ag

erD

ep

en

den

cies)

init

ialis

eU

RLs

(use

r :

Use

r) :

void

start

Th

read

s()

: void

dis

covere

dN

ew

Url

(en

try :

Url

Qu

eu

eEn

try)

: b

oole

an

thre

ad

IsFi

nis

hed

(th

read

: W

eb

Sp

iderT

hre

ad

) :

void

getN

extE

ntr

y()

: U

rlQ

ueu

eEn

try

rep

lace

Sess

ion

Ids(

en

try :

Url

Qu

eu

eEn

try)

: void

setC

urr

en

tSess

ion

Id(s

ess

ion

Id :

Nam

eValu

ePa

ir)

: void

getO

pen

Url

s()

: in

t

Web

Sp

iderT

hre

ad

<<

inte

rface

>>

setU

rlQ

ueu

eEn

try(e

ntr

y :

Url

Qu

eu

eEn

try)

: void

getU

rlQ

ueu

eEn

try()

: U

rlQ

ueu

eEn

try

clean

Up

Befo

reExit

() :

void

AbstractThreadManager

<<

create

>>

Ab

stra

ctTh

read

Man

ag

er(

dep

s :

Th

read

Man

ag

erD

ep

end

en

cies)

init

ialis

e()

: v

oid

start

Th

read

s()

: void

dis

covere

dN

ew

Url

(en

try :

Url

Qu

eu

eEn

try)

: b

oole

an

thre

ad

IsFi

nis

hed

(th

read

: W

eb

Sp

iderT

hre

ad

) :

void

getN

extE

ntr

y()

: U

rlQ

ueu

eEn

try

isS

pid

eri

ng

Con

stra

intV

iola

ted

(entr

y :

Url

Qu

eu

eEn

try)

: b

oole

an

init

ialis

eS

tart

Tim

e()

: v

oid

isD

iskF

ull(

) :

boole

an

isM

axLe

velR

each

ed

(cu

rren

tLevel :

int)

: b

oole

an

isTi

meU

p()

: b

oole

an

dele

teU

rlQ

ueu

eEn

try(e

ntr

y :

Url

Qu

eu

eEn

try)

: void

up

date

Url

Qu

eu

eEn

try(e

ntr

y :

Url

Qu

eu

eEn

try)

: void

setE

ntr

yA

sBein

gFe

d(e

ntr

y :

Url

Qu

eu

eEn

try)

: void

saveU

rlQ

ueu

eEn

try(e

ntr

y :

Url

Qu

eu

eEn

try)

: in

tlo

ad

Url

Qu

eu

eEn

try(i

d :

in

t) :

Url

Qu

eu

eEn

try

path

Exis

ts(e

ntr

y :

Url

Queu

eEn

try,p

are

nt

: U

rlQ

ueu

eEn

try)

: b

oole

an

getO

pen

Url

s()

: in

tsa

veR

ed

irect

(entr

y :

Url

Qu

eu

eEn

try)

: void

getT

hre

ad

Cou

nt(

) :

int

clean

Up

() :

void

getU

ser(

) :

Use

rse

tUse

r(u

: U

ser)

: v

oid

isK

illThre

ad

s()

: b

oole

an

setK

illTh

read

s(ki

ll :

boole

an

) :

void

setS

avin

gN

ew

Url

s(fl

ag

: b

oole

an

) :

void

isS

avin

gN

ew

Url

s()

: b

oole

an

getA

ccess

Den

ied

Reg

ex()

: S

trin

gg

etT

est

Data

() :

Test

Data

setT

est

Data

(tD

ata

: T

est

Data

) :

void

getO

pti

on

s()

: G

lob

alO

pti

ons

getU

rlValid

ato

r()

: W

eb

Sp

iderU

rlValid

ato

r

Norm

alW

eb

Sp

iderT

hre

ad

Man

ag

er

<<

create

>>

Norm

alW

eb

Sp

iderT

hre

ad

Man

ag

er(

dep

s :

Th

read

Man

ag

erD

ep

en

den

cies)

init

ialis

eFi

rstH

tmlP

ag

e(p

ag

e :

Htm

lPag

e)

: void

init

ialis

eU

rlS

eed

s(co

nfig

: M

eta

LayerC

on

fig

) :

void

start

Th

read

s()

: void

thre

ad

IsFi

nis

hed

(th

read

: W

eb

Sp

iderT

hre

ad

) :

void

dis

covere

dN

ew

Url

(en

try :

Url

Qu

eu

eEn

try)

: b

oole

an

getN

extE

ntr

y()

: U

rlQ

ueu

eEn

try

Th

read

Man

ag

er

<<

inte

rface

>>

setK

illTh

read

s(t

: b

oole

an)

: void

getT

hre

ad

Cou

nt(

) :

int

clean

Up

() :

void

getO

pen

Url

s()

: in

tse

tSavin

gN

ew

Url

s(fl

ag

: b

oole

an

) :

void

<<

realiz

e>

>

Mis

sin

gU

rlW

eb

Sp

iderT

hre

ad

Man

ag

er

<<

create

>>

Mis

sin

gU

rlW

eb

Sp

iderT

hre

ad

Man

ag

er(

dep

s :

Th

read

Man

ag

erD

ep

en

den

cies)

init

ialis

eM

issi

ng

Url

s(u

sers

: U

sers

) :

void

start

Th

read

s()

: void

dis

covere

dN

ew

Url

(en

try :

Url

Qu

eu

eEn

try)

: b

oole

an

thre

ad

IsFi

nis

hed

(th

read

: W

eb

Sp

iderT

hre

ad

) :

void

getN

extE

ntr

y()

: U

rlQ

ueu

eEn

try

getC

hain

(en

try :

Url

Qu

eu

eEn

try)

: U

rlQ

ueu

eEn

try

rep

lace

Sess

ion

Ids(

en

try :

Url

Qu

eu

eEn

try)

: void

setC

urr

en

tSess

ion

Id(s

ess

ion

Id :

Nam

eValu

ePa

ir)

: void

Mis

sin

gU

rlW

eb

Sp

iderT

hre

ad

Imp

lN

orm

alW

eb

Sp

iderT

hre

ad

Imp

l

AbstractWebSpiderThread

<<

create

>>

Ab

stra

ctW

eb

Sp

iderT

hre

ad

(ext

: Li

st<

Url

Extr

act

or>

,wFa

ctory

: W

eb

Clie

ntF

act

ory

)ru

n()

: v

oid

sleep

() :

void

clean

Up

(pag

e :

Pag

e)

: void

clean

Up

() :

void

clean

Up

Befo

reExit

() :

void

saveW

eb

Req

uest

An

dR

esp

on

se()

: v

oid

saveW

eb

Req

uest

An

dR

esp

on

se(r

eq

uest

: W

eb

Req

uest

,resp

on

se :

Web

Resp

on

se)

: void

saveW

eb

Req

uest

An

dR

esp

on

seO

nC

riti

calE

rror(

) :

void

getW

eb

Pag

e(p

are

nt

: U

rlQ

ueu

eEn

try)

: Pa

ge

isA

ccess

Den

ied

Pag

e(a

ccess

Den

ied

Reg

ex :

Str

ing

,pag

e :

Pag

e)

: b

oole

an

pri

ntI

nfo

rmati

on

ForU

ser(

) :

void

setU

rlQ

ueu

eEn

try(q

En

try :

Url

Qu

eu

eEn

try)

: void

getU

rlQ

ueu

eEn

try()

: U

rlQ

ueu

eEn

try

<<

realiz

e>

>

<<

use

s>>

<<

use

s>>

<<

use

s>>

Ap

pen

dix

C -

UM

L c

lass

des

ign

of

the

web

sp

ider

ing

co

mp

on

ent

Ap

pen

dix

D -

UM

L c

lass

des

ign

of

the

UR

L e

xtra

cto

rs

OnM

ouse

EventE

xtr

act

or

AbstractUrlExtractor

<<

create

>>

Abst

ract

Url

Extr

act

or(

val :

WebSpid

erU

rlValid

ato

r)co

llect

Url

s(cu

rrentU

rl :

Str

ing,h

tmlP

age :

Htm

lPage)

: in

taddD

om

ChangeLi

stener(

htm

lPage :

Htm

lPage)

: void

rem

oveD

om

ChangeLi

stener(

htm

lPage :

Htm

lPage)

: void

getD

om

ChangeLi

stener(

) :

Dom

ChangeC

olle

ctor

addD

isco

verU

rlLi

steners

(lis

tener

: D

isco

vere

dU

rlLi

stener)

: v

oid

callD

isco

vere

dU

rl(u

rl :

Str

ing,r

esp

onse

: W

ebR

esp

onse

) :

void

callD

isco

vere

dU

rl(u

rl :

Str

ing)

: void

getU

rlValid

ato

r()

: W

ebSpid

erU

rlValid

ato

rgetF

ully

Qualifi

edU

rlA

sStr

ing(r

ela

tivetU

rl :

Str

ing,p

age :

Htm

lPage)

: Str

ing

getF

ully

Qualifi

edU

rlA

sUR

L(re

lati

vetU

rl :

Str

ing,p

age :

Htm

lPag

e)

: U

RL

getW

ebC

lient(

) :

WebC

lientW

rapper

setW

ebC

lient(

clie

nt

: W

ebC

lientW

rapper)

: v

oid

getO

rder(

) :

int

com

pare

To(o

: U

rlExtr

act

or)

: int

getT

est

Data

() :

Test

Data

setT

est

Data

(data

: T

est

Data

) :

void

getU

ser(

) :

Use

rse

tUse

r(u :

Use

r) :

void

Fram

eExtr

act

or

OnC

lickE

xtr

act

or

RegexU

rlExtr

act

or

Hre

fExtr

act

or

Url

Extr

act

or

<<

inte

rface

>>

colle

ctU

rls(

curr

entU

rl :

Str

ing,h

tmlP

age :

Htm

lPage)

: in

tgetW

ebC

lient(

) :

WebC

lientW

rapper

setW

ebC

lient(

clie

nt

: W

ebC

lientW

rapper)

: v

oid

addD

isco

verU

rlLi

steners

(lis

tener

: D

isco

vere

dU

rlLi

stener)

: v

oid

getO

rder(

) :

int

getT

est

Data

() :

Test

Data

setT

est

Data

(data

: T

est

Data

) :

void

getU

ser(

) :

Use

rse

tUse

r(u :

Use

r) :

void

<<

realiz

e>

>

OnC

hangeEventE

xtr

act

or

Form

Extr

act

or

Htm

lCom

mentE

xtr

act

or

JavaScript DOM Modifications Performance Test

Kevin Denver 15.06.2010 Page 1

Web Browser createElement createTextNode cloneNode appendChild insertBefore innerHTML Total

[ms] [ms] [ms] [ms] [ms] [ms] [ms]

HtmlUnit 2.7 #1 269 92 584 65 66 300 1376

#2 216 97 537 60 62 292 1264

#3 242 177 607 62 68 314 1470

#4 202 74 612 30 52 258 1228

#5 244 106 564 60 55 301 1330

234.6 109.2 580.8 55.4 60.6 293 1333.6

Firefox 3.6.3 #1 26 8 32 80 95 96 337

#2 25 11 29 83 151 85 384

#3 26 10 32 85 141 99 393

#4 25 11 30 84 105 87 342

#5 25 9 30 81 94 91 330

25.4 9.8 30.6 82.6 117.2 91.6 357.2 Δ 4

Chromium 6.0.432.0 #1 3 4 14 11 11 30 73

#2 3 4 8 10 10 33 68

#3 2 4 10 12 13 31 72

#4 2 3 10 12 10 28 65

#5 2 3 12 10 10 32 69

2.4 3.6 10.8 11 10.8 30.8 69.4 Δ 19

Appendix F – HtmlUnit JavaScript Performance (Code)

HtmlUnit Java Benchmarkimport java.io.IOException; import java.net.MalformedURLException; import java.util.ArrayList; import java.util.List;

import com.gargoylesoftware.htmlunit.CollectingAlertHandler; import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException; import com.gargoylesoftware.htmlunit.WebClient; import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class HtmlUnitBenchmark { public static void main(final String[] args) throws FailingHttpStatusCodeException,

MalformedURLException, IOException, InterruptedException { final WebClient webClient = new WebClient(); final List<String> collectedAlerts = new ArrayList<String>(); webClient.setCssEnabled(false); webClient.waitForBackgroundJavaScriptStartingBefore(9000); webClient.setThrowExceptionOnScriptError(false); webClient.setAlertHandler(new CollectingAlertHandler(collectedAlerts)); final HtmlPage page =

webClient.getPage("file:///home/kevin/Desktop/dommodify.html"); while(true) { if (page.getTitleText().equalsIgnoreCase("Finished")) { break; } synchronized (page) { page.wait(1); } } System.out.println(collectedAlerts); } }

HTML Page with embedded JavaScript...<script> var timeDiff = { setStartTime:function (){ d = new Date(); time = d.getTime(); },

getDiff:function (){ d = new Date(); return (d.getTime()time); } }

// Try to force real results var ret, tmp, str;

var elems = []; var htmlstr = document.body.innerHTML; var div = document.createElement("div"); var num = 400;

for (var i = 0; i < 1; i++) { str += String.fromCharCode( (25 * Math.random()) + 97 ); timeDiff.setStartTime(); for ( var i = 0; i < num; i++ ) {

ret = document.createElement("div"); ret = document.createElement("span"); ret = document.createElement("table");

ret = document.createElement("tr"); ret = document.createElement("select");

} alert("createElement(): " + timeDiff.getDiff()); timeDiff.setStartTime(); for ( var i = 0; i < num; i++ ) {

ret = document.createTextNode(str); ret = document.createTextNode(str + "2"); ret = document.createTextNode(str + "3"); ret = document.createTextNode(str + "4"); ret = document.createTextNode(str + "5");

} alert("createTextNode(): " + timeDiff.getDiff());

timeDiff.setStartTime(); document.body.innerHTML = htmlstr; alert("document.body.innerHTML: " +timeDiff.getDiff()); elems = []; var telems = document.body.childNodes; for ( var i = 0; i < telems.length; i++ ) { elems.push( telems[i] ); }

timeDiff.setStartTime(); for ( var i = 0; i < elems.length; i++ ) { ret = elems[i].cloneNode(false);

ret = elems[i].cloneNode(true); ret = elems[i].cloneNode(true);

} alert("cloneNode(): " + timeDiff.getDiff());

timeDiff.setStartTime(); for ( var i = 0; i < elems.length; i++ ) { document.body.appendChild( elems[i] );

} alert("appendChild(): " + timeDiff.getDiff());

timeDiff.setStartTime(); for ( var i = 0; i < elems.length; i++ ) { document.body.insertBefore( elems[i], document.body.firstChild ); }

alert("insertBefore(): " + timeDiff.getDiff()); document.title = "Finished"; }

alert("Finished"); </script>

using declarative information in web application frameworks to improve security

Documents