Project:Http BomberFinal Presentation

Liat Ben-AmiYonathan Perez

Instructor: Roy Mitrany

Presentation OverviewGoalMarket Survey – where does HTTP Bomber fitHTTP BackgroundHow does HTTP Bomber work?TechnologiesDesign OverviewImportant ModulesConcurrencyGUISummary & Gained Knowledge

GoalsGenerate Http traffic load in order to test

server performanceAnalyze server responses and provide

analysis results with a clear graphic display

Market survey: existing productsSeveral traffic generators: generate traffic

according to user-defined workload. HP httperf , curl-loader project,..

Http Analyzers / Monitors : monitor all http traffic from and to a user-agent, analyze it and provide graphic display of the results.Http Analyzer V5 by IE Inspector, Http Debugger

Pro by Softx,…HTTP Bomber is an HTTP traffic generator that

also provides analysis and graphic display of the traffic to and from the tested server.

HTTP ProtocolMost significant application-layer protocol in

the World Wide Web.Operates over reliable transport layer

protocol (TCP)Goal: Transfer data of various types

(Resources) over the Internet. Resources are identified by URIs.

A stateless request-response protocol

HTTP Messages<initial line>Header 1: value 1Header 2: value2…Header N: value N

<Optional Message body>

HTTP Messages – cont.Request initial line: Method + URI + protocol versionMethods supported by HTTP Bomber: GET & POST

Response initial line: protocol version + status code + textual explanation

status codes:1xx: informational2xx: success3xx: redirection4xx: Client error5xx: Server error

How does HTTP Bomber work?User sets execution parameters (including

tested server’s name)User builds a list of requests to be sent to

serverUser starts simulationVirtual users send requests to server and

collect responses. Responses are analyzed and statistical info is gathered and displayed.

TechnologiesJavaEclipseGUI: javax.swing, JiglooJfreechartApache Http components

Design OverviewHTTP Bomber

Generator

Analyzer

(including Graphic display)

Response Recorder

write read

Server

GUI / API

Req

uest L

ist

Design Overview – Cont.Modular designHttpBomber – Top hierarchy module. Its

public methods are used as user interface – as API or through GUI.

Generator: generate workload and communicate with the server

Analyzer: gather statistical info, write report, display graphs

Generator and Analyzer interact through a common synchronized data structure

HttpBomber – Sequence Diagram

HttpGenerator - PurposeSend requests to server according to user-defined workload (execution parameters and request list) and produce records of server’s responses.

HttpGenerator - ImplementationRequests are sent by virtual users (threads)

through a multi-threaded Apache HttpClient.

Number of virtual users is defined in execution parameters.

Each virtual user iterates over the request list in a cyclic order. For each request: send request, receive response, generate a record and update the response recorder.

Custom handling for IO and protocol errors

Generator – Sequence Diagram

Analyzer - PurposeGather and produce statistical info from response records, produce report file and display graphs.

Analyzer - ImplementationgetData thread copies new data from the

synchronized response recorder, updates counters and local data structures and writes record content to the report file.

Gathered info: requestsSent, errorNum, dataReceived, dataSent, number of responses per each status code

Separate threads run graphs – one thread per graph. Updated according to gathered info.

at end of simulation – statistical info is computed and added to report and final graphs are displayed

Report File Example******Start Report******Start Time: Tue Apr 26 02:05:33 IDT 2011Workload Parameters:Tested Server: www.google.co.il, port: 80, Number of virtual users: 1, max time cap: 9223372036854775807 msecmax requests per user: 5, max timeout: 60000 msec, max allowed retrials per request: 2Response Records:Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: 11983 bytes, Response Time: 921 msec, Time stamp: Tue Apr 26

02:05:34 IDT 2011Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: 11999 bytes, Response Time: 115 msec, Time stamp: Tue Apr 26

02:05:34 IDT 2011Request index: 2, Status Code: 200, Data Sent: 60 bytes, Data Received: 10597 bytes, Response Time: 332 msec, Time stamp: Tue Apr 26

02:05:34 IDT 2011Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: 11983 bytes, Response Time: 125 msec, Time stamp: Tue Apr 26

02:05:34 IDT 2011Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: 11999 bytes, Response Time: 131 msec, Time stamp: Tue Apr 26

02:05:35 IDT 2011Total requests sent: 5Total data received: 58.56 KBTotal data sent: 0.33 KBTotal errors: 0Average traffic rate: 31.97 KB/secResponse Time Statistics: Minimal - 115 msec, Maximal - 921 msec, Median - 131 msec, Average - 324 msecStatus Code Distribution (zero indicates IO or protocol errors): 200 : 20%404 : 40%405 : 40%Time: Tue Apr 26 02:05:35 IDT 2011Simulation ended:simulation ended*********End of report********

Synchronized Response Recorder

1 2 N

N - Number of requests in request list

Records of responses to request i

HttpBomber Class Diagram

Graph Display – Online Graphs

Graph Display – Offline Graphs

Graph Display - Implementation

Graph Display – Sequence Diagram

ConcurrencyHTTP Bomber is a highly-concurrent

applicationSignificant need for Synchronization.

Minimize access to mutable shared data.Several “virtual users” threads send requests

simultaneously to server and write to the Response Recorder

Analyzer: Analyzer main thread, getData thread and a thread for each of the graphs

Main thread

Simulation start Analyzer thread

Virtual Users Threads….

startstart

join

start

getData ThreadGraph Threads

start

donedone

joinjoin

join

User Guide - GUI

User Guide - GUI

User Guide - GUI

User Guide – using API example public class MainTest{public static void main(String[] args) {

URI uri1 = null; URI uri2 = null;try {

uri1 = new URI("http://www.google.co.il/");uri2 = new URI("/jiadb");

} catch (URISyntaxException e) { return; }HttpBomber bomber = new HttpBomber();/* set execution parameters */bomber.setExecutionParameters("www.google.co.il", 10, 60000, 100, Long.MAX_VALUE, 2);/* build request list */try{

bomber.addGetRequest(uri2);bomber.addStringPostRequest(uri1, "bla bla", "text/plain; charset = UTF-8");bomber.addGetRequest(uri1);bomber.addHeader(0, "User-Agent", "HTTP Bomber");bomber.addHeader(1, "User-Agent", "HTTP Bomber");bomber.addHeader(2, "User-Agent", "HTTP Bomber");}catch(Exception e){ e.printStackTrace();}

/* set graphic display */bomber.setGraphDisplay(true, true, true, true);/* run simulation */bomber.Init();}

}

Summary & Gained KnowledgeWhat we did:Created a reliable and user-friendly application

to analyze performance of web serversWhat we learned: JavaHTTP Protocol and Apache Http ComponentsGUI designGraphic library APIDesign and implementation of a complex

multi-threaded software tool

?