Botnet Detection Tools and Techniques: A

Review

A report submitted in partial fulfillment of the requirements for theaward of

IASc-INSA-NASI SUMMER RESEARCH FELLOWSHIP

sponsored by Indian Academy of Sciences, Bangalore

By

KANCHAN M. BHALE

Reg No.- ENGT89

Under the guidance of

Dr. B.M.Mehtre

Centre for Cyber Security

Institute for Development and Research in Banking

TechnologyEstablished by Reserve Bank of India

Castle Hills, Road No.1, Masab Tank, Hyderabad-57, Telangana.

June-2016

DECLARATION

I hereby declare that the project work entitled Botnet Detection Tools and Tech-

niques : A Review submitted to Indian Academy of Sciences, Banglore and at IDRBT,

Hyderabad is prepared by me and was not submitted to any other institution for award

of any other fellowship for the best of my knowledge.

Kanchan M. Bhale

Reg. No. ENGT89

i

CERTIFICATE OF APPROVAL

This is to certify that the project report entitled Botnet Detection Tools and

Techniques : A Review submitted to the Indian Academy of Sciences, Banglore

and at IDRBT by KANCHAN MOHANIRAJ BHALE, bearing Registration No.:

ENGT89, in the partial fulfillment for the requirement for the award of IASc-INSA-

NASI SUMMER RESEARCH FELLOWSHIP is a bonafide work carried out by

her under my supervision and guidance.The matter submitted in this report is original

and has not been submitted for the award of any other fellowship.

Dr. B. M. Mehtre

(Project Guide)

Associate Professor,

Center for Cyber Security,

IDRBT, Hyderabad

ii

Abstract

A Bot is a type of malware that allows an attacker to take control of infected machine.

The Botnet is a network of bots. A Bot infected machine is often called as zombie and

cybercriminals who control these bots are called Botherders or Botmasters. Bots are often

spread themselves across internet by searching for vulnerable machines to expand. The

way the bots are controlled depends upon architecture of botnet Command and Control

(C&C) mechanism which may be based on Internet Relay Chat (IRC) or HTTP or Peer to

Peer(P2P). Botnet is widely used to carry out malicious activities like Distributed Denial

of Service(DDoS) attacks, sending spam mails and click frauds. In recent years, botnet

based attacks have become more sophisticated and can bypass all security safeguards.

Botnet detection techniques are broadly based on either setting up of a honeypot to

collect bot binaries or developing intrusion detection system. The intrusion detection

system (IDS) identify botnet traffic by monitoring network and system logs. It can be

based on anomaly behavior or signature or DNS. The Netflow analyzer is popular tool

for detecting botnet anomaly based detection.The Snort, Suricata, Ntop, Bothunter are

other tools which are based on signatures of botnet. The DNS based botnet traffic is

monitored by Wireshark. The BotMiner tool uses clustering algorithm to detect botnet.

Zeus toolkit is popular among hackers community for analysis of botnet internals.

We tested and analyzed Zeus toolkit and Snort IDS for botnet detection. The performance

of Snort IDS evaluated on CTU-13 datasets. The CTU-13 contains thirteen datasets of

different botnets. The overall efficiency of the present Snort rules for botnet detection is

70 % for all datasets but for some datasets like BOTNET-44, 47, and 49 is very less. The

Snort rules are revised and tested on the same datasets. These revised rules contains the

new botnet signatures that was not present in old Snort rules. Because of addition of new

signatures, the botnet detection efficiency improved to upto 80% and there is significant

improvement for datasets like BOTNET-44, 47, and 49.

iii

ACKNOWLEDGMENT

I would like to thank God who gave me the grace and privilege to pursue this pro-

gram and successfully complete it in spite of many challenges faced. I express my deepest

gratitude to my beloved guide, Dr. B. M. Mehtre, for his precious guidance, valuable

suggestions and time that he invested throughout the work. His inspiring suggestions and

encouragement helped me in all the time of my research and writing of this report. I would

like to express special thanks to, Prof. V.U. Deshmukh, Principal, Vidya Pratishthan’s

College of Engineering, for encouraging me to attend this program.

Finally, I thankful to my family members for their silent sacrifice and heartening inspira-

tion that helped me lot.

Kanchan M. Bhale

Reg.No.- ENGT89

iv

Contents

Abstract iii

List of Figures vi

1 Introduction 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Organization of the Report . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Literature Survey 5

2.1 Botnet in DDoS Attacks: Trends and Challenges . . . . . . . . . . . . . . . 5

2.2 Botnet Detection Techniques: Review, Future Trends, and Issues . . . . . . 6

2.3 Detecting Botnet by Anomalous Traffic . . . . . . . . . . . . . . . . . . . . 7

2.4 An Empirical Comparison of Botnet Detection Methods . . . . . . . . . . . 8

3 Evaluation of Botnet Detection Tools and Techniques 11

3.1 Zeus Toolkit Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Zeus Botnet Network Analysis . . . . . . . . . . . . . . . . . . . . . 12

3.2 Snort Intrusion Detection System . . . . . . . . . . . . . . . . . . . . . . . 13

3.2.1 Snort Rule Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3 CTU-13 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 Proposed Approach 17

4.1 Proposed System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Conclusion 20

Bibliography

List of Figures

1.1 Bot Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Botent Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1 DDoS attack using Botnet . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Taxonomy of Botnet Detection Techniques . . . . . . . . . . . . . . . . . . 10

3.1 Zeus Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2 Network Traffic Capture of Zeus bot . . . . . . . . . . . . . . . . . . . . . 13

3.3 Snort Rule Header Format . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.1 Evaluation System Architecture . . . . . . . . . . . . . . . . . . . . . . . . 17

Chapter 1

Introduction

1.1 Introduction

A Bot is an autonomous program automatically perform task without knowing to a

real user. A collection of machines which run such autonomous bot is called as botnets.

Bot is remotely controlled by command and control server. The black-hat developers cre-

ated highly sophisticated malwares that are difficult to detect and remove. Bot program is

stealthy during its whole life cycle. They had generated relatively small network footprint

and most of time remains ideal for stealing information. The concept of remote-controlled

computer bot originated from Internet Relay Chat (IRC). It provides one to many com-

munications channels and support very large number of concurrent users. Eggdrop was

first bot developed in 1993.

As internet connects billions of computers, tablets, smart phones together to share the

information across the globe, peoples are relying on these technologies to share their per-

sonal as well as business information.The black hat hackers used its vulnerabilities to

perform attacks. The initial intention of these cyber criminals was just to gain fame but

over the period they are doing criminal activities to earn money.

The bot lifecycle consists of following phases shown in figure 1.1

• Creation: Firstly, botmaster develop his software mostly by extending previous code

or by adding new features. This is very well tested in isolated environment.

• Infection: There are many ways for infecting victims machine through software vul-

nerabilities, email attachments and trojan horse. Once victims machine is infected

by this software then it is called zombie.

• Rallying: After infection, zombie machine attempts first and try to contact com-

1

Botmaster (C&C Server)

Creation

Infection

Rallying

Waiting

Executing

Figure 1.1: Bot Lifecycle

mand and control machine. This process is called Rallying. In centralized botnet

topology, this could be IRC or HTTP servers whereas in P2P topology zombie

tries to locate peer machine and join the network. Bot program contains multi-

ple addresses of servers. Some C&C servers are configured in such a way that it

immediately reply to bots initial request.

• Waiting: After joining to network, bot waits for command from C&C server. During

this phase very little traffic is found between bot and its master.

• Executing: Once the bot received command from its master, it starts executing it.

After execution it sends result to bot master via C&C network. Typical commands

are: scanning for new victims, sending spam, and sending DoS flood.

There are two main botnet topologies: centralized and peer to peer (P2P). In centralized

botnets, IRC is still pre dominant protocol of C&C channel. Now this trend is decreasing

2

and new bots come with HTTP for their C&C channel. The major drawback of centralized

botnets is single point failure. If centralized entity is removed the entire network is

unusable. But, modern botnets overcome this problem by using fast-flux DNS techniques.

In fast flux DNS techniques it is very difficult to trace central entity. The compromised

hosts are used as proxies to hide identities of true C&C servers. These hosts constantly

alternates DNS configuration to resolve one hostname with multiple IP addresses. Popular

examples of IRC bots are Agobot, Spybot, and Sdbot.

The botnet architecture is explained by figure 1.2 There are many protocols available for

Botnet Architectures

CentralizedPeer to Peer Hybrid

Web Based Application based

HTTPSSocial VPNs

HTTP IRC

Figure 1.2: Botent Architecture

P2P networks, each differing in the way nodes first join the network and the role they

later play in passing traffic along. Some popular protocols are BitTorrent, WASTE, and

Kademlia.

1.2 Motivation

Botnets are network of compromised hosts and remotely controlled computer system. In

recent years, the diversity of malware has grown almost exponentially. The main goal of

botnet master is to gain financial profit from the activities they allow and other include

political or even military interests. In the last few years, some applications related to

botnets have taken a leading role which motivates researchers to resolve these issues. The

major applications based on botnets are listed below:

3

1. Identity Theft:

The major aim of the botnet master is gaining financial benefits. Botmaster auto-

matically extract users data and credentials from infected hosts. Its main targets

include passwords for various services like e-mail accounts, web shops, banking plat-

forms or social networking platforms.

2. Spam Email:

The popular use of botnets is for unsolicited mass mailing, also known as spamming.

Recently, spammers are attract towards botnets which own high computation power

network of compromised computers.

3. Click Frauds:

As botmaster has full control on infected machine, the attacker take help of the

controlled bots to visit the pages and to generate clicks on the target banners. In

this case, the attacker gains money directly from the advertising company.

4. Distributed Denial of Service attack(DDoS):

Botnets usually consist of large numbers of remote machines,their cumulative band-

width can reach multiple gigabytes of upstream traffic per second. This enables

botmasters to start targeted sabotage attacks against websites.

5. Botnets may also be used in political or military contexts.

1.3 Problem Statement

The main objective is to study and analyze botnet detection tools and techniques.

1.4 Organization of the Report

The work presented in this report is review of botnet detection techniques and tools.The

chapter 2 gives literature survey already exist related to botnet detection. The Chapter

3 put lights on evaluation of existing botnet detection tools and methods. The proposed

method and experimental results are discuss in chapter 4. Finally, the conclusion of the

work presented in chapter 5.

4

Chapter 2

Literature Survey

As more cyber criminals are using botnet to perform sophisticated attacks, there

is need to develop strong defense mechanism against it. Lots of research articles were

available related to botnet detection. Some of the important papers are summarized

below:

2.1 Botnet in DDoS Attacks: Trends and Challenges

Nazrul Hoque, [1] presented comprehensive overview of DDoS attack. The paper also

contains detail discussion of botnet architecture, tools developed using botnet architec-

tures to perform DDos attack. This paper also summarized important issues and research

challenges. The figure 2.1 illustrate the DDoS attack using botnet architecture [1] . In

context to DDoS , there are two categories of botnet, DDoS attack using stationary bot-

net and DDoS attack using mobile botnet.There are four reasons behind using botnet for

performing DDoS attack:

1. Large number of zombie nodes allow generation of powerful flood attacks quickly

2. Difficulty to identify the main attacker

3. Ability to use protocols to bypass security mechanisms

4. Difficulty in real time detection

Botnet based DDoS attack is basically launched using three basic models: Agent handler

model, web based model and IRC based model. It summarize all existing stationary and

mobile botnets. Botnet detection methods are typically classified into two categories:

analysis of passive traffic and traffic generated by honeynet. This paper raises issues in

existing DDoS detection methods: Existing detection method are capable to detect low

5

Figure 2.1: DDoS attack using Botnet

rate DDoS or high rate DDoS attack.The performance of most method depend on network

conditions and parameters.

2.2 Botnet Detection Techniques: Review, Future

Trends, and Issues

Ahmad Karim, [2] presents a comprehensive review of the latest state-of-the-art tech-

niques for botnet detection and figures out the trends of previous and current research.

The author also discuss future direction of botnet detection techniques.

Researchers have developed many architectures and botnet detection taxonomies.The

figure 2.2 explains taxonomy of botnet detection techniques [2] based on their implemen-

tation. The honeynets are used to collect information about bots for analysis such as

finding botnet characteristics, finding tools used behind attack and motivation behind

the attack. Intrusion detection system is a software application or hardware to monitor

6

system services for malicious activities or policy violations and accordingly generate re-

ports.

This paper [2] also explains future trends of botnets:

• Social botnets:- Botmasters now capture a huge audience while remaining hidden

from it.They try to exploit social media sites such as Facebook and Twitter. Botnet

Butterfly is one of the profitable botnets which damaged 12 millions PCs worldwide.

• Mobile botnets: Mobile botnets are a serious threat to smart phones. Hacker’s

objective is to perform illegal phone calls,sending emails,illegal photo access. The

most popular moblie botnets are Dreamdroid, Zeus and Tigerbot.

• Botnets to Botclouds: Dark clouds are controlled by cyber criminals which are

silently infect networks.

The author also put light on open issues in botnet detection techniques.

• Most of techniques are not accurately measure the size of the botnet.

• Researchers face difficulty in obtaining real trace. They also find difficulties in

comparing their result with previously published benchmark because datasets to

full extent are not easily accessible to research domain.

• Mobile botnets detection research is at initial stage only.

2.3 Detecting Botnet by Anomalous Traffic

Chia-Mei Chen, [3] explained anomaly score based botnet detection to identify the botnet

activities.The author uses the similarity measurement and the periodic characteristics of

botnets to employs two-level correlation relating the set of hosts with same anomaly be-

haviors.This method can differentiate the malicious network traffic generated by infected

hosts (bots) from that by normal IRC clients. This method is also applicable for small

size of botnets.

The author observers IRC traffic within an organization network domain and identifies the

infected host and suspicious C&C server. This method identifies infected machine even

if it generate small traffic. It is also useful to detect C&C server. The author proposed

method perform following steps:

1. IRC bot traffic collection at organization gateway.

7

2. The attributes of network traffic are extracted from packet header and payload, it

is called as feature extraction. here the following flow attributes are selected for

further analysis: Source Ip, Destination Ip, Source port,Destination port, Times-

tamp,Payload.

3. Traffic Correlation: It employs the homogeneous response and the group activity

patterns to identify such anomalous machines. Normal machines responds randomly

where as infected bot machines respond at some regular interval and exhibit similar

response pattern. The author proposed two levels of correlation.

4. Anomaly Scoring: Among different group flows, the group flow occurring in a shorter

time span is more likely to be a botnet.If anomaly scoring exceeds certain threshold

it generate alerts to administrator.

2.4 An Empirical Comparison of Botnet Detection

Methods

S. Garciaa, [4] compares the output of three different botnet detection methods by exe-

cuting them over a new, real, labeled and large botnet dataset.The results of two methods

(BClus and CAMNEP) and BotHunter were compared using a methodology and a novel

error metric designed for botnet detections methods.

The Cooperative Adaptive Mechanism for Network Protection (CAMNEP) is network

behaviour analysis system. It process network flow generated by routers and identify

anomalous traffic using different anomaly detection method. The system architecture of

this method is comprised of three layers:

• Anomaly Detector: This layer analyze the netflows using different anomaly detection

methods. The output of these methods are aggregated as events using statistical

methods and passed to Trust Model.

• Trust Model: This layer maps netflow into traffic clusters based on their behavioral

patterns. The trust models act as persistence memory.

• Aggregation: This layer creates one output that integrates the individual opinion

of each anomaly detection method. The result of aggregation is to provide anomaly

score to user.

BClus method is behavioral based method which does not use any anomaly detection

methods. This method is use to cluster each network traffic generated by each IP addresses

8

and to recognize which cluster have behavior similar to botnet traffic. The basic schema

is as follows:

• Separate netflows in time window

• Aggregate netflows based on IP

• Cluster formation

• Train classification model on ground truth labels

• Use the classification model to test bot clusters

The author creates new well labeled dataset for botnet researchers and publicly avail-

able on website called as CTU Malware Capture Botnets [4]. The author also proposed

comparison methodology and error metrics.

9

Figure 2.2: Taxonomy of Botnet Detection Techniques

10

Chapter 3

Evaluation of Botnet Detection

Tools and Techniques

3.1 Zeus Toolkit Evaluation

The Zeus is a well known banking trojan which act as man in browser attack. This was

originally called as Zbot and infected about 3.6 millions of computers in united States.

The Zeus crime ware toolkit has user friendly interface and is available in public domain.

So it become one of the favorite tools for hackers. This tool allows attackers to configure

and create malicious binaries, which are mainly used to steal user’s Internet banking user

id and password. The figure 3.1 shows components of zeus toolkit.

Figure 3.1: Zeus Toolkit

The Zeus toolkit consist of:

• The control panel(PHP) scripts for displaying user friendly GUI to botmaster. This

11

helps botmaster to monitor the bots remotely. It uses MYSQL database for storing

results.

• The configuration files to customize botnet parameters. Config.txt contains basic

configuration whereas webinject.txt contains targeted websites and possible injection

attacks.

• The builder executable file which binds config.txt with webinject.txt and encrypted

using encryption key to generate config.bin and bot.exe.

• The config.bin is encrypted configuration file and bot.exe is bot executable file to

infect any machine.

3.1.1 Zeus Botnet Network Analysis

We built a sandbox environment for testing network traffic between C&C server and in-

fected machine.The Windows virtual machine act as C&C server. XAMPP with MYSQL

is installed on C&C server The following steps were carried out during testing:

1. Download zeus remote admin toolkit.This toolkit contains three subfolders: builder,other

and server.

2. Create folder bot in xampp/htdocs and copy server.php into it.

3. Run install script.

4. Configure and create Zeus bot client by modifying config and webinject text file.

5. The builder program helps to create encrypted config file and bot binaries.

6. Send bot executable file to targeted machines for infection.

7. When victim machine execute bot executable, it tries to communicate with the C&C

server.

After initial infection, bot send request packet to C&C server for configuration file. The

C&C server send the encrytped configuration file to bot. Then bot perform the task

defined in configuration file and return result to botmaster by gate.php file.The figure 3.2

shows traffic capture between infected machine and C&C server.

Bot remain in sleep mode and wakeup at regular interval of time and send reply to

botmaster. The sleeping time is defined in configuration file. Our C&C server IP is

172.21.21.110 and infected zeus bot machine IP is 172.21.21.115. This network analysis

of zeus bot help to write snort rules.

12

Figure 3.2: Network Traffic Capture of Zeus bot

3.2 Snort Intrusion Detection System

Snort is a free, simple, fast, and flexible network IDS. It has been ported to various

Unix platforms and also the Win32 platform. It is one of the most active open source

projects in the field of security. It is a signature based network intrusion detection system

capable of logging every possible trace of intrusion attempts. Snort logs alerts into a

text file, syslog, XML, libpcap format, or a database. Snort can find traces of possible

intrusion attempts by pattern matching with existing rule files specified at initial setup

or by detecting statistically anomaly on network traffic (using SPADE plugin). In alert

mode, snort requires a configuration file located at /etc/snort.conf file. Snort logs alerts

into /var/log/snort directory. As snort is signature based, to detect latest attack methods

you need to keep rules up to date.There are set of rules in snort to detect botnet traffic.

13

3.2.1 Snort Rule Sets

Typical snort rule is composed of two separate elements :Rule Headers and Rule Options.

The rule header can be considered a brief description of the network connection. The rule

header format is shown in figure 3.3

The rule header action field values are as follows:

Figure 3.3: Snort Rule Header Format

• alert: logs and alerts the packet when triggered.

• log: only logs the packet when triggered

• pass: ignores or drops the packet or traffic matching

• activate: alerts then activate a dynamic rule or rules

• dynamic: Ignores, until started by activate rule

The second half of the rule is rule options defines what is involved in the network

packet.These options are triggered only if the rule header matches certain packet con-

tent. If there is a match,snort writes as alert message to the alert file in the snort logging

directory. Packet data is also logged. Once alert is issued, the administrator can go back,

review the packets and confirm or deny it was an intrusion attempt. The common rule

options are: msg, falgs, content, offset, depth, ttl, classtype, priority, and reference.

Snort rules are categorized based on botnet detection into 3 categories. These rules are

summarized into table 3.1

Malware-CNC.rules contains rules which identifies command and control channel, out-

bound connections, possible Zeus user agent and dirtjumpers of DDoS. Malware-Tool

14

Table 3.1: Snort Rules Detects Botnet

No. Type of Botnet Rule File No. of Rules

1. Malware-CNC.rules 31772. Malware-Tools.rules 1443. Blacklist.rules 273

rulesets identifies http flood attempts to known urls. Blacklist ruleset generates an alert

if any machine try to connect black listed websites.Some samples of these three categories

of rules listed in table 3.2

Table 3.2: Samples of Snort Rules

No. Type of Botnet Rules

1. alert tcp any any −> any HTTP PORTS (msg:”MALWARECNC Possible Zeus UserAgent Download”; flow:to server,established; content:”UserAgent|3A| Download|0D 0A|”; fast pattern:only; http header;pcre:”x2E(bin|exe|php)([x5c]|)smiU”; metadata:impact flag red, policy securityips drop, service http;reference:url,en.wikipedia.org/wiki/Zeus(trojan horse); classtype:trojan-activity; sid:16441; rev:9;)

2. alert tcp any any −> any HTTP PORTS (msg:”MALWARE-TOOLS Dirt Jumper toolkit variant http flood attempt”; flow:to server,established; content:”Mozilla/5.0 |28|compatible|3B|SummizeBot +http://www.”; fast pattern:28,20; http header;content:!”summize.com”; within:11;http header; metadata:impact flag red, policy security-ips drop, service http;reference:url,www.virustotal.com/en/file/9cc7baa0756743cebbcd7fab977495e652bda32762e9c1b8367aa38fdfaf5440/analysis/; classtype:attempted-dos; sid:25920; rev:2;)

3. alert tcp any any −> any HTTP PORTS (msg:”BLACKLIST USER-AGENT Win.32.Sramler.A runtime traffic detected”;flow:to server,established; content:”User|2D|Agent|3A| QvodDown”; nocase; http header;content:”/qd.jpg”; nocase; http uri;metadata:policy securityips drop, service http;reference:url,www.virustotal.com/latest-report.html?resource=d549699a392c6e45cff7ed3621849867; classtype:trojan-activity; sid:21380; rev:2;)

3.3 CTU-13 Datasets

The CTU-13 [4] is a dataset of botnet traffic that was captured in the CTU University,

Czech Republic, in 2011. The goal of the dataset was to have a large capture of real

botnet traffic mixed with normal traffic and background traffic. The CTU-13 dataset

consists in thirteen captures (called scenarios) of different botnet samples. Each capture

performs different action. The total datasets are 13 , but we use 7 datasets. We analyzed

seven datasets of different bot traffic. First two datasets contain bot traffic which uses

15

HTTP protocol to perform click fraud attack. Dataset 3 used IRC channel to perform

DDoS attack. This traffic illustrate Valentine frauder attack. Dataset 4 and 7 used fast

flux techniques to hide C&C server. Dataset 5 and 6 contain bot traffic which generate

spam mails.The infected machine is SARUMAN (IP: 147.32.84.165)

The analysis of Botnet traffic captures is explained in table 3.3 below.

Table 3.3: CTU-13 Dataset Analysis

No. Dataset Bot #Packets Remarks

1 CTU-MALWARE- Neris 323154 The botnet used an HTTP basedCAPTURE-BOTNET-42 C&C channel and not an IRC C&C channel.

The bot sent spam, and do some ClickFraud.2 CTU-MALWARE- Neris 176064 The bot sent spam, connected to an

CAPTURE-BOTNET-43 HTTP CC, and use HTTP to do some ClickFraud.3 CTU-MALWARE- Rbot 495056 ICMP DoS attackPRIVMSG

CAPTURE-BOTNET-44 #zarasa48 :.login zarasa484 CTU-MALWARE- Virut 45853 Bot used a fast-flux DNS

CAPTURE-BOTNET-46 technique to hide botmaster5 CTU-MALWARE- Menti 24764 Sending stock based spam emails

CAPTURE-BOTNET-476 CTU-MALWARE Murlo 85735 Chinese Trojan

-CAPTURE-BOTNET-497 CTU-MALWARE- Virut 440625 Bot used a fast-flux DNS

CAPTURE-BOTNET-54 technique to hide botmaster

16

Chapter 4

Proposed Approach

4.1 Proposed System Architecture

As the performance efficiency of existing snort rules for botnet detection is approximately

70%, we analyzed and reviewed the CTU-13 botnet traffic using wireshark . The attacks

signatures were obtained to revise snort rules for botnet detection. The proposed system

architecture is shown in fig 4.1 Some samples of proposed rules described in table 4.1

CTU-13 Dataset(BOTNET Traffic)

Analyze Packet CaptureUsing Wireshark

Update Snort Rules

Run Snort IDS

Alert DatabaseMonitor

Report

Figure 4.1: Evaluation System Architecture

.

17

Table 4.1: Modified Snort Rule Samples

No. Revised Botnet Rules

1 alert tcp any any −> any HTTP PORTS (msg:”MALWARE-CNC Click Fraud variant outbound connection”;flow:to server,established; content:”/getjson”; nocase; http uri; content:”data=”;nocase; http client body; metadata:impact flag red, policy balanced-ips drop, policy security-ips drop,service http; reference:url,http://podwine.com/getjson;classtype:trojan-activity; sid:80009; rev:9;)

2 alert tcp any any −> any 6667 (msg:”Possible IRC-EXE access (PRIV)”; flow:to server,established;content:”|50 52 49 56 4d 53 47|”; content:”|2e|exe|20|”;sid:800013; rev:7; )

3 alert tcp any any −> any 80 (msg:” Possible QVOD CNC Commmand”; flow:to server,established; content:”GET /QvodSetupPlus5 5.0.69.exe”; nocase:; reference:url,http://qd.qvod.com/QvodSetupPlus5 5.0.69.exe;classtype:trojan-activity;sid:800020;rev:1;)

4.2 Experimental Results

For experimental evaluation, Snort IDS version 2.9.2.2 installed on the Ubuntu 14.0 op-

erating system. The snort configuration file modified to incorporate new rules.

The command used to run dataset is as follows:

snort -c /etc/snort/snort.conf -N -r /Downloads/botnet-capture-20110810-neris.pcap

snort is command to run snort IDS. -c option indicate location of snort.conf file, -r option

is used to read network traffic from given location, -N option indicate do not log packets

to terminal.

The Snort-IDS detected traffic packet which matches to the Snort rule then it will

generate alert and saved into database. The efficiency of botnet detection of Snort IDS

is calculated by comparing alert log with actual input bot traffic. The efficiency of de-

tection describe the effectiveness of the system in terms of botnet detection. The Snort

IDS efficiency is based on parameters like True Positive (TP), True Negative (TN), False

Positive (FP), False Negative(FN).

Efficiency = (TP+TN)(TP+TN+FP+FN)

∗ 100

TP: True Positive means correctly identified Botnet traffic

TN: True Negative means incorrectly identified Botnet traffic

FP: False Positive means correctly rejected Botnet traffic

FN: False Negative means incorrectly rejected Botnet traffic.

The comparison of present rules and proposed rules efficiency is presented in table 4.2

18

Table 4.2: Comparison of Present and Revised Rules

Efficiency of Efficiency ofNo. Dataset Total Botnets Present Rules(%) Revised Rules(%)

1 BOTNET-42 323154 91.3 91.52 BOTNET-43 176064 63.7 72.53 BOTNET-44 495056 60.6 81.04 BOTNET-46 45853 88.6 88.65 BOTNET-47 24764 41.8 82.26 BOTNET-49 85735 58.3 82.57 BOTNET-54 440625 90.9 91.0

The detection efficiency of existing Snort rules is very less for BOTNET -47 and BOTNET-

49 datasets. The detection efficiency is improved after revising rules. The remarkable

improvement in detection of BOTNET -47 and BOTNET-49 datasets.

19

Chapter 5

Conclusion

Botnet is a major security threat and difficult to discover its existence. We reviewed

different botnet tools and detection techniques. The intrusion detection system is widely

used for botnet detection. There are anomaly based and signature based tools to detect

botnet like Netflow, Snort, Suricata, Ntop, Wireshark. The other category of tools are

based on mining like Botminer, Botsnifffer, Botfinder. Bothunter is driven by Snort. It

monitor two way communication between internal asset and external entity. Zeus Toolkit

is most popular in hacker community for understanding botnet internals. It is publicly

available, so many variant of Zeus malware exists in internet domain.

We tested and analyzed Zeus toolkit and Snort IDS on CTU-13 dataset for botnet de-

tection. We tested performance of Zeus bot in sandbox environment. The existing Snort

rules evaluated on CTU-13 datasets. The overall efficiency of the present Snort rules for

botnet detection is 70 % for all datasets but for some datasets like BOTNET-44, 47, and

49 is very less. The Snort rules are revised and tested on the same datasets.The Snort

rules are revised and tested on the same datasets. These revised rules contains the new

botnet signatures that was not present in old Snort rules. Because of addition of new

signatures, the botnet detection efficiency improved to upto 80% and there is significant

improvement for datasets like BOTNET-44, 47, and 49.

20

Bibliography

[1] N. Hoque, D. K. Bhattacharyya, and J. K. Kalita, “Botnet in ddos attacks:

Trends and challenges,” Communications Surveys & Tutorials, IEEE, vol. 17, no. 4,

pp. 2242–2270.

[2] A. Karim, R. B. Salleh, M. Shiraz, S. A. A. Shah, I. Awan, and N. B. Anuar,

“Botnet detection techniques: review, future trends, and issues,” Journal of Zhejiang

University SCIENCE C, vol. 15, no. 11, pp. 943–983, 2014.

[3] C.-M. Chen and H.-C. Lin, “Detecting botnet by anomalous traffic,” journal of in-

formation security and applications, vol. 21, pp. 42–51, 2015.

[4] S. Garcıa, M. Grill, J. Stiborek, and A. Zunino, “An empirical comparison of botnet

detection methods,” computers & security, vol. 45, pp. 100–123, 2014.

[5] K. Shanthi and D. Seenivasan, “Detection of botnet by analyzing network traffic flow

characteristics using open source tools,” in Intelligent Systems and Control (ISCO),

2015 IEEE 9th International Conference on, pp. 1–5, IEEE, 2015.

[6] H. Binsalleeh, T. Ormerod, A. Boukhtouta, P. Sinha, A. Youssef, M. Debbabi, and

L. Wang, “On the analysis of the zeus botnet crimeware toolkit,” in Privacy Security

and Trust (PST), 2010 Eighth Annual International Conference on, pp. 31–38, IEEE,

2010.

[7] G. Gu, P. A. Porras, V. Yegneswaran, M. W. Fong, and W. Lee, “Bothunter: De-

tecting malware infection through ids-driven dialog correlation.,” in Usenix Security,

vol. 7, pp. 1–16, 2007.

[8] M. Stevanovic and J. M. Pedersen, “An efficient flow-based botnet detection us-

ing supervised machine learning,” in Computing, Networking and Communications

(ICNC), 2014 International Conference on, pp. 797–801, IEEE, 2014.

21

[9] F. Rahat and S. N. Ahsan, “Comparative study of machine learning techniques for

pre-processing of network intrusion data,” in 2015 International Conference on Open

Source Systems & Technologies (ICOSST), pp. 46–51, IEEE, 2015.

[10] N. Khamphakdee, N. Benjamas, and S. Saiyod, “Improving intrusion detection sys-

tem based on snort rules for network probe attacks detection with association rules

technique of data mining,” Journal of ICT Research and Applications, vol. 8, no. 3,

pp. 234–250, 2015.

[11] F. Tegeler, X. Fu, G. Vigna, and C. Kruegel, “Botfinder: Finding bots in network

traffic without deep packet inspection,” in Proceedings of the 8th international con-

ference on Emerging networking experiments and technologies, pp. 349–360, ACM,

2012.

22