improvement of tcp packet reassembly in libnids advisor : shyh-in hwang presenter : chun-hui hwang...

Improvement of TCP Packet Reassembly in Libnids

Advisor : Shyh-In HwangPresenter : Chun-Hui HwangE-mail: [email protected] 2009.07.01

2/28

Outline

• Motivation

• Goals

• Libnids Introduction

• System architecture

• Approaches

• Implementation

• Experiment Result

• Conclusion

• Future work

3/28

Motivation

• Network security monitor is important

• API libraries are convenient

• Libnids is often used by network monitor systems

• Libnids drawback :

– when packet lose, it can’t reassemble following packets

– It consumes a lot of memory to store packets

4/28

Goals

• To modify libnids - add a packet dispatch mechanism

• Let libnids can analyze and reassemble packets which already received

• Memory can be released normally

• Packet header informations delivered to AP layer

5/28

Libnids Introduction(1/2)

• Library Network Intrusion Detection System

• Emulates the IP stack of Linux 2.0.x

• Libnids capability:– IP defragmentation

– TCP stream reassembly

– TCP port scan detection

6/28

Libnids Introduction(2/2)

• Libnids applications:– Network Protocol Analysis

– Sniffer

– Network Intrusion Detection System

– Other SNMP traffic analyze (May,2007)

data reassembly Combine with dsniff (Nov.2006 & 2007)

check connection state and session data Network tracing system (April,2009)

IP defragmentation, TCP stream reassembly

7/28

System architecture

Internet

SnifferPC PC

Router

Switch

PC PC

8/28

Libnids process

Sniffer

Libnids initialization

Packet complete?

Catch packets

Packet reassembly

Data recovery

Analyze dataNo

Yes

Packet reassembly

IP defragmen-tation

TCP stream reassembly

9/28

Approaches

Improvement of libnids start

Sniffer program call libnids

Write a sniffer program

Read offline packets

Packets reassembly

Add packet dispatch

mechanism

Finish improvement

Packet header information to AP

10/28

Packet dispatch & Packet header informations

• Packet dispatch mechanism– A FIN or RESET packet has been received

– Packet sequence number falls outside of the current sliding window

– Users define timeout period for packets

• Packet header informations– An additional option

11/28

Implementation

• Use a sniffer program read offline packets

• Packet proceed to IP defragmentation

• Packet proceed to TCP stream reassembly– Check packet header length 、 IP address

– Check packet header flag

– TCP packet or not

– Check time stamp

– Check TCP connection

– Check data length add packet flag-FIN greater than 0

• Packets go into TCP queue

12/28

Implementation

Start TCP stream assembly

IP address

NoCheck TCPheader length

Check ACK flag

TCP information hash

table

TCP packet format

Decide packet form client or server

Data length sequence num

Check timestamp

Data length + FIN flag

TCP stream assembly end

TCP queue function

No

No

Yes

Yes

No

Yes

Euqal 0

Check packet from a new TCP

connection

Check RST flag

Not euqal 0

Euqal 1

Not euqal 1

Check SYN flag

Yes

No

Yes

Check SYN flag

Greater than 1

Not greater than 1

Implementation

C 3

Client

Server

C 4 C 5 C 6 C 7

S 3 S 4 S 5 S 6 S 7

C 8

S 8

C 9 C 10

S 9 S 10

C 1 C 2

S 1 S 2

SYN ACK ACK ACK FIN

SYN

ACK ACK ACK ACKFIN

ACK

C 11

ACK

close

close

Libnids queue for client

Application Layer

Libnids queue for server

C 3

Client

Server

C 4 C 5 C 6 C 7

S 3 S 4 S 5 S 6 S 7

C 8

S 8

C 9 C 10

S 9 S 10

C 1 C 2

S 1 S 2

SYN ACK ACK ACK FIN

SYN

ACK ACK ACK ACKFIN

ACK

C 11

ACK

close

close


Application Layer


lost packets

C 3

Client

Server

C 4 C 5 C 6 C 7

S 3 S 4 S 5 S 6 S 7

C 8

S 8

C 9 C 10

S 9 S 10

C 1 C 2

S 1 S 2

SYN ACK ACK ACK FIN

SYN

ACK ACK ACK ACKFIN

ACK

C 11

ACK

close

close


Application Layer


queued packets in libnidslost packets

C 3

Client

Server

C 4 C 5 C 6 C 7

S 3 S 4 S 5 S 6 S 7

C 8

S 8

C 9 C 10

S 9 S 10

C 1 C 2

S 1 S 2

SYN ACK ACK ACK FIN

SYN

ACK ACK ACK ACKFIN

ACK

C 11

ACK

close

close


Application Layer



14/28

Packet dispatch mechanism

• A FIN or RESET packet has been received

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10

FIN

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10

FIN

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10

FIN

lost packets queued packets in libnids packets delivered to AP

15/28


• Packet sequence number falls outside of the current sliding window

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10

ACK

Sliding window

queued packets in libnids lost packets

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 10

Sliding windowSliding window

packets delivered to AP

16/28


• Users define timeout period for packets

C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

C 8 C 9C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

C 8 C 9C 3

Client

C 4 C 5

ACK


Application Layer

C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

lost packets

C 3

Client

C 4 C 5

ACK


Application Layer

C 6

queued packets in libnids

C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

C 8


C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

C 8 C 9C 3

Client

C 4 C 5

ACK


Application Layer

C 6 C 7

ACK

C 8 C 9

May be retransmitted after 60s + User defined waiting time

17/28

Packet header informations

• Use option choice– Payload

– Packet header informations• payload

• source/destination IP

• source/destination port

• data length

• all packets byte

• data offset

18/28

Experiment Analyze

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

packets delivered to APqueued packets in libnidslost packets

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

19/28

Experiment Analyze

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

queued packets in libnidslost packets packets delivered to AP

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

20/28

Experiment Analyze

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

queued packets in libnidslost packets packets delivered to AP

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

21/28

Experiment Analyze

C 4 C 5 C 6 C 7C 3

ACK

TCP session 1

#1-C 4

Client

#1-C5 #2-C3 #2-C4 #2-C5 #2-C6 #2-C7 #1-C6#1-C3

ACK ACK

#1-C7


Application Layer

C 4 C 5 C 6 C 7C 3

ACK

TCP session 2

Client


Application Layer

queued packets in libnids

lost packets


22/28

Experiment Result

Packet lost

Packet with information

Result of analysis

Original libnids Improved libnids

Success Analysis Success Analysis

1 6 3 50% 6 100%

2 13 8 62% 13 100%

3 20 17 85% 20 100%

4 21 15 71% 21 100%

5 60 54 90% 60 100%

23/28

Experiment Result

0

20

40

60

80

100

6 13 20 21 60

Packets with information

Ana

lysi

s(%

)

Original libnids

Improved libnids

24/28

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application LayerC 5

Experiment Analyze



C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

sliding windowsliding window

C 5

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

C 3

Client

C 4 C 5 C 6 C 7 C 8 C 9

ACK ACK


Application Layer

late packets

25/28

Experiment Result

Packet late

Packet with information

Result of analysis

Original libnids Improved libnids

Success Analysis Success Analysis

1 14 8 57% 13 93%

2 23 15 68% 22 96%

3 61 54 89% 60 98%

4 25 23 92% 24 96%

5 86 77 90% 84 98%

26/28

Experiment Result

0

20

40

60

80

100

14 23 61 25 86

Packets with information

Ana

lysi

s

Original libnids

Improved libnids

27/28

Conclusion

• Libnids packet dispatch mechanism

• Libnids can reassemble suspended packets

• Do not consume a lot of memory

• Packet header informations delivered to AP layer

28/28

Thank you

improvement of tcp packet reassembly in libnids advisor : shyh-in hwang presenter : chun-hui hwang...

Documents

implementation slide

experiment result slide

time slide

libnids process slide

offline packets packet

ip defragmentation packet

improvement of tcp packet

tcp queue slide