1

Migratory TCP:Connection Migration for Service

Continuity in the Internet*

Florin Sultan, Kiran Srinivasan, Deepa Iyer, Liviu IftodeDepartment of Computer Science

Rutgers University

*Work supported in part by the NSF under grant CCR-0133366

2

Today: TCP-based Internet Services

TCP : reliable byte-stream

Client Server

3

The Problem with TCP Adverse conditions

internetwork congestion or failure server overloaded, failed or under DoS attack

TCP response network delays => packet loss => retransmission

TCP limitations addressing: implicitly binds the service to a server error recovery: limited to one server

4

Highly Available Server? Not Enough!

To a TCP client: connect() will succeed most of the time

To an end-user: may not help beyond connect() connectivity failures, server overload, etc.

Service continuity = uninterrupted end-to-end delivery of a service as important to end-user as the high availability of

the server!

5

Our Solution: Cooperative Service Model

Client

Server 1

Server 2

congestion

Logically equivalent servers cooperate in sustaining the service by migration of live connections

6

Migratory TCP (M-TCP)

A client connection transparently migrates to different servers during its lifetime server applications cooperate to support the

handoff client application does not change servers can be geographically distributed

Designed and implemented as a TCP extension

Mechanism decoupled from migration policy policies are not the focus of this talk

7

Per-connection State Server 1 Server

2Application

M-TCP

Application

M-TCP

Application state

Protocol state

8

Application - M-TCP Service Contract

Application Define fine-grained per-connection application

state Export a per-connection application state

snapshot to the protocol Import per-connection state after migration and

resume service M-TCP

Transfer per-connection state Synchronize application state with protocol state

9

Server Migration API

export_state(conn, state_buff) origin server, periodically during service on

connection import_state(conn, state_buff)

destination server, after accepting connectionorigin destinationApplication Application

export() import()

M-TCP M-TCP

10

Connection Migration Mechanism

C

C’

< S

tat e

Requ

est

> (2)

< S

tate

Reply

> (3)

Client

Server 1

Server 2

<SYN + MIGRATE C > (1)<SYN/ACK> (4)

11

Prototype Implementation

Modified TCP/IP stack in the FreeBSD kernel

Real applications Icecast audio streaming server

migrate audio stream PostgreSQL transactional DB server front-end

migrate sequence of transactions with ACID semantics Apache web server

migrate transfer of static files, dynamic content

12

Experimental Setup

S1 S2

C

Hub

Control interface

Service interface128.6.171.161 128.6.171.162

10.10.10.10 10.10.10.11

13

Streaming Server Experiment

Effective throughput close to average rate seen before server performance degrades

14

Related Work HTTP server fail-over by connection migration

[Snoeren ‘00] soft TCP and HTTP state maintained at back-up servers application-specific: relies on HTTP features

Fault-tolerant TCP [Alvisi ‘00] persistent connections across server crashes failures masked using TCP wrappers

Stream Control Transmission Protocol [RFC 2960] multi-homing preserves connectivity if network fails

15

Conclusions Transport layer protocol that supports service continuity

dynamic, light-weight, transparent connection migration

Migration API for server applications

Working prototype for FreeBSD & real applications

16

Future Work

Multi-process state migration application state spans multiple processes that

communicate via IPC channels Non-intrusive state transfer by remote

memory access (VIA, InfiniBand) A distribution of the M-TCP software for

FreeBSD will be available this fall

17

http://discolab.rutgers.edu/mtcp

Google: Migratory TCP or M-TCP

Thank you!