UFFT: Ultra-Fast File Transfer

Sandeep KakumanuCheng-Lin Tsao

Muhammad Raza Khan

Background Current bulk file

transfer Either single path from

single server Or multiple paths from

multiple servers (one from each) in a costly CDN

Multi-path routing Existence of multiple

paths from single server to single client

Potential benefits: increased end-to-end throughput, loss/failure tolerance

Source

Client

Source 2

Source 3

Data Streams

Paths

Proposed architecture Multi-path routing using overlay

networks One direct path from server to client Multiple indirect paths through overlay

nodes

Source

Client

Indirect paths

Direct path

Our solution Use overlay networks to setup multiple pat

hs Use digital fountain codes to minimize prob

ing Fountain codes: a class of rateless codes Generate huge number of encoded packets Decode from any subset of K encoded packets Inherent functionality of integrating traffic

Milestones Finished implementing 3 different rateless codes in

C++ # symbols K = 10000, size of symbol = 1000 bytes Random codes: high decoding complexity O(K3) LT codes: O(K) complexity, 2 seconds Online codes: also O(K)

Implement overlay network on PlanetLab Send encoded packets over multiple paths on the

overlay Performance measurements

Select symbols

Choose degree

Degree Prob.

1 0.0040

2 0.4738

3 0.1585

4 0.0796

… …

10000 9.44e-9

Degree distribution (k=10000,c=0.03,δ=0.01)

Original symbols

Random generation

XOR encoding

Header Encoded symbol

Original data

LT codes (encoding)

Random seed

LT codes (decoding)

Original symbols

Original data

Encoded symbols

Encoded symbols

Measurements

Throughput and overhead vs. # of paths

0

20

40

60

80

100

1 2 3 4Number of paths

Thro

ughp

ut (M

bps)

0.00

2.00

4.00

6.00

8.00

10.00

Ove

rhea

d (%

)

throughputoverhead

Throughput and overhead vs. file size

0

200

400

600

800

1000

1200

1 5 10 20File size (MByte)

Thro

ughp

ut (M

bps)

0

5

10

15

20

Ove

rhea

d (%

)

throughputoverhead

UDP Throughput using multiple paths

0

20

40

60

80

100

120

1 2 3 4

No of paths

Th

rou

gh

pu

t an

d G

oo

dp

ut

Mb

ps

Throughput

Goodput

Improvements in LT Codes

LT Codes on the average require 500 more encoded blocks then the original number of blocks

Can we some how reduce this upper bound? Online Codes

Online Codes

Select symbols

Choose degree

Degree Prob.

1 0.0040

2 0.4738

3 0.1585

4 0.0796

… …

F 9.44e-9

Original symbols

Random generation

XOR encoding

Header Encoded symbol

Original data

Random seed

ε = sub-optimality

Q= success probability

F= Function(ε)

Composite symbols X XX

Aux symbols

XOR encoding

12 … Q

Step 1

Step 2

Step3 Step4

Step5

Step6

Online Codes (Decoding)

Original symbols

Original data

Encoded symbols

X XX

No of Encoded Blocks needed to decode a file of Blocks n = (1+ 3*ε )n

Aux symbols

Arrows indicate decoding from Aux blocks

Decoded Blocks vs Received Blocks

Overhead of Online Codes Encoding

Overhead

0

0.2

0.4

0.6

0.8

1

1.2

1.4

2557 5238 9869 16873

No of Original blocks

% overhead

Conclusions Diversity of multi-path was exploited to

increase capacity Digital fountain codes are used to prevent

adverse effects of blasting packets Implemented 2 different Digital fountain codes in C+

+ Digital Fountain Codes can be used as Poor

Man solution for transmitting data on multiple paths. Instead of first analyzing different paths just encode data using Digital Fountain Codes and send.

Issues and Future Work Issues

Computation overhead Not suitable for slow machines

Inconsistent data transfer Difference in file system?

Future work Complete implementation

Control mechanism Segmentation into blocks

Offline processing Primary thread: decoding decision, secondary thread: XOR Reserve network resources

Rate control No rate control in this project Not TCP-friendly