UDT: UDP based Data Transfer Protocol Breaking the Data Transfer Bottleneck

UIC
National Center for Data Mining
http://udt.sourceforge.net University of Illinois at Chicago

• Bulk data transfer in data intensive x

e
d
applications Background
The Problems

n 1
I
Earth data in s 0.95
• Cooperation between large number of Chicago
s
Inexpensive storage and high bandwidth optical networks have facilitated the rapid e
data flows n 0.9
increase of distributed data intensive applications, especially in the field of E- r
i
a 0.85
• High computation overhead, large data Science. F UDT
TCP
copy, bursting disk-network IO 0.8 -2
TCP’s problem: poor bandwidth utilization in high BDP networks, RTT bias, and
-1 0 1 2 3
10 10 10 10 10 10
RTT (ms)
• TCP is ineffective Earth data in prone to queuing and reverse traffic.
Amsterdam Jain’s fairness index of UDT and TCP
Figure on the left: two earth observation data streams from Chicago and Amsterdam
were joined and analyzed in real time during iGrid 2002. x
e
d
• Easy to deploy: user space and end-to- n
The Challenges

I 0.8
end approach without router feedback
Symbols and Abbreviations
UDT
y TCP
t 0.6
i
• High performance, fast data transfer SYN: Synchronization Time. A UDT constant time that is 0.01 seconds. R: Packet Sending Rate (in packets/s) ACK: Acknowledgement
l
i 0.4
b
a
• Intra-protocol fairness without RTT bias RTT: Round Trip Time W: Congestion Window Size (in packets) NAK: Negative Acknowledgement (loss report) t 0.2
S

MSS: Maximum Segment Size (in bytes) B: Estimated Bandwidth (in bits/s) AS: Packet Arrival Speed (in packets/s)
• TCP friendliness 0 -2
10 10
-1
10
0
10
1
10
2
10
3

RTT (ms)

UDT Architecture Application/Protocol Buffer

Stability Index of UDT and TCP: smaller value
Application Buffer
UDT is an application level transport protocol over UDP. It is duplex. Each UDT entity has both a DATA Seq. No | User Data is more stable
sender and a receiver. Two UDT entities communicate through a pair of UDP ports. Protocol Buffer
Control ACK | RTT | BW | CW | ACK seq. s
• UDP-based, application level protocol UDT uses packet-based sequencing and timer-based selective acknowledgement. ACK is sent every NAK | loss list s
e
1
n
SYN and NAK is sent once a packet loss event is detected.
The Solutions

r 0.98
• Protocol design to support efficient Control ACK2 | ACK Seq i
a 0.96
packet processing UDT uses a hybrid rate-window congestion control. Rate control is triggered every SYN, whereas
F

T 0.94
Loss List
• Configurable congestion control window control is triggered every ACK. Loss List UDP Connection T
R 0.92
sender receiver
0.9
• Efficient native congestion control
0 1 2 3
10 10 10 10

Rate Control Window Control

RTT (ms)
algorithm UDT’s RTT fairness: values are the throughput
B (Mb/s) Increment x

• Optimized implementation
Rate control tunes the packet sending rate. No more than one packet can be (packets) Window control limits the number of unacknowledged e
d ratio of two flows, one has a fixed RTT of 1ms,
n
sent during each packet sending period. B ≤ 0.1 0.00067 packets. It is done at the receiver side. Once an ACK is to I the other ranges its RTT from 1ms to 1000ms.
be sent, update the window size to: s

Additive Increase: Every SYN, if there is no NAK, but there are ACKs received, 0.1 < B ≤ 1 0.001 s
e 1.4
• Fast. n
the increment of next SYN is given by: 1 < B ≤ 10 0.01 W = W * a + AS * (RTT + SYN) * (1 - a) 0< a <1 i
l
1.2
1
The Features

d
• Fair and friendly.
inc = max(10 log10 B − 9 , 1 / 1500) × 1500 / MSS 10 < B ≤ 100 0.1 n
The minimum value between W and the receiver’s e
i
0.8
0.6
• Easy to use. 100 < B ≤ 1000 1 available buffer size (flow control) is sent to the sender in
r
F 0.4
Multiplicative Decrease: For a random chosen NAK
1000 < B ≤10000 10 ACK. P 0.2

• Highly configurable.
C
T 0 -2
R = R * 8/9 … … 10 10
-1
10
0

RTT (ms)
10
1
10
2
10
3

• Firewall friendly. * MSS = 1500 bytes

UDT’s TCP friendliness index: values are
Bandwidth Estimation De-Synchronization Implementation )
s
mean throughout of 10 TCP flows with 5 other
p
b
UDT flows vs. with 5 other TCP flows.
UDT uses receiver based packet pairs (RBPP) A randomization method is used to remove the negative impact of loss UDP multiplexer allows multiple UDT M
(
to estimate link capacity L. Suppose the synchronization. It also avoids drastic changes in the sending rate. connections to share single port. t
1000

• SC03: 10 UDT flows and 200 TCP current sending rate is C, then congestion epochs u 800

flows from Amsterdam to Phoenix – fair Efficient processing on protocol data structures, p
h 600
The Results

and friendly if C is less than the last decreased sending including memory copy avoidance, self-clocking, g
u 400 to Chicago, 1Gbps, 0.04ms
o
rate and fast scheduling of UDT connections. r 200
to Canarie, OC-12, 16ms
to Amsterdam, 1Gbps, 110ms

• SC06: transfer SDSS data disk-disk B=L–C

h
T
between Chicago and Tampa at 8Gb/s time Supports multi-core processing, buffer auto- 0
0 10 20 30 40 50 60 70 80 90 100
else Time (s)
using Sector & UDT. BWC winner. decreases NAKs sizing, rendezvous connection setup, etc.
B = min {L-C, L/9}
M=5, N=2 UDT’s efficiency in real networks
• SC08: Large area cloud computing with )
s
Sector/Sphere. UDT supports 120*120 p

Demonstration Reference
b
flows in the system. M
( 600

• Open source BSD license Figure on the left: 50 TCP flows and 4 UDT flows UDT web site: http://udt.sourceforge.net t
u
400
share a 1Gb/s, 180ms RTT link. The TCP window size p
Internet Draft: Yunhong Gu, Robert L. Grossman, UDT: A high h
The Software

• User level C++ library is configured so that the maximum TCP throughout is g
u
performance data transfer protocol, draft-gg-udt-02.txt o
200

• Support Linux, BSD, UNIX, and 5Mb/s. The size of each green square represents the r
h
Windows current transfer speed of that flow. The size of each Paper: Yunhong Gu and Robert L. Grossman, UDT: UDP-based T 0
0 10 20 30 40 50 60 70 80 90 100

blue square in the TCP group is 5Mb/s, and it is Data Transfer for High-Speed Wide Area Networks, Computer Time (s)
• API very similar to BSD Socket 250Mb/s in the UDT group. Networks (Elsevier). Volume 51, Issue 7. May 2007
330
UDT fairness in real networks: 3 UDT flows in
328
• 20,000 downloads so far, used in real networks with different RTTs (0.04ms,
326
numerous commercial and research 16ms, 110ms) and bottlenecks (OC12, 1Gb/s,
324

products. 1Gb/s)
322

320
0 10 20 30 40 50 60 70 80 90 100