TCP 18 2 27
17 TCP (ACK:ACKnowledge) (RTT:Round Trip Time) TCP (Transmission Control Protocol) PSPacer (Precise Software Pacer) JGN2 TCP FAST TCP UDP PSPacer
1 1 2 TCP 2 2.1.................................. 3 2.2......................... 3 3 PSPacer 5 3.1........................................ 5 3.1.1 iproute2.................................. 7 3.1.2 Qdisc................................... 7 4 1 4.1................................. 1 4.1.1............................. 11 4.2 IPG-aware packet scheduling.......................... 11 4.3........................................ 12 5 13 5.1.................... 13 5.2........................... 13 5.3...................................... 14 5.3.1 iperf.................................... 14 5.3.2 FAST TCP................................ 15 5.3.3......................... 16 5.3.4 PSPacer.................................. 17 6 18 6.1...................... 18 i
6.2......................... 22 6.2.1 Standard TCP.............................. 23 6.2.2 FAST TCP................................ 28 6.3.................................. 33 7 34 35 36 ii
1 DNA. IP PC PSPacer / TCP ( ) 2 3 PSPacer 4 5 PSPacer 6 7 1
2 TCP IP Transmission Cotrol Protocol (TCP) User Datagram Protocol (UDP) WWW TCP TCP IP TCP TCP (ACK: ACKnowledgement) (RTT: Round Trip Time) RTT 2
2.1, linux 1 1ms 2.2 RTT RTT ACK ACK HighSpeed TCP Scalable TCP BIC-TCP AIMD(Additive Increase and Multiplicative Decrease) 3
FAST TCP 4
3 PSPacer PSPacer ( ) PSPacer PSPacer ( ) TCP PSPacer iproute2 Qdisc tc PSPacer 3.1 PSPacer (IPG Inter Packet GAP) tc tc TCP PSPacer Linux iproute2 3 qdisc class( ) filter( ) qdisc 5
PSPacer 2 qdisc tc PSPacer IEEE82.3x PAUSE / PAUSE PAUSE PAUSE 512 PC 1 IP 8Kbps 93Mbps 1 PSPacer Linux PSPacer IP PSPacer PSPacer TCP/IP TCP/IP RTT PSPacer 9 6
PSPacer 3.1.1 iproute2 iproute2 Linux arp ifconfig route Linux2.2 GRE(generic Routing Encapsulation) GRE Cisco IPv6 Linux OS 1 iproute2 2 1 LinuxBox 3.1.2 Qdisc PSP(PSPacer) qdisc CBQ(Class-Based Queueing ) qdisc PSP qdisc 1. 2. PSP qdisc 7
parent major qdisc handle major qdisc qdisc default minor-id rate rate PCI(Peripheral Component Interconnect) PCI CPU Central Processing Unit CPU parent major qdisc classid major qdisc ( qdisc) qdisc 8
rate rate mode mode 2 1 2 PSPacer 5 9
4 2.,. IPG-aware packet scheduling., IPG-aware packet scheduling.. 4.1... NIC(Network Interface Card) IFQ(Interface Queue). IFQ NIC..,.,. NIC. 4.1. 1
4.1: 4.1.1. cwnd targetrate = cwnd packetsizu RT T (4.1) maxrate NIC. targetrate = maxrate packetsize packetsize + gapsize (4.2) 2. gapsize = maxrate RT T cwnd packetsize (4.3) 4.2 IPG-aware packet scheduling IPG-aware packet scheduling IPG..,. 11
. 4.3,TCP/IP NIC IFQ Linux kernel QoS qdisk qdisc IFQ sch-ipg 2 Target Rate Estimator Gap Packet Injector 12
5.1: 5 PSPacer 5.1 PC1 PC2 1 1 5.1 PC1 Standard(linux-2.6.1-41msmp) FAST(linux-2.6.7) PSPacer version 1.1 5.2 PC1 PC2 JGN2 5.2 13
JGN2 sender router JGN2 receiver 5.2: PC1 Standard (linux-2.6.1-41msmp) FAST(linux-2.6.7) PSPacer version 1.1 5.3 TCP Standard TCP PSPacer iperf RTT 5ms 1ms 15ms 2ms PSPacer 93Mbps Standard TCP FAST TCP 1 2 UDP UDP 1Mb/s 2Mb/s PSPacer 93Mbps 5.3.1 iperf TCP UDP iperf iperf TCP UDP UDP bandwidth iperf 14
( ) iperf % iperf -s ( ) % iperf -c < ( IP )> TCP iperf 5.3.2 FAST TCP FAST TCP 23 1Gbps FAST TCP echo 496 33554432 33554432 > /proc/sys/net/ipv4/tcp_mem echo 496 33554432 33554432 > /proc/sys/net/ipv4/tcp_rmem echo 496 33554432 33554432 > /proc/sys/net/ipv4/tcp_wmem echo 496 33554432 33554432 > /proc/sys/net/core/wmem_max echo 496 33554432 33554432 > /proc/sys/net/core/rmem_max echo 1 > /proc/sys/net/ipv4/tcp_fast echo 1 > /proc/sys/net/ipv4/tcp_fast_bc echo 1 > /proc/sys/net/ipv4/tcp_fast_kmon_rtt 15
echo 4 > /proc/sys/net/ipv4/tcp_fast_kmon_t /sbin/ifconfig eth txqueuelen 4 echo 15 > /proc/sys/net/core/netdev_max_backlog modprobe e1 RxDescriptors=496,496 6 9 5.3.3 TCP TCP Standard TCP sysctl -w net.ipv4.route.flush=1 sysctl -w net.core.rmem_max=33554432 sysctl -w net.core.wmem_max=33554432 sysctl -w net.core.rmem_default=65536 sysctl -w net.core.wmem_default=65536 sysctl -w net.ipv4.tcp_rmem= 496 33554432 33554432 sysctl -w net.ipv4.tcp_wmem= 496 33554432 33554432 16
sysctl -w net.ipv4.tcp_mem= 33554432 33554432 33554432 sysctl -w net.core.netdev_max_backlog=15 madprobe e1 RxDescriptors=496,496 madprobe e1 TxDescriptors=496,496 5.3.4 PSPacer PSPacer ethtool -K eth tso off /sbin/tc qdisc add dev eth root handle 1: psp default 2 /sbin/tc class add dev eth parent 1: classid 1:1 psp rate 93mbit /sbin/tc class add dev eth parent 1: classod 1:2 psp mode /sbin/tc qdisc add dev eth parent 1:1 handle 1: pfifo /sbin/tc qdisc add dev eth parent 1:2 handle 2: pfifo /sbin/tc filter add dev eth parent 1: protocol ip pref 1 u32 match ip dst < IP > classid 1:1 17
6 PSPacer 6.1 3 Standard TCP 6 1 6 2 6 3 6 4 Standard TCP RTT PSPacer FAST TCP 6 5 6 6 6 7 6 8 FAST TCP RTT PSPacer 18
1 8 throughput[mbps] 6 4 2 without pacing with pacing 5 1 15 2 25 3 6.1: Standard TCP (RTT 5) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.2: Standard TCP (RTT 1) 19
1 8 throughput[mbps] 6 4 2 without pacing with pacing 5 1 15 2 25 3 6.3: Standard TCP (RTT 15) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.4: Standard TCP (RTT 2) 2
1 8 throughput[mbps] 6 4 2 without pacing with pacing 5 1 15 2 25 3 6.5: FAST TCP (RTT 5) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.6: FAST TCP (RTT 1) 21
1 8 throughput[mbps] 6 4 2 without pacing with pacing 5 1 15 2 25 3 6.7: FAST TCP (RTT 15) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 with spacing without spacing 6.8: FAST TCP (RTT 2) 6.2 Standard TCP FAST TCP 1 2 UDP UDP 1Mb/s 2Mb/s PSPacer 93Mbps 22
6.2.1 Standard TCP Standard TCP 6.9 6.1 6.11 6.12 6.13 6.14 6.15 6.16 6.17 23
1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.9: Standard TCP 1 1 8 iperf1 iperf2 SUM throughput[mbps] 6 4 2 5 1 15 2 25 3 6.1: Standard TCP 2 (PSPacer ) 24
1 8 throughput[mbps] 6 4 2 iperf1 iperf2 SUM 5 1 15 2 25 3 6.11: Standard TCP 2 (PSPacer ) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.12: Standard TCP+UDP(1Mbps) 25
1 8 throughput[mbps] 6 4 2 with spacing without spacing 5 1 15 2 25 3 6.13: Standard TCP+UDP(2Mbps) 1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.14: Standard TCP+UDP(1Mbps) (PSPacer 26
1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.15: StandardTCP+UDP(1Mbps) (PSPacer ) 1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.16: StandardTCP+UDP(2Mbps) (PSPacer ) 27
1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.17: StandardTCP+UDP(2Mbps) (PSPacer ) 6.2.2 FAST TCP FAST TCP 6.18 6.19 6.2 6.21 6.22 6.23 6.24 6.25 6.26 28
1 8 throughput[mbps] 6 4 2 with spacing without spacing 5 1 15 2 25 3 6.18: FAST TCP 1 ) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 iperf1 iperf2 SUM 6.19: FASTTCP 2 (PSPacer ) 29
1 8 throughput[mbps] 6 4 2 iperf1 iperf2 SUM 5 1 15 2 25 3 6.2: FAST TCP 2 (PSPacer ) 1 8 throughput[mbps] 6 4 2 5 1 15 2 25 3 without pacing with pacing 6.21: FAST TCP+UDP(1Mbps) 3
1 8 throughput[mbps] 6 4 2 with spacing without spacing 5 1 15 2 25 3 6.22: FAST TCP+UDP(2Mbps) 1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.23: FAST TCP+UDP(1Mbps) (PSPacer ) 31
1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.24: FAST TCP+UDP(1Mbps) (PSPacer ) 1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.25: FAST TCP+UDP(2Mbps) (PSPacer ) 32
1 8 loss jitter 2 1.5 loss[%] 6 4 1 jitter[ms] 2.5 5 1 15 2 25 3 6.26: FAST TCP+UDP(2Mbps) (PSPacer ) 6.3 Standard TCP PSPacer FAST TCP PSPacer Standard TCP FAST TCP PSPacer FAST TCP PSPacer FAST TCP UDP 2Mbps PSPacer 33
7 PSPacer Standdard TCP TCP FAST TCP PSPacer Standard TCP PSPacer FAST TCP PSPacer Standard TCP FAST TCP PSPacer FAST TCP+UDP UDP 34
NICT JGN2 35
[1] Ryousei Takano,Tomohiro Kudoh,Yuetsu kodama,motohiko Matsuda,Hiroshi Tezeka and Yutaka Ishikawa: Design and Evaluation of Precise Software Pacing Mechanisms for Fast Long-Distance Networks [2] : TCP,vol.44,No.9,pp.951 957 [3] http://www.gridmpi.org/pspacer-1./index.ja.jsp [4] http://www.aist.go.jp/aist-j/aistinfo/aist-today/vol5-9/vol5-9-topics/ vol5-9- topics.html 36