research-article

Improving GridFTP performance using the Phoebus session layer

Authors:

Aaron BrownAuthors Info & Claims

SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis

Article No.: 34, Pages 1 - 10

https://doi.org/10.1145/1654059.1654094

Published: 14 November 2009 Publication History

Abstract

Phoebus is an infrastructure for improving end-to-end throughput in high-bandwidth, long-distance networks by using a "session layer" protocol and "gateways" in the network. Phoebus has the ability to dynamically allocate network resources and to use segmentspecific transport protocols between gateways, as well as to apply other performance-improving techniques on behalf of the user.

One of the key data movement applications in high-performance and Grid computing is GridFTP from the Globus project. GridFTP features a modular library interface called XIO that allows it to use alternative transport mechanisms. To facilitate use of the Phoebus system, we have implemented a Globus XIO driver for Phoebus.

This paper presents tests of the Phoebus-enabled GridFTP over a network testbed that allows us to modify latency and loss rates. We discuss use of various transport connections, both end-to-end and hop-by-hop, and evaluate the performance of a variety of cases. We demonstrate that Phoebus can easily improve performance in a diverse set of scenarios and of cases, in many instance it outperforms the state of the art.

References

[1]

Enabling high performance data transfers. http://www.psc.edu/networking/projects/tcptune/.

[2]

Net:netem. http://www.linux-foundation.org/en/Net:Netem.

[3]

TCP tuning guide. http://dsd.lbl.gov/TCP-tuning/linux.html.

[4]

Vfer: High performance data-transfer in user-space. http://vfer.sourceforge.net/cgi-bin/index.cgi?page=home.

[5]

W. Allcock, J. Bresnahan, R. Kettimuthu, and M. Link. The globus striped gridftp framework and server. In Proceedings of the ACM/IEEE SC 2005 Conference, page 54, 2005.

Digital Library

[6]

A. Brown, E. Kissel, M. Swany, and G. Almes. Phoebus: A session protocol for dynamic and heterogeneous networks. UDCIS Technical Report 2008:334, http://damsl.cis.udel.edu/projects/phoebus/phoebus_tech_report.pdf.

[7]

S. F. E. Kohler, M. Handley. Datagram congestion control protocol (DCCP). RFC 4340, March 2006.

[8]

B. Ford and J. Iyengar. Breaking up the transport logjam. In Proceedings of ACM HotNets, 2008.

[9]

I. Foster and C. Kesselman. Globus: A metacomputing infrastructure toolkit. International Journal of Supercomputer Applications, 11(2):115--128, 1997.

Digital Library

[10]

GridFTP. http://www.globus.org/datagrid/gridftp.html.

[11]

Y. Gu and R. Grossman. UDT: UDP-based data transfer for high-speed wide area networks. Computer Networks (Elsevier) Volume 51, Issue 7., 2007.

Digital Library

[12]

T. J. Hacker, B. D. Athey, and B. Noble. The end-to-end performance effects of parallel TCP sockets on a lossy wide-area network. In IPDPS '02: Proceedings of the 16th International Parallel and Distributed Processing Symposium, page 314, Washington, DC, USA, 2002. IEEE Computer Society.

Digital Library

[13]

E. He, J. Leigh, O. Yu, and T. A. DeFanti. Reliable blast UDP: predictable high performance bulk data transfer. In Proc. of theIEEE Cluster Computing 2002, pages 317--324, 2002.

Digital Library

[14]

K. Kaneko and J. Katto. TCP-fusion: A hybrid congestion control algorithm for high-speed networks. In International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet), pages 31--36, 2007.

[15]

M. Mathis, J. Semke, J. Mahdavi, and T. Ott. The macroscopic behavior of the TCP congestion avoidance algorithm. Computer Communications Review, 27(3), July 1997.

Digital Library

[16]

A. P. Mudambi, X. Zheng, and M. Veeraraghavan. A transport protocol for dedicated end-to-end circuits. In Communications, 2006. ICC '06. IEEE International Conference on Communications, pages 18--23, 2006.

[17]

L. Ong and J. Yoakum. An introduction to the stream control transmission protocol (SCTP). RFC 3286, May 2002.

[18]

J. H. Saltzer, D. P. Reed, and D. D. Clark. End-to-end arguments in system design. ACM Transactions on Computer Systems, 2(4):277--288, 1984.

Digital Library

[19]

M. Swany. Improving Throughput for Grid Applications with Network Logistics. In Supercomputing 2004, November 2004.

Digital Library

[20]

K. Tan, J. Song, Q. Zhang, and M. Sridharan. Compound TCP: A scalable and TCP-friendly congestion control for high-speed networks. In International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet), 2006.

[21]

F. Vacirca, A. Baiocchi, and A. Castellani. Yeah-TCP: yet another highspeed TCP. In International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet), pages 37--42, 2007.

[22]

Web100. http://www.web100.org.

[23]

D. Wei, C. Jin, S. Low, and S. Hegde. FAST TCP: Motivation, architecture, algorithms, performance. In EEE/ACM Transactions on Networking (TON) Volume 14, Issue 6, pages 1246--1259, 2006.

Digital Library

[24]

Z. Zhang, G. Hasegawa, and M. Murata. Reasons not to parallelize TCP connections for long fat networks. In Proceedings of SPECTS 2006, 2006.

[25]

X. Zheng, A. P. Mudambi, and M. Veeraraghavan. FRTP: Fixed rate transport protocol -- a modified version of SABUL for end-to-end circuits. In Proceedings of the 1st International Workshop on Provisioning and Transport for Hybrid Networks (PATHNETS), 2004.

Cited By

Nadig DRamamurthy BBockelman B(2022)SNAG: SDN-Managed Network Architecture for GridFTP Transfers Using Application-AwarenessIEEE/ACM Transactions on Networking10.1109/TNET.2022.315000030:4(1585-1598)Online publication date: Aug-2022
https://doi.org/10.1109/TNET.2022.3150000
Huang CNakasan CIchikawa KWatashiba YIida H(2017)A Multipath OpenFlow Controller for Multiple TCP Stream ApplicationsJournal of Information Processing10.2197/ipsjjip.25.92425(924-933)Online publication date: 2017
https://doi.org/10.2197/ipsjjip.25.924
Sen SRao NLiu QImam NKettimuthu RFoster I(2017)On Analytics of File Transfer Rates over Dedicated Wide-Area Connections2017 IEEE 13th International Conference on e-Science (e-Science)10.1109/eScience.2017.93(576-585)Online publication date: Oct-2017
https://doi.org/10.1109/eScience.2017.93
Show More Cited By

Index Terms

Improving GridFTP performance using the Phoebus session layer
1. General and reference
  1. Cross-computing tools and techniques
    1. Performance
2. Networks
  1. Network protocols
  2. Network services
    1. Network management

Recommendations

Improving GridFTP Performance with Split TCP Connections
E-SCIENCE '05: Proceedings of the First International Conference on e-Science and Grid Computing

Many grid computing projects involve sharing and transferring large volumes of data. It is often important to maximise data throughput. Grid computing environments can be highly heterogeneous in nature and could involve large round trip times between ...
Globus GridFTP: what's new in 2007
GridNets '07: Proceedings of the first international conference on Networks for grid applications

GridFTP is a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. It is based on the Internet FTP protocol, and it defines extensions for high-performance operation and security. The Globus ...
Performance of a GridFTP overlay network

TCP is widely deployed and is used by GridFTP for data transfers within grid environments. Due to TCP dynamics, throughput of data transfers can be much lower than the available bandwidth in wide area networks. Splitting a TCP connection into segments ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis

November 2009

778 pages

ISBN:9781605587448

DOI:10.1145/1654059

Conference Chair:
Wilfred Pinfold

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SC '09

Sponsor:

SIGARCH
IEEE-CS

SC '09: International Conference for High Performance Computing, Networking, Storage and Analysis

November 14 - 20, 2009

Oregon, Portland

Acceptance Rates

SC '09 Paper Acceptance Rate 59 of 261 submissions, 23%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
25
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nadig DRamamurthy BBockelman B(2022)SNAG: SDN-Managed Network Architecture for GridFTP Transfers Using Application-AwarenessIEEE/ACM Transactions on Networking10.1109/TNET.2022.315000030:4(1585-1598)Online publication date: Aug-2022
https://doi.org/10.1109/TNET.2022.3150000
Huang CNakasan CIchikawa KWatashiba YIida H(2017)A Multipath OpenFlow Controller for Multiple TCP Stream ApplicationsJournal of Information Processing10.2197/ipsjjip.25.92425(924-933)Online publication date: 2017
https://doi.org/10.2197/ipsjjip.25.924
Sen SRao NLiu QImam NKettimuthu RFoster I(2017)On Analytics of File Transfer Rates over Dedicated Wide-Area Connections2017 IEEE 13th International Conference on e-Science (e-Science)10.1109/eScience.2017.93(576-585)Online publication date: Oct-2017
https://doi.org/10.1109/eScience.2017.93
Yildirim EArslan EKim JKosar T(2016)Application-Level Optimization of Big Data Transfers through Pipelining, Parallelism and ConcurrencyIEEE Transactions on Cloud Computing10.1109/TCC.2015.24158044:1(63-75)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1109/TCC.2015.2415804
Kettimuthu RAgrawal GSadayappan PFoster I(2016)Differentiated Scheduling of Response-Critical and Best-Effort Wide-Area Data Transfers2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2016.97(1113-1122)Online publication date: May-2016
https://doi.org/10.1109/IPDPS.2016.97
Huang CNakasan CIchikawa KIida H(2015)A Multipath Controller for Accelerating GridFTP Transfer over SDNProceedings of the 2015 IEEE 11th International Conference on e-Science10.1109/eScience.2015.37(439-447)Online publication date: 31-Aug-2015
https://dl.acm.org/doi/10.1109/eScience.2015.37
Su YAgrawal GWoodring JMyers KWendelberger JAhrens J(2014)Effective and efficient data sampling using bitmap indicesCluster Computing10.1007/s10586-014-0360-517:4(1081-1100)Online publication date: 1-Dec-2014
https://dl.acm.org/doi/10.1007/s10586-014-0360-5
Su YWang YAgrawal GKettimuthu RGropp WMatsuoka S(2013)SDQuery DSIProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.1145/2503210.2503270(1-12)Online publication date: 17-Nov-2013
https://dl.acm.org/doi/10.1145/2503210.2503270
Su YAgrawal GWoodring JMyers KWendelberger JAhrens JParashar MWeissman JEpema DFigueiredo R(2013)Taming massive distributed datasetsProceedings of the 22nd international symposium on High-performance parallel and distributed computing10.1145/2493123.2462906(13-24)Online publication date: 17-Jun-2013
https://dl.acm.org/doi/10.1145/2493123.2462906
Su YAgrawal GWoodring JMyers KWendelberger JAhrens JParashar MWeissman JEpema DFigueiredo R(2013)Taming massive distributed datasetsProceedings of the 22nd international symposium on High-performance parallel and distributed computing10.1145/2462902.2462906(13-24)Online publication date: 17-Jun-2013
https://dl.acm.org/doi/10.1145/2462902.2462906
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents