Article

The Globus Striped GridFTP Framework and Server

Authors:

William Allcock,

John Bresnahan,

Rajkumar Kettimuthu,

Catalin Dumitrescu,

Ian FosterAuthors Info & Claims

SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing

Page 54

https://doi.org/10.1109/SC.2005.72

Published: 12 November 2005 Publication History

Abstract

The GridFTP extensions to the File Transfer Protocol define a general-purpose mechanism for secure, reliable, high-performance data movement. We report here on the Globus striped GridFTP framework, a set of client and server libraries designed to support the construction of data-intensive tools and applications. We describe the design of both this framework and a striped GridFTP server constructed within the framework. We show that this server is faster than other FTP servers in both single-process and striped configurations, achieving, for example, speeds of 27.3 Gbit/s memory-to-memory and 17 Gbit/s disk-to-disk over a 60 millisecond round trip time, 30 Gbit/s network. In another experiment, we show that the server can support 1800 concurrent clients without excessive load. We argue that this combination of performance and modular structure make the Globus GridFTP framework both a good foundation on which to build tools and applications, and a unique testbed for the study of innovative data management techniques and network protocols.

References

[1]

1. Bonnie file system benchmark, 2005. www.textuality.com/bonnie.

[2]

2. Distributed Optical Testbed, 2004. www.dotresearch.org.

[3]

3. General Parallel File System (GPFS), 2004. www- 1.ibm.com/servers/eserver/clusters/software/gpfs.html.

[4]

4. NcFTPd Server, 2005. www.ncftp.com.

[5]

5. Tsunami Network Protocol Implementation, 2004. www.indiana.edu/~anml/anmlresearch.html.

[6]

6. Washington University FTP Daemon (WU-FTPD), 2005. www.wu-ftpd.org.

[7]

7. Allcock, W. GridFTP: Protocol Extensions to FTP for the Grid. Global Grid ForumGFD-R-P. 020, 2003.

[8]

8. Allman, M., Paxson, V. and Stevens, W. TCP Congestion Control. IETF, RFC-2581, 1999.

Digital Library

[9]

9. Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnal, D., Tuecke, S., Data Management and Transfer in High Performance Computational Grid Environments. Parallel Computing Journal, 28 (5), 749-771. 2002.

Digital Library

[10]

10. Allcock, W., Bresnahan, J., Kettimuthu, R. and Link, J., The Globus eXtensible Input/Output System (XIO): A Protocol-Independent I/O System for the Grid. Joint Workshop on High-Performance Grid Computing and High-Level Parallel Programming Models held in conjunction with International Parallel and Distributed Processing Symposium, 2005.

Digital Library

[11]

11. Bavier, A., Bowman, M., Chun, B., Culler, D., Karlin, S., Muir, S., Peterson, L., Roscoe, T., Spalink, T. and Wawrzoniak, M., Operating System Support for Planetary-Scale Services. 1st Symposium on Network Systems Design and Implementation, 2004.

Digital Library

[12]

12. Beck, M., Moore, T. and Plank, J., An End-to-End Approach to Globally Scalable Network Storage. ACM SIGCOMM, 2002.

Digital Library

[13]

13. Bent, J., Venkataramani, V., LeRoy, N., Roy, A., Stanley, J., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H. and Livny, M., Flexibility, Manageability, and Performance in a Grid Storage Appliance. 11th IEEE International Symposium on High Performance Distributed Computing, 2002, IEEE Computer Society Press.

Digital Library

[14]

14. Catlett, C. The TeraGrid: A Primer, 2002. www.teragrid.org.

[15]

15. Chien, A., Faber, T., Falk, A., Bannister, J., Grossman, R. and Leigh, J. Transport Protocols for High Performance: Whither TCP? Communications of the ACM, 46 (11). 42-49. 2003.

Digital Library

[16]

16. Clark, D., Lambert, M. and Zhang, L. NETBLT: A Bulk Data Transfer Protocol. IETF, RFC 998, 1987.

[17]

17. Cohen, B. Incentives Build Robustness in BitTorrent. 2003.

[18]

18. Czajkowski, K., Foster, I., Karonis, N., Kesselman, C., Martin, S., Smith, W. and Tuecke, S. A Resource Management Architecture for Metacomputing Systems. 4th Workshop on Job Scheduling Strategies for Parallel Processing, Springer-Verlag, 1998, 62-82.

Digital Library

[19]

19. Dumitrescu, C., Raicu, I., Ripeanu, M. and Foster, I., DiPerF: Automated DIstributed PERformance testing Framework. 5th International Workshop in Grid Computing, 2004.

Digital Library

[20]

20. Dunigan, T., Mathis, M. and Tierney, B., A TCP Tuning Daemon. IEEE Supercomputing 2002, Baltimore, Maryland, 2002.

Digital Library

[21]

21. Floyd, S. HighSpeed TCP for Large Congestion Windows. IETF, RFC 3649, 2003.

Digital Library

[22]

22. Foster, I., Alpert, E., Chervenak, A., Drach, B., Kesselman, C., Nefedova, V., Middleton, D., Shoshani, A., Sim, A. and Williams, D., The Earth System Grid II: Turning Climate Datasets Into Community Resources. Annual Meeting of the American Meteorological Society, 2002.

[23]

23. Foster, I., D. R. Kohr, J., Krishnaiyer, R. and Choudhary, A. A Library-Based Approach to Task Parallelism in a Data-Parallel Language. Journal of Parallel and Distributed Computing, 45(2). 148-158. 1998.

Digital Library

[24]

24. Foster, I., Fidler, M., Roy, A., Sander, V. and Winkler, L. End-to-End Quality of Service for High-end Applications. Computer Communications, 27(14). 1375- 1388. 2004.

Digital Library

[25]

25. Foster, I. and Kesselman, C. Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications, 11 (2). 115-128. 1997.

[26]

26. Foster, I., Kesselman, C., Tsudik, G. and Tuecke, S., A Security Architecture for Computational Grids. 5th ACM Conference on Computer and Communications Security, 1998, 83-91.

Digital Library

[27]

27. Foster, I., Kohr, D., Krishnaiyer, R. and Mogill, J. Remote I/O: Fast Access to Distant Storage. IOPADS'97, 1997, 14-25.

Digital Library

[28]

28. Gropp, W., Lusk, E. and Skjellum, A. Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press, 1994.

Digital Library

[29]

29. Gu, Y. and Grossman, R. L., UDT: An Application Level Transport Protocol for Grid Computing. Second International Workshop on Protocols for Fast Long-Distance Networks, 2003.

[30]

30. Hacker, T., Athey, B. and Noble, B., The end-to-end performance effects of parallel tcp sockets on a lossy wide-area network. 16th IEEECS/ACM International Parallel and Distributed Processing Symposium, 2002.

Digital Library

[31]

31. Hacker, T. J., Noble, B. D. and Athey, B. D., Improving Throughput and Maintaining Fairness using Parallel TCP. IEEE InfoCom, 2004.

[32]

32. He, E., Leigh, J., Yu, O. and DeFanti, T. A., Reliable Blast UDP: Predictable High Performance Bulk Data Transfer. IEEE Cluster Computing, 2002.

Digital Library

[33]

33. Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanan, M., Sidebotham, R. N. and West, M. J. Scale and Performance in a Distributed File System. ACM Transactions on Computer Systems, 6 (1). 51-81. 1988.

Digital Library

[34]

34. Hutchinson, N. C. and Peterson, L. L. The x-Kernel: An architecture for implementing network protocols. IEEE Transactions on Software Engineering, 17 (1). 64-76. 1991.

Digital Library

[35]

35. Jin, C., Wei, D. X. and Low, S. H., FAST TCP: motivation, architecture, algorithms, performance. IEEE Infocom, 2004.

[36]

36. Johnston, W., Greiman, W., Hoo, G., Lee, J., Tierney, B., Tull, C. and Olson, D., High-Speed Distributed Data Handling for On-Line Instrumentation Systems. ACM/IEEE SC97: High Performance Networking and Computing, 1997.

Digital Library

[37]

37. Karonis, N., Supinski, B. d., Foster, I., Gropp, W., Lusk, E. and Bresnahan, J., Exploiting Hierarchy in Parallel Computer Networks to Optimize Collective Operation Performance. 14th International Parallel and Distributed Processing Symposium, 2000, 377-384.

Digital Library

[38]

38. Katabi, D., Handley, M. and Rohrs, C., Congestion Control for High Bandwidth-Delay Product Networks. Sigcomm, 2002.

Digital Library

[39]

39. Kelly, T., Scalable TCP: Improving Performance in High-Speed Wide Area Networks. First International Workshop on Protocols for Fast Long Distance Networks, 2003.

Digital Library

[40]

40. Kielmann, T., Hofman, R., Bal, H., Plaat, A. and Bhoedjang, R., MagPIe: MPI's Collective Communication Operations for Clustered Wide Area Systems. 7th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 1999, 131-140.

Digital Library

[41]

41. Kosar, T. and Livny, M. A Framework for Reliable and Efficient Data Placement in Distributed Computing Systems. Journal of Parallel and Distributed Computing. 2005.

Digital Library

[42]

42. Mandrichenko, I. GridFTP Protocol Improvements. Global Grid Forum, GWD-E-21, 2003.

[43]

43. Neuman, B.C. and Ts'o, T. Kerberos: An Authentication Service for Computer Networks. IEEE Communications Magazine, 32 (9). 33-88. 1994.

[44]

44. Pearlman, L., Welch, V., Foster, I., Kesselman, C. and Tuecke, S., A Community Authorization Service for Group Collaboration. IEEE 3rd International Workshop on Policies for Distributed Systems and Networks, 2002.

Digital Library

[45]

45. Popek, G.J., Guy, R.G., Thomas W. Page, J. and Heidemann, J.S., Replication in Ficus Distributed File Systems. Workshop on Management of Replicated Data, 1990, IEEE, 20-25.

[46]

46. Postel, J. Transmission Control Protocol. Internet Engineering Task Force, RFC 793, 1981.

[47]

47. Postel, J. and Reynolds, J. File Transfer Protocol. Internet Engineering Task Force, RFC 959, 1985.

[48]

48. Qiu, L., Zhang, Y. and Keshav, S., On Individual and Aggregate TCP Performance. 7th International Conference on Network Protocols, 1999.

Digital Library

[49]

49. Ramaswamy, S. and Banerjee, P. Automatic Generation of Efficient Array Redistribution Routines for Distributed Memory Multicomputers. Frontiers '95: The 5th Symposium on the Frontiers of Massively Parallel Computation, McLean, Va., 1995, 342-349.

Digital Library

[50]

50. Rosario, J.d., Bordawekar, R. and Choudhary, A. Improved Parallel I/O via a Two-Phase Runtime Access Strategy. Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS'93, April 1993, 56- 70.

[51]

51. Seamons, K.E., Chen, Y., Jones, P., Jozwiak, J. and Winslett, M. Server-Directed Collective I/O in Panda. Supercomputing '95, 1995.

Digital Library

[52]

52. Sherwood, R., Braud, R. and Bhattacharjee, B., Slurpie: A Cooperative Bulk Data Transfer Protocol. InfoComm, 2004.

[53]

53. Sivakumar, H., Grossman, R.L., Mazzucco, M., Pan, Y. and Zhang, Q. Simple Available Bandwidth Utilization Library for High-Speed Wide Area Networks. Journal of Supercomputing. 2004.

Digital Library

[54]

54. Strayer, W.T., Lewis, M.J. and Cline Jr., R.E. XTP as Transport Protocol for Distributed Parallel Processing. USENIX Symposium on High-speed Networking. 1994.

Digital Library

[55]

55. Swany, M., Improving Throughput for Grid Applications with Network Logistics. SC'04, 2004.

Digital Library

[56]

56. Thain, D., Basney, J., Son, S.-C. and Livny, M., The Kangaroo Approach to Data Movement on the Grid. 10th IEEE International Symposium on High Performance Distributed Computing, 2001, IEEE Computer Society Press, 7-9.

Digital Library

[57]

57. Tuecke, S., Welch, V., Engert, D., Pearlman, L. and Thompson, M. Internet X.509 Public Key Infrastructure Proxy Certificate Profile. Internet Engineering Task Force, RFC 3820, 2004.

[58]

58. WebDAV Web-based Distributed Authoring and Versioning, 2004. http://webdav.org.

[59]

59. Weigle, E. and Chien, A.A., The Composite Endpoint Protocol (CEP): Scalable Endpoints for Terabit Flows. CCGrid, 2005.

Digital Library

Cited By

Pápay LPustelnik JRzadca KStrack BStradomski PWołowiec BZasadzinski MSekar VYu MSeneviratne AVeitch D(2024)An exabyte a day: throughput-oriented, large scale, managed data transfers with EffingoProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672262(970-982)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672262
Arifuzzaman MArslan EGallivan KNikolopoulos DBeivide RGallopoulos E(2023)Use Only What You Need: Judicious Parallelism For File Transfers in High Performance NetworksProceedings of the 37th International Conference on Supercomputing10.1145/3577193.3593722(122-132)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3577193.3593722
Benninger KJordan CLambert MSamuel TLiming LWheeler D(2022)Experiences in network and data transfer across large virtual organizations—a retrospectivePractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3530763(1-8)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3530763
Show More Cited By

Recommendations

Globus GridFTP: what's new in 2007
GridNets '07: Proceedings of the first international conference on Networks for grid applications

GridFTP is a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. It is based on the Internet FTP protocol, and it defines extensions for high-performance operation and security. The Globus ...
The zebra striped network file system
Managed GridFTP
IPDPSW '11: Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum

GridFTP extends the standard FTP protocol to provide a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. The Globus GridFTP implementation has become the preeminent high-performance data transfer ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing

November 2005

829 pages

ISBN:1595930612

General Chair:
William Kramer

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Computer Society

United States

Publication History

Published: 12 November 2005

Check for updates

Qualifiers

Article

Conference

SC '05

Sponsor:

SIGARCH
IEEE-CS

SC '05: International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 18, 2005

Acceptance Rates

SC '05 Paper Acceptance Rate 62 of 260 submissions, 24%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

145
Total Citations
View Citations
840
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pápay LPustelnik JRzadca KStrack BStradomski PWołowiec BZasadzinski MSekar VYu MSeneviratne AVeitch D(2024)An exabyte a day: throughput-oriented, large scale, managed data transfers with EffingoProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672262(970-982)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672262
Arifuzzaman MArslan EGallivan KNikolopoulos DBeivide RGallopoulos E(2023)Use Only What You Need: Judicious Parallelism For File Transfers in High Performance NetworksProceedings of the 37th International Conference on Supercomputing10.1145/3577193.3593722(122-132)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3577193.3593722
Benninger KJordan CLambert MSamuel TLiming LWheeler D(2022)Experiences in network and data transfer across large virtual organizations—a retrospectivePractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3530763(1-8)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3530763
Cleveland SArisdakessian CNelson CBelcaid MFrank KJacobs G(2022)The C-MĀIKI Gateway: A Modern Science Platform for Analyzing Microbiome DataPractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3530291(1-7)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3530291
Arifuzzaman MArslan Ede Supinski BHall MGamblin T(2021)Online optimization of file transfers in high-speed networksProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476208(1-13)Online publication date: 14-Nov-2021
https://dl.acm.org/doi/10.1145/3458817.3476208
Liu ZKettimuthu RChung JAnanthakrishnan RLink MFoster I(2021)Design and Evaluation of a Simple Data Interface for Efficient Data Transfer across Diverse StorageACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/34520076:1(1-25)Online publication date: 29-May-2021
https://dl.acm.org/doi/10.1145/3452007
Yang YCao QYao JJiang HYang L(2020)Batch-file Operations to Optimize Massive Files AccessingACM Transactions on Storage10.1145/339428616:3(1-25)Online publication date: 16-Jul-2020
https://dl.acm.org/doi/10.1145/3394286
Arisdakessian CCleveland SBelcaid M(2020)MetaFlow|mics: Scalable and Reproducible Nextflow Pipelines for the Analysis of Microbiome Marker DataPractice and Experience in Advanced Research Computing 2020: Catch the Wave10.1145/3311790.3396664(120-124)Online publication date: 26-Jul-2020
https://dl.acm.org/doi/10.1145/3311790.3396664
Allcock WAllen BAnanthakrishnan RBlaiszik BChard KChard RFoster ILacinski LPapka MWagner RFurlani T(2019)PetrelPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3332241(1-7)Online publication date: 28-Jul-2019
https://dl.acm.org/doi/10.1145/3332186.3332241
Meade PRiedel BSchultz DFurlani T(2019)IceCube's Long Term Archive SoftwarePractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3332196(1-5)Online publication date: 28-Jul-2019
https://dl.acm.org/doi/10.1145/3332186.3332196
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents