research-article

Combining ECN and RTT for Datacenter Transport

Authors:

Yibo ZhuAuthors Info & Claims

APNet '17: Proceedings of the First Asia-Pacific Workshop on Networking

Pages 36 - 42

https://doi.org/10.1145/3106989.3107002

Published: 03 August 2017 Publication History

Abstract

Datacenter transports should provide low average and tail flow completion times (FCT) to achieve desired application performance. While most prior datacenter transports take either ECN or RTT as congestion signal, this paper makes a case that both signals are indispensable: ECN, as a per-hop signal, is more effective to prevent packet loss; while RTT, as an end-to-end signal, controls end-to-end queueing delay better. As persistent low flow completion times imply low queueing delay and near zero packet loss, we introduce EAR, a new datacenter transport that hears and reacts to both ECN and RTT. Our preliminary results show that: 1) compared to delay-based DCTCP, EAR achieves up to 91% lower packet losses and 93% fewer timeouts; 2) compared to ECN-based DCTCP, EAR reduces RTT by up to 32% for cross-rack traffic in a 4-level fattree. As a result, EAR delivers persistent low average and tail completion times under various scenarios in large scale simulations.

References

[1]

Google Data Centers. https://www.google.com/about/datacenters.

[2]

Microsoft Datacenter Infrastructure. https://www.microsoft.com/en-us/cloud-platform/global-datacenters.

[3]

The Network Simulator NS-2. http://www.isi.edu/nsnam/ns.

[4]

Mohammad Al-Fares, Alexander Loukissas, and Amin Vahdat. 2008. A scalable, commodity data center network architecture. In SIGCOMM.

Digital Library

[5]

Mohammad Alizadeh, Albert Greenberg, David A Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, and Murari Sridharan. 2010. Data center tcp (dctcp). In SIGCOMM.

Digital Library

[6]

Mohammad Alizadeh, Abdul Kabbani, Tom Edsall, Balaji Prabhakar, Amin Vahdat, and Masato Yasuda. 2012. Less is more: trading a little bandwidth for ultra-low latency in the data center. In NSDI.

Digital Library

[7]

Mohammad Alizadeh, Shuang Yang, Milad Sharif, Sachin Katti, Nick McKeown, Balaji Prabhakar, and Scott Shenker. 2013. pfabric: Minimal near-optimal datacenter transport. In SIGCOMM.

Digital Library

[8]

Wei Bai, Kai Chen, Hao Wang, Li Chen, Dongsu Han, and Chen Tian. 2015. Information-Agnostic Flow Scheduling for Commodity Data Centers. In NSDI.

Digital Library

[9]

Wei Bai, Li Chen, Kai Chen, and Haitao Wu. 2016. Enabling ecn in multi-service multi-queue data centers. In NSDI.

Digital Library

[10]

Sally Floyd. 2003. HighSpeed TCP for large congestion windows. In RFC 3649.

Digital Library

[11]

Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: a new TCP-friendly high-speed TCP variant. In SIGOPS.

Digital Library

[12]

Dongsu Han, Robert Grandl, Aditya Akella, and Srinivasan Seshan. 2013. FCP: A Flexible Transport Framework for Accommodating Diversity. In SIGCOMM.

Digital Library

[13]

Chi-Yao Hong, Matthew Caesar, and P Godfrey. 2012. Finishing flows quickly with preemptive scheduling. In SIGCOMM.

Digital Library

[14]

Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew Goldberg. 2009. Quincy: fair scheduling for distributed computing clusters. In SOSP.

Digital Library

[15]

Van Jacobson. 1988. Congestion avoidance and control. In SIGCOMM.

Digital Library

[16]

Virajith Jalaparti, Peter Bodik, Ishai Menache, Sriram Rao, Konstantin Makarychev, and Matthew Caesar. 2015. Network-aware scheduling for data-parallel jobs: Plan when you can. In SIGCOMM.

Digital Library

[17]

Glenn Judd. 2015. Attaining the Promise and Avoiding the Pitfalls of TCP in the Datacenter. In NSDI.

Digital Library

[18]

Changhyun Lee, Chunjong Park, Keon Jang, Sue Moon, and Dongsu Han. 2016. DX: Latency-Based Congestion Control for Datacenters. In ToN.

Digital Library

[19]

Vincent Liu, Daniel Halperin, Arvind Krishnamurthy, and Thomas E Anderson. 2013. F10: A Fault-Tolerant Engineered Network. In NSDI.

Digital Library

[20]

Radhika Mittal, Nandita Dukkipati, Emily Blem, Hassan Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, David Zats, and others. 2015. TIMELY: RTT-based Congestion Control for the Datacenter. In SIGCOMM.

Digital Library

[21]

Ali Munir, Ghufran Baig, Syed M Irteza, Ihsan A Qazi, Alex X Liu, and Fahad R Dogar. 2014. Friends, not foes: synthesizing existing transport strategies for data center networks. In SIGCOMM.

Digital Library

[22]

Ali Munir, Ting He, Ramya Raghavendra, Franck Le, and Alex X Liu. 2016. Network Scheduling Aware Task Placement in Datacenters. In CoNEXT.

Digital Library

[23]

Ali Munir, Ihsan A Qazi, Zartash A Uzmi, Aisha Mushtaq, Saad N Ismail, M Safdar Iqbal, and Basma Khan. 2013. Minimizing flow completion times in data centers. In INFOCOM.

[24]

Rong Pan, Balaji Prabhakar, and Ashvin Laxmikantha. 2007. QCN: Quantized congestion notification. In IEEE802.

[25]

Arjun Roy, Hongyi Zeng, Jasmeet Bagga, George Porter, and Alex C Snoeren. 2015. Inside the social network's (datacenter) network. In SIGCOMM.

Digital Library

[26]

Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, and others. 2015. Jupiter rising: A decade of clos topologies and centralized control in google's datacenter network. In SIGCOMM.

Digital Library

[27]

Balajee Vamanan, Jahangir Hasan, and TN Vijaykumar. 2012. Deadline-aware datacenter tcp (d2tcp). In SIGCOMM.

Digital Library

[28]

Christo Wilson, Hitesh Ballani, Thomas Karagiannis, and Ant Rowtron. 2011. Better never than late: Meeting deadlines in datacenter networks. In SIGCOMM.

Digital Library

[29]

Lisong Xu, Khaled Harfoush, and Injong Rhee. 2004. Binary increase congestion control (BIC) for fast long-distance networks. In INFOCOM.

[30]

Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. 2010. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In EuroSys.

Digital Library

[31]

Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, and Ming Zhang. 2015. Congestion control for large-scale RDMA deployments. In SIGCOMM.

Digital Library

[32]

Yibo Zhu, Monia Ghobadi, Vishal Misra, and Jitendra Padhye. 2016. ECN or Delay: Lessons Learnt from Analysis of DCQCN and TIMELY. In CoNEXT.

Digital Library

Cited By

Lu YMa XCui C(2024)DCCSComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110457247:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110457
Hu JLuo WHe YWang JZhang D(2024)Deep Reinforcement Learning Based Load Balancing for Heterogeneous Traffic in Datacenter NetworksAlgorithms and Architectures for Parallel Processing10.1007/978-981-97-0798-0_16(270-289)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0798-0_16
Hu JHuang JLi ZWang JHe T(2023)A Receiver-Driven Transport Protocol With High Link Utilization Using Anti-ECN Marking in Data Center NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2022.321834320:2(1898-1912)Online publication date: Jun-2023
https://doi.org/10.1109/TNSM.2022.3218343
Show More Cited By

Index Terms

Combining ECN and RTT for Datacenter Transport
1. Networks
  1. Network protocols
    1. Transport protocols

Recommendations

ECN or Delay: Lessons Learnt from Analysis of DCQCN and TIMELY
CoNEXT '16: Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies

Data center networks, and especially drop-free RoCEv2 networks require efficient congestion control protocols. DCQCN (ECN-based) and TIMELY (delay-based) are two recent proposals for this purpose. In this paper, we analyze DCQCN and TIMELY using fluid ...
RELD, RTT ECN Loss Differentiation to optimize the performance of transport protocols on wireless networks

One major yet unsolved problem in wired-cum-wireless networks is the classification of losses, which might result from wireless temporary interferences or from network congestion. The transport protocol response to losses should be different for these ...
Combining RTT and ECN for RoCEv2 Protocol
HPCCT & BDAI '20: Proceedings of the 2020 4th High Performance Computing and Cluster Technologies Conference & 2020 3rd International Conference on Big Data and Artificial Intelligence

Remote direct memory access (RDMA) has the advantages of direct user-level access to HW, asynchronous communication, etc. RoCEv2 protocol enables RDMA technology to be used in large-scale data centers over Ethernet. It provides high throughput and low ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

APNet '17: Proceedings of the First Asia-Pacific Workshop on Networking

August 2017

127 pages

ISBN:9781450352444

DOI:10.1145/3106989

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

APNet'17

APNet'17: First Asia-Pacific Workshop on Networking

August 3 - 4, 2017

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 50 of 118 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu YMa XCui C(2024)DCCSComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110457247:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110457
Hu JLuo WHe YWang JZhang D(2024)Deep Reinforcement Learning Based Load Balancing for Heterogeneous Traffic in Datacenter NetworksAlgorithms and Architectures for Parallel Processing10.1007/978-981-97-0798-0_16(270-289)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0798-0_16
Hu JHuang JLi ZWang JHe T(2023)A Receiver-Driven Transport Protocol With High Link Utilization Using Anti-ECN Marking in Data Center NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2022.321834320:2(1898-1912)Online publication date: Jun-2023
https://doi.org/10.1109/TNSM.2022.3218343
Ma HXu D(2023)An INT-based TCP window modulator for congestion control in data center networksJournal of Network and Computer Applications10.1016/j.jnca.2023.103688217(103688)Online publication date: Aug-2023
https://doi.org/10.1016/j.jnca.2023.103688
Zeng GBai WChen GChen KHan DZhu YCui L(2022)Congestion Control for Cross-Datacenter NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2022.316158030:5(2074-2089)Online publication date: Oct-2022
https://doi.org/10.1109/TNET.2022.3161580
Hu SZeng GBai WWang ZQiao BChen KTan KWang Y(2022)Aeolus: A Building Block for Proactive Transport in Datacenter NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2021.311998630:2(542-556)Online publication date: Apr-2022
https://doi.org/10.1109/TNET.2021.3119986
Zeng GChen LYi BChen K(2022)Cutting Tail Latency in Commodity Datacenters with CloudburstIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796898(600-609)Online publication date: 2-May-2022
https://dl.acm.org/doi/10.1109/INFOCOM48880.2022.9796898
Bhutta AAhmed H(2022)Edge-Based Congestion-Aware Datacenter Load Balancing with Smart ProbingThe Computer Journal10.1093/comjnl/bxac13566:12(2908-2920)Online publication date: 21-Oct-2022
https://doi.org/10.1093/comjnl/bxac135
Huang JWang SLi SZou SHu JWang J(2021)HTPC: heterogeneous traffic-aware partition coding for random packet spraying in data center networksJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-021-00248-410:1Online publication date: 5-Jun-2021
https://dl.acm.org/doi/10.1186/s13677-021-00248-4
Biswas JGupta JKant KPal AMinturn D(2021)Provisioning Differentiated QoS for NVMe over Fabrics2021 IEEE 46th Conference on Local Computer Networks (LCN)10.1109/LCN52139.2021.9524967(154-161)Online publication date: 4-Oct-2021
https://doi.org/10.1109/LCN52139.2021.9524967
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents