Conflict-free Replicated Priority Queue: Design, Verification and Evaluation
Pages 302 - 312
Abstract
Internet-scale distributed systems often rely on replication to achieve fault-tolerance and load distribution. To provide low latency and high availability, the systems are often required to accept updates on one replica immediately and then propagate the updates among replicas asynchronously. Conflict-free Replicated Data Type (CRDT) is a principled approach to addressing the challenge for these systems to resolve conflicts among concurrent updates. Although many CRDTs have been studied, little research has been done on Conflict-free Replicated Priority Queue (CRPQ), which is a collection of elements that focuses on maintaining element orderings based on their priority values, and can be used in many applications scenarios such as task scheduling and network routing. In this work, we discuss the design rationales of CRPQs and introduce two CRPQ designs: Add-Win CRPQ and Remove-Win CRPQ. The correctness of the designs is formally verified using TLA+. We also demonstrate the effectiveness of our designs by implementing them over Redis. Our evaluation shows that both CRPQs perform well in terms of data consistency and memory overhead.
References
[1]
2022. The TLA+ Home Page. https://lamport.azurewebsites.net/tla/tla.html
[2]
2022. TLA+ – the Brontinus Release. Retrieved Apr. 21, 2022 from https://github.com/tlaplus/tlaplus/releases/tag/v1.7.1
[3]
2023. Azure Cosmos DB – NoSQL Database, Microsoft Azure. Retrieved Feb. 16, 2023 from https://azure.microsoft.com/en-us/products/cosmos-db
[4]
2023. Commands for sorted set of Redis. https://redis.io/commands/?group=sorted-set
[5]
2023. Conflict-Free Replicated Data Types Based on Redis. Retrieved Feb. 11, 2023 from https://github.com/elem-azar-unis/CRDT-Redis
[6]
2023. Main page of Redis. https://redis.io/
[7]
2023. Redis Active-Active Geo-Distribution (CRDTs-Based). https://redis.com/redis-enterprise/technology/active-active-geo-distribution/
[8]
2023. Riak Enterprise NoSQL Database, Scalable Database Solutions. Retrieved Feb. 16, 2023 from https://riak.com/
[9]
Mehdi Ahmed-Nacer, Claudia-Lavinia Ignat, Gérald Oster, Hyun-Gul Roh, and Pascal Urso. 2011. Evaluating CRDTs for Real-Time Document Editing. In Proceedings of the 11th ACM Symposium on Document Engineering(DocEng ’11). ACM, New York, NY, USA, 103–112. https://doi.org/10.1145/2034691.2034717
[10]
Dan Alistarh, Justin Kopinsky, Jerry Li, and Nir Shavit. 2015. The SprayList: A Scalable Relaxed Priority Queue. In Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (San Francisco, CA, USA) (PPoPP 2015). Association for Computing Machinery, New York, NY, USA, 11–20. https://doi.org/10.1145/2688500.2688523
[11]
Alexandr Andoni, Dumitru Daniliuc, Sarfraz Khurshid, and Darko Marinov. 2002. Evaluating the "Small Scope Hypothesis". (2002).
[12]
Ratan Bajpai, Krishna Kishore Dhara, and Venkatesh Krishnaswamy. 2008. QPID: A Distributed Priority Queue with Item Locality. In 2008 IEEE International Symposium on Parallel and Distributed Processing with Applications. 215–223. https://doi.org/10.1109/ISPA.2008.90
[13]
Kenneth Birman, André Schiper, and Pat Stephenson. 1991. Lightweight Causal and Atomic Group Multicast. ACM Transactions on Computer Systems 9, 3 (Aug. 1991), 272–314. https://doi.org/10.1145/128738.128742
[14]
Eric A. Brewer. 2000. Towards Robust Distributed Systems (Abstract). In Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing(PODC’00). ACM, New York, NY, USA, 7–. https://doi.org/10.1145/343477.343502
[15]
Martin A. Brown. 2020. Traffic Control HOWTO. http://tldp.org/HOWTO/Traffic-Control-HOWTO/index.html
[16]
Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages(POPL ’14). ACM, New York, NY, USA, 271–284. https://doi.org/10.1145/2535838.2535848
[17]
Vitor Enes, Paulo Sergio Almeida, Carlos Baquero, and Joao Leitao. 2019. Efficient Synchronization of State-Based CRDTs. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). 148–159. https://doi.org/10.1109/ICDE.2019.00022
[18]
Michael Feldmann and Christian Scheideler. 2019. Skeap & Seap: Scalable Distributed Priority Queues for Constant and Arbitrary Priorities. In The 31st ACM Symposium on Parallelism in Algorithms and Architectures (Phoenix, AZ, USA) (SPAA ’19). Association for Computing Machinery, New York, NY, USA, 287–296. https://doi.org/10.1145/3323165.3323193
[19]
Seth Gilbert and Nancy A. Lynch. 2012. Perspectives on the CAP Theorem. Computer 45, 2 (2012), 30–36. https://doi.org/10.1109/MC.2011.389
[20]
Léon Gondelman, Simon Oddershede Gregersen, Abel Nieto, Amin Timany, and Lars Birkedal. 2021. Distributed Causal Memory: Modular Specification and Verification in Higher-Order Distributed Separation Logic. Proc. ACM Program. Lang. 5, POPL, Article 42 (Jan. 2021). https://doi.org/10.1145/3434323
[21]
Travis Hance, Marijn Heule, Ruben Martins, and Bryan Parno. 2021. Finding Invariants of Distributed Systems: It’s a Small (Enough) World after All. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21). USENIX Association, 115–131. https://www.usenix.org/conference/nsdi21/presentation/hance
[22]
Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Communications of The Acm 21, 7 (July 1978), 558–565. https://doi.org/10.1145/359545.359563
[23]
Leslie Lamport. 2009. The PlusCal Algorithm Language. In Theoretical Aspects of Computing - ICTAC 2009, Martin Leucker and Carroll Morgan (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 36–60.
[24]
Wyatt Lloyd, Michael J. Freedman, Michael Kaminsky, and David G. Andersen. 2013. Stronger Semantics for Low-Latency Geo-Replicated Storage. In Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation(Nsdi’13). USENIX Association, Berkeley, CA, USA, 313–328. http://dl.acm.org/citation.cfm?id=2482626.2482657
[25]
Yue Ma, Runbo Ni, Xiaofeng Gao, and Guihai Chen. 2021. Mixed Priority Queue Scheduling Based on Spectral Clustering in Spatial Crowdsourcing. In 2021 IEEE International Conference on Web Services (ICWS). 293–302. https://doi.org/10.1109/ICWS53863.2021.00047
[26]
Johannes Oetsch, Michael Prischink, Jörg Pührer, Martin Schwengerer, and Hans Tompits. 2012. On the Small-Scope Hypothesis for Testing Answer-Set Programs. In Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning(KR’12). AAAI Press, Rome, Italy, 43–53.
[27]
Gérald Oster, Pascal Urso, Pascal Molli, and Abdessamad Imine. 2006. Data Consistency for P2P Collaborative Editing. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work(CSCW ’06). ACM, New York, NY, USA, 259–268. https://doi.org/10.1145/1180875.1180916
[28]
Anastasiia Postnikova, Nikita Koval, Giorgi Nadiradze, and Dan Alistarh. 2022. Multi-Queues Can Be State-of-the-Art Priority Schedulers. In Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (Seoul, Republic of Korea) (PPoPP ’22). Association for Computing Machinery, New York, NY, USA, 353–367. https://doi.org/10.1145/3503221.3508432
[29]
Nuno Preguiça. 2018. Conflict-Free Replicated Data Types: An Overview. arXiv preprint arXiv:1806.10254 (2018). arxiv:1806.10254
[30]
Yue Qian, Zhonghai Lu, and Qiang Dou. 2010. QoS scheduling for NoCs: Strict Priority Queueing versus Weighted Round Robin. In 2010 IEEE International Conference on Computer Design. 52–59. https://doi.org/10.1109/ICCD.2010.5647577
[31]
Hyun-Gul Roh, Myeongjae Jeon, Jin-Soo Kim, and Joonwon Lee. 2011. Replicated Abstract Data Types: Building Blocks for Collaborative Applications. J. Parallel Distrib. Comput. 71, 3 (March 2011), 354–368. https://doi.org/10.1016/j.jpdc.2010.12.006
[32]
Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. A Comprehensive Study of Convergent and Commutative Replicated Data Types. Research Report RR-7506. Inria – Centre Paris-Rocquencourt ; INRIA, "". 50 pages. https://hal.inria.fr/inria-00555588
[33]
Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-Free Replicated Data Types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems(SSS’11). Springer-Verlag, Berlin, Heidelberg, 386–400. http://dl.acm.org/citation.cfm?id=2050613.2050642
[34]
Chengzheng Sun, Yi Xu, and Agustina Agustina. 2014. Exhaustive Search of Puzzles in Operational Transformation. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW ’14). Association for Computing Machinery, New York, NY, USA, 519–529. https://doi.org/10.1145/2531602.2531630
[35]
Stephane Weiss, Pascal Urso, and Pascal Molli. 2010. Logoot-Undo: Distributed Collaborative Editing System on P2P Networks. IEEE Transactions on Parallel and Distributed Systems 21, 8 (2010), 1162–1174. https://doi.org/10.1109/TPDS.2009.173
[36]
Yi Xu, Chengzheng Sun, and Mo Li. 2014. Achieving Convergence in Operational Transformation: Conditions, Mechanisms and Systems. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW ’14). Association for Computing Machinery, New York, NY, USA, 505–518. https://doi.org/10.1145/2531602.2531629
[37]
Serif Yesil, Azin Heidarshenas, Adam Morrison, and Josep Torrellas. 2019. Understanding Priority-Based Scheduling of Graph Algorithms on a Shared-Memory Platform. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’19). Association for Computing Machinery, New York, NY, USA, Article 46, 14 pages. https://doi.org/10.1145/3295500.3356160
[38]
Ding Yuan, Yu Luo, Xin Zhuang, Guilherme Renna Rodrigues, Xu Zhao, Yongle Zhang, Pranay U. Jain, and Michael Stumm. 2014. Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation(OSDI’14). USENIX Association, USA, 249–265.
[39]
Marek Zawirski. 2015. Dependable Eventual Consistency with Replicated Data Types. Theses. Universite Pierre et Marie Curie. https://tel.archives-ouvertes.fr/tel-01248051
[40]
Peter Zeller, Annette Bieniusa, and Arnd Poetzsch-Heffter. 2014. Formal Specification and Verification of CRDTs. In Formal Techniques for Distributed Objects, Components, and Systems, Erika Ábrahám and Catuscia Palamidessi (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 33–48.
Index Terms
- Conflict-free Replicated Priority Queue: Design, Verification and Evaluation
Recommendations
Approaches to Conflict-free Replicated Data Types
Conflict-free Replicated Data Types (CRDTs) allow optimistic replication in a principled way. Different replicas can proceed independently, being available even under network partitions and always converging deterministically: Replicas that have received ...
Geometric tail of queue length of low-priority customers in a nonpreemptive priority MAP/PH/1 queue
We consider a MAP/PH/1 queue with two priority classes and nonpreemptive discipline, focusing on the asymptotic behavior of the tail probability of queue length of low-priority customers. A sufficient condition under which this tail probability decays ...
Model‐checking‐driven explorative testing of CRDT designs and implementations
AbstractInternet‐scale distributed systems often replicate data at multiple geographic locations to provide low latency and high availability, despite node and network failures. According to the CAP theorem, low latency and high availability can only be ...
Comments
Information & Contributors
Information
Published In
August 2023
332 pages
ISBN:9798400708947
DOI:10.1145/3609437
- Editors:
- Hong Mei,
- Jian Lv,
- Zhi Jin,
- Xuandong Li,
- Xiaohu Yang,
- Xin Xia
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- Cooperation Fund of Huawei-Nanjing University Next Generation Programming Innovation Lab
Conference
Internetware 2023
Acceptance Rates
Overall Acceptance Rate 55 of 111 submissions, 50%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 76Total Downloads
- Downloads (Last 12 months)67
- Downloads (Last 6 weeks)2
Reflects downloads up to 06 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format