Article

HQ replication: a hybrid quorum protocol for byzantine fault tolerance

Authors:

Barbara Liskov,

Rodrigo Rodrigues,

Liuba ShriraAuthors Info & Claims

OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation

Pages 177 - 190

Published: 06 November 2006 Publication History

Abstract

There are currently two approaches to providing Byzantine-fault-tolerant state machine replication: a replica-based approach, e.g., BFT, that uses communication between replicas to agree on a proposed ordering of requests, and a quorum-based approach, such as Q/U, in which clients contact replicas directly to optimistically execute operations. Both approaches have shortcomings: the quadratic cost of inter-replica communication is un-necessary when there is no contention, and Q/U requires a large number of replicas and performs poorly under contention.

We present HQ, a hybrid Byzantine-fault-tolerant state machine replication protocol that overcomes these problems. HQ employs a lightweight quorum-based protocol when there is no contention, but uses BFT to resolve contention when it arises. Furthermore, HQ uses only 3f + 1 replicas to tolerate f faults, providing optimal resilience to node failures.

We implemented a prototype of HQ, and we compare its performance to BFT and Q/U analytically and experimentally. Additionally, in this work we use a new implementation of BFT designed to scale as the number of faults increases. Our results show that both HQ and our new implementation of BFT scale as f increases; additionally our hybrid approach of using BFT to handle contention works well.

References

[1]

Abd-El-Malek, M., Ganger, G. R., Goodson, G. R., Reiter, M. K., and Wylie, J. J. Fault-scalable byzantine fault-tolerant services. In SOSP '05: Proceedings of the twentieth ACM symposium on Operating systems principles (New York, NY, USA, 2005), ACM Press, pp. 59--74.

Digital Library

[2]

Castro, M., and Liskov, B. Practical Byzantine Fault Tolerance and Proactive Recovery. ACM Transactions on Computer Systems 20, 4 (Nov. 2002), 398--461.

Digital Library

[3]

Chockler, G., Malkhi, D., and Reiter, M. Backoff protocols for distributed mutual exclusion and ordering. In Proc. of the IEEE International Conference on Distributed Computing Systems (2001).

Digital Library

[4]

Cowling, J., Myers, D., Liskov, B., Rodrigues, R., and Shrira, L. Hq replication: Properties and optimizations. Technical Memo In Prep., MIT Computer Science and Artificial Laboratory, Cambridge, Massachusetts, 2006.

[5]

Fry, C., and Reiter, M. Nested objects in a byzantine Quorum-replicated System. In Proc. of the IEEE Symposium on Reliable Distributed Systems (2004).

Digital Library

[6]

Herlihy, M. P., and Wing, J. M. Axioms for Concurrent Objects. In Conference Record of the 14th Annual ACM Symposium on Principles of Programming Languages (1987).

Digital Library

[7]

Kistler, J. J., and Satyanarayanan, M. Disconnected Operation in the Coda File System. In Thirteenth ACM Symposium on Operating Systems Principles (Asilomar Conference Center, Pacific Grove, CA., Oct. 1991), pp. 213--225.

Digital Library

[8]

Lamport, L., Shostak, R., and Pease, M. The Byzantine Generals Problem. ACM Transactions on Programming Languages and Systems 4, 3 (July 1982), 382--401.

Digital Library

[9]

Lamport, L. L. The implementation of reliable distributed multiprocess systems. Computer Networks 2 (1978), 95--114.

[10]

Liskov, B., Ghemawat, S., Gruber, R., Johnson, P., Shrira, L., and Williams, M. Replication in the Harp File System. In Proceedings of the Thirteenth ACM Symposium on Operating System Principles (Pacific Grove, California, 1991), pp. 226--238.

Digital Library

[11]

Liskov, B., and Rodrigues, R. Byzantine clients rendered harmless. Tech. Rep. MIT-LCS-TR-994 and INESC-ID TR-10-2005, July 2005.

Digital Library

[12]

Malkhi, D., and Reiter, M. Byzantine Quorum Systems. Journal of Distributed Computing 11, 4 (1998), 203--213.

Digital Library

[13]

Malkhi, D., and Reiter, M. An Architecture for Survivable Coordination in Large Distributed Systems. IEEE Transactions on Knowledge and Data Engineering 12, 2 (Apr. 2000), 187--202.

Digital Library

[14]

Martin, J.-P., and Alvisi, L. Fast byzantine consensus. In International Conference on Dependable Systems and Networks (2005), IEEE, pp. 402--411.

Digital Library

[15]

Merkle, R. C. A Digital Signature Based on a Conventional Encryption Function. In Advances in Cryptology - Crypto'87, C. Pomerance, Ed., no. 293 in Lecture Notes in Computer Science. Springer-Verlag, 1987, pp. 369--378.

Digital Library

[16]

National Institute of Standards and Technology. Fips 198: The keyed-hash message authentication code (hmac), March 2002.

[17]

National Institute of Standards and Tecnology. Fips 180--2: Secure hash standard, August 2002.

[18]

Reiter, M. The Rampart toolkit for building high-integrity services. Theory and Practice in Distributed Systems (Lecture Notes in Computer Science 938) (1995), 99--110.

Digital Library

[19]

Schneider, F. B. Implementing fault-tolerant services using the state machine approach: a tutorial. ACM Comput. Surv. 22, 4 (1990), 299--319.

Digital Library

[20]

White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., and Joglekar, A. An integrated experimental environment for distributed systems and networks. In Proc. of the Fifth Symposium on Operating Systems Design and Implementation (Boston, MA, Dec. 2002), USENIX Association, pp. 255--270.

Digital Library

[21]

Yin, J., Martin, J., Venkataramani, A., Alvisi, L., and Dahlin, M. Separating agreement from execution for byzantine fault tolerant services. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (Oct. 2003).

Digital Library

Cited By

Aguilera MBurgelin CGuerraoui RMurat AXygkis AZablotchi IGavrilovska ATerry D(2024)DSigProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691974(667-685)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691974
Wang FJi YLiu MLi YLi XZhang XShi XGai KRaymond Choo K(2021)An Optimization Strategy for PBFT Consensus Mechanism Based On Consortium BlockchainProceedings of the 3rd ACM International Symposium on Blockchain and Secure Critical Infrastructure10.1145/3457337.3457843(71-76)Online publication date: 24-May-2021
https://dl.acm.org/doi/10.1145/3457337.3457843
Gandhi NRoth ESandler BHaeberlen APhan LBarbalace ABhatotia PAlvisi LCadar C(2021)REBOUNDProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456257(523-539)Online publication date: 21-Apr-2021
https://dl.acm.org/doi/10.1145/3447786.3456257
Show More Cited By

HQ replication: a hybrid quorum protocol for byzantine fault tolerance

Recommendations

HQ replication: a hybrid quorum protocol for byzantine fault tolerance
OSDI '06: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7

There are currently two approaches to providing Byzantine-fault-tolerant state machine replication: a replica-based approach, e.g., BFT, that uses communication between replicas to agree on a proposed ordering of requests, and a quorum-based approach, ...
Database Replication
Branch replication scheme: A new model for data replication in large scale data grids

Data replication is a practical and effective method to achieve efficient and fault-tolerant data access in grids. Traditionally, data replication schemes maintain an entire replica in each site where a file is replicated, providing a read-only model. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation

November 2006

407 pages

ISBN:1931971471

Program Chairs:
Brian Bershad
University of Washington
,
Jeff Mogul
Hewlett-Packard Labs

Sponsors

VMware
NSF: National Science Foundation
Google Inc.
Infosys
SIGOPS: ACM Special Interest Group on Operating Systems
Sun Microsystems
Intel: Intel
Microsoft Research: Microsoft Research
DoCoMo USA Labs
USENIX Assoc: USENIX Assoc
HP invent
Ask.com
IBM: IBM

Publisher

USENIX Association

United States

Publication History

Published: 06 November 2006

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
688
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Aguilera MBurgelin CGuerraoui RMurat AXygkis AZablotchi IGavrilovska ATerry D(2024)DSigProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691974(667-685)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691974
Wang FJi YLiu MLi YLi XZhang XShi XGai KRaymond Choo K(2021)An Optimization Strategy for PBFT Consensus Mechanism Based On Consortium BlockchainProceedings of the 3rd ACM International Symposium on Blockchain and Secure Critical Infrastructure10.1145/3457337.3457843(71-76)Online publication date: 24-May-2021
https://dl.acm.org/doi/10.1145/3457337.3457843
Gandhi NRoth ESandler BHaeberlen APhan LBarbalace ABhatotia PAlvisi LCadar C(2021)REBOUNDProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456257(523-539)Online publication date: 21-Apr-2021
https://dl.acm.org/doi/10.1145/3447786.3456257
Gupta SRahnama SHellings JSadoghi M(2020)ResilientDBProceedings of the VLDB Endowment10.14778/3380750.338075713:6(868-883)Online publication date: 11-Mar-2020
https://dl.acm.org/doi/10.14778/3380750.3380757
Zhang ISharma NSzekeres AKrishnamurthy APorts D(2018)Building Consistent Transactions with Inconsistent ReplicationACM Transactions on Computer Systems10.1145/326998135:4(1-37)Online publication date: 16-Dec-2018
https://dl.acm.org/doi/10.1145/3269981
Duan SReiter MZhang HLie DMannan MBackes MWang X(2018)BEATProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security10.1145/3243734.3243812(2028-2041)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3243734.3243812
Zheng JLin QXu JWei CZeng CYang PZhang Y(2017)PaxosStoreProceedings of the VLDB Endowment10.14778/3137765.313777810:12(1730-1741)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.14778/3137765.3137778
Nikolaou SVan Renesse RLea RGopalakrishnan STilevich EMurphy ABlackstock M(2015)Turtle ConsensusProceedings of the 16th Annual Middleware Conference10.1145/2814576.2814811(185-196)Online publication date: 24-Nov-2015
https://dl.acm.org/doi/10.1145/2814576.2814811
Moraru IAndersen DKaminsky MLazowska ETerry DArpaci-Dusseau RGehrke J(2014)Paxos Quorum LeasesProceedings of the ACM Symposium on Cloud Computing10.1145/2670979.2671001(1-13)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1145/2670979.2671001
Jaffe AMoscibroda TSen SKowalski DPanconesi A(2012)On the price of equivocation in byzantine agreementProceedings of the 2012 ACM symposium on Principles of distributed computing10.1145/2332432.2332491(309-318)Online publication date: 16-Jul-2012
https://dl.acm.org/doi/10.1145/2332432.2332491
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents