research-article

Software transactional memory for large scale clusters

Authors:

Robert L. Bocchino,

Vikram S. Adve,

Bradford L. ChamberlainAuthors Info & Claims

PPoPP '08: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming

Pages 247 - 258

https://doi.org/10.1145/1345206.1345242

Published: 20 February 2008 Publication History

Abstract

While there has been extensive work on the design of software transactional memory (STM) for cache coherent shared memory systems, there has been no work on the design of an STM system for very large scale platforms containing potentially thousands of nodes. In this work, we present Cluster-STM, an STM designed for high performance on large-scale commodity clusters. Our design addresses several novel issues posed by this domain, including aggregating communication, managing locality, and distributing transactional metadata onto the nodes. We also re-evaluate several STM design choices previously studied for cache-coherent machines and conclude that, in some cases, different choices are appropriate on clusters. Finally, we show that our design scales well up to 512 processors. This is because on a cluster, the main barrier to STM scalability is the remote communication overhead imposed by the STM operations, and our design aggregates most of that communication with the communication of the underlying data.

References

[1]

A.-R. Adl-Tabatabai, B. T. Lewis, V. Menon, B. R. Murphy, B. Saha, and T. Shpeisman. Compiler and runtime support for efficient software transactional memory. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 26--37, New York, NY, 2006. ACM Press.

Digital Library

[2]

C. Amza, A. L. Cox, S. Dwarkadas, P. Keleher, H. Lu, R. Rajamony, W. Yu, and W. Zwaenepoel. TreadMarks: Shared memory computing on networks of workstations. Computer, 29(2):18--28, 1996.

Digital Library

[3]

D. A. Bader and K. Madduri. Design and implementation of the HPCS graph analysis benchmark on symmetric multiprocessors. Technical report, Georgia Institute of Technology, Atlanta, GA, September 2005.

[4]

P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley, 1987.

Digital Library

[5]

C. Blundell, J. Devietti, E. C. Lewis, and M. M. K. Martin. Making the fast case common and the uncommon case simple in unbounded transactional memory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 24--34, New York, NY, USA, 2007. ACM Press.

Digital Library

[6]

D. Bonachea. Gasnet specification, v1.1. Technical Report No. UCB/CSD-02-1207, University of California at Berkeley, October 2002.

Digital Library

[7]

B. D. Carlstrom, A. McDonald, H. Chafi, J. Chung, C. C. Minh, C. Kozyrakis, and K. Olukotun. The Atomos transactional programming language. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 1--13, New York, NY, 2006. ACM Press.

Digital Library

[8]

B. Chamberlain, D. Callahan, and H. Zima. Parallel programmability and the Chapel language. In Int'l. J. High Performance Comp. Applications, volume 21, pages 291--312, Thousand Oaks, CA, USA, 2007. Sage Publications, Inc.

Digital Library

[9]

P. Charles, C. Grothoff, V. Saraswat, C. Donawa, A. Kielstra, K. Ebcioglu, C. von Praun, and V. Sarkar. X10: An object-oriented approach to non-uniform cluster computing. In Proc. ACM SIGPLAN Conf. on Object-Oriented Prog., Systs., Langs., and Applications (OOPSLA), pages 519--538, New York, NY, 2005. ACM Press.

Digital Library

[10]

C. Cole and M. Herlihy. Snapshots and software transactional memory. J. Sci. Comp. Prog., 58(3):310--324, 2005.

Digital Library

[11]

D. Dice and N. Shavit. Understanding tradeoffs in software transactional memory. In Proc. Int'l. Symp. on Code Gen. and Optimization (CGO), pages 21--33, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[12]

T. El-Ghazawi, W. Carlson, T. Sterling, and K. Yelick. UPC: Distributed Shared-Memory Programming. Wiley-Interscience, June 2005.

Digital Library

[13]

K. P. Eswaran, J. N. Gray, R. A. Lorie, and I. L. Traiger. The notions of consistency and predicate locks in a database system. Commun. of the ACM, 19(11):624--633, 1976.

Digital Library

[14]

K. Fraser. Practical lock freedom. PhD thesis, Cambridge University Computer Laboratory, 2003.

[15]

T. Harris and K. Fraser. Language support for lightweight transactions. In Proc. ACM SIGPLAN Conf. on Object-Oriented Prog., Systs., Langs., and Applications (OOPSLA), pages 388--402, New York, NY, 2003. ACM Press.

Digital Library

[16]

T. Harris, M. Plesko, A. Shinnar, and D. Tarditi. Optimizing memory transactions. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 14--25, New York, NY, 2006. ACM Press.

Digital Library

[17]

M. Herlihy, V. Luchangco, M. Moir, and W. N. Scherer III. Software transactional memory for dynamic-sized data structures. In Symp. on Principles of Distrib. Comp., pages 92--101, New York, NY, 2003. ACM Press.

Digital Library

[18]

M. Herlihy and J. E. B. Moss. Transactional memory: Architectural support for lock-free data structures. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 289--300, New York, NY, USA, 1993. ACM Press.

Digital Library

[19]

M. Herlihy and Y. Sun. Distributed transactional memory for metric-space networks. In Symp. on Distrib. Comp., volume 3724. Springer, Sept. 2005.

Digital Library

[20]

S. Kumar, M. Chu, C. J. Hughes, P. Kundu, and A. Nguyen. Hybrid transactional memory. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 209--220, New York, NY, 2006. ACM Press.

Digital Library

[21]

H. T. Kung and J. T. Robinson. On optimistic methods for concurrency control. ACM Trans. on Database Systs., 6(2):213--226, 1981.

Digital Library

[22]

K. Manassiev, M. Mihailescu, and C. Amza. Exploiting distributed version concurrency in a transactional memory cluster. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 198--208, New York, NY, 2006. ACM Press.

Digital Library

[23]

V. J. Marathe, W. N. Scherer, and M. L. Scott. Design tradeoffs in modern software transactional memory systems. In Proc. Workshop on Langs., Compilers, and Run-Time Support for Scalable Systs., pages 1--7, New York, NY, 2004. ACM Press.

Digital Library

[24]

V. J. Marathe, W. N. Scherer III, and M. L. Scott. Adaptive software transactional memory. In Proc. Int'l. Symp. on Distrib. Comp., Cracow, Poland, Sep 2005. Earlier but expanded version available as TR 868, University of Rochester Computer Science Dept., May 2005.

Digital Library

[25]

M. Martin, C. Blundell, and E. Lewis. Subtleties of transactional memory atomicity semantics. IEEE Comp. Arch. Letters, 5(2), 2006.

Digital Library

[26]

C. C. Minh, M. Trautmann, J. Chung, A. McDonald, N. Bronson, J. Casper, C. Kozyrakis, and K. Olukotun. An effective hybrid transactional memory system with strong isolation guarantees. SIGARCH Comput. Archit. News, 35(2):69--80, 2007.

Digital Library

[27]

C. C. Minh, M. Trautmann, J. Chung, A. McDonald, N. Bronson, J. Casper, C. Kozyrakis, and K. Olukotun. An effective hybrid transactional memory system with strong isolation guarantees. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 69--80, New York, NY, USA, 2007. ACM Press.

Digital Library

[28]

R. W. Numerich and J. Reid. Co-array fortran for parallel programming. SIGPLAN Fortran Forum, 17(2):1--31, 1998.

Digital Library

[29]

C. H. Papadimitriou. The serializability of concurrent database updates. J. ACM, 26(4):631--653, 1979.

Digital Library

[30]

R. Rajwar, M. Herlihy, and K. Lai. Virtualizing transactional nemory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 494--505, Washington, DC, 2005. IEEE Computer Society.

Digital Library

[31]

D. P. Reed. Implementing atomic actions on decentralized data. ACM Trans. on Comp. Systs., 1(1):3--23, 1983.

Digital Library

[32]

B. Saha, A.-R. Adl-Tabatabai, R. L. Hudson, C. C. Minh, and B. Hertzberg. McRT-STM: A high performance software transactional memory system for a multi-core runtime. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 187--197, New York, NY, 2006. ACM Press.

Digital Library

[33]

B. Saha, A.-R. Adl-Tabatabai, and Q. Jacobson. Architectural support for software transactional memory. In Proc. ACM/IEEE Int'l. Symp. on Microarch. (MICRO), pages 185--196, Washington, DC, USA, 2006. IEEE Computer Society.

Digital Library

[34]

N. Shavit and D. Touitou. Software transactional memory. In Symp. on Principles of Distrib. Comp., pages 204--213, New York, NY, 1995. ACM Press.

Digital Library

[35]

T. Shpeisman, V. Menon, A.-R. Adl-Tabatabai, S. Balensiefer, D. Grossman, R. L. Hudson, K. F. Moore, and B. Saha. Enforcing isolation and ordering in STM. ACM Conf. on Prog. Lang. Design and Implementation (PLDI), 42(6):78--88, 2007.

Digital Library

[36]

A. Shriraman, M. F. Spear, H. Hossain, V. J. Marathe, S. Dwarkadas, and M. L. Scott. An integrated hardware-software approach to flexible transactional memory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 104--115, New York, NY, USA, 2007. ACM Press.

Digital Library

[37]

Sun Microsystems, Inc. The Fortress language specification, version 1.0 β. Technical report, Sun Microsystems, Inc., March 2007.

[38]

K. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. Hilfinger, S. Graham, D. Gay, P. Colella, and A. Aiken. Titanium: A high-performance Java dialect. Concurrency -- Practice and Experience, 10(11-13):825--836, Sept-Nov 1998.

Cited By

Lopes ACastro DRomano PTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)PIM-STM: Software Transactional Memory for Processing-In-Memory SystemsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640428(897-911)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640428
Busch CChlebus BHerlihy MPopovic MPoudel PSharma G(2023)Flexible scheduling of transactional memory on treesTheoretical Computer Science10.1016/j.tcs.2023.114184(114184)Online publication date: Sep-2023
https://doi.org/10.1016/j.tcs.2023.114184
Poudel PRai SGuragain SSharma G(2023)Ordered Scheduling in Control-Flow Distributed Transactional MemoryDistributed Computing and Intelligent Technology10.1007/978-3-031-24848-1_5(67-83)Online publication date: 8-Jan-2023
https://doi.org/10.1007/978-3-031-24848-1_5
Show More Cited By

Index Terms

Software transactional memory for large scale clusters
1. Computing methodologies
  1. Distributed computing methodologies
    1. Distributed programming languages
  2. Parallel computing methodologies
    1. Parallel programming languages
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language features
        Concurrent programming structures
      2. Language types
        Distributed programming languages
        Parallel programming languages

Recommendations

Lowering Conflicts of High Contention Software Transactional Memory
CSSE '08: Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 03

Two concurrent transactions are said to conflict based on linearizability semantics if they access the same shared data and at least one of them modifies that data. In many applications enforcing the strict linearizability semantics over the entire read-...
Software Distributed Shared Memory with Transactional Coherence - A Software Engine to Run Transactional Shared-memory Parallel Applications on Clusters
PDP '10: Proceedings of the 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing

Transactional Memory is a novel, promising approach for simplifying parallel programming and increasing its acceptance and diffusion. Until now, almost all the research work on TM has been focused on shared-memory architectures, while very limited ...
Software transactional memory for multicore embedded systems
LCTES '09

Embedded systems, like general-purpose systems, can benefit from parallel execution on a symmetric multicore platform. Unfortunately, concurrency issues present in general-purpose programming also apply to embedded systems, protection from which is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PPoPP '08: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming

February 2008

308 pages

ISBN:9781595937957

DOI:10.1145/1345206

General Chair:
Siddhartha Chatterjee
IBM Research USA
,
Program Chair:
Michael L. Scott
University of Rochester USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PPoPP08

Sponsor:

PPoPP08: ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

February 20 - 23, 2008

UT, Salt Lake City, USA

Acceptance Rates

Overall Acceptance Rate 230 of 1,014 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

87
Total Citations
View Citations
1,118
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lopes ACastro DRomano PTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)PIM-STM: Software Transactional Memory for Processing-In-Memory SystemsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640428(897-911)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640428
Busch CChlebus BHerlihy MPopovic MPoudel PSharma G(2023)Flexible scheduling of transactional memory on treesTheoretical Computer Science10.1016/j.tcs.2023.114184(114184)Online publication date: Sep-2023
https://doi.org/10.1016/j.tcs.2023.114184
Poudel PRai SGuragain SSharma G(2023)Ordered Scheduling in Control-Flow Distributed Transactional MemoryDistributed Computing and Intelligent Technology10.1007/978-3-031-24848-1_5(67-83)Online publication date: 8-Jan-2023
https://doi.org/10.1007/978-3-031-24848-1_5
Busch CChlebus BHerlihy MPopovic MPoudel PSharma G(2022)Flexible Scheduling of Transactional Memory on TreesStabilization, Safety, and Security of Distributed Systems10.1007/978-3-031-21017-4_10(146-163)Online publication date: 9-Nov-2022
https://doi.org/10.1007/978-3-031-21017-4_10
Poudel PSharma G(2021)Adaptive Versioning in Transactional Memory SystemsAlgorithms10.3390/a1406017114:6(171)Online publication date: 31-May-2021
https://doi.org/10.3390/a14060171
Busch CHerlihy MPopovic MSharma G(2021)Dynamic scheduling in distributed transactional memoryDistributed Computing10.1007/s00446-021-00410-wOnline publication date: 20-Nov-2021
https://doi.org/10.1007/s00446-021-00410-w
Williams BLeidel JWang XDonofrio DChen Y(2020)CircusTent: A Benchmark Suite for Atomic Memory OperationsProceedings of the International Symposium on Memory Systems10.1145/3422575.3422789(144-157)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3422575.3422789
Poudel PSharma G(2020)GraphTMProceedings of the 21st International Conference on Distributed Computing and Networking10.1145/3369740.3369774(1-10)Online publication date: 4-Jan-2020
https://dl.acm.org/doi/10.1145/3369740.3369774
Busch CHerlihy MPopovic MSharma G(2020)Dynamic Scheduling in Distributed Transactional Memory2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS47924.2020.00094(874-883)Online publication date: May-2020
https://doi.org/10.1109/IPDPS47924.2020.00094
Busch CHerlihy MPopovic MSharma G(2020)Fast Scheduling in Distributed Transactional MemoryTheory of Computing Systems10.1007/s00224-020-10008-7Online publication date: 23-Oct-2020
https://doi.org/10.1007/s00224-020-10008-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents