Article

Accurate data redistribution cost estimation in software distributed shared memory systems

Authors:

Donald G. Morris,

David K. LowenthalAuthors Info & Claims

PPoPP '01: Proceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming

Pages 62 - 71

https://doi.org/10.1145/379539.379570

Published: 18 June 2001 Publication History

Abstract

Distributing data is one of the key problems in implementing efficient distributed-memory parallel programs. The problem becomes more difficult in programs where data redistribution between computational phases is considered. The global data distribution problem is to find the optimal distribution in multi-phase parallel programs. Solving this problem requires accurate knowledge of data redistribution cost.

We are investigating this problem in the context of a software distributed shared memory (SDSM) system, in which obtaining accurate redistribution cost estimates is difficult. This is because SDSM communication is implicit: It depends on access patterns, page locations, and the SDSM consistency protocol.

We have developed integrated compile- and run-time analysis for SDSM systems to determine accurate redistribution cost estimates with low overhead. Our resulting system, SUIF-Adapt, can efficiently and accurately estimate execution time, including redistribution, to within 5% of the actual time in all of our test cases and is often much closer. These precise costs enable SUIF-Adapt to find efficient global data distributions in multiple-phase programs.

References

[1]

J. Anderson and M. Lam. Global optimizations for parallelism and locality on scalable parallel machines. In Proceedings of the SIGPLAN '93 Conference on Program Language Design and Implementation, pages 112-125, June 1993.]]

Digital Library

[2]

J. B. Carter, J. K. Bennett, and W. Zwaenepoel. Implementation and performance of Munin. In Proceedings of 13th ACM Symposium On Operating Systems, pages 152-164, Oct. 1991.]]

Digital Library

[3]

S. Chandra and J. R. Larus. Optimizing communication in HPF programs on fine-grain distributed shared memory. InSixth Symposium on Principles and Practice ofParallel Programming, pages 100-111, June 1997.]]

Digital Library

[4]

M. W. Hall, J. M. Anderson, S. P. Amarasinghe, B. R. Murphy, S.-W. Liao, E. Bugnion, and M. S. Lam. Maximizing multiprocessor performance with the SUIF compiler. IEEE Computer, 29(12):84-89, Dec. 1996.]]

Digital Library

[5]

P. Havlak and K. Kennedy. An implementation of interprocedural bounded regular section analysis. IEEE Transactions on Parallel and Distributed Systems, 2(3):350-360, 1991.]]

Digital Library

[6]

G. M. Howard and D. K. Lowenthal. An integrated compiler/run-time system for global data distribution in distributed shared memory systems. In Second Workshop on Software DSM, May 2000.]]

[7]

L. Iftode. Home-Based Shared Virtual Memory. PhD thesis, Princeton University, June 1998.]]

Digital Library

[8]

S. Ioannidis and S. Dwarkadas. Compiler and run-time support for adaptive load balancing in software distributed shared memory systems. In Proceedings of the Fourth Workshop on Languages, Compilers, and Run-Time Systems for Parallel Computing, pages 107-122, May 1998.]]

Digital Library

[9]

P. Keleher, S. Dwarkadas, A. Cox, and W. Zwaenepoel. TreadMarks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the 1994 Winter Usenix Conference, pages 115-131, Jan. 1994.]]

Digital Library

[10]

K. Kennedy and U. Kremer. Automatic data layout for distributed-memory machines. ACM TOPLAS, 20(4):869-916, 1998.]]

Digital Library

[11]

K. Li and P. Hudak. Memory coherence in shared virtual memory systems. ACM Transactions on Computer Systems, 7(4), Nov. 1989.]]

Digital Library

[12]

D. K. Lowenthal and G. R. Andrews. An adaptive approach to data placement. In Proceedings of the 10th International Symposium on Parallel Processing, pages 349-353, Apr. 1996.]]

Digital Library

[13]

D. K. Lowenthal, V. W. Freeh, and G. R. Andrews. Using fine-grain threads and run-time decision making in parallel computing. Journal of Parallel and Distributed Computing, 37:41-54, Nov. 1996.]]

Digital Library

[14]

D. K. Lowenthal and F. Lowenthal. Supporting regular data distributions on nondedicated parallel machines (submitted to Supercomputing '01). May 2001.]]

[15]

H.Lu, A. L.Cox, S. Dwarkadas, R. Rajamony, and W. Zwaenepoel. Compiler and distributed shared memory support for irregular applications. In Sixth Symposium on Principles and Practice of Parallel Programming, pages 48-56, June 1997.]]

Digital Library

[16]

U. Rencuzogullari and S. Dwarkadas. Dynamic adaptation to available resources for parallel computing in an autonomous network of workstations. In Eighth Conference on Principles and Practice of Parallel Programming (to appear), June 2001.]]

Digital Library

[17]

C.-W. Tseng. An Optimizing Fortran D Compiler for MIMD Distributed-Memory Machines. PhD thesis, Rice University, Jan. 1993.]]

Digital Library

[18]

K. Zhang, J. Mellor-Crummey, and R. J. Fowler. Compilation and runtime optimizations for software distributed shared memory. InFifth Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, pages83-88, May 2000.]]

Digital Library

Cited By

Freeh VKappiah NLowenthal DBletsch T(2008)Just-in-time dynamic voltage scalingJournal of Parallel and Distributed Computing10.1016/j.jpdc.2008.04.00768:9(1175-1185)Online publication date: 1-Sep-2008
https://dl.acm.org/doi/10.1016/j.jpdc.2008.04.007
Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Kwok YMaciejewski ASiegel HAhmad IGhafoor A(2006)A semi-static approach to mapping dynamic iterative tasks onto heterogeneous computing systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2005.06.01566:1(77-98)Online publication date: 1-Jan-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2005.06.015
Show More Cited By

Index Terms

Accurate data redistribution cost estimation in software distributed shared memory systems

Recommendations

Accurate data redistribution cost estimation in software distributed shared memory systems

Distributing data is one of the key problems in implementing efficient distributed-memory parallel programs. The problem becomes more difficult in programs where data redistribution between computational phases is considered. The global data ...
Compiling shared-memory applications for distributed-memory systems
Scalable data redistribution services for distributed-memory machines

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PPoPP '01: Proceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming

June 2001

142 pages

ISBN:1581133464

DOI:10.1145/379539

Chairmen:
Michael Heath
Univ. of Illinois, Illinois, IN
,
Andrew Lumsdaine
Indiana Univ.

ACM SIGPLAN Notices Volume 36, Issue 7
July 2001
143 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/568014
Issue’s Table of Contents

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

PPoPP01

Sponsor:

SIGPLAN

PPoPP01: ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

Utah, Snowbird, USA

Acceptance Rates

Overall Acceptance Rate 230 of 1,014 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
405
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Freeh VKappiah NLowenthal DBletsch T(2008)Just-in-time dynamic voltage scalingJournal of Parallel and Distributed Computing10.1016/j.jpdc.2008.04.00768:9(1175-1185)Online publication date: 1-Sep-2008
https://dl.acm.org/doi/10.1016/j.jpdc.2008.04.007
Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002
Kwok YMaciejewski ASiegel HAhmad IGhafoor A(2006)A semi-static approach to mapping dynamic iterative tasks onto heterogeneous computing systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2005.06.01566:1(77-98)Online publication date: 1-Jan-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2005.06.015
Freeh VLowenthal DPingali KYelick KGrimshaw A(2005)Using multiple energy gears in MPI programs on a power-scalable clusterProceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming10.1145/1065944.1065967(164-173)Online publication date: 15-Jun-2005
https://dl.acm.org/doi/10.1145/1065944.1065967
Nakazawa MLowenthal DZhou WKramer W(2005)The MHETA Execution Model for Heterogeneous ClustersProceedings of the 2005 ACM/IEEE conference on Supercomputing10.1109/SC.2005.73Online publication date: 12-Nov-2005
https://dl.acm.org/doi/10.1109/SC.2005.73
Kappiah NFreeh VLowenthal DKramer W(2005)Just In Time Dynamic Voltage ScalingProceedings of the 2005 ACM/IEEE conference on Supercomputing10.1109/SC.2005.39Online publication date: 12-Nov-2005
https://dl.acm.org/doi/10.1109/SC.2005.39
Weatherly DLowenthal DNakazawa MLowenthal FMcGraw J(2003)Dyn-MPIProceedings of the 2003 ACM/IEEE conference on Supercomputing10.1145/1048935.1050156Online publication date: 15-Nov-2003
https://dl.acm.org/doi/10.1145/1048935.1050156
Rencuzogullari UDwardadas S(2001)Dynamic adaptation to available resources for parallel computing in an autonomous network of workstationsACM SIGPLAN Notices10.1145/568014.37957236:7(72-81)Online publication date: 18-Jun-2001
https://dl.acm.org/doi/10.1145/568014.379572
Rencuzogullari UDwardadas SHeath MLumsdaine A(2001)Dynamic adaptation to available resources for parallel computing in an autonomous network of workstationsProceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming10.1145/379539.379572(72-81)Online publication date: 18-Jun-2001
https://dl.acm.org/doi/10.1145/379539.379572
Weatherly DLowenthal DNakazawa MLowenthal F(2006)Dyn-MPIJournal of Parallel and Distributed Computing10.1016/j.jpdc.2006.02.00266:6(822-838)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1016/j.jpdc.2006.02.002

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents