poster

A novel shared memory framework for distributed deep learning in high-performance computing architecture

Authors:

Shinyoimg Ahn,

Joongheon Kim,

Sungwon KangAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Pages 191 - 192

https://doi.org/10.1145/3183440.3195091

Published: 27 May 2018 Publication History

Get Access

Abstract

This paper proposes a novel virtual shared memory framework, Soft Memory Box (SMB), which directly shares the memory of remote nodes among distributed processes to improve communication performance/speed via deep learning parameter sharing.

References

[1]

He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR'16.

Google Scholar

[2]

Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., and Kingsbury, B. Deep neural networks for acoustic modeling in speech recognition. In IEEE Signal Process. Mag., 2012.

Google Scholar

[3]

Krizhevsky, A., Sutskever, I., and Hinton, G. Imagenet classification with deep convolutional neural networks. In NIPS'12.

Digital Library

Google Scholar

[4]

Shi, S., and Chu, X. Performance modeling and evaluation of distributed deep learning frameworks on GPUs. In arXiv:1711.05979v2 {cs.DC}.

Google Scholar

[5]

Simonyan, K., and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In ICLR'15.

Google Scholar

[6]

Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI'17.

Google Scholar

[7]

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In CVPR'15.

Google Scholar

[8]

Yangqing, J., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. Caffe: Convolutional architecture for fast feature embedding. In MM'14.

Google Scholar

Cited By

View all

Raju MGulhane KA. GChandra SDwaramwar PBoopathi S(2024)Architectures of High-Performance Computing Systems for Machine Learning WorkloadsIntegrating Machine Learning Into HPC-Based Simulations and Analytics10.4018/978-1-6684-3795-7.ch017(435-460)Online publication date: 13-Dec-2024
https://doi.org/10.4018/978-1-6684-3795-7.ch017
Shan ZYang HYang XChen X(2022)Object Detection Based On Multi-Process2022 IEEE International Conference on Networking, Sensing and Control (ICNSC)10.1109/ICNSC55942.2022.10004179(1-5)Online publication date: 15-Dec-2022
https://doi.org/10.1109/ICNSC55942.2022.10004179
Sundari KNarmadha R(2022)Design energy efficient shared distributed memory management system on SoC’s to improve memory performanceApplied Nanoscience10.1007/s13204-021-02114-w13:2(1691-1701)Online publication date: 18-Jan-2022
https://doi.org/10.1007/s13204-021-02114-w
Show More Cited By

Index Terms

A novel shared memory framework for distributed deep learning in high-performance computing architecture
1. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

Distributed shared memory: experience with Munin
EW 5: Proceedings of the 5th workshop on ACM SIGOPS European workshop: Models and paradigms for distributed systems structuring

Distributed shared memory (DSM) is the provision in software of a shared memory programming model on a distributed memory machine [6]. We are exploring the use of DSM in a cluster-based computing environment of workstations and servers connected by a ...
Distributed shared memory for high performance networks
ICWET '10: Proceedings of the International Conference and Workshop on Emerging Trends in Technology

This paper describes the design and implementation of a distributed shared memory facility for high performance networks. Distributed Shared Memory (DSM) also known as a distributed global address space (DGAS) is a term in computer science that refers ...
Distributed shared persistent memory
SoCC '17: Proceedings of the 2017 Symposium on Cloud Computing

Next-generation non-volatile memories (NVMs) will provide byte addressability, persistence, high density, and DRAM-like performance. They have the potential to benefit many datacenter applications. However, most previous research on NVMs has focused on ...

Comments

Information & Contributors

Information

Published In

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

May 2018

231 pages

ISBN:9781450356633

DOI:10.1145/3183440

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
214
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Raju MGulhane KA. GChandra SDwaramwar PBoopathi S(2024)Architectures of High-Performance Computing Systems for Machine Learning WorkloadsIntegrating Machine Learning Into HPC-Based Simulations and Analytics10.4018/978-1-6684-3795-7.ch017(435-460)Online publication date: 13-Dec-2024
https://doi.org/10.4018/978-1-6684-3795-7.ch017
Shan ZYang HYang XChen X(2022)Object Detection Based On Multi-Process2022 IEEE International Conference on Networking, Sensing and Control (ICNSC)10.1109/ICNSC55942.2022.10004179(1-5)Online publication date: 15-Dec-2022
https://doi.org/10.1109/ICNSC55942.2022.10004179
Sundari KNarmadha R(2022)Design energy efficient shared distributed memory management system on SoC’s to improve memory performanceApplied Nanoscience10.1007/s13204-021-02114-w13:2(1691-1701)Online publication date: 18-Jan-2022
https://doi.org/10.1007/s13204-021-02114-w
Shafiq SMashkoor AMayr-Dorn CEgyed A(2021)A Literature Review of Using Machine Learning in Software Development Life Cycle StagesIEEE Access10.1109/ACCESS.2021.31197469(140896-140920)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3119746
Ahn SLim E(2020)SoftMemoryBox II: A Scalable, Shared Memory Buffer Framework for Accelerating Distributed Training of Large-Scale Deep Neural NetworksIEEE Access10.1109/ACCESS.2020.30381128(207097-207111)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3038112
Lim EAhn SPark YChoi W(2018)Distributed Deep Learning Framework based on Shared Memory for Fast Deep Neural Network Training2018 International Conference on Information and Communication Technology Convergence (ICTC)10.1109/ICTC.2018.8539420(1239-1242)Online publication date: Oct-2018
https://doi.org/10.1109/ICTC.2018.8539420

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Distributed shared memory: experience with Munin

Distributed shared memory for high performance networks

Distributed shared persistent memory