The Minos Computing Library: efficient parallel programming for extremely heterogeneous systems

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/ Software available from tensorflow.org.

C. Augonnet, S. Thibault, R. Namyst, and P. A. Wacrenier. 2011. StarPU: a unified platform for task scheduling on heterogeneous multicore architectures. Concurrency and Computation: Practice and Experience 23, 2 (2011), 187--198.

D. F. Bacon, R. Rabbah, and S. Shukla. 2013. FPGA programming for the masses. Commun. ACM 56, 4 (2013), 56--63.

M. Bauer, S. Treichler, E. Slaughter, and A. Aiken. 2012. Legion: Expressing locality and independence with logical regions. International Conference for High Performance Computing, Networking, Storage and Analysis, SC (2012).

M. E. Belviranli, L. N. Bhuyan, and R. Gupta. 2013. A dynamic self-scheduling scheme for heterogeneous multiprocessor architectures. ACM Transactions on Architecture and Code Optimization 9, 4 (2013), 1--20.

R. Blumofe, C. Joerg, B. Kuszmaul, C. Leiserson, K. Randall, and Y. Zhou. 1995. Cilk: An efficient multithreaded runtime system. In ACM SIGPLAN symposium on Principles and Practice of Parallel Programming (PPOPP). ACM New York, NY, USA, 207--216.

J. Bueno, J. Planas, A. Duran, R. M. Badia, X. Martorell, E. Ayguade, and J. Labarta. 2012. Productive Programming of GPU Clusters with OmpSs. In Parallel & Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International. 557--568.

L. Dagum and R. Menon. 1998. OpenMP: : An Industry-Standard API for Shared-Memory Programming. IEEE Computational Science AND Engineering 5, 1 (1998), 46--55.

B. Dally. 2010. GPU Computing to Exascale and Beyond.

M. Frigo, C. E. Leiserson, and K. H. Randall. 1998. The implementation of the Cilk-5 multithreaded language. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM Press New York, NY, USA, 212--223.

Intel. [n. d.]. Threading Building Blocks. [Online]. Available: https://software.intel.com/en-us/intel-tbb. (Accessed Feb. 1, 2019).

Pekka Jääskeläinen, Carlos Sánchez de La Lama, Erik Schnetter, Kalle Raiskila, Jarmo Takala, and Heikki Berg. 2015. pocl: A Performance-Portable OpenCL Implementation. International Journal of Parallel Programming 43, 5 (01 Oct 2015), 752--785.

H. Kaiser, T. Heller, B. Adelstein-Lelbach, A. Serio, and D. Fey. 2014. HPX: A task based programming model in a global address space. In Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models. ACM, 6.

L. V. Kale and S. Krishnan. 1993. CHARM++: a portable concurrent object oriented system based on C++. Vol. 28. ACM.

J. Kim, H. Kim, J. H. Lee, and J. Lee. 2011. Achieving a single compute device image in OpenCL for multiple GPUs. In 16th ACM symposium on Principles and practice of parallel programming. ACM, San Antonio, TX, USA, 277--288.

M. Kotsifakou, P. Srivastava, M.D. Sinclair, R. Komuravelli, V. Adve, and S. Adve. 2018. HPVM: heterogeneous parallel virtual machine. In Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, Vienna, Austria, 68--80.

S. Mittal and J. S. Vetter. 2015. A Survey of CPU-GPU Heterogeneous Computing Techniques. Comput. Surveys 47, 4 (2015), 1--35.

J. Nickolls and I. Buck. 2007. NVIDIA CUDA software and GPU parallel computing architecture. In Microprocessor Forum.

OpenACC. 2015. OpenACC: Directives for Accelerators.

J. Planas, R.M. Badia, E. Ayguadé, and J. Labarta. 2013. Self-Adaptive OmpSs Tasks in Heterogeneous Environments. In IEEE 27th International Symposium on Parallel and Distributed Processing. 138--149.

N. Ravi, Y. Yang, T. Bao, and S. Chakradhar. 2013. Semi-automatic restructuring of offloadable tasks for many-core accelerators. (2013), 1--12.

C.J. Rossbach, J. Currey, M. Silberstein, B. Ray, and E. Witchel. 2011. PTask: operating system abstractions to manage GPUs as compute devices. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles. ACM, Cascais, Portugal, 233--248.

C. J. Rossbach, Y. Yu, J. Currey, J.-P. Martin, and D. Fetterly. 2013. Dandelion. (2013), 49--68.

T. R. W. Scogland, B. Rountree, W. C. Feng, and B. R. De Supinski. 2012. Heterogeneous task scheduling for accelerated OpenMP. Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012 (2012), 144--155.

K. Spafford, J. Meredith, and J. S. Vetter. 2010. Maestro: Data Orchestration and Tuning for OpenCL Devices. In Euro-Par 2010 - Parallel Processing, Pasqua D'Ambra, Mario Guarracino, and Domenico Talia (Eds.), Vol. 6272. Springer Berlin Heidelberg, 275--286.

J. E. Stone, D. Gohara, and G. Shi. 2010. OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems. Computing in Science and Engineering 12, 3 (2010), 66--73.

J.S. Vetter, R. Brightwell, M. Gokhale, P. McCormick, R. Ross, J. Shalf, K. Antypas, D. Donofrio, T. Humble, C. Schuman, B. Van Essen, S. Yoo, A. Aiken, D. Bernholdt, S. Byna, K. Cameron, F. Cappello, B. Chapman, A. Chien, M. Hall, R. Hartman-Baker, Z. Lan, M. Lang, J. Leidel, S. Li, R. Lucas, J. Mellor-Crummey, P. Peltz Jr., T. Peterka, M. Strout, and J. Wilke. 2018. Extreme Heterogeneity 2018 - Productive Computational Science in the Era of Extreme Heterogeneity: Report for DOE ASCR Workshop on Extreme Heterogeneity. Technical Report. USDOE Office of Science (SC) (United States).

Wikipedia. [n. d.]. NVLink. [Online]. Available: https://en.wikipedia.org/wiki/NVLink. (Accessed Feb. 1,2019).