skip to main content
10.1145/3458817.3487400acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article
Public Access

Billion atom molecular dynamics simulations of carbon at extreme conditions and experimental time and length scales

Published: 13 November 2021 Publication History

Editorial Notes

The editors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected VoR was published on December 14, 2021. For reference purposes the VoR may still be accessed via the Supplemental Material section on this page.

Abstract

Billion atom molecular dynamics (MD) using quantum-accurate machine-learning Spectral Neighbor Analysis Potential (SNAP) observed long-sought high pressure BC8 phase of carbon at extreme pressure (12 Mbar) and temperature (5,000 K). 24-hour, 4650 node production simulation on OLCF Summit demonstrated an unprecedented scaling and unmatched real-world performance of SNAP MD while sampling 1 nanosecond of physical time. Efficient implementation of SNAP force kernel in LAMMPS using the Kokkos CUDA backend on NVIDIA GPUs combined with excellent strong scaling (better than 97% parallel efficiency) enabled a peak computing rate of 50.0 PFLOPs (24.9% of theoretical peak) for a 20 billion atom MD simulation on the full Summit machine (27,900 GPUs). The peak MD performance of 6.21 Matom-steps/node-s is 22.9 times greater than a previous record for quantum-accurate MD. Near perfect weak scaling of SNAP MD highlights its excellent potential to advance the frontier of quantum-accurate MD to trillion atom simulations on upcoming exascale platforms.

Supplementary Material

3487400-vor (3487400-vor.pdf)
Version of Record for "Billion atom molecular dynamics simulations of carbon at extreme conditions and experimental time and length scales" by Nguyen-Cong et al., Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '21).
MP4 File (Billion atom molecular dynamics simulations of carbon at extreme conditions and experimental time and length scales.mp4)
Presentation video

References

[1]
Didier Queloz. Nobel Lecture: 51 Pegasi b and the exoplanet revolution. Rev. Mod. Phys., 92:30503, 2020.
[2]
Michel Mayor. Nobel Lecture: Plurality of worlds in the cosmos: A dream of antiquity, a modern reality of astrophysics. Rev. Mod. Phys., 92:30502, 2020.
[3]
J. R. Rygg et al. X-ray diffraction at the National Ignition Facility. Rev. Sci. Instrum., 91, 2020.
[4]
D. B. Sinars et al. Review of pulsed power-driven high energy density physics research on Z at Sandia. Phys. Plasmas, 27:070501, 2020.
[5]
Malcolm I. McMahon. Synchrotron and FEL Studies of Matter at High Pressures, pages 1857--1896. Springer International Publishing, Cham, 2020.
[6]
Martin French and Thomas R. Mattsson. Thermodynamically constrained correction to ab initio equations of state. J. Appl. Phys., 116:013510, 2014.
[7]
Lorin X. Benedict, Kevin P. Driver, Sebastien Hamel, Burkhard Militzer, Tingting Qi, Alfredo A. Correa, A. Saul, and Eric Schwegler. Multiphase equation of state for carbon addressing high pressures and temperatures. Phys. Rev. B, 89:224109, 2014.
[8]
Thomas S. Duffy and Raymond F. Smith. Ultra-high pressure dynamic compression of geological materials. Front. Earth Sci., 7:1--20, 2019.
[9]
Steven J. Plimpton and Aidan P. Thompson. Computational aspects of many-body potentials. MRS Bull., 37:513--521, 2012.
[10]
Volker L. Deringer, Miguel A. Caro, and Gábor Csányi. Machine Learning Interatomic Potentials as Emerging Tools for Materials Science. Advanced Materials, 31:1902765, 2019.
[11]
Jörg Behler. Perspective: Machine learning potentials for atomistic simulations. J. Chem. Phys., 145, 2016.
[12]
Yunxing Zuo, Chi Chen, Xiangguo Li, Zhi Deng, Yiming Chen, Jörg Behler, Gábor Csányi, Alexander V. Shapeev, Aidan P. Thompson, Mitchell A. Wood, and Shyue Ping Ong. Performance and cost assessment of machine learning interatomic potentials. J. Phys. Chem. A, 124:731--745, 2020.
[13]
Aidan P Thompson, Laura P Swiler, Christian R Trott, Stephen M Foiles, and Garritt J Tucker. Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials. Journal of Computational Physics, 285:316--330, 2015.
[14]
Miguel A. Caro, Volker L. Deringer, Jari Koskinen, Tomi Laurila, and Gábor Csányi. Growth Mechanism and Origin of High sp3 Content in Tetrahedral Amorphous Carbon. Phys. Rev. Lett., 120:166101, 2018.
[15]
Volker L. Deringer, Noam Bernstein, Gábor Csányi, Chiheb Ben Mahmoud, Michele Ceriotti, Mark Wilson, David A. Drabold, and Stephen R. Elliott. Origins of structural and electronic transitions in disordered silicon. Nature, 589:59--64, 2021.
[16]
M. A. Wood, M. A. Cusentino, B. D. Wirth, and A. P. Thompson. Data-driven material models for atomistic simulation. Phys. Rev. B, 99:1--12, 2019.
[17]
Conrad W. Rosenbrock, Konstantin Gubaev, Alexander V. Shapeev, Livia B. Pártay, Noam Bernstein, Gábor Csányi, and Gus L. W. Hart. Machine-learned interatomic potentials for alloys and alloy phase diagrams. npj Computational Materials, 7:24, 2021.
[18]
Jonathan T Willman, Ashley S Williams, Kien Nguyen-Cong, Aidan P Thompson, Mitchell A Wood, Anatoly B Belonoshko, and Ivan I Oleynik. Quantum accurate SNAP carbon potential for MD shock simulations. AIP Conf. Proc., 2272:070055, 2020.
[19]
S. Plimpton. Fast parallel algorithms for short-range molecular dynamics. J. Comp. Phys., 117:1--19, 1995. https://lammps.org/.
[20]
C. Hakim, K.; van Westrenen, W.; Dominik, Kaustubh Hakim, Rob Spaargaren, Damanveer S. Grewal, Arno Rohrbach, Jasper Berndt, Carsten Dominik, and Wim Van Westrenen. Mineralogy, Structure, and Habitability of Carbon-Enriched Rocky Exoplanets: A Laboratory Approach. Astrobiology, 19:867--884, 2019.
[21]
J. H. Eggert, D. G. Hicks, P. M. Celliers, D. K. Bradley, R. S. McWilliams, R. Jeanloz, J. E. Miller, T. R. Boehly, and G. W. Collins. Melting temperature of diamond at ultrahigh pressure. Nat. Phys., 6:40--43, 2010.
[22]
R. F. Smith, J. H. Eggert, R. Jeanloz, T. S. Duffy, D. G. Braun, J. R. Patterson, R. E. Rudd, J. Biener, A. E. Lazicki, A. V. Hamza, J. Wang, T. Braun, L. X. Benedict, P. M. Celliers, and G. W. Collins. Ramp compression of diamond to five terapascals. Nature, 511:330--333, 2014.
[23]
A. Lazicki, D. McGonegle, J. R. Rygg, D. G. Braun, D. C. Swift, M. G. Gorman, R. F. Smith, P. G. Heighway, A. Higginbotham, M. J. Suggit, D. E. Fratanduono, F. Coppari, C. E. Wehrenberg, R. G. Kraus, D. Erskine, J. V. Bernier, J. M. McNaney, R. E. Rudd, G. W. Collins, J. H. Eggert, and J. S. Wark. Metastability of diamond ramp-compressed to 2 terapascals. Nature, 589:532--535, 2021.
[24]
Top500. https://www.top500.org/lists/2021/06/. [Online; accessed 23-July-2021].
[25]
Jörg Behler. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys., 134:074106, 2011.
[26]
Albert P. Bartók, Mike C. Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett., 104:136403, 2010.
[27]
Justin S. Smith, Ben Nebgen, Nicholas Lubbers, Olexandr Isayev, and Adrian E. Roitberg. Less is more: Sampling chemical space with active learning. The Journal of Chemical Physics, 148:241733, 2018.
[28]
Justin S Smith, Benjamin Nebgen, Nithin Mathew, Jie Chen, Nicholas Lubbers, Leonid Burakovsky, Sergei Tretiak, Hai Ah Nam, Timothy Germann, Saryu Fensin, et al. Automated discovery of a robust interatomic potential for aluminum. Nature communications, 12:1--13, 2021.
[29]
Nicholas Lubbers, Justin S. Smith, and Kipton Barros. Hierarchical modeling of molecular energies using a deep neural network. The Journal of Chemical Physics, 148:241715, 2018.
[30]
Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and Weinan E. Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett., 120:143001, 2018.
[31]
Rebecca K. Lindsey, Laurence E. Fried, and Nir Goldman. Chimes: A force matched potential with explicit three-body interactions for molten carbon. Journal of Chemical Theory and Computation, 13:6222--6229, 2017.
[32]
Alexander V. Shapeev. Moment tensory potentials: a class of systematically improvable interatomic potentials. Multiscale Model. Simul., 14:1153, 2016.
[33]
Jonathan Vandermause, Steven B Torrisi, Simon Batzner, Yu Xie, Lixin Sun, Alexie M Kolpak, and Boris Kozinsky. On-the-fly active learning of interpretable bayesian force fields for atomistic rare events. npj Computational Materials, 6:1--11, 2020.
[34]
Ralf Drautz. Atomic cluster expansion of scalar, vectorial, and tensorial properties including magnetism and charge transfer. Phys. Rev. B, 102:024104, 2020.
[35]
Yury Lysogorskiy, Cas van der Oord, Anton Bochkarev, Sarath Menon, Matteo Rinaldi, Thomas Hammerschmidt, Matous Mrovec, Aidan Thompson, Gábor Csányi, Christoph Ortner, and Ralf Drautz. Performant implementation of the atomic cluster expansion (pace) and application to copper and silicon. npj Computational Materials, 7:97, 2021.
[36]
B. M. Adams, K. R. Dalbey, M. S. Eldred, D. M. Gay, L. P. Swiler, W. J. Bohnhoff, J. P. Eddy, and K. Haskell. Dakota users manual version 5.1. Sandia Technical Report SAND2010-2183, Sandia National Laboratories, Albuquerque, NM, 2011.
[37]
Christian R. Trott, Simon D. Hammond, and Aidan P. Thompson. SNAP: Strong scaling high fidelity molecular dynamics simulations on leadership-class computing platforms. In Julian Martin Kunkel, Thomas Ludwig, and Hans Werner Meuer, editors, Supercomputing, pages 19--34, Cham, 2014. Springer International Publishing.
[38]
Weile Jia, Han Wang, Mohan Chen, Denghui Lu, Lin, Roberto Car, E Weinan, and Linfeng Zhang. Pushing the Limit of Molecular Dynamics with Ab Initio Accuracy to 100 Million Atoms with Machine Learning. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--14. IEEE, 2020.
[39]
Denghui Lu, Han Wang, Mohan Chen, Lin Lin, Roberto Car, Weinan E, Weile Jia, and Linfeng Zhang. 86 pflops deep potential molecular dynamics simulation of 100 million atoms with ab initio accuracy. Computer Physics Communications, 259:107624, 2021.
[40]
LAMMPS. Github commit. https://github.com/lammps/lammps/commit/39a09d3, 2018.
[41]
H. Carter Edwards, Christian R. Trott, and Daniel Sunderland. Kokkos: Enabling manycore performance portability through polymorphic memory access patterns. Journal of Parallel and Distributed Computing, 74:3202 -- 3216, 2014. Domain-Specific Languages and High-Level Frameworks for High-Performance Computing.
[42]
LAMMPS. Github commit. https://github.com/lammps/lammps/commit/c9742056d, 2021.
[43]
LAMMPS. Github commit. https://github.com/lammps/lammps/commit/e363b4a, 2021.
[44]
Stuart Slattery, Christoph Junghans, Damien Lebrun-Grandie, Shane Fogerty, Robert Bird, Sam Reeve, Guangye Chen, Rene Halver, Aaron Scheinberg, Cameron Smith, and Evan Weinberg. Ecp-copa/cabana: Version 0.3.0. 2020.
[45]
LAMMPS. Github commit. https://github.com/lammps/lammps/commit/fe9f7f4, 2019.
[46]
Mitchell A Wood and Aidan P Thompson. Extending the accuracy of the SNAP interatomic potential form. The Journal of Chemical Physics, 148:241721, 2018.
[47]
Andreas Götz, Mark Williamson, Dong Xu, Duncan Poole, Scott Grand, and Ross Walker. Routine microsecond molecular dynamics simulations with amber on gpus. 1. generalized born. Journal of chemical theory and computation, 8:1542--1555, 2012.
[48]
M. Höhnerbach, A. E. Ismail, and P. Bientinesi. The vectorization of the tersoff multi-body potential: An exercise in performance portability. In SC '16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 69--81, 2016.
[49]
B. Homerding. Preparing applications for aurora. Argonne technical report, Argonne National Laboratory, Chicago, IL, 2021. https://ecpannualmeeting.com/assets/overview/sessions/Preparing_Applications_for_Aurora.pdf.
[50]
P. C. Roth. Developing software for olcf frontier. Oak ridge technical report, Oak Ridge National Laboratory, Oak Ridge, TN, 2021. https://ecpannualmeeting.com/assets/overview/sessions/Roth-Path-to-Frontier-20200206.pdf.
[51]
M.A. Cusentino, M.A. Wood, and A.P. Thompson. Beryllium-driven structural evolution at the divertor surface. Nuclear Fusion, 61:046049, 2021.
[52]
Mary Alice Cusentino, Mitchell A Wood, and Aidan P Thompson. Explicit multielement extension of the spectral neighbor analysis potential for chemically complex systems. The Journal of Physical Chemistry A, 124:5456--5464, 2020.

Cited By

View all

Index Terms

  1. Billion atom molecular dynamics simulations of carbon at extreme conditions and experimental time and length scales
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          SC '21: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
          November 2021
          1493 pages
          ISBN:9781450384421
          DOI:10.1145/3458817
          © 2021 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

          Sponsors

          In-Cooperation

          • IEEE CS

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 13 November 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. carbon
          2. extreme conditions
          3. machine-learning interatomic potentials
          4. molecular dynamics

          Qualifiers

          • Research-article

          Funding Sources

          Conference

          SC '21
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

          Upcoming Conference

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)623
          • Downloads (Last 6 weeks)75
          Reflects downloads up to 06 Nov 2024

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)GPU-based molecular dynamics of fluid flowsInternational Journal of High Performance Computing Applications10.1177/1094342023121301338:1(34-49)Online publication date: 30-Jan-2024
          • (2024)Extreme Metastability of Diamond and its Transformation to the BC8 Post-Diamond Phase of CarbonThe Journal of Physical Chemistry Letters10.1021/acs.jpclett.3c03044(1152-1160)Online publication date: 25-Jan-2024
          • (2024)Scalable Parallel Algorithm for Graph Neural Network Interatomic Potentials in Molecular Dynamics SimulationsJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0019020:11(4857-4868)Online publication date: 30-May-2024
          • (2024)Diamond under extremesMaterials Science and Engineering: R: Reports10.1016/j.mser.2024.100857161(100857)Online publication date: Dec-2024
          • (2023)О влиянии морфологии поверхности сажевых частиц на кинетику их роста. Молекулярно-динамическое исследованиеТеплофизика высоких температур10.31857/S004036442301017961:1(68-74)Online publication date: 1-Jan-2023
          • (2023)FitSNAP: Atomistic machine learning with LAMMPSJournal of Open Source Software10.21105/joss.051188:84(5118)Online publication date: Apr-2023
          • (2023)Machine-learned interatomic potentials: Recent developments and prospective applicationsJournal of Materials Research10.1557/s43578-023-01239-838:24(5079-5094)Online publication date: 8-Dec-2023
          • (2023)Exploring model complexity in machine learned potentials for simulated propertiesJournal of Materials Research10.1557/s43578-023-01152-038:24(5136-5150)Online publication date: 18-Sep-2023
          • (2023)Frontier: Exploring ExascaleProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607089(1-16)Online publication date: 12-Nov-2023
          • (2023)Transonic dislocation propagation in diamondScience10.1126/science.adh5563382:6666(69-72)Online publication date: 6-Oct-2023
          • Show More Cited By

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Get Access

          Login options

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media