research-article

Performance potential of optimization phase selection during dynamic JIT compilation

Authors:

Michael R. Jantz,

Prasad A. KulkarniAuthors Info & Claims

ACM SIGPLAN Notices, Volume 48, Issue 7

Pages 131 - 142

https://doi.org/10.1145/2517326.2451539

Published: 16 March 2013 Publication History

Abstract

Phase selection is the process of customizing the applied set of compiler optimization phases for individual functions or programs to improve performance of generated code. Researchers have recently developed novel feature-vector based heuristic techniques to perform phase selection during online JIT compilation. While these heuristics improve program startup speed, steady-state performance was not seen to benefit over the default fixed single sequence baseline. Unfortunately, it is still not conclusively known whether this lack of steady-state performance gain is due to a failure of existing online phase selection heuristics, or because there is, indeed, little or no speedup to be gained by phase selection in online JIT environments. The goal of this work is to resolve this question, while examining the phase selection related behavior of optimizations, and assessing and improving the effectiveness of existing heuristic solutions.

We conduct experiments to find and understand the potency of the factors that can cause the phase selection problem in JIT compilers. Next, using long-running genetic algorithms we determine that program-wide and method-specific phase selection in the HotSpot JIT compiler can produce ideal steady-state performance gains of up to 15% (4.3% average) and 44% (6.2% average) respectively. We also find that existing state-of-the-art heuristic solutions are unable to realize these performance gains (in our experimental setup), discuss possible causes, and show that exploiting knowledge of optimization phase behavior can help improve such heuristic solutions. Our work develops a robust open-source production-quality framework using the HotSpot JVM to further explore this problem in the future.

References

[1]

F. Agakov, E. Bonilla, J. Cavazos, B. Franke, G. Fursin, M. F. P. O'Boyle, J. Thomson, M. Toussaint, and C. K. I. Williams. Using machine learning to focus iterative optimization. In CGO '06: Proceedings of the Symposium on Code Generation and Optimization, pages 295--305, 2006.

Digital Library

[2]

L. Almagor, K. D. Cooper, A. Grosul, T. J. Harvey, S. W. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Finding effective compilation sequences. In Proceedings of the 2004 Conference on Languages, Compilers, and Tools for Embedded Systems, pages 231--239, 2004.

Digital Library

[3]

. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Inc., New York, NY, USA, 1995.

Digital Library

[4]

S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, B. Moss, A. Phansalkar, D. Stefanović, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, OOPSLA '06, pages 169--190, 2006.

Digital Library

[5]

. Cavazos and M. F. P. O'Boyle. Method-specific dynamic compilation using logistic regression. In Proceedings of the conference on Object-oriented programming systems, languages, and applications, pages 229--240, 2006.

Digital Library

[6]

. Chow and Y. Wu. Feedback-directed selection and characterization of compiler optimizatons. Proc. 2nd Workshop on Feedback Directed Optimization, 1999.

[7]

K. D. Cooper, P. J. Schielke, and D. Subramanian. Optimizing for reduced code space using genetic algorithms. In Proceedings of the ACM SIGPLAN 1999 workshop on Languages, compilers, and tools for embedded systems, pages 1--9, 1999.

Digital Library

[8]

G. Fursin, Y. Kashnikov, A. Memon, Z. Chamski, O. Temam, M. Namolaru, E. Yom-Tov, B. Mendelson, A. Zaks, E. Courtois, F. Bodin, P. Barnard, E. Ashton, E. Bonilla, J. Thomson, C. Williams, and M. O'Boyle. Milepost gcc: Machine learning enabled self-tuning compiler. International Journal of Parallel Programming, 39: 296--327, 2011.

[9]

A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous java performance evaluation. In Proceedings of the conference on Object-oriented programming systems and applications, OOPSLA '07, pages 57--76, 2007.

Digital Library

[10]

D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1st edition, 1989.

Digital Library

[11]

M. Haneda, P. M. W. Knijnenburg, and H. A. G. Wijshoff. Optimizing general purpose compiler optimization. In Proceedings of the 2nd conference on Computing frontiers, CF '05, pages 180--188, New York, NY, USA, 2005. ACM. ISBN 1-59593-019-1.

Digital Library

[12]

K. Hoste and L. Eeckhout. Cole: compiler optimization level exploration. In Proceedings of the 6th annual IEEE/ACM international symposium on Code generation and optimization, CGO '08, pages 165--174, New York, NY, USA, 2008.

Digital Library

[13]

http://www.oracle.com/technetwork/java/javase/memorymanagement-whitepaper150215.pdf. Memory Management in the Java HotSpot Virtual Machine, April 2006.

[14]

K. Ishizaki, M. Kawahito, T. Yasue, M. Takeuchi, T. Ogasawara, T. Suganuma, T. Onodera, H. Komatsu, and T. Nakatani. Design, implementation, and evaluation of optimizations in a just-in-time compiler. In Proceedings of the ACM 1999 conference on Java Grande, JAVA '99, pages 119--128, 1999.

Digital Library

[15]

K. Ishizaki, M. Takeuchi, K. Kawachiya, T. Suganuma, O. Gohda, T. Inagaki, A. Koseki, K. Ogata, M. Kawahito, T. Yasue, T. Ogasawara, T. Onodera, H. Komatsu, and T. Nakatani. Effectiveness of cross-platform optimizations for a java just-in-time compiler. In Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, pages 187--204, 2003.

Digital Library

[16]

P. A. Kulkarni. JIT compilation policy for modern machines. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications, pages 773--788, 2011.

Digital Library

[17]

P. A. Kulkarni, D. B. Whalley, and G. S. Tyson. Evaluating heuristic optimization phase order search algorithms. In CGO '07: Proceedings of the International Symposium on Code Generation and Optimization, pages 157--169, 2007.

Digital Library

[18]

P. A. Kulkarni, M. R. Jantz, and D. B. Whalley. Improving both the performance benefits and speed of optimization phase sequence searches. In Proceedings of the ACM SIGPLAN/SIGBED 2010 conference on Languages, compilers, and tools for embedded systems, LCTES '10, pages 95--104, 2010. ISBN 978-1-60558-953-4.

Digital Library

[19]

S. Kulkarni and J. Cavazos. Mitigating the compiler optimization phase-ordering problem using machine learning. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications, OOPSLA '12, pages 147--162. ACM, 2012.

Digital Library

[20]

H. Lee, D. von Dincklage, A. Diwan, and J. E. B. Moss. Understanding the behavior of compiler optimizations. Software Practice & Experience, 36 (8): 835--844, July 2006.

Digital Library

[21]

M. Paleczny, C. Vick, and C. Click. The Java hotspottm server compiler. In Proceedings of the Symposium on Java™ Virtual Machine Research and Technology Symposium, pages 1--12, Berkeley, CA, USA, 2001. USENIX.

Digital Library

[22]

Z. Pan and R. Eigenmann. PEAK: a fast and effective performance tuning system via compiler optimization orchestration. ACM Trans. Program. Lang. Syst., 30: 17:1--17:43, May 2008.

Digital Library

[23]

R. Sanchez, J. Amaral, D. Szafron, M. Pirvu, and M. Stoodley. Using machines to learn method-specific compilation strategies. In Code Generation and Optimization, pages 257--266, April 2011.

Digital Library

[24]

PEC98. Specjvm98 benchmarks. http://www.spec.org/jvm98/, 1998.

[25]

S. Triantafyllis, M. Vachharajani, N. Vachharajani, and D. I. August. Compiler optimization-space exploration. In Proceedings of the International Symposium on Code Generation and Optimization, pages 204--215. IEEE, 2003.

Digital Library

Cited By

Wu JMeng XXu JZhang H(2020)Reliable Compilation Optimization Phase-ordering Exploration with Reinforcement Learning2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC42975.2020.9283132(4058-4065)Online publication date: 11-Oct-2020
https://dl.acm.org/doi/10.1109/SMC42975.2020.9283132
Nobre RMartins LCardoso J(2016)A graph-based iterative compiler pass selection and phase ordering approachACM SIGPLAN Notices10.1145/2980930.290795951:5(21-30)Online publication date: 13-Jun-2016
https://dl.acm.org/doi/10.1145/2980930.2907959
Nobre RMartins LCardoso JKuo TWhalley D(2016)A graph-based iterative compiler pass selection and phase ordering approachProceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded Systems10.1145/2907950.2907959(21-30)Online publication date: 13-Jun-2016
https://dl.acm.org/doi/10.1145/2907950.2907959
Show More Cited By

Index Terms

Performance potential of optimization phase selection during dynamic JIT compilation
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Logical and relational learning
        Inductive logic learning
2. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

Performance potential of optimization phase selection during dynamic JIT compilation
VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Phase selection is the process of customizing the applied set of compiler optimization phases for individual functions or programs to improve performance of generated code. Researchers have recently developed novel feature-vector based heuristic ...
Making collection operations optimal with aggressive JIT compilation
SCALA 2017: Proceedings of the 8th ACM SIGPLAN International Symposium on Scala

Functional collection combinators are a neat and widely accepted data processing abstraction. However, their generic nature results in high abstraction overheads -- Scala collections are known to be notoriously slow for typical tasks. We show that ...
Mitigating JIT compilation latency in virtual execution environments
VEE 2019: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Many Virtual Execution Environments (VEEs) rely on Just-in-time (JIT) compilation technology for code generation at runtime, e.g. in Dynamic Binary Translation (DBT) systems or language Virtual Machines (VMs). While JIT compilation improves native ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 48, Issue 7

VEE '13

July 2013

194 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/2517326

Issue’s Table of Contents

VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
March 2013
210 pages
ISBN:9781450312660
DOI:10.1145/2451512
General Chair:
Steve Muir
VMware, USA
,
Program Chairs:
Gernot Heiser
NICTA and University of New South Wales, Australia
,
Steve Blackburn
Australian National University, Australia

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2013

Published in SIGPLAN Volume 48, Issue 7

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
329
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu JMeng XXu JZhang H(2020)Reliable Compilation Optimization Phase-ordering Exploration with Reinforcement Learning2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC42975.2020.9283132(4058-4065)Online publication date: 11-Oct-2020
Nobre RMartins LCardoso J(2016)A graph-based iterative compiler pass selection and phase ordering approachACM SIGPLAN Notices10.1145/2980930.290795951:5(21-30)Online publication date: 13-Jun-2016
Nobre RMartins LCardoso JKuo TWhalley D(2016)A graph-based iterative compiler pass selection and phase ordering approachProceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded Systems10.1145/2907950.2907959(21-30)Online publication date: 13-Jun-2016
Kuo TWhalley D(2016)Proceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded SystemsundefinedOnline publication date: 13-Jun-2016
Burgstaller TGarber DLe VFelfernig A(2024)Optimization Space Learning: A Lightweight, Noniterative Technique for Compiler AutotuningProceedings of the 28th ACM International Systems and Software Product Line Conference10.1145/3646548.3672588(36-46)Online publication date: 2-Sep-2024
Ferres BMuller ORousseau F(2023)A Chisel Framework for Flexible Design Space Exploration through a Functional ApproachACM Transactions on Design Automation of Electronic Systems10.1145/359076928:4(1-31)Online publication date: 5-Apr-2023
Zhu MHao D(2023)Compiler Auto-Tuning via Critical Flag Selection2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00209(1000-1011)Online publication date: 11-Sep-2023
Wu JXu JMeng XZhang HZhang Z(2021)Enabling Reliability-Driven Optimization Selection with Gate Graph Attention Neural NetworkInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402040024030:11n12(1641-1665)Online publication date: 21-Jan-2021
Wu JXu JMeng XZhang HZhang ZLi L(2020)Compilation Optimization Pass Selection Using Gate Graph Attention Neural Network for Reliability ImprovementIEEE Access10.1109/ACCESS.2020.30167588(150422-150434)Online publication date: 2020
Prokopec ADuboscq GLeopoldseder DWürthinger TKandemir MJimborean AMoseley T(2019)An optimization-driven incremental inline substitution algorithm for just-in-time compilersProceedings of the 2019 IEEE/ACM International Symposium on Code Generation and Optimization10.5555/3314872.3314893(164-179)Online publication date: 16-Feb-2019
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents