Article

Comparison of two activity analyses for automatic differentiation: context-sensitive flow-insensitive vs. context-insensitive flow-sensitive

Authors:

Paul D. HovlandAuthors Info & Claims

SAC '07: Proceedings of the 2007 ACM symposium on Applied computing

Pages 1323 - 1329

https://doi.org/10.1145/1244002.1244287

Published: 11 March 2007 Publication History

Abstract

Automatic differentiation (AD) is a family of techniques to generate derivative code from a mathematical model expressed in a programming language. AD computes partial derivatives for each operation in the input code and combines them to produce the desired derivative by applying the chain rule. Activity analysis is a compiler analysis used to find active variables in automatic differentiation. By lifting the burden of computing partial derivatives for passive variables, activity analysis can reduce the memory requirement and run time of the generated derivative code. This paper compares a new context-sensitive flow-insensitive (CSFI) activity analysis with an existing context-insensitive flow-sensitive (CIFS) activity analysis in terms of execution time and the quality of the analysis results. Our experiments with eight benchmarks show that the new CSFI activity analysis runs up to 583 times faster and overestimates up to 18.5 times fewer active variables than does the existing CIFS activity analysis.

References

[1]

http://mitgcm.org/.

[2]

Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Techniques, and Tools. Addison-Wesley, 1986.

Digital Library

[3]

Brett M. Averick, Richard G. Garter, and Jorge J. More. MINPACK-2 Test Problem Collection. Technical Memorandum ANL/MCS-TM-150, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, May 1991.

[4]

Martin Berz, Christian H. Bischof, George F. Corliss, and Andreas Griewank, editors. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, PA, 1996.

[5]

Christian Bischof, Alan Carle, Peyvand Khademi, and Andrew Mauer. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering, 3(3):18--32, 1996.

Digital Library

[6]

Mike Fagan and Alan Carle. Activity Analysis in ADIFOR: Algorithms and Effectiveness. Technical Report TR04-21, Department of Computational and Applied Mathematics, Rice University, Houston, TX, November 2004.

[7]

Message Passing Interface Forum. MPI: A message-passing interface standard. International Journal of Supercomputer Applications, 8(3/4):165--414, 1994.

[8]

Andreas Griewank and George F. Corliss, editors. Automatic Differentiation of Algorithms: Theory, Implementation, and Application. SIAM, Philadelphia, PA, 1991.

[9]

L. Hascoët, U. Naumann, and V. Pascual. "To be recorded" analysis in reverse-mode automatic differentiation. Future Generation Computer Systems, 21(8), 2004.

Digital Library

[10]

Uday P. Khedker and Dhananjay M. Dhamdhere. A generalized theory of bit vector data flow analysis. ACM Transactions on Programming Languages and Systems, 16(5):1472--1511, 1994.

Digital Library

[11]

Barbara Kreaseck, Luis Ramos, Scott Easterday, Michelle Strout, and Paul Hovland. Hybrid static/dynamic activity analysis. In Proceedings of the 3rd International Workshop on Automatic Differentiation Tools and Applications (ADTA '04), Reading, England, 2006.

[12]

Arun Lakhotia. Rule-based approach to computing module cohesion. In Proceedings of the 15th International Conference on Software Engineering, pages 35--44, Baltimore, MD, 1993.

Digital Library

[13]

Thomas Reps, Susan Horwitz, Mooly Sagiv, and Genevieve Rosay. Speeding up slicing. In Proceedings of the 2nd ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 11--20, 1994.

Digital Library

[14]

Thomas Reps and Genevieve Rosay. Precise interprocedural chopping. In Proceedings of the 3rd ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 41--52, 1995.

Digital Library

[15]

Rice University. Open64 project. http://www.hipersoft.rice.edu/open64/.

[16]

Marc Shapiro and Susan Horwitz. The effects of the precision of pointer analysis. In International Symposium on Static Analysis, pages 16--34, 1997. Lecture Notes in Computer Science, Vol. 1302, Pascal Van Hentenryck (ed.), Springer-Verlag, New York, NY.

Digital Library

[17]

Michelle Mills Strout, Barbara Kreaseck, and Paul D. Hovland. Data-flow analysis for MPI programs. In International Conference on Parallel Processing, 2006.

Digital Library

[18]

Michelle Mills Strout, John Mellor-Crummey, and Paul D. Hovland. Representation-independent program analysis. In Proceedings of The Sixth ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, 2005.

Digital Library

[19]

Jean Utke. OpenAD: Algorithm implementation user guide. Technical Memorandum ANL/MCS-TM-274, Mathematics and Computer Science Division, Argonne National Laboratory, 2004.

[20]

Rob F. van der Wijngaart. NAS parallel benchmarks version 2.4. Technical Report NAS-02-007, NASA Advanced Supercomputing (NAS) Division, October 2002.

[21]

Mark Weiser. Program slicing. In Proceedings of the 5th International Conference on Software Engineering, pages 439--449, 1981.

Digital Library

Cited By

Moses WChuravy VLarochelle HRanzato MHadsell RBalcan MLin H(2020)Instead of rewriting foreign code for machine learning, automatically synthesize fast gradientsProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496770(12472-12485)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496770
Hückelheim JHascoët LMüller J(2017)Algorithmic Differentiation of Code with Multiple Context-Specific ActivitiesACM Transactions on Mathematical Software10.1145/301546443:4(1-21)Online publication date: 9-Jan-2017
https://dl.acm.org/doi/10.1145/3015464
Shin JMalusare PHovland P(2008)Design and Implementation of a Context-Sensitive, Flow-Sensitive Activity Analysis Algorithm for Automatic DifferentiationAdvances in Automatic Differentiation10.1007/978-3-540-68942-3_11(115-125)Online publication date: 2008
https://doi.org/10.1007/978-3-540-68942-3_11

Index Terms

Comparison of two activity analyses for automatic differentiation: context-sensitive flow-insensitive vs. context-insensitive flow-sensitive
1. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Automatic differentiation
2. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

Algorithmic Differentiation of Code with Multiple Context-Specific Activities

Algorithmic differentiation (AD) by source-transformation is an established method for computing derivatives of computational algorithms. Static dataflow analysis is commonly used by AD tools to determine the set of active variables, that is, variables ...
Data-Flow Analysis for MPI Programs
ICPP '06: Proceedings of the 2006 International Conference on Parallel Processing

Message passing via MPI is widely used in singleprogram, multiple-data (SPMD) parallel programs. Existing data-flow frameworks do not model the semantics of message-passing SPMD programs, which can result in less precise and even incorrect analysis ...
Practical escape analyses: how good are they?
VEE '07: Proceedings of the 3rd international conference on Virtual execution environments

A key analysis developed for the compilation of parallel programs is thread escape analysis (hereafter referred to as escape analysis), which determines what objects are accessed in more than one thread, and which references within a program are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '07: Proceedings of the 2007 ACM symposium on Applied computing

March 2007

1688 pages

ISBN:1595934804

DOI:10.1145/1244002

Conference Chairs:
Yookun Cho
Seoul National University, Seoul, Korea
,
Roger L. Wainwright
University of Tulsa, Tulsa, Oklahoma
,
Hisham M. Haddad
Kennesaw State University, Kennesaw, Georgia
,
Sung Y. Shin
South Dakota State University, Brookings, South Dakota
,
Program Chair:
Yong Wan Koo
The University of Suwon, Gyeongggi-do, Korea

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SAC07

Sponsor:

SIGAPP

SAC07: The 2007 ACM Symposium on Applied Computing

March 11 - 15, 2007

Seoul, Korea

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Moses WChuravy VLarochelle HRanzato MHadsell RBalcan MLin H(2020)Instead of rewriting foreign code for machine learning, automatically synthesize fast gradientsProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496770(12472-12485)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496770
Hückelheim JHascoët LMüller J(2017)Algorithmic Differentiation of Code with Multiple Context-Specific ActivitiesACM Transactions on Mathematical Software10.1145/301546443:4(1-21)Online publication date: 9-Jan-2017
https://dl.acm.org/doi/10.1145/3015464
Shin JMalusare PHovland P(2008)Design and Implementation of a Context-Sensitive, Flow-Sensitive Activity Analysis Algorithm for Automatic DifferentiationAdvances in Automatic Differentiation10.1007/978-3-540-68942-3_11(115-125)Online publication date: 2008
https://doi.org/10.1007/978-3-540-68942-3_11

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents