article

Implementation of automatic differentiation tools

Authors:

Christian H. Bischof,

Paul D. Hovland,

Boyana NorrisAuthors Info & Claims

ACM SIGPLAN Notices, Volume 37, Issue 3

Pages 98 - 107

https://doi.org/10.1145/509799.503047

Published: 14 January 2002 Publication History

Abstract

Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its derivatives. Derivatives play an important role in a wide variety of scientific computing applications, including optimization, solution of nonlinear equations, sensitivity analysis, and nonlinear inverse problems. We describe a simple component architecture for developing tools for automatic differentiation and other mathematically oriented semantic transformations of scientific software. This architecture consists of a compiler-based, language-specific front-end for source transformation, loosely coupled with one or more language-independent "plug-in" transformation modules. The coupling mechanism between the front-end and transformation modules is provided by the XML Abstract Interface Form (XAIF). XAIF provides an abstract, language-independent representation of language constructs common in imperative languages, such as C and Fortran. We describe the use of this architecture in constructing tools for automatic differentiation of Fortran 77 and ANSI C, and we discuss how access to compiler optimization techniques can enable more efficient derivative augmentation.

References

[1]

ABATE, J., BISCHOF, C., CARLE, A., AND ROH, L. Algorithms and design for a second-order automatic differentiation module. In Proc. Int. Symposium on Symbolic and Algebraic Computing (ISSAC) '97 (New York, 1997), Association of Computing Machinery, pp. 149-155.

Digital Library

[2]

AIF Developer's Page. www-unix.mcs.anl.gov/autodiff/AIF.

[3]

BARTHOLOMEW-BIGGS, M. OPFAD - a users guide to the OPtima Forward Automatic Differentiation tool. Tech. rep., Numerical Optimization Centre, University of Hertfordsshire, 1995.

[4]

BENDTSEN, C., AND STAUNING, O. FADBAD, A Flexible C++ Package for Automatic Differentiation, Using the Forward and Backward Methods. Tech. Rep. IMM-REP-1996-17, Department of Mathematical Modelling, Technical University of Denmark, Aug. 1996.

[5]

BERZ, M. High-order computation and normal form analysis of repetitive systems. In Physics of Particle Accelerators, M. Month, Ed., vol. AIP 249. American Institute of Physics, Woodbury, NY, 1991, p. 456.

[6]

BERZ, M., BISCHOF, C., CORLISS, G., AND GRIEWANK, A. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, 1996.

[7]

BISCHOF, C. Automatic differentiation and numerical software design. In The Quality of Numerical Software: Assessment and Enhancement (London, 1997), R. Boisvert, Ed., Chapman & Hall, pp. 287-299.

Digital Library

[8]

BISCHOF, C., CARLE, A., CORLISS, G., GRIEWANK, A., AND HOVLAND, P. ADIFOR: Generating derivative codes from Fortran programs. Scientific Programming 1, 1 (1992), 11-29.

Digital Library

[9]

BISCHOF, C., CARLE, A., KHADEMI, P., AND MAUER, A. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering 3, 3 (1996), 18-32.

Digital Library

[10]

BISCHOF, C., CARLE, A., KHADEMI, P., MAUER, A., AND HOVLAND, P. ADIFOR 2.0 user's guide (Revision C). Tech. Rep. ANL/MCS-TM-192, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.

[11]

BISCHOF, C., CORLISS, G., AND GRIEWANK, A. ADIFOR exception handling. Tech. Rep. ANL/MCS-TM-159, Mathematics and Computer Science Division, Argonne National Laboratory, 1991.

[12]

BISCHOF, C., CORLISS, G., AND GRIEWANK, A. Structured second- and higher-order derivatives through univariate Taylor series. Optimization Methods and Software 2 (1993), 211-232.

[13]

BISCHOF, C., AND ROH, L. The automatic differentiation intermediate form (AIF), 1996. Unpublished Information.

[14]

BISCHOF, C., ROH, L., AND MAUER, A. ADIC - An extensible automatic differentiation tool for ANSI-C. Software-Practice and Experience 27, 12 (1997), 1427-1456.

Digital Library

[15]

BISCHOF, C. H., AND HAGHIGHAT, M. R. On hierarchical differentiation. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 83-94.

[16]

BROWN, S. OPRAD - a users guide to the OPtima Reverse Automatic Differentiation tool. Tech. rep., Numerical Optimization Centre, University of Hertfordsshire, 1995.

[17]

GIERING, R. Tangent linear and adjoint model compiler, users manual. Unpublished Information, Max-Planck Institut f~r Meteorologie, Hamburg, Germany, 1996.

[18]

GILL, P. E., MURRAY, W., AND WRIGHT, M. H. Practical Optimization. Academic Press, London, 1981.

[19]

GOLDMAN, V. V., AND CATS, G. Automatic adjoint modeling within a program generation framework: A case study for a weather forecasting grid-point model. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 185-194.

[20]

GRIEWANK, A. On automatic differentiation. In Mathematical Programming: Recent Developments and Applications (Amsterdam, 1989), Kluwer Academic Publishers, pp. 83-108.

[21]

GRIEWANK, A. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, Philadelphia, 2000.

Digital Library

[22]

GRIEWANK, A., JUEDES, D., AND UTKE, J. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software 22, 2 (1996), 131-167.

Digital Library

[23]

GRIEWANK, A., AND REESE, S. On the calculation of Jacobian matrices by the Markowitz rule. In Automatic Differentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. F. Corliss, Eds. SIAM, Philadelphia, 1991, pp. 126-135.

[24]

MONAGAN, M., AND RODONI, R. R. An implementation of the forward and reverse mode of automatic differentiation in Maple. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 353-362.

[25]

PRESS, W. H., AND TEUKOLSKY, S. A. Numerical calculation of derivatives. Computers in Physics 5, 1 (Jan./Feb. 1991), 88-89.

[26]

PRYCE, J. D., AND REID, J. K. AD01 - a Fortran 90 code for automatic differentiation. Unpublished information, Rutherford Appleton Laboratory, Oxon, U.K., 1996.

[27]

RALL, L. B. Automatic Differentiation: Techniques and Applications, vol. 120 of Lecture Notes in Computer Science. Springer Verlag, Berlin, 1981.

[28]

ROH, L., HOVLAND, P. D., ABATE, J., AND BISCHOF, C. AIF component system. Unpublished Information.

[29]

ROSTAING, N., DALMAS, S., AND GALLIGO, A. Automatic differentiation in Odyssee. Tellus 45a, 5 (October 1993), 558-568.

[30]

SHIRIAEV, D., AND GRIEWANK, A. ADOL-F: Automatic differentiation of Fortran codes. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, 1996, pp. 375-384.

Cited By

Zubair MRanjan DWalden ANastac GNielsen EDiskin BPaterno MJung SDavis J(2023)Efficient GPU Implementation of Automatic Differentiation for Computational Fluid Dynamics2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC58850.2023.00055(377-386)Online publication date: 18-Dec-2023
https://doi.org/10.1109/HiPC58850.2023.00055
Hück ABischof CSagebaum MGauger NJurgelucks BLarour EPerez G(2017)A usability case study of algorithmic differentiation tools on the ISSM ice sheet modelOptimization Methods and Software10.1080/10556788.2017.139660233:4-6(844-867)Online publication date: 8-Nov-2017
https://doi.org/10.1080/10556788.2017.1396602
Vassilev VVassilev MPenev AMoneta LIlieva V(2015)Clad — Automatic Differentiation Using Clang and LLVMJournal of Physics: Conference Series10.1088/1742-6596/608/1/012055608(012055)Online publication date: 22-May-2015
https://doi.org/10.1088/1742-6596/608/1/012055
Show More Cited By

Index Terms

Implementation of automatic differentiation tools
1. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language features
        Modules / packages
  2. Software organization and properties
    1. Extra-functional properties
      1. Interoperability
    2. Software system structures
      1. Software architectures

Recommendations

Implementation of automatic differentiation tools
PEPM '02: Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulation

Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
On the implementation of automatic differentiation tools

Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
A methodology for the development of discrete adjoint solvers using automatic differentiation tools

A methodology for the rapid development of adjoint solvers for computational fluid dynamics (CFD) models is presented. The approach relies on the use of automatic differentiation (AD) tools to almost completely automate the process of development of ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 37, Issue 3

March 2002

142 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/509799

Issue’s Table of Contents

PEPM '02: Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulation
January 2002
146 pages
ISBN:158113455X
DOI:10.1145/503032
Program Chair:
Peter Thiemann
Univeristy of Freiburg, Germany

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 January 2002

Published in SIGPLAN Volume 37, Issue 3

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
751
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zubair MRanjan DWalden ANastac GNielsen EDiskin BPaterno MJung SDavis J(2023)Efficient GPU Implementation of Automatic Differentiation for Computational Fluid Dynamics2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC58850.2023.00055(377-386)Online publication date: 18-Dec-2023
https://doi.org/10.1109/HiPC58850.2023.00055
Hück ABischof CSagebaum MGauger NJurgelucks BLarour EPerez G(2017)A usability case study of algorithmic differentiation tools on the ISSM ice sheet modelOptimization Methods and Software10.1080/10556788.2017.139660233:4-6(844-867)Online publication date: 8-Nov-2017
https://doi.org/10.1080/10556788.2017.1396602
Vassilev VVassilev MPenev AMoneta LIlieva V(2015)Clad — Automatic Differentiation Using Clang and LLVMJournal of Physics: Conference Series10.1088/1742-6596/608/1/012055608(012055)Online publication date: 22-May-2015
https://doi.org/10.1088/1742-6596/608/1/012055
Šolinc UKorelc J(2015)A simple way to improved formulation of $$\hbox {FE}^2$$FE2 analysisComputational Mechanics10.1007/s00466-015-1208-456:5(905-915)Online publication date: 1-Nov-2015
https://dl.acm.org/doi/10.1007/s00466-015-1208-4
Chapoutot A(2010)Interval slopes as a numerical abstract domain for floating-point variablesProceedings of the 17th international conference on Static analysis10.5555/1882094.1882106(184-200)Online publication date: 14-Sep-2010
https://dl.acm.org/doi/10.5555/1882094.1882106
Martel MPuebla GVidal G(2009)Program transformation for numerical precisionProceedings of the 2009 ACM SIGPLAN workshop on Partial evaluation and program manipulation10.1145/1480945.1480960(101-110)Online publication date: 19-Jan-2009
https://dl.acm.org/doi/10.1145/1480945.1480960
Korelc J(2009)Automation of primal and sensitivity analysis of transient coupled problemsComputational Mechanics10.1007/s00466-009-0395-244:5(631-649)Online publication date: 26-May-2009
https://doi.org/10.1007/s00466-009-0395-2
Bücker HPetera MVehreschild A(2008)Code Optimization Techniques in Source Transformations for Interpreted LanguagesAdvances in Automatic Differentiation10.1007/978-3-540-68942-3_20(223-233)Online publication date: 2008
https://doi.org/10.1007/978-3-540-68942-3_20
Pascual VHascoët L(2008)TAPENADE for CAdvances in Automatic Differentiation10.1007/978-3-540-68942-3_18(199-209)Online publication date: 2008
https://doi.org/10.1007/978-3-540-68942-3_18
Liang SQin J(2008)Data Assimilation Methods for Land Surface Variable EstimationAdvances in Land Remote Sensing10.1007/978-1-4020-6450-0_12(313-339)Online publication date: 2008
https://doi.org/10.1007/978-1-4020-6450-0_12
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents