skip to main content
research-article
Open access

An approach to generate correctly rounded math libraries for new floating point variants

Published: 04 January 2021 Publication History

Abstract

Given the importance of floating point (FP) performance in numerous domains, several new variants of FP and its alternatives have been proposed (e.g., Bfloat16, TensorFloat32, and posits). These representations do not have correctly rounded math libraries. Further, the use of existing FP libraries for these new representations can produce incorrect results. This paper proposes a novel approach for generating polynomial approximations that can be used to implement correctly rounded math libraries. Existing methods generate polynomials that approximate the real value of an elementary function 𝑓 (𝑥) and produce wrong results due to approximation errors and rounding errors in the implementation. In contrast, our approach generates polynomials that approximate the correctly rounded value of 𝑓 (𝑥) (i.e., the value of 𝑓 (𝑥) rounded to the target representation). It provides more margin to identify efficient polynomials that produce correctly rounded results for all inputs. We frame the problem of generating efficient polynomials that produce correctly rounded results as a linear programming problem. Using our approach, we have developed correctly rounded, yet faster, implementations of elementary functions for multiple target representations.

References

[1]
Denis Arzelier, Florent Bréhard, and Mioara Joldes. 2019. Exchange Algorithm for Evaluation and Approximation ErrorOptimized Polynomials. In 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). 30-37. https://doi.org/10.1109/ ARITH. 2019.00014
[2]
Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (Beijing, China) ( PLDI '12). ACM, New York, NY, USA, 453-462. https://doi.org/10.1145/2345156.2254118
[3]
Jeremy Bernstein, Jiawei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, and Yisong Yue. 2020. Learning compositional functions via multiplicative weight updates. arXiv: 2006. 14560 [cs.NE]
[4]
Sylvie Boldo, Marc Daumas, and Ren-Cang Li. 2009. Formally Verified Argument Reduction with a Fused Multiply-Add. In IEEE Transactions on Computers, Vol. 58. 1139-1145. https://doi.org/10.1109/TC. 2008.216
[5]
Peter Borwein and Tamas Erdelyi. 1995. Polynomials and Polynomial Inequalities. Springer New York. https://doi.org/10. 1007/978-1-4612-0793-1
[6]
Nicolas Brisebarre and Sylvvain Chevillard. 2007. Eficient polynomial L-approximations. In 18th IEEE Symposium on Computer Arithmetic (ARITH '07). https://doi.org/10.1109/ARITH. 2007.17
[7]
Nicolas Brisebarre, Jean-Michel Muller, and Arnaud Tisserand. 2006. Computing Machine-Eficient Polynomial Approximations. In ACM ACM Transactions on Mathematical Software, Vol. 32. Association for Computing Machinery, New York, NY, USA, 236-256. https://doi.org/10.1145/1141885.1141890
[8]
Nicolas Brunie, Florent de Dinechin, Olga Kupriianova, and Christoph Lauter. 2015. Code Generators for Mathematical Functions. In 2015 IEEE 22nd Symposium on Computer Arithmetic. 66-73. https://doi.org/10.1109/ARITH. 2015.22
[9]
Hung Tien Bui and Sofiene Tahar. 1999. Design and synthesis of an IEEE-754 exponential function. In Engineering Solutions for the Next Millennium. 1999 IEEE Canadian Conference on Electrical and Computer Engineering, Vol. 1. 450-455 vol. 1. https://doi.org/10.1109/CCECE. 1999.807240
[10]
Sylvain Chevillard, John Harrison, Mioara Joldes, and Christoph Lauter. 2011. Eficient and accurate computation of upper bounds of approximation errors. Theoretical Computer Science 412. https://doi.org/10.1016/j.tcs. 2010. 11.052
[11]
Sylvain Chevillard, Mioara Joldes, and Christoph Lauter. 2010. Sollya: An Environment for the Development of Numerical Codes. In Mathematical Software-ICMS 2010 (Lecture Notes in Computer Science, Vol. 6327 ). Springer, Heidelberg, Germany, 28-31. https://doi.org/10.1007/978-3-642-15582-6_5
[12]
Sylvain Chevillard and Christopher Lauter. 2007. A Certified Infinite Norm for the Implementation of Elementary Functions. In Seventh International Conference on Quality Software (QSIC 2007 ). 153-160. https://doi.org/10.1109/QSIC. 2007.4385491
[13]
Sangeeta Chowdhary, Jay P. Lim, and Santosh Nagarakatte. 2020. Debugging and Detecting Numerical Errors in Computation with Posits. In 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'20). https: //doi.org/10.1145/3385412.3386004
[14]
William J Cody and William M Waite. 1980. Software manual for the elementary functions. Prentice-Hall, Englewood Clifs, NJ.
[15]
Mike Cowlishaw. 2008. IEEE Standard for Floating-Point Arithmetic. IEEE 754-2008. IEEE Computer Society. 1-70 pages. https://doi.org/10.1109/IEEESTD. 2008.4610935
[16]
Nasrine Damouche and Matthieu Martel. 2018. Salsa: An Automatic Tool to Improve the Numerical Accuracy of Programs. In Automated Formal Methods (Kalpa Publications in Computing, Vol. 5 ), Natarajan Shankar and Bruno Dutertre (Eds.). 63-76. https://doi.org/10.29007/j2fd
[17]
Catherine Daramy, David Defour, Florent Dinechin, and Jean-Michel Muller. 2003. CR-LIBM: A correctly rounded elementary function library. In Proceedings of SPIE Vol. 5205 : Advanced Signal Processing Algorithms, Architectures, and Implementations XIII, Vol. 5205. https://doi.org/10.1117/12.505591
[18]
Marc Daumas, Guillaume Melquiond, and Cesar Munoz. 2005. Guaranteed proofs using interval arithmetic. In 17th IEEE Symposium on Computer Arithmetic (ARITH'05). 188-195. https://doi.org/10.1109/ARITH. 2005.25
[19]
Florent de Dinechin, Christopher Lauter, and Guillaume Melquiond. 2011. Certifying the Floating-Point Implementation of an Elementary Function Using Gappa. In IEEE Transactions on Computers, Vol. 60. 242-253. https://doi.org/10.1109/TC. 2010.128
[20]
Florent de Dinechin, Christoph Quirin Lauter, and Guillaume Melquiond. 2006. Assisted Verification of Elementary Functions Using Gappa. In Proceedings of the 2006 ACM Symposium on Applied Computing (Dijon, France) (SAC '06). Association for Computing Machinery, New York, NY, USA, 1318-1322. https://doi.org/10.1145/1141277.1141584
[21]
Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. 2007. MPFR: A Multipleprecision Binary Floating-point Library with Correct Rounding. ACM Trans. Math. Software 33, 2, Article 13 ( June 2007 ). https://doi.org/10.1145/1236463.1236468
[22]
Zhoulai Fu and Zhendong Su. 2019. Efective Floating-point Analysis via Weak-distance Minimization. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) ( PLDI 2019). ACM, New York, NY, USA, 439-452. https://doi.org/10.1145/3314221.3314632
[23]
Ambros Gleixner, Daniel E. Stefy, and Kati Wolter. 2015. Iterative Refinement for Linear Programming. Technical Report 15-15. ZIB, Takustr. 7, 14195 Berlin. https://doi.org/10.1287/ijoc. 2016.0692
[24]
Ambros M. Gleixner, Daniel E. Stefy, and Kati Wolter. 2012. Improving the Accuracy of Linear Programming Solvers with Iterative Refinement. In Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation (Grenoble, France) (ISSAC '12). Association for Computing Machinery, New York, NY, USA, 187-194. https://doi.org/10. 1145/2442829.2442858
[25]
Eric Goubault. 2001. Static Analyses of the Precision of Floating-Point Operations. In Proceedings of the 8th International Symposium on Static Analysis (SAS). Springer, 234-259. https://doi.org/10.1007/3-540-47764-0_14
[26]
John Gustafson. 2017. Posit Arithmetic. https://posithub.org/docs/Posits4.pdf
[27]
John Gustafson. 2020. The Minefield Method: A Uniformly Fast Solution to the Table-Maker's Dilemma. https://bit.ly/2ZP4kHj
[28]
John Gustafson and Isaac Yonemoto. 2017. Beating Floating Point at Its Own Game: Posit Arithmetic. Supercomputing Frontiers and Innovations: an International Journal 4, 2 ( June 2017 ), 71-86. https://doi.org/10.14529/jsfi170206
[29]
John Harrison. 1997a. Floating point verification in HOL light: The exponential function. In Algebraic Methodology and Software Technology, Michael Johnson (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 246-260. https: //doi.org/10.1007/BFb0000475
[30]
John Harrison. 1997b. Verifying the Accuracy of Polynomial Approximations in HOL. In International Conference on Theorem Proving in Higher Order Logics. https://doi.org/10.1007/BFb0028391
[31]
John Harrison. 2009. HOL Light: An Overview. In Proceedings of the 22nd International Conference on Theorem Proving in Higher Order Logics, TPHOLs 2009 (Lecture Notes in Computer Science, Vol. 5674 ), Stefan Berghofer, Tobias Nipkow, Christian Urban, and Makarius Wenzel (Eds.). Springer-Verlag, Munich, Germany, 60-66. https://doi.org/10.1007/978-3-642-03359-9_4
[32]
IBM. 2008. Accurate Portable MathLib. http://oss.software.ibm.com/mathlib/
[33]
Intel. 2019. Delivering a New Intelligence with AI at Scale. https://www.intel.com/content/www/us/en/artificial-intelligence/ posts/nnp-aisummit.html
[34]
Claude-Pierre Jeannerod, Hervé Knochel, Christophe Monat, and Guillaume Revy. 2011. Computing Floating-Point Square Roots via Bivariate Polynomial Evaluation. IEEE Trans. Comput. 60. https://doi.org/10.1109/TC. 2010.152
[35]
Jef Johnson. 2018. Rethinking floating point for deep learning. http://export.arxiv.org/abs/ 1811.01721
[36]
William Kahan. 2004. A Logarithm Too Clever by Half. https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT
[37]
Dhiraj D. Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. 2019. A Study of BFLOAT16 for Deep Learning Training. arXiv: 1905.12322
[38]
Olga Kupriianova and Christoph Lauter. 2014. Metalibm: A Mathematical Functions Code Generator. In 4th International Congress on Mathematical Software. https://doi.org/10.1007/978-3-662-44199-2_106
[39]
Wonyeol Lee, Rahul Sharma, and Alex Aiken. 2017. On Automatically Proving the Correctness of Math.h Implementations. Proceedings of the ACM on Programming Languages 2, POPL, Article 47 ( Dec. 2017 ), 32 pages. https://doi.org/10.1145/ 3158135
[40]
Vincent Lefèvre and Jean-Michel Muller. 2001. Worst Cases for Correct Rounding of the Elementary Functions in Double Precision. In 15th IEEE Symposium on Computer Arithmetic (Arith '01). 111-118. https://doi.org/10.1109/ARITH. 2001. 930110
[41]
Vincent Lefèvre, Jean-Michel Muller, and Arnaud Tisserand. 1998. Toward correctly rounded transcendentals. IEEE Trans. Comput. 47, 11 ( 1998 ), 1235-1243. https://doi.org/10.1109/12.736435
[42]
Cerlane Leong. 2019. SoftPosit-Math. https://gitlab.com/cerlane/softposit-math
[43]
Jay P. Lim, Mridul Aanjaneya, John Gustafson, and Santosh Nagarakatte. 2020a. A Novel Approach to Generate Correctly Rounded Math Libraries for New Floating Point Representations. arXiv: 2007. 05344 [cs.MS]
[44]
Jay P. Lim and Santosh Nagarakatte. 2020a. RLibm. https://github.com/rutgers-apl/rlibm
[45]
Jay P. Lim and Santosh Nagarakatte. 2020b. RLibm-generator. https://github.com/rutgers-apl/rlibm-generator
[46]
Jay P. Lim, Matan Shachnai, and Santosh Nagarakatte. 2020b. Approximating Trigonometric Functions for Posits Using the CORDIC Method. In Proceedings of the 17th ACM International Conference on Computing Frontiers (Catania, Sicily, Italy) ( CF '20). Association for Computing Machinery, New York, NY, USA, 19-28. https://doi.org/10.1145/3387902.3392632
[47]
Guillaume Melquiond. 2019. Gappa. http://gappa.gforge.inria.fr
[48]
Sun Microsystems. 2008. LIBMCR 3 " 16 February 2008 " "libmcr-0.9". http://www.math.utah.edu/cgi-bin/man2html.cgi?/usr/ local/man/man3/libmcr.3
[49]
Jean-Michel Muller. 2005. Elementary Functions: Algorithms and Implementation. Birkhauser. https://doi.org/10.1007/978-1-4899-7983-4
[50]
NVIDIA. 2020. TensorFloat-32 in the A100 GPU Accelerates AI Training, HPC up to 20x. https://blogs.nvidia.com/blog/2020/ 05/14/tensorfloat-32-precision-format/
[51]
Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Vol. 50. Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/ 2813885.2737959
[52]
Eugene Remes. 1934. Sur un procédé convergent d'approximations successives pour déterminer les polynômes d'approximation. Comptes rendus de l' Académie des Sciences 198 ( 1934 ), 2063-2065.
[53]
Alex Sanchez-Stern, Pavel Panchekha, Sorin Lerner, and Zachary Tatlock. 2018. Finding Root Causes of Floating Point Error. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (Philadelphia, PA, USA) ( PLDI 2018). ACM, New York, NY, USA, 256-269. https://doi.org/10.1145/3296979.3192411
[54]
Joe Sawada. 2002. Formal verification of divide and square root algorithms using series calculation. In 3rd International Workshop on the ACL2 Theorem Prover and its Applications.
[55]
Giuseppe Tagliavini, Stefan Mach, Davide Rossi, Andrea Marongiu, and Luca Benin. 2018. A transprecision floating-point platform for ultra-low power computing. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE). 1051-1056. https://doi.org/10.23919/DATE. 2018.8342167
[56]
Ping-Tak Peter Tang. 1990. Table-Driven Implementation of the Logarithm Function in IEEE Floating-Point Arithmetic. ACM Trans. Math. Software 16, 4 (Dec. 1990 ), 378-400. https://doi.org/10.1145/98267.98294
[57]
Lloyd N. Trefethen. 2012. Approximation Theory and Approximation Practice (Other Titles in Applied Mathematics). Society for Industrial and Applied Mathematics, USA.
[58]
Shibo Wang and Pankaj Kanwar. 2019. BFloat16: The secret to high performance on Cloud TPUs. https://cloud.google.com/ blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus
[59]
Xin Yi, Liqian Chen, Xiaoguang Mao, and Tao Ji. 2019. Eficient Automated Repair of High Floating-Point Errors in Numerical Libraries. Proceedings of the ACM on Programming Languages 3, POPL, Article 56 ( Jan. 2019 ), 29 pages. https://doi.org/10.1145/3290369
[60]
Abraham Ziv. 1991. Fast Evaluation of Elementary Mathematical Functions with Correctly Rounded Last Bit. ACM Trans. Math. Software 17, 3 (Sept. 1991 ), 410-423. https://doi.org/10.1145/114697.116813
[61]
Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2019. Detecting Floating-Point Errors via Atomic Conditions. Proceedings of the ACM on Programming Languages 4, POPL, Article 60 ( Dec. 2019 ), 27 pages. https://doi.org/10.1145/3371128

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages
Proceedings of the ACM on Programming Languages  Volume 5, Issue POPL
January 2021
1789 pages
EISSN:2475-1421
DOI:10.1145/3445980
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 January 2021
Published in PACMPL Volume 5, Issue POPL

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. correctly rounded math libraries
  2. floating point
  3. posits

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)211
  • Downloads (Last 6 weeks)25
Reflects downloads up to 06 Feb 2025

Other Metrics

Citations

Cited By

View all

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Full Access

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media