article

The design and implementation of the MRRR algorithm

Authors:

Inderjit S. Dhillon,

Beresford N. Parlett,

Christof VömelAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 32, Issue 4

Pages 533 - 560

https://doi.org/10.1145/1186785.1186788

Published: 01 December 2006 Publication History

Get Access

Abstract

In the 1990's, Dhillon and Parlett devised the algorithm of multiple relatively robust representations (MRRR) for computing numerically orthogonal eigenvectors of a symmetric tridiagonal matrix T with O(n²) cost. While previous publications related to MRRR focused on theoretical aspects of the algorithm, a documentation of software issues has been missing. In this article, we discuss the design and implementation of the new MRRR version STEGR that will be included in the next LAPACK release. By giving an algorithmic description of MRRR and identifying governing parameters, we hope to make STEGR more easily accessible and suitable for future performance tuning. Furthermore, this should help users understand design choices and tradeoffs when using the code.

References

[1]

Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Croz, J. D., Greenbaum, A., Hammarling, S., McKenney, A., and Sorensen, D. 1999. LAPACK User's Guide, 3rd. ed. SIAM, Philadelphia, PA.

Digital Library

Google Scholar

[2]

ANSI/IEEE 1985. IEEE Standard for Binary Floating Point Arithmetic. Std 754-1985 ed. ANSI/IEEE, New York.

Google Scholar

[3]

Antonelli, D. and Vömel, C. 2005. LAPACK working note 168: PDSYEVR. ScaLAPACK's parallel MRRR algorithm for the symmetric eigenvalue problem. Tech. Rep. UCBCSD-05-1399, University of California, Berkeley.

Google Scholar

[4]

Barlow, J. and Demmel, J. 1990. Computing accurate eigensystems of scaled diagonally dominant matrices. SIAM J. Numer. Anal. 27, 3, 762--791.

Digital Library

Google Scholar

[5]

Bientinesi, P., Dhillon, I. S., and van de Geijn, R. A. 2005. A parallel eigensolver for dense symmetric matrices based on multiple relatively robust representations. SIAM J. Sci. Comput. 27, 1, 43--66.

Digital Library

Google Scholar

[6]

Blackford, L. S., Demmel, J., Dongarra, J. J., Duff, I. S., Hammarling, S., Henry, G., Heroux, M., Kaufman, L., Lumsdaine, A., Petitet, A., Pozo, R., Remington, K., and Whaley, R. C. 2002. An updated set of basic linear algebra subprograms (BLAS). ACM Trans. Math. Softw. 28, 2, 135--151.

Digital Library

Google Scholar

[7]

Bunch, J., Nielsen, P., and Sorensen, D. 1978. Rank-One modification of the symmetric eigenproblem. Numer. Math. 31, 31--48.

Digital Library

Google Scholar

[8]

Choi, J., Demmel, J., Dhillon, I., Dongarra, J., Ostrouchov, S., Petitet, A., Stanley, K., Walker, D., and Whaley, R. C. 1996. ScaLAPACK: A portable linear algebra library for distributed memory computers---Design issues and performance. Comput. Phys. Commun. 97, 1--15.

Crossref

Google Scholar

[9]

Choi, J., Dongarra, J. J., Ostrouchov, S., Petitet, A., and Walker, D. 1995. A proposal for a set of parallel basic linear algebra subprograms. Tech. Rep. UT-CS-95-292, University of Tennessee, Knoxville, TN.

Digital Library

Google Scholar

[10]

Cuppen, J. J. M. 1981. A divide and conquer method for the symmetric tridiagonal eigenproblem. Numer. Math. 36, 177--195.

Digital Library

Google Scholar

[11]

Davis, C. and Kahan, W. 1970. The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal. 7, 1, 1--47.

Digital Library

Google Scholar

[12]

Demmel, J. W. 1997. Applied Numerical Linear Algebra. SIAM, Philadelphia, PA.

Digital Library

Google Scholar

[13]

Demmel, J. W., Dhillon, I. S., Marques, O. A., Parlett, B. N., and Vömel, C. 2005. Performance and accuracy of the symmetric eigensolvers in LAPACK. University of California, Berkeley. In preparation.

Google Scholar

[14]

Demmel, J. W., Dhillon, I. S., and Ren, H. 1995. On the correctness of some bisection-like parallel eigenvalue algorithms in floating point arithmetic. Electron. Trans. Num. Anal. 3, 116--140.

Google Scholar

[15]

Demmel, J. W. and Kahan, W. 1990. Accurate singular values of bidiagonal matrices. SIAM J. Sci. Stat. Comput. 11, 5, 873--912.

Digital Library

Google Scholar

[16]

Demmel, J. W. and Li, X. S. 1994. Faster numerical algorithms via exception handling. IEEE Trans. Comp. 43, 8, 983--992.

Digital Library

Google Scholar

[17]

Dhillon, I. S. 1997. A new O(n2) algorithm for the symmetric tridiagonal eigenvalue/eigenvector problem. Ph.D. thesis, University of California, Berkeley, California, USA.

Digital Library

Google Scholar

[18]

Dhillon, I. S. and Parlett, B. N. 2004b. Multiple representations to compute orthogonal eigenvectors of symmetric tridiagonal matrices. Linear Algebra Appl. 387, 1--28.

Crossref

Google Scholar

[19]

Dhillon, I. S. and Parlett, B. N. 2004b. Orthogonal eigenvectors and relative gaps. SIAM J. Matrix Anal. Appl. 25, 3, 858--899.

Digital Library

Google Scholar

[20]

Dhillon, I. S., Parlett, B. N., and Vömel, C. 2005. Glued matrices and the MRRR algorithm. SIAM J. Sci. Comput. 27, 2, 496--510.

Digital Library

Google Scholar

[21]

Dongarra, J. and Whaley, R. C. 1995. A User's Guide to the BLACS v1.1. Tech. Rep. UT-CS-95-281, University of Tennessee, Knoxville, TN, USA.

Digital Library

Google Scholar

[22]

Dongarra, J. J., Du Croz, J., Duff, I. S., and Hammarling, S. 1990. A set of level 3 basic linear algebra subprograms. ACM Trans. Math. Softw. 16, 1--17.

Digital Library

Google Scholar

[23]

Dongarra, J. J., Duff, I. S., Sorensen, D. C., and van der Vorst, H. A. 1998. Numerical Linear Algebra for High-Performance Computers. SIAM Press, Philadelphia, PA.

Digital Library

Google Scholar

[24]

Dongarra, J. J. and van de Geijn, R. A. 1991. Lapack working note 37: Two dimensional basic linear algebra communication subprograms. Tech. Rep. UT-CS-91-138, University of Tennessee, Knoxville, TN, USA.

Digital Library

Google Scholar

[25]

Fernando, K. V. and Parlett, B. N. 1994. Accurate singular values and differential qd algorithms. Numeri. Math. 67, 191--229.

Google Scholar

[26]

Gu, M. and Eisenstat, S. C. 1995. A divide-and-conquer algorithm for the symmetric tridiagonal eigenproblem. SIAM J. Matrix Anal. Appl. 16, 1, 172--191.

Digital Library

Google Scholar

[27]

Ipsen, I. C. F. 1997. Computing an eigenvector with inverse iteration. SIAM Rev. 39, 2, 254--291.

Digital Library

Google Scholar

[28]

Marques, O. A., Parlett, B. N., and Vömel, C. 2005. LAPACK working note 167: Subset computations with the MRRR algorithm. Tech. Rep. UCBCSD-05-1392, University of California, Berkeley, Calfornia, USA.

Google Scholar

[29]

Marques, O. A., Riedy, E. J., and Vömel, C. 2005. Lapack working note 172: Benefits of IEEE-754 features in modern symmetric tridiagonal eigensolvers. Tech. Rep. UCBCSD-05-1414, University of California, Berkeley, California, USA.

Google Scholar

[30]

Overton, M. 2001. Numerical Computing with IEEE Floating Point Arithmetic. SIAM Press, Philadelphia, PA.

Digital Library

Google Scholar

[31]

Parlett, B. N. 1995. Acta Numerica. Cambridge University Press, Cambridge, UK. 459--491.

Google Scholar

[32]

Parlett, B. N. 1998. The Symmetric Eigenvalue Problem. SIAM Press, Philadelphia, PA.

Digital Library

Google Scholar

[33]

Parlett, B. N. 2000. For tridiagonals T replace T with LDLt. J. Comp. Appl. Math. 123, 117--130.

Digital Library

Google Scholar

[34]

Parlett, B. N. 2003. Perturbation of eigenpairs of factored symmetric tridiagonal matrices. Found. Comput. Math. 3, 2, 207--223.

Crossref

Google Scholar

[35]

Parlett, B. N. 2005. A bidiagonal matrix determines its hyperbolic SVD to varied relative accuracy. SIAM J. Matrix Anal. Appl. 26, 4, 1022--1057.

Digital Library

Google Scholar

[36]

Parlett, B. N. and Dhillon, I. S. 1997. Fernando's solution to Wilkinson's problem: An application of double factorization. Linear Algebra Appl. 267, 247--279.

Crossref

Google Scholar

[37]

Parlett, B. N. and Dhillon, I. S. 2000a. Relatively robust representations of symmetric tridiagonals. Linear Algebra Appl. 309, 1--3, 121--151.

Crossref

Google Scholar

[38]

Parlett, B. N. and Marques, O. 2000. An implementation of the dqds algorithm (positive case). Linear Algebra and Appl. 309, 217--259.

Crossref

Google Scholar

[39]

Rutishauser, H. 1954. Der quotienten-differenzen-algorithmus. Z. Angew. Math. Phys. 5, 233--251.

Crossref

Google Scholar

[40]

Rutishauser, H. 1976. Vorlesungen über Numerische Mathematik. Birkhäuser, Basel.

Google Scholar

[41]

Snir, M., Otto, S., Huss-Lederman, S., Walker, D., and Dongarra, J. J. 1996. MPI: The Complete Reference. MIT Press, Cambridge, MA.

Digital Library

Google Scholar

[42]

van de Geijn, R. A. 1997. Using PLAPACK: Parallel Linear Algebra Package. MIT Press, Cambridge, MA.

Digital Library

Google Scholar

Cited By

View all

Ferrari MCavalli FHarake HLompa CLo Russo NEvans KSchenk O(2024)Arrowhead Factorization of Real Symmetric Matrices and its Applications in Optimized EigendecompositionProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3659914.3659918(1-12)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3659914.3659918
Uchino YImamura T(2024)High-Performance Eigensolver Combining EigenExa and Iterative RefinementProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00213(1703-1712)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00213
Ryabinin KSarras GLöffler WBiermann M(2024)Direct Solver Aiming at Elimination of Systematic Errors in 3D Stellar PositionsComputational Science – ICCS 202410.1007/978-3-031-63783-4_23(309-323)Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-63783-4_23
Show More Cited By

Index Terms

The design and implementation of the MRRR algorithm
1. Computing methodologies
  1. Symbolic and algebraic manipulation
    1. Symbolic and algebraic algorithms
      1. Linear algebra algorithms
2. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computations on matrices

Recommendations

Bidiagonal SVD Computation via an Associated Tridiagonal Eigenproblem

The Singular Value Decomposition (SVD) is widely used in numerical analysis and scientific computing applications, including dimensionality reduction, data compression and clustering, and computation of pseudo-inverses. In many cases, a crucial part of ...
ScaLAPACK's MRRR algorithm

The (sequential) algorithm of Multiple Relatively Robust Representations, MRRR, is a more efficient variant of inverse iteration that does not require reorthogonalization. It solves the eigenproblem of an unreduced symmetric tridiagonal matrix T ∈ R^{n × ...}
Algorithm 880: A testing infrastructure for symmetric tridiagonal eigensolvers

LAPACK is often mentioned as a positive example of a software library that encapsulates complex, robust, and widely used numerical algorithms for a wide range of applications. At installation time, the user has the option of running a (limited) number ...

Reviews

Reviewer: Jesse Louis Barlow

The multiple relatively robust representation (MRRR) algorithm is a long, detailed answer to the question of how to find the eigenvectors of an n × n symmetric tridiagonal matrix T given its computed eigenvalues. In theory, inverse iteration should yield these eigenvectors in O(n2) operations, but the eigenvectors of clustered eigenvalues may not come out orthogonal without, say, Gram-Schmidt orthogonalization [1]. For an n × n matrix T, MRRR avoids Gram-Schmidt in computing the eigenvectors of how it represents the shifted matrix T-λ I for inverse iteration. It stores the factorization T - λ I = LDLT where D and L are either bidiagonal or twisted bidiagonal. MRRR manipulates this factorization using the dqds algorithms [2], making the representation (D,L) very accurate and allowing for fast accurate changes in the shift λ. It also uses a smart initial guess for inverse iteration, and then uses sophisticated shift and recompute strategies (called representation trees) to separate out clusters of eigenvalues. The paper summarizes the literature on MRRR and gives details about its software. Tracking down the items in the bibliography of this paper is necessary for truly understanding MRRR. Since the MRRR algorithm is very complex and the associated analysis is even more complex, its details are for specialists only. For nonspecialists, it is important to understand that these algorithms are a major advance in methods for finding the eigenvectors of symmetric tridiagonals, that the software is available, and that symmetric eigenvalue decompositions and singular value decompositions can now be computed more quickly. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 32, Issue 4

December 2006

145 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/1186785

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2006

Published in TOMS Volume 32, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

102
Total Citations
View Citations
853
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)2

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ferrari MCavalli FHarake HLompa CLo Russo NEvans KSchenk O(2024)Arrowhead Factorization of Real Symmetric Matrices and its Applications in Optimized EigendecompositionProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3659914.3659918(1-12)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3659914.3659918
Uchino YImamura T(2024)High-Performance Eigensolver Combining EigenExa and Iterative RefinementProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00213(1703-1712)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00213
Ryabinin KSarras GLöffler WBiermann M(2024)Direct Solver Aiming at Elimination of Systematic Errors in 3D Stellar PositionsComputational Science – ICCS 202410.1007/978-3-031-63783-4_23(309-323)Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-63783-4_23
Kobayashi MHirota YKudo SHoshi TYamamoto Y(2023)Automatic performance tuning using the ATMathCoreLib tool: Two experimental studies related to dense symmetric eigensolversConcurrency and Computation: Practice and Experience10.1002/cpe.784936:10Online publication date: 30-Jun-2023
https://doi.org/10.1002/cpe.7849
Wu XDavidović DAchilles SDi Napoli ERobinson T(2022)ChASEProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3539781.3539792(1-12)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3539781.3539792
Nikiforovskii DDeineka ISharkov IMeshkovskii I(2022)A Method for Fiber Optic Gyroscope Temperature Drift Compensation Using Correlations between the Readings of the Gyroscope and Several Temperature SensorsGyroscopy and Navigation10.1134/S207510872202005513:2(105-109)Online publication date: 4-Oct-2022
https://doi.org/10.1134/S2075108722020055
Davidovic D(2021)An overview of dense eigenvalue solvers for distributed memory systems2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO)10.23919/MIPRO52101.2021.9596900(265-271)Online publication date: 27-Sep-2021
https://doi.org/10.23919/MIPRO52101.2021.9596900
Sierra-Arriaga FBranco RLee B(2020)Security Issues and Challenges for Virtualization TechnologiesACM Computing Surveys10.1145/338219053:2(1-37)Online publication date: 19-May-2020
https://dl.acm.org/doi/10.1145/3382190
Zolfaghari BSrivastava GRoy SNemati HAfghah FKoshiba TRazi ABibak KMitra PRai B(2020)Content Delivery NetworksACM Computing Surveys10.1145/338061353:2(1-34)Online publication date: 17-Apr-2020
https://dl.acm.org/doi/10.1145/3380613
Suaboot JFahad ATari ZGrundy JMahmood AAlmalawi AZomaya ADrira K(2020)A Taxonomy of Supervised Learning for IDSs in SCADA EnvironmentsACM Computing Surveys10.1145/337949953:2(1-37)Online publication date: 17-Apr-2020
https://dl.acm.org/doi/10.1145/3379499
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Bidiagonal SVD Computation via an Associated Tridiagonal Eigenproblem

ScaLAPACK's MRRR algorithm

Algorithm 880: A testing infrastructure for symmetric tridiagonal eigensolvers

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations