HIFIR: Hybrid Incomplete Factorization with Iterative Refinement for Preconditioning Ill-Conditioned and Singular Systems
Authors: 
Qiao Chen, Xiangmin JiaoAuthors Info & Claims
Qiao Chen, Xiangmin JiaoAuthors Info & ClaimsArticle No.: 32, Pages 1 - 33
Abstract
We introduce a software package called Hybrid Incomplete Factorization with Iterative Refinement (HIFIR) for preconditioning sparse, unsymmetric, ill-conditioned, and potentially singular systems. HIFIR computes a hybrid incomplete factorization (HIF), which combines multilevel incomplete LU factorization with a truncated, rank-revealing QR (RRQR) factorization on the final Schur complement. This novel hybridization is based on the new theory of ϵ-accurate approximate generalized inverse (AGI). It enables near-optimal preconditioners for consistent systems and enables flexible GMRES to solve inconsistent systems when coupled with iterative refinement. In this article, we focus on some practical algorithmic and software issues of HIFIR. In particular, we introduce a new inverse-based rook pivoting (IBRP) into ILU, which improves the robustness and the overall efficiency for some ill-conditioned systems by significantly reducing the size of the final Schur complement for some systems. We also describe the software design of HIFIR in terms of its efficient data structures for supporting rook pivoting in a multilevel setting, its template-based generic programming interfaces for mixed-precision real and complex values in C++, and its user-friendly high-level interfaces in MATLAB and Python. We demonstrate the effectiveness of HIFIR for ill-conditioned or singular systems arising from several applications, including the Helmholtz equation, linear elasticity, stationary incompressible Navier–Stokes (INS) equations, and time-dependent advection-diffusion equation.
References
[1]
Martin Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring, Marie E. Rognes, and Garth N. Wells. 2015. The FEniCS project version 1.5. Archive of Numerical Software 3, 100 (2015), 9–23.
[2]
Patrick R. Amestoy, Timothy A. Davis, and Iain S. Duff. 2004. Algorithm 837: AMD, an approximate minimum degree ordering algorithm. ACM Transactions on Mathematical Software 30, 3 (2004), 381–388.
[3]
Patrick R. Amestoy, Iain S. Duff, Jean-Yves L’Excellent, and Jacko Koster. 2000. MUMPS: A general purpose distributed memory sparse solver. In Proceedings of the International Workshop on Applied Parallel Computing. Springer, 121–130.
[4]
James Baglama and Lothar Reichel. 2005. Augmented implicitly restarted Lanczos bidiagonalization methods. SIAM Journal on Scientific Computing 27, 1 (2005), 19–42.
[5]
Satish Balay, Shrirang Abhyankar, Mark F. Adams, Jed Brown, Peter Brune, Kris Buschelman, Lisandro Dalcin, Alp Dener, Victor Eijkhout, William D. Gropp, Dmitry Karpeyev, Dinesh Kaushik, Matthew G. Knepley, Dave A. May, Lois Curfman McInnes, Richard Tran Mills, Todd Munson, Karl Rupp, Patrick Sanan, Barry F. Smith, Stefano Zampini, Hong Zhang, and Hong Zhang. 2020. PETSc Users Manual. Technical Report ANL-95/11 - Revision 3.14. Argonne National Laboratory. Retrieved from https://www.mcs.anl.gov/petsc.
[6]
Randolph E. Bank and Craig C. Douglas. 1993. Sparse matrix multiplication package (SMMP). Advances in Computational Mathematics 1, 1 (1993), 127–137.
[7]
Stefan Behnel, Robert Bradshaw, Craig Citro, Lisandro Dalcin, Dag Sverre Seljebotn, and Kurt Smith. 2010. Cython: The best of both worlds. Computing in Science & and Engineering 13, 2 (2010), 31–39.
[8]
Michele Benzi. 2002. Preconditioning techniques for large linear systems: A survey. Journal of Computational Physics 182, 2 (2002), 418–477.
[9]
Michele Benzi and Maxim A. Olshanskii. 2006. An augmented Lagrangian-based approach to the Oseen problem. SIAM Journal on Scientific Computing 28, 6 (2006), 2095–2113.
[10]
Michele Benzi, Daniel B. Szyld, and Arno Van Duin. 1999. Orderings for incomplete factorization preconditioning of nonsymmetric problems. SIAM Journal on Scientific Computing 20, 5 (1999), 1652–1670.
[11]
Michele Benzi and Miroslav Tůma. 2003. A robust incomplete factorization preconditioner for positive definite matrices. Numerical Linear Algebra with Applications 10, 5–6 (2003), 385–400.
[12]
Michele Benzi and Miroslav Tůma. 2003. A robust preconditioner with low memory requirements for large sparse least squares problems. SIAM Journal on Scientific Computing 25, 2 (2003), 499–512.
[13]
Christian H. Bischof. 1990. Incremental condition estimation. SIAM Journal on Matrix Analysis and Applications 11, 2 (1990), 312–322.
[14]
Åke Björck. 1996. Numerical Methods for Least Squares Problems. SIAM.
[15]
Matthias Bollhöfer, José I. Aliaga, Alberto F. Martín, and Enrique S. Quintana-Ortí. 2011. ILUPACK. In Proceedings of the Encyclopedia of Parallel Computing. Springer, 917–926.
[16]
Matthias Bollhöfer and Yousef Saad. 2006. Multilevel preconditioners constructed from inverse-based ILUs. SIAM Journal on Scientific Computing 27, 5 (2006), 1627–1650.
[17]
John Charles Butcher and Nicolette Goodwin. 2008. Numerical Methods for Ordinary Differential Equations. Vol. 2. Wiley Online Library.
[18]
Tony F. Chan. 1987. Rank revealing QR factorizations. Linear Algebra and Its Applications 88 (1987), 67–82.
[19]
Wing-Man Chan and Alan George. 1980. A linear time implementation of the reverse Cuthill-McKee algorithm. BIT Numerical Mathematics 20, 1 (1980), 8–14.
[20]
Qiao Chen, Aditi Ghai, and Xiangmin Jiao. 2021. HILUCSI: Simple, robust, and fast multilevel ILU for large-scale saddle-point problems from PDEs. Numerical Linear Algebra with Applications 28, 6 (2021), e2400. DOI:DOI:
[21]
Qiao Chen, Xiangmin Jiao, and Oliver Yang. 2021. Robust and efficient multilevel-ILU preconditioning of hybrid Newton–GMRES for incompressible Navier–Stokes equations. International Journal for Numerical Methods in Fluids 93, 12 (2021), 3405–3423. DOI:
[22]
Edmond Chow and Yousef Saad. 1997. Experimental study of ILU preconditioners for indefinite matrices. Journal of Computational and Applied Mathematics 86, 2 (1997), 387–414.
[23]
Timothy A. Davis and Yifan Hu. 2011. The University of Florida sparse matrix collection. ACM Transactions on Mathematical Software 38, 1 (2011), 1–25.
[24]
Ron S. Dembo, Stanley C. Eisenstat, and Trond Steihaug. 1982. Inexact Newton methods. SIAM Journal on Numerical Analysis 19, 2 (1982), 400–408.
[25]
James W. Demmel, Michael T. Heath, and Henk A. van der Vorst. 1993. Parallel numerical linear algebra. Acta Numerica 2 (1993), 111–197. DOI:
[26]
Iain S. Duff and Jacko Koster. 2001. On algorithms for permuting large entries to the diagonal of a sparse matrix. SIAM Journal on Matrix Analysis and Applications 22, 4 (2001), 973–996.
[27]
Stanley C. Eisenstat, Martin H. Schultz, and Andrew H. Sherman. 1981. Algorithms and data structures for sparse symmetric Gaussian elimination. SIAM Journal on Scientific and Statistical Computing 2, 2 (1981), 225–237.
[28]
Howard Elman, Victoria E. Howle, John Shadid, Robert Shuttleworth, and Ray Tuminaro. 2006. Block preconditioners based on approximate commutators. SIAM Journal on Scientific Computing 27, 5 (2006), 1651–1668.
[29]
Oliver G. Ernst and Martin J. Gander. 2012. Why it is difficult to solve Helmholtz problems with classical iterative methods. In Proceedings of the Numerical Analysis of Multiscale Problems. Springer, 325–363.
[30]
David Chin-Lung Fong and Michael Saunders. 2011. LSMR: An iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing 33, 5 (2011), 2950–2971.
[31]
John Alan George. 1971. Computer implementation of the finite element method. Ph.D. Dissertation. Stanford University, Stanford, CA. AAI7205916.
[32]
Christophe Geuzaine and Jean-François Remacle. 2009. Gmsh: A 3-D finite element mesh generator with built-in pre-and post-processing facilities. International Journal for Numerical Methods in Engineering 79, 11 (2009), 1309–1331.
[33]
Aditi Ghai, Cao Lu, and Xiangmin Jiao. 2019. A comparison of preconditioned Krylov subspace methods for large-scale nonsymmetric linear systems. Numerical Linear Algebra with Applications 26, 1 (2019), e2215.
[34]
Gene H. Golub and Charles F. Van Loan. 2013. Matrix Computations (4th ed.). Johns Hopkins.
[35]
Nicholas Gould and Jennifer Scott. 2017. The state-of-the-art of preconditioners for sparse linear least-squares problems. ACM Transactions on Mathematical Software 43, 4 (2017), 1–35.
[36]
Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. (2010). Retrieved from http://eigen.tuxfamily.org.
[37]
Anshul Gupta. 2021. WSMP: Watson Sparse Matrix Package (Part-III: Iterative Solution of Sparse Systems) Version 20.12. Technical Report. IBM T. J. Watson Research Center. Retrieved from http://www.research.ibm.com/projects/wsmp.
[38]
Anshul Gupta and Thomas George. 2010. Adaptive techniques for improving the performance of incomplete factorization preconditioning. SIAM Journal on Scientific Computing 32, 1 (2010), 84–110.
[39]
Anshul Gupta, Mahesh Joshi, and Vipin Kumar. 2001. WSMP: A High-performance Shared- and Distributed-memory Parallel Sparse Linear Equation Solver. Technical Report. IBM T. J. Watson Research Center.
[40]
Ken Hayami, Jun-Feng Yin, and Tokushi Ito. 2010. GMRES methods for least squares problems. SIAM Journal on Matrix Analysis and Applications 31, 5 (2010), 2400–2430.
[41]
P. Heggernes, S. Eisenstat, G. Kumfert, and A. Pothen. 2001. The computational complexity of the minimum degree algorithm. In Proceedings of the 14th Norwegian Computer Science Conference. 98–109.
[42]
Michael A. Heroux, Roscoe A. Bartlett, Vicki E. Howle, Robert J. Hoekstra, Jonathan J. Hu, Tamara G. Kolda, Richard B. Lehoucq, Kevin R. Long, Roger P. Pawlowski, Eric T. Phipps, Andrew G. Salinger, Heidi K. Thornquist, Ray S. Tuminaro, James M. Willenbring, Alan Williams, and Kendall S. Stanley. 2005. An overview of the Trilinos project. ACM Transactions on Mathematical Software 31, 3 (2005), 397–423.
[43]
Magnus Rudolph Hestenes and Eduard Stiefel. 1952. Methods of conjugate gradients for solving linear systems. Journal of research of the National Bureau of Standards 49, 1 (1952), 409–436.
[44]
Charles A. R. Hoare. 1961. Algorithm 65: Find. Communications of the ACM 4, 7 (1961), 321–322.
[45]
hypre Developers. 2021. hypre Documentation Release 2.21.0. Lawrence Livermore National Laboratory.
[46]
Klaus Iglberger, Georg Hager, Jan Treibig, and Ulrich Rüde. 2012. Expression templates revisited: A performance analysis of current methodologies. SIAM Journal on Scientific Computing 34, 2 (2012), C42–C69.
[47]
A. Jennings and M. A. Ajiz. 1984. Incomplete methods for solving \(A^{T}Ax=b\). SIAM Journal on Scientific Computing 5, 4 (1984), 978–987.
[48]
Xiangmin Jiao and Qiao Chen. 2022. Approximate generalized inverses with iterative refinement for \(\epsilon\)-accurate preconditioning of singular systems. SIAM Journal on Matrix Analysis and Applications 43, 1 (2022), 40–67.
[49]
Xiangmin Jiao, Xuebin Wang, and Qiao Chen. 2021. Optimal and low-memory near-optimal preconditioning of fully implicit Runge-Kutta schemes for parabolic PDEs. SIAM Journal on Scientific Computing 43, 5 (2021), A3527–A3551. DOI:
[50]
Alex Kanevsky, Mark H. Carpenter, David Gottlieb, and Jan S. Hesthaven. 2007. Application of implicit-explicit high order Runge–Kutta methods to discontinuous-Galerkin schemes. Journal of Computational Physics 225, 2 (2007), 1753–1781. DOI:
[51]
David Kay, Daniel Loghin, and Andrew Wathen. 2002. A preconditioner for the steady-state Navier–Stokes equations. SIAM Journal on Scientific Computing 24, 1 (2002), 237–256.
[52]
Miroslav Kuchta, Kent-Andre Mardal, and Mikael Mortensen. 2019. On the singular Neumann problem in linear elasticity. Numerical Linear Algebra with Applications 26, 1 (2019), e2212.
[53]
STFC Rutherford Appleton Laboratory. 2021. HSL_MC64. Permute and scale a sparse unsymmetric or rectangular matrix to put large entries on the diagonal. (2021). Retrieved from https://www.hsl.rl.ac.uk/catalogue/hsl_mc64.html. Accessed: 2021-3-16
[54]
Na Li and Yousef Saad. 2005. Crout versions of ILU factorization with pivoting for sparse symmetric matrices. Electronic Transactions on Numerical Analysis 20 (2005), 75–85.
[55]
Na Li, Yousef Saad, and Edmond Chow. 2003. Crout versions of ILU for general sparse matrices. SIAM Journal on Scientific Computing 25, 2 (2003), 716–728.
[56]
Xiaoye S. Li. 2005. An overview of SuperLU: Algorithms, implementation, and user interface. ACM Transactions on Mathematical Software 31, 3 (2005), 302–325.
[57]
Xiaoye S. Li and Meiyue Shao. 2011. A supernodal approach to incomplete LU factorization with partial pivoting. ACM Transactions on Mathematical Software 37, 4 (2011), 1–20.
[58]
Anders Logg and Garth N. Wells. 2010. DOLFIN: Automated finite element computing. ACM Transactions on Mathematical Software 37, 2 (2010), 1–28.
[59]
Jan Mayer. 2006. Alternative weighted dropping strategies for ILUTP. SIAM Journal on Scientific Computing 27, 4 (2006), 1424–1437.
[60]
Jan Mayer. 2007. ILU++: A new software package for solving sparse linear systems with iterative methods. PAMM: Proceedings in Applied Mathematics and Mechanics. Wiley Online Library, 2020123–2020124.
[61]
Keiichi Morikuni and Ken Hayami. 2015. Convergence of inner-iteration GMRES methods for rank-deficient least squares problems. SIAM Journal on Matrix Analysis and Applications 36, 1 (2015), 225–250.
[62]
Johann Moulin, Pierre Jolivet, and Olivier Marquet. 2019. Augmented Lagrangian preconditioner for large-scale hydrodynamic stability analysis. Computer Methods in Applied Mechanics and Engineering 351 (2019), 718–743.
[63]
Christopher C. Paige and Michael A. Saunders. 1975. Solution of sparse indefinite systems of linear equations. SIAM Journal on Numerical Analysis 12, 4 (1975), 617–629.
[64]
Christopher C. Paige and Michael A. Saunders. 1982. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software 8, 1 (1982), 43–71.
[65]
Will Pazner and Per-Olof Persson. 2017. Stage-parallel fully implicit Runge–Kutta solvers for discontinuous Galerkin fluid simulations. Journal of Computational Physics 335 (2017), 700–717. DOI:DOI:
[66]
George Poole and Larry Neal. 2000. The rook’s pivoting strategy. Journal of Computational and Applied Mathematics 123, 1–2 (2000), 353–369.
[67]
C. Radhakrishna Rao and Sujit Kumar Mitra. 1972. Generalized inverse of a matrix and its applications. In Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Theory of Statistics. The Regents of the University of California, 601–620.
[68]
Youcef Saad. 1988. Preconditioning techniques for nonsymmetric and indefinite linear systems. Journal of Computational and Applied Mathematics 24, 1–2 (1988), 89–105.
[69]
Youcef Saad. 1993. A flexible inner-outer preconditioned GMRES algorithm. SIAM Journal on Scientific Computing 14, 2 (1993), 461–469.
[70]
Yousef Saad. 2003. Iterative Methods for Sparse Linear Systems. SIAM.
[71]
Youcef Saad and Martin H. Schultz. 1986. GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM Journal on Scientific Computing 7, 3 (1986), 856–869.
[72]
Yousef Saad and Brian Suchomel. 2002. ARMS: An algebraic recursive multilevel solver for general sparse linear systems. Numerical Linear Algebra with Applications 9, 5 (2002), 359–378.
[73]
Michael Schäfer, Stefan Turek, Franz Durst, Egon Krause, and Rolf Rannacher. 1996. Benchmark computations of laminar flow around a cylinder. In Proceedings of the Flow Simulation with High-Performance Computers II. Springer, 547–566.
[74]
David Silvester, Howard Elman, David Kay, and Andrew Wathen. 2001. Efficient preconditioning of the linearized Navier–Stokes equations for incompressible flow. Journal of Computational and Applied Mathematics 128, 1–2 (2001), 261–279.
[75]
Barry F. Smith, Petter E. Bjørstad, and William D. Gropp. 1996. Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press.
[76]
Cedric Taylor and Paul Hood. 1973. A numerical solution of the Navier–Stokes equations using the finite element technique. Computers & and Fluids 1, 1 (1973), 73–100.
[77]
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C. J. Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. 2020. SciPy 1.0: Fundamental algorithms for scientific computing in python. Nature Methods 17, 3 (2020), 261–272. DOI:
Index Terms
- HIFIR: Hybrid Incomplete Factorization with Iterative Refinement for Preconditioning Ill-Conditioned and Singular Systems
Recommendations
Approximate Generalized Inverses with Iterative Refinement for $\epsilon$-Accurate Preconditioning of Singular Systems
We introduce a new class of preconditioners to enable flexible GMRES to find a least-squares solution, and potentially the pseudoinverse solution, of large-scale sparse, asymmetric, singular, and potentially inconsistent systems. We develop the ...
Parallel Hybrid Preconditioning: Incomplete Factorization with Selective Sparse Approximate Inversion
We consider parallel preconditioning to solve large sparse linear systems $Ax=b$ using conjugate gradients when $A$ is symmetric and positive definite. We develop a preconditioner that can be viewed as a hybrid of incomplete factorization and sparse ...
Multistep matrix splitting iteration preconditioning for singular linear systems
Multistep matrix splitting iterations serve as preconditioning for Krylov subspace methods for solving singular linear systems. The preconditioner is applied to the generalized minimal residual (GMRES) method and the flexible GMRES (FGMRES) method. We ...
Comments
Information & Contributors
Information
Published In
September 2022
357 pages
ISSN:0098-3500
EISSN:1557-7295
DOI:10.1145/3551652
- Editors:
- Zhaojun Bai,
- Wolfgang Bangerth
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 September 2022
Online AM: 10 May 2022
Accepted: 02 May 2022
Revised: 30 November 2021
Received: 17 June 2021
Published in TOMS Volume 48, Issue 3
Check for updates
Author Tags
Qualifiers
- Research-article
- Refereed
Funding Sources
- Empire State Development grant NYS
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 247Total Downloads
- Downloads (Last 12 months)42
- Downloads (Last 6 weeks)1
Reflects downloads up to 02 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format