Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods

doi:10.1007/s42967-023-00315-4

Communications on Applied Mathematics and Computation ›› 2024, Vol. 6 ›› Issue (1): 705-738.doi: 10.1007/s42967-023-00315-4

• ORIGINAL PAPERS • Previous Articles Next Articles

Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods

Ben Burnett¹, Sigal Gottlieb¹, Zachary J. Grant²

1. Center for Scientific Computing and Data Science Research, UMass Dartmouth, North Dartmouth, Massachusetts, USA;
2. CSME, Michigan State University, East Lansing, Michigan, USA

Received:2022-12-16 Revised:2023-09-07 Published:2024-04-16
Contact: Sigal Gottlieb,E-mail:sgottlieb@umassd.edu;Ben Burnett,E-mail:bburnett@umassd.edu;Zachary J. Grant,E-mail:zack.j.grant@gmail.com E-mail:sgottlieb@umassd.edu;bburnett@umassd.edu;zack.j.grant@gmail.com
Supported by:
Burnett and Gottlieb’s work was partially supported by ONR UMass Dartmouth Marine and UnderSea Technology (MUST) grant N00014-20-1-2849 under the project S31320000049160, by DOE grant DE-SC0023164 sub-award RC114586-UMD, and by AFOSR grants FA9550-18-1-0383 and FA9550-23-1-0037. Grant’s work was partially supported by Michigan State University, by AFOSR grants FA9550-19-1-0281 and FA9550-18-1-0383 and by DOE grant DE-SC0023164.

Abstract

Abstract: Additive Runge-Kutta methods designed for preserving highly accurate solutions in mixed-precision computation were previously proposed and analyzed. These specially designed methods use reduced precision for the implicit computations and full precision for the explicit computations. In this work, we analyze the stability properties of these methods and their sensitivity to the low-precision rounding errors, and demonstrate their performance in terms of accuracy and efficiency. We develop codes in FORTRAN and Julia to solve nonlinear systems of ODEs and PDEs using the mixed-precision additive Runge-Kutta (MP-ARK) methods. The convergence, accuracy, and runtime of these methods are explored. We show that for a given level of accuracy, suitably chosen MP-ARK methods may provide significant reductions in runtime.

Key words: Mixed precision, Runge-Kutta methods, Additive methods, Accuracy

Ben Burnett, Sigal Gottlieb, Zachary J. Grant. Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods[J]. Communications on Applied Mathematics and Computation, 2024, 6(1): 705-738.

TrendMD

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://www.camc.shu.edu.cn/EN/10.1007/s42967-023-00315-4

https://www.camc.shu.edu.cn/EN/Y2024/V6/I1/705

References

[1] Abdelfattah, A., Anzt, H., Boman, E., Carson, E., Cojean, T., Dongarra, J., Fox, A., Gates, M., Higham, N.J., Li, X.S., Liu, Y., Loe, J., Luszczek, P., Pranesh, S., Rajamanickam, S., Ribizel, T., Smith, B., Swirydowicz, K., Thomas, S., Tomov, S., Tzai, M., Yamazaki, I., Yang, U.M.:A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. Int. J. High Performance Comput. Appl. 35(4), 344-369 (2021)
[2] Abdelfattah, A., Tomov, S., Dongarra, J.J.:Fast batched matrix multiplication for small sizes using half-precision arithmetic on GPUs. In:2019 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019, Rio de Janeiro, Brazil, May 20-24, pp. 111-122. IEEE (2019)
[3] Alexander, R.:Diagonally implicit Runge-Kutta methods for stiff ODEs. SIAM J. Numer. Anal. 14(6), 1006-1021 (1977)
[4] Burnett, B., Gottlieb, S., Grant, Z.J., Heryudono, A.:Evaluation, performance, of mixed-precision Runge-Kutta methods. In:IEEE High Performance Extreme Computing Conference (HPEC). Waltham, MA, USA 2021, 1-6 (2021). https://doi.org/10.1109/HPEC49654.2021.9622803
[5] Butcher, J.C.:Numerical Methods for Ordinary Differential Equations. Wiley, Hoboken (2016)
[6] Butcher, J.C.:B-series:algebraic analysis of numerical methods. In:Springer Series in Computational Mathematics, 55. Springer, Cham, Switzerland (2021)
[7] Croci, M., Fasi, M., Higham, N.J., Mary, T., Mikaitis, M.:Stochastic rounding:implementation, error analysis and applications. R. Soc. Open Sci. 9(3), 211631 (2021)
[8] Croci, M., Giles, M.B.:Effects of round-to-nearest and stochastic rounding in the numerical solution of the heat equation in low precision. IMA J. Numer. Anal. 43(3), 1358-1390 (2023)
[9] Croci, M., Rosilho de Souza, G.:Mixed-precision explicit stabilized Runge-Kutta methods for single-and multi-scale differential equations. J. Comput. Phys. 464(1), 111349 (2022)
[10] Grant, Z.J.:Perturbed Runge-Kutta methods for mixed precision applications. J. Sci. Comput. 92(1), 1-20 (2022)
[11] Gupta, S., Agrawal, A., Gopalakrishnan, K., Narayanan, P.:Deep learning with limited numerical precision. In:Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37, ICML'15, pp. 1737-1746. JMLR.org (2015)
[12] Haidar, A., Tomov, S., Dongarra, J., Higham, N.J.:Harnessing GPU tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. In:Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC'18, pp. 47:1-47:11, Piscataway, NJ, USA. IEEE Press (2018)
[13] Higham, N.J.:Error analysis for standard and GMRES-based iterative refinement in two and three-precisions. MIMS EPrint 2019.19, Manchester Institute for Mathematical Sciences, The University of Manchester, November (2019)
[14] Higham, N.J., Mary, T.:A new approach to probabilistic rounding error analysis. SIAM J. Sci. Comput. 41(5), 2815-2835 (2019)
[15] Higham, N.J., Pranesh, S.:Simulating low precision floating-point arithmetic. SIAM J. Sci. Comput. 41(5), C585-C602 (2019)
[16] Higham, N.J., Pranesh, S., Zounon, M.:Squeezing a matrix into half precision, with an application to solving linear systems. SIAM J. Sci. Comput. 41(4), A2536-A2551 (2019)
[17] Higueras, I., Ketcheson, D.I., Kocsis, T.A.:Optimal monotonicity-preserving perturbations of a given Runge-Kutta method. J. Sci. Comput. 76(3), 1337-1369 (2018)
[18] Kennedy, C.A., Carpenter, M.H.:Additive Runge-Kutta schemes for convection-diffusion-reaction equations. Appl. Numer. Math. 44(1/2), 139-181 (2003)
[19] Paxton, E.A., Chantry, M., Klöwer, M., Saffin, L., Palmer, T.:Climate modeling in low precision:effects of both deterministic and stochastic rounding. J. Clim. 35(4), 1215-1229 (2022)
[20] Petschow, M., Quintana-Ort, E.S., Bientinesi, P.:Improved accuracy and parallelism for MRRR-based eigensolvers-a mixed precision approach. SIAM J. Sci. Comput. 36(2), C240-C263 (2014). https://doi.org/10.1137/130911561
[21] Richter, C., Schops, S., Clemens, M.:GPU-accelerated mixed-precision algebraic multigrid preconditioners for discrete elliptic field problems. IEEE Transact. Magnetics 50(2), 83-90 (2014)
[22] Sandu, A., Gunther, M.:A generalized-structure approach to additive Runge-Kutta methods. SIAM J. Numer. Anal. 53(1), 17-42 (2015)
[23] Zounon, M., Higham, N.J., Lucas, C., Tisseur, F.:Performance impact of precision reduction in sparse linear systems solvers. J. Comput. Sci. 8, e778 (2022)

Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Tarek Aboelenen. Efficient and Accurate Spectral Method for Solving Fractional Differential Equations on the Half Line Using Orthogonal Generalized Rational Jacobi Functions [J]. Communications on Applied Mathematics and Computation, 2025, 7(4): 1419-1443.
[2]	Guangyao Zhu, Yan Jiang, Mengping Zhang. Inverse Lax-Wendroff Boundary Treatment for Solving Conservation Laws with Finite Volume Methods [J]. Communications on Applied Mathematics and Computation, 2025, 7(3): 885-909.
[3]	Lei Yang, Shun Li, Yan Jiang, Chi-Wang Shu, Mengping Zhang. Inverse Lax-Wendroff Boundary Treatment of Discontinuous Galerkin Method for 1D Conservation Laws [J]. Communications on Applied Mathematics and Computation, 2025, 7(2): 796-826.
[4]	Mengjiao Jiao, Yan Jiang, Mengping Zhang. A Provable Positivity-Preserving Local Discontinuous Galerkin Method for the Viscous and Resistive MHD Equations [J]. Communications on Applied Mathematics and Computation, 2024, 6(1): 279-310.
[5]	Zepeng Liu, Yan Jiang, Mengping Zhang, Qingyuan Liu. High Order Finite Difference WENO Methods for Shallow Water Equations on Curvilinear Meshes [J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 485-528.
[6]	Kunlei Zhao, Yulong Du, Li Yuan. A New Sixth-Order WENO Scheme for Solving Hyperbolic Conservation Laws [J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 3-30.
[7]	Hendrik Ranocha, Lisandro Dalcin, Matteo Parsani, David I. Ketcheson. Optimized Runge-Kutta Methods with Automatic Step Size Control for Compressible Computational Fluid Dynamics [J]. Communications on Applied Mathematics and Computation, 2022, 4(4): 1191-1228.
[8]	Xiaozhou Li. How to Design a Generic Accuracy-Enhancing Filter for Discontinuous Galerkin Methods [J]. Communications on Applied Mathematics and Computation, 2022, 4(3): 759-782.
[9]	Jennifer K. Ryan. Capitalizing on Superconvergence for More Accurate Multi-Resolution Discontinuous Galerkin Methods [J]. Communications on Applied Mathematics and Computation, 2022, 4(2): 417-436.
[10]	Qi Tao, Yan Xu, Xiaozhou Li. Negative Norm Estimates for Arbitrary Lagrangian-Eulerian Discontinuous Galerkin Method for Nonlinear Hyperbolic Equations [J]. Communications on Applied Mathematics and Computation, 2022, 4(1): 250-270.
[11]	Liyao Lyu, Zheng Chen. Local Discontinuous Galerkin Methods with Novel Basis for Fractional Difusion Equations with Non-smooth Solutions [J]. Communications on Applied Mathematics and Computation, 2022, 4(1): 227-249.
[12]	Giacomo Albi, Lorenzo Pareschi. High Order Semi-implicit Multistep Methods for Time-Dependent Partial Diferential Equations [J]. Communications on Applied Mathematics and Computation, 2021, 3(4): 701-718.
[13]	F. L. Romeo, M. Dumbser, M. Tavelli. A Novel Staggered Semi-implicit Space-Time Discontinuous Galerkin Method for the Incompressible Navier-Stokes Equations [J]. Communications on Applied Mathematics and Computation, 2021, 3(4): 607-647.
[14]	Daniel J. Frean, Jennifer K. Ryan. Superconvergence and the Numerical Flux: a Study Using the Upwind-Biased Flux in Discontinuous Galerkin Methods [J]. Communications on Applied Mathematics and Computation, 2020, 2(3): 461-486.
[15]	Christiane Helzel. A Third-Order Accurate Wave Propagation Algorithm for Hyperbolic Partial Diferential Equations [J]. Communications on Applied Mathematics and Computation, 2020, 2(3): 403-427.