A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations

doi:10.1007/s42967-021-00129-2

Communications on Applied Mathematics and Computation ›› 2023, Vol. 5 ›› Issue (1): 97-115.doi: 10.1007/s42967-021-00129-2

• ORIGINAL PAPERS • 上一篇下一篇

A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations

Scott E. Field¹, Sigal Gottlieb¹, Zachary J. Grant², Leah F. Isherwood¹, Gaurav Khanna^1,3

1 University of Massachusetts Dartmouth, North Dartmouth, MA 02747, USA;
2 Department of Computational and Applied Mathematics, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA;
3 University of Rhode Island, Kingston, RI 02881, USA

收稿日期:2020-10-02 修回日期:2021-01-22 出版日期:2023-03-20 发布日期:2023-03-08
通讯作者: Gaurav Khanna,E-mail:gkhanna@umassd.edu E-mail:gkhanna@umassd.edu
基金资助:
Many of the computations were performed on the MIT/IBM Satori GPU supercomputer supported by the Massachusetts Green High Performance Computing Center (MGHPCC) and Oak Ridge National Laboratory (ORNL)’s Summit leadership-class supercomputer under Director Discretionary allocation AST166. We thank Sebastiano Bernuzzi for helpful discussions. The authors acknowledge support of NSF Grants No. PHY-2010685 (G.K) and No. DMS-1912716 (S.F, S.G, and G.K), AFOSR Grant No. FA9550-18-1-0383 (S.G) and Office of Naval Research/Defense University Research Instrumentation Program (ONR/DURIP) Grant No. N00014181255. This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while a subset of the authors were in residence at the Institute for Computational and Experimental Research in Mathematics in Providence, RI, during the Advances in Computational Relativity program. A part of this research is sponsored by the Office of Advanced Scientific Computing Research; US Department of Energy, and was performed at the Oak Ridge National Laboratory, which is managed by UT-Battelle, LLC under Contract no. De-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.

A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations

Scott E. Field¹, Sigal Gottlieb¹, Zachary J. Grant², Leah F. Isherwood¹, Gaurav Khanna^1,3

1 University of Massachusetts Dartmouth, North Dartmouth, MA 02747, USA;
2 Department of Computational and Applied Mathematics, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA;
3 University of Rhode Island, Kingston, RI 02881, USA

Received:2020-10-02 Revised:2021-01-22 Online:2023-03-20 Published:2023-03-08
Contact: Gaurav Khanna,E-mail:gkhanna@umassd.edu E-mail:gkhanna@umassd.edu
Supported by:
Many of the computations were performed on the MIT/IBM Satori GPU supercomputer supported by the Massachusetts Green High Performance Computing Center (MGHPCC) and Oak Ridge National Laboratory (ORNL)’s Summit leadership-class supercomputer under Director Discretionary allocation AST166. We thank Sebastiano Bernuzzi for helpful discussions. The authors acknowledge support of NSF Grants No. PHY-2010685 (G.K) and No. DMS-1912716 (S.F, S.G, and G.K), AFOSR Grant No. FA9550-18-1-0383 (S.G) and Office of Naval Research/Defense University Research Instrumentation Program (ONR/DURIP) Grant No. N00014181255. This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while a subset of the authors were in residence at the Institute for Computational and Experimental Research in Mathematics in Providence, RI, during the Advances in Computational Relativity program. A part of this research is sponsored by the Office of Advanced Scientific Computing Research; US Department of Energy, and was performed at the Oak Ridge National Laboratory, which is managed by UT-Battelle, LLC under Contract no. De-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.

摘要/Abstract

摘要： We develop and use a novel mixed-precision weighted essentially non-oscillatory (WENO) method for solving the Teukolsky equation, which arises when modeling perturbations of Kerr black holes. We show that WENO methods outperform higher-order finite-difference methods, standard in the discretization of the Teukolsky equation, due to the need to add dissipation for stability purposes in the latter. In particular, as the WENO scheme uses no additional dissipation, it is well suited for scenarios requiring long-time evolution such as the study of price tails and gravitational wave emission from extreme mass ratio binaries. In the mixed-precision approach, the expensive computation of the WENO weights is performed in reduced floating-point precision that results in a significant speedup factor of ≈ 3.3. In addition, we use state-of-the-art Nvidia general-purpose graphics processing units and cluster parallelism to further accelerate the WENO computations. Our optimized WENO solver can be used to quickly generate accurate results of significance in the field of black hole and gravitational wave physics. We apply our solver to study the behavior of the Aretakis charge—a conserved quantity, that if detected by a gravitational wave observatory like LIGO/Virgo would prove the existence of extremal black holes.

关键词: Numerical methods, Finite differencing, Hyperbolic, Partial differential equations, Black holesblack holes, WENO

Abstract: We develop and use a novel mixed-precision weighted essentially non-oscillatory (WENO) method for solving the Teukolsky equation, which arises when modeling perturbations of Kerr black holes. We show that WENO methods outperform higher-order finite-difference methods, standard in the discretization of the Teukolsky equation, due to the need to add dissipation for stability purposes in the latter. In particular, as the WENO scheme uses no additional dissipation, it is well suited for scenarios requiring long-time evolution such as the study of price tails and gravitational wave emission from extreme mass ratio binaries. In the mixed-precision approach, the expensive computation of the WENO weights is performed in reduced floating-point precision that results in a significant speedup factor of ≈ 3.3. In addition, we use state-of-the-art Nvidia general-purpose graphics processing units and cluster parallelism to further accelerate the WENO computations. Our optimized WENO solver can be used to quickly generate accurate results of significance in the field of black hole and gravitational wave physics. We apply our solver to study the behavior of the Aretakis charge—a conserved quantity, that if detected by a gravitational wave observatory like LIGO/Virgo would prove the existence of extremal black holes.

Key words: Numerical methods, Finite differencing, Hyperbolic, Partial differential equations, Black holesblack holes, WENO

中图分类号:

Scott E. Field, Sigal Gottlieb, Zachary J. Grant, Leah F. Isherwood, Gaurav Khanna. A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 97-115.

参考文献

1. Abbott, B.P., et al.: Observation of gravitational waves from a binary black hole merger. Phys. Rev. Lett. 116(6), 061102 (2016)
2. Abbott, B.P., et al.: Tests of general relativity with GW150914. Phys. Rev. Lett. 116(22), 221101 (2016)
3. Abbott, B.P., et al.: GWTC-1: a gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs. Phys. Rev. X9(3), 031040 (2019)
4. Akiyama, K., et al.: First m87 event horizon telescope results. IV. Imaging the central supermassive black hole. Astrophys. J. Lett. 875(1), L4 (2019)
5. Angelopoulos, Y., Aretakis, S., Gajic, D.: Horizon hair of extremal black holes and measurements at null infinity. Phys. Rev. Lett. 121(13), 131102 (2018)
6. Aretakis, S.: A note on instabilities of extremal black holes under scalar perturbations from afar. Class. Quantum Grav. 30(9), 095010 (2013)
7. Burko, L.M., Khanna, G., Zenginoğlu, A.: Cauchy-horizon singularity inside perturbed Kerr black holes. Phys. Rev. D 93(4), 041501 (2016). [Erratum: Phys. Rev. D 96, 129903 (2017)]
8. Burko, L.M., Khanna, G., Sabharwal, S.: (Transient) Scalar hair for (nearly) extreme black holes. Phys. Rev. Res. 1, 033106 (2019)
9. Burko, L.M., Khanna, G., Sabharwal, S.: Scalar and gravitational hair for extreme Kerr black holes. arXiv: 2005. 07294 (2020)
10. Dong, B., Gottlieb, S., Hristova, Y., Jiang, Y,. Wang, H.: The effect of the sensitivity parameter in weighted essentially non-oscillatory methods. In: Brenner, S. (ed.) Topics in Numerical Partial Differential Equations and Scientific Computing. The IMA Volumes in Mathematics and Its Applications, vol. 160, pp. 23–50. Springer, New York, NY (2016)
11. Gottlieb, S., Ketcheson, D.I., Shu, C.-W.: Strong Stability Preserving Runge-Kutta and Multistep Time Discretizations. World Scientific, Singapore (2011)
12. Harms, E., Bernuzzi, S., Brügmann, B.: Numerical solution of the 2+1 Teukolsky equation on a hyperboloidal and horizon penetrating foliation of Kerr and application to late-time decays. Class. Quantum Grav. 30(11), 115013 (2013)
13. Harms, E., Bernuzzi, S., Nagar, A., Zenginoğlu, A.: A new gravitational wave generation algorithm for particle perturbations of the Kerr spacetime. Class. Quantum. Grav. 31(24), 245004 (2014)
14. Jiang, G.-S., Shu, C.-W.: Efficient implementation of weighted ENO schemes. J. Comput. Phys. 126(1), 202–228 (1996)
15. Khanna, G..: High-precision numerical simulations on a CUDA GPU: Kerr black hole tails. J. Sci. Comput. 56(2), 366–380 (2013)
16. Kreiss, H.O., Oliger, J.: Methods for the approximate solution of time dependent problems. Global Atm. Res. Prog. Publ. 10, GARP (1973)
17. Poisson, E.: Absorption of mass and angular momentum by a black hole: time-domain formalisms for gravitational perturbations, and the small-hole or slow-motion approximation. Phys. Rev. D 70(8), 084044 (2004)
18. Regge, T., Wheeler, J.A.: Stability of a Schwarzschild singularity. Phys. Rev. 108, 1063–1069 (1957)
19. Sundararajan, P.A., Khanna, G., Hughes, S.A.: Towards adiabatic waveforms for inspiral into Kerr black holes: a new model of the source for the time domain perturbation equation. Phys. Rev. D 76(10), 104005 (2007)
20. Sundararajan, P.A., Khanna, G., Hughes, S.A., Drasco, S.: Towards adiabatic waveforms for inspiral into Kerr black holes. II. Dynamical sources and generic orbits. Phys. Rev. D 78(2), 024022 (2008)
21. Sundararajan, P.A., Khanna, G., Hughes, S.A.: Binary black hole merger gravitational waves and recoil in the large mass ratio limit. Phys. Rev. D 81(10), 104009 (2010)
22. Teukolsky, S.A.: Perturbations of a rotating black hole. I. Fundamental equations for gravitational electromagnetic and neutrino field perturbations. Astrophys. J. 185, 635–647 (1973)
23. Wysocki, D., Lange, J., O’Shaughnessy, R.: Reconstructing phenomenological distributions of compact binaries via gravitational wave observations. Phys. Rev. D 100(4), 043012 (2019)
24. Zenginoğlu, A.: Hyperboloidal evolution with the Einstein equations. Class. Quantum Grav. 25, 195025 (2008)
25. Zenginoğlu, A.: A hyperboloidal study of tail decay rates for scalar and Yang-Mills fields. Class. Quantum Grav. 25, 175013 (2008)
26. Zenginoğlu, A.: Hyperboloidal foliations and scri-fixing. Class. Quantum Grav. 25, 145002 (2008)
27. Zenginoğlu, A.: Asymptotics of black hole perturbations. Class. Quantum Grav. 27, 045015 (2010)
28. Zenginoğlu, A.: A geometric framework for black hole perturbations. Phys. Rev. D 83, 127502 (2011)
29. Zenginoğlu, A.: Hyperboloidal layers for hyperbolic equations on unbounded domains. J. Comput. Phys. 230, 2286–2302 (2011)
30. Zenginoğlu, A., Khanna, G.: Null infinity waveforms from extreme-mass-ratio inspirals in Kerr spacetime. Phys. Rev. X 1(2), 021017 (2011)
31. Zenginoğlu, A., Khanna, G., Burko, L.M.: Intermediate behavior of Kerr tails. Gen. Relativ. Gravit. 46, 1672 (2014)
32. Zenginoğlu, A., Nunez, D., Husa, S.: Gravitational perturbations of Schwarzschild spacetime at null infinity and the hyperboloidal initial value problem. Class. Quantum Grav. 26, 035009 (2009)
33. Zenginoğlu, A., Tiglio, M.: Spacelike matching to null infinity. Phys. Rev. D 80, 024044 (2009)

[1]	Kunlei Zhao, Yulong Du, Li Yuan. A New Sixth-Order WENO Scheme for Solving Hyperbolic Conservation Laws[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 3-30.
[2]	Jun Zhu, Jianxian Qiu. New Finite Difference Mapped WENO Schemes with Increasingly High Order of Accuracy[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 64-96.
[3]	Andrew Christlieb, Matthew Link, Hyoseon Yang, Ruimeng Chang. High-Order Semi-Lagrangian WENO Schemes Based on Non-polynomial Space for the Vlasov Equation[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 116-142.
[4]	M. Semplice, E. Travaglia, G. Puppo. One- and Multi-dimensional CWENOZ Reconstructions for Implementing Boundary Conditions Without Ghost Cells[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 143-169.
[5]	Linjin Li, Jingmei Qiu, Giovanni Russo. A High-Order Semi-Lagrangian Finite Difference Method for Nonlinear Vlasov and BGK Models[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 170-198.
[6]	Yifei Wan, Yinhua Xia. A New Hybrid WENO Scheme with the High-Frequency Region for Hyperbolic Conservation Laws[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 199-234.
[7]	Bao-Shan Wang, Wai Sun Don, Alexander Kurganov, Yongle Liu. Fifth-Order A-WENO Schemes Based on the Adaptive Diffusion Central-Upwind Rankine-Hugoniot Fluxes[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 295-314.
[8]	Xucheng Meng, Yaguang Gu, Guanghui Hu. A Fourth-Order Unstructured NURBS-Enhanced Finite Volume WENO Scheme for Steady Euler Equations in Curved Geometries[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 315-342.
[9]	G. Puppo, M. Semplice, G. Visconti. Quinpi: Integrating Conservation Laws with CWENO Implicit Methods[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 343-369.
[10]	R. Abgrall. A Combination of Residual Distribution and the Active Flux Formulations or a New Class of Schemes That Can Combine Several Writings of the Same Hyperbolic Problem: Application to the 1D Euler Equations[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 370-402.
[11]	Liang Li, Jun Zhu, Chi-Wang Shu, Yong-Tao Zhang. A Fixed-Point Fast Sweeping WENO Method with Inverse Lax-Wendroff Boundary Treatment for Steady State of Hyperbolic Conservation Laws[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 403-427.
[12]	Zepeng Liu, Yan Jiang, Mengping Zhang, Qingyuan Liu. High Order Finite Difference WENO Methods for Shallow Water Equations on Curvilinear Meshes[J]. Communications on Applied Mathematics and Computation, 2023, 5(1): 485-528.
[13]	Qi Tao, Yan Xu, Xiaozhou Li. Negative Norm Estimates for Arbitrary Lagrangian-Eulerian Discontinuous Galerkin Method for Nonlinear Hyperbolic Equations[J]. Communications on Applied Mathematics and Computation, 2022, 4(1): 250-270.
[14]	Yuan Xu, Qiang Zhang. Superconvergence Analysis of the Runge-Kutta Discontinuous Galerkin Method with Upwind-Biased Numerical Flux for Two-Dimensional Linear Hyperbolic Equation[J]. Communications on Applied Mathematics and Computation, 2022, 4(1): 319-352.
[15]	Zheng Sun, Chi-Wang Shu. Enforcing Strong Stability of Explicit Runge-Kutta Methods with Superviscosity[J]. Communications on Applied Mathematics and Computation, 2021, 3(4): 671-700.

A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations

A GPU-Accelerated Mixed-Precision WENO Method for Extremal Black Hole and Gravitational Wave Physics Computations

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价