Discovery of Governing Equations with Recursive Deep Neural Networks

doi:10.1007/s42967-023-00270-0

Abstract

Abstract: Model discovery based on existing data has been one of the major focuses of mathematical modelers for decades. Despite tremendous achievements in model identification from adequate data, how to unravel the models from limited data is less resolved. This paper focuses on the model discovery problem when the data is not efficiently sampled in time, which is common due to limited experimental accessibility and labor/resource constraints. Specifically, we introduce a recursive deep neural network (RDNN) for data-driven model discovery. This recursive approach can retrieve the governing equation efficiently and significantly improve the approximation accuracy by increasing the recursive stages. In particular, our proposed approach shows superior power when the existing data are sampled with a large time lag, from which the traditional approach might not be able to recover the model well. Several examples of dynamical systems are used to benchmark this newly proposed recursive approach. Numerical comparisons confirm the effectiveness of this recursive neural network for model discovery. The accompanying codes are available at https:// github. com/ c2fd/ RDNNs.

Key words: Deep neural networks (DNNs), Recursive, Model discovery, Dynamical systems

CLC Number:

Jarrod Mau, Jia Zhao. Discovery of Governing Equations with Recursive Deep Neural Networks[J]. Communications on Applied Mathematics and Computation, 2025, 7(1): 239-263.

TrendMD

References

1. Anitescu, C., Atroshchenko, E., Alajlan, N., Rabczuk, T.: Artificial neural network methods for the solution of second order boundary value problems. Comput. Mater. Contin. 59(1), 345–359 (2019)
2. Berg, J., Nystrom, K.: A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41 (2018)
3. Berg, J., Nystrom, K.: Data-driven discovery of PDEs in complex datasets. J. Comput. Phys. 284, 239– 252 (2019)
4. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. 113(15), 3932–3937 (2016)
5. Chen, Z., Xiu, D.: On generalized residue network for deep learning of unknown dynamical systems. J. Comput. Phys. 438, 110362 (2021) 6. Daniels, B.C., Nemenman, I.: Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression. PLoS One 10(3), e0119821 (2015)
7. Dockhorn, T.: A discussion on solving partial differential equations using neural networks. arXiv:
1904. 07200 (2019)
8. E, W., Yu, B.: The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 1–12 (2018)
9. Han, J., Jentzen, A., E, W.: Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. 115(34), 8505–8510 (2018)
10. Higham, C., Higham, D.: Deep learning: an introduction for applied mathematicians. SIAM Rev. 61(4), 860–891 (2019)
11. Jin, P., Zhu, A., Karniadakis, G.E., Tang, Y.: Symplectic networks: intrinsic structure-preserving networks for identifying Hamiltonian systems. Neural Netw. 132, 166–179 (2020)
12. Koopman, B.O., Neumann, J.V.: Dynamical systems of continuous spectra. Proc. Natl. Acad. Sci. USA 18(3), 255–263 (1932)
13. Kutz, J.N., Fu, X., Brunton, S.L.: Multi-resolution dynamic mode decomposition. SIAM J. Appl. Dyn. Syst. 15(2), 713–735 (2016)
14. Li, M., Jiang, L.: Deep learning nonlinear multiscale dynamic problems using Koopman operator. J. Comput. Phys. 446, 110660 (2021)
15. Liu, Y., Kutz, J.N., Brunton, S.L.: Hierarchical deep learning of multiscale differential equation timesteppers. Philos. Trans. A 380, 20210200 (2021)
16. Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: learning PDEs from data. In: Proceedings of the 35th International Conference on Machine Learning, vol 80, pp. 3208–3216 (2018)
17. Lu, L., Meng, X., Mao, Z., Karniadakis, G.E.: DeepXDE: a deep learning library for solving differential equations. SIAM Rev. 63(1), 208–228 (2021)
18. Nguyen-Thanh, V., Zhuang, X., Rabczuk, T.: A deep energy method for finite deformation hyperelasticity. Eur. J. Mech. 80, 103874 (2020)
19. Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numer. 8, 143–195 (1999)
20. Qin, T., Chen, Z., Jakeman, J., Xiu, D.: Data-driven learning of non-autonomous systems. SIAM J. Sci. Comput. 43(3), A1607–A1624 (2021)
21. Qin, T., Wu, K., Xiu, D.: Data driven governing equations approximation using deep neural networks. J. Comput. Phys. 395, 620–635 (2019)
22. Raissi, M.: Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res. 19, 1–24 (2018)
23. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv: 1801. 01236 (2018)
24. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
25. Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)
26. Wang, Y.-J., Lin, C.-T.: Runge-Kutta neural network for identification of continuous systems. In: SMC’98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics, vol. 4, pp. 3277–3282 (1998)
27. Wang, Y., Lin, C.: Runge-Kutta neural network for identification of dynamical systems in high accuracy. IEEE Trans. Neural Netw. 9(2), 294–307 (1998)
28. Wight, C.L., Zhao, J.: Solving Allen-Cahn and Cahn-Hilliard equations using the adaptive physicsinformed neural networks. Commun. Comput. Phys. 29, 930–954 (2021)
29. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In: IEEE 2019 American Control Conference (ACC), pp. 4832– 4839 (2019)
30. Yu, H., Tian, X., Li, Q.: OnsagerNet: learning stable and interpretable dynamics using a generalized Onsager principle. Phys. Rev. Fluids 6, 114402 (2021)
31. Zhao, J.: Discovering phase field models from image data with the pseudo-spectral physics-informed neural networks. Commun. Appl. Math. Comput. 3, 357–369 (2021)