Communications on Applied Mathematics and Computation ›› 2024, Vol. 6 ›› Issue (2): 1175-1188.doi: 10.1007/s42967-023-00302-9

• ORIGINAL PAPERS • Previous Articles     Next Articles

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Wes Whiting1, Bao Wang2, Jack Xin1   

  1. 1. Department of Mathematics, University of California, Irvine, CA, USA;
    2. Department of Mathematics, Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA
  • Received:2022-10-31 Revised:2023-07-28 Accepted:2023-07-31 Online:2023-10-05 Published:2023-10-05
  • Contact: Wes Whiting,E-mail:wwhiting@uci.edu E-mail:wwhiting@uci.edu
  • Supported by:
    The work was partially supported by NSF Grants DMS-1854434, DMS-1952644, and DMS-2151235 at UC Irvine, and Bao Wang is supported by NSF Grants DMS-1924935, DMS-1952339, DMS-2110145, DMS-2152762, and DMS-2208361, and DOE Grants DE-SC0021142 and DE-SC0002722.

PDF

18

Abstract: We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Key words: Hyperbolic neural network, Riemannian gradient descent, Riemannian Adam(RAdam), Training convergence

Copyright © Editorial By Communications on Applied Mathematics and Computation
沪ICP备09014157