Communications on Applied Mathematics and Computation ›› 2024, Vol. 6 ›› Issue (2): 1175-1188.doi: 10.1007/s42967-023-00302-9

• ORIGINAL PAPERS • 上一篇    下一篇

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Wes Whiting1, Bao Wang2, Jack Xin1   

  1. 1. Department of Mathematics, University of California, Irvine, CA, USA;
    2. Department of Mathematics, Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA
  • 收稿日期:2022-10-31 修回日期:2023-07-28 接受日期:2023-07-31 出版日期:2023-10-05 发布日期:2023-10-05
  • 通讯作者: Wes Whiting,E-mail:wwhiting@uci.edu E-mail:wwhiting@uci.edu
  • 基金资助:
    The work was partially supported by NSF Grants DMS-1854434, DMS-1952644, and DMS-2151235 at UC Irvine, and Bao Wang is supported by NSF Grants DMS-1924935, DMS-1952339, DMS-2110145, DMS-2152762, and DMS-2208361, and DOE Grants DE-SC0021142 and DE-SC0002722.

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Wes Whiting1, Bao Wang2, Jack Xin1   

  1. 1. Department of Mathematics, University of California, Irvine, CA, USA;
    2. Department of Mathematics, Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA
  • Received:2022-10-31 Revised:2023-07-28 Accepted:2023-07-31 Online:2023-10-05 Published:2023-10-05
  • Contact: Wes Whiting,E-mail:wwhiting@uci.edu E-mail:wwhiting@uci.edu
  • Supported by:
    The work was partially supported by NSF Grants DMS-1854434, DMS-1952644, and DMS-2151235 at UC Irvine, and Bao Wang is supported by NSF Grants DMS-1924935, DMS-1952339, DMS-2110145, DMS-2152762, and DMS-2208361, and DOE Grants DE-SC0021142 and DE-SC0002722.

PDF

18

摘要： We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

关键词: Hyperbolic neural network, Riemannian gradient descent, Riemannian Adam(RAdam), Training convergence

Abstract: We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Key words: Hyperbolic neural network, Riemannian gradient descent, Riemannian Adam(RAdam), Training convergence

Copyright © Editorial By Communications on Applied Mathematics and Computation
沪ICP备09014157