Discrete-Time Stochastic LQ Optimal Control Problem with Random Coefficients

doi:10.1007/s42967-024-00462-2

Communications on Applied Mathematics and Computation ›› 2026, Vol. 8 ›› Issue (2): 640-663.doi: 10.1007/s42967-024-00462-2

• ORIGINAL PAPERS • Previous Articles Next Articles

Discrete-Time Stochastic LQ Optimal Control Problem with Random Coefficients

Yiwei Wu¹, Maoning Tang², Qingxin Meng²

1. Department of Mathematics, Zhejiang Normal University, Jinhua, 321004, Zhejiang, China;
2. Department of Mathematical Sciences, Huzhou University, Huzhou, 313000, Zhejiang, China

Received:2024-07-15 Revised:2024-10-08 Online:2026-04-07 Published:2026-04-07
Contact: Qingxin Meng,E-mail:mqx@zjhu.edu.cn E-mail:mqx@zjhu.edu.cn
Supported by:
Qingxin Meng was supported by the Key Projects of the Natural Science Foundation of Zhejiang Province of China (No. Z22A013952) and the National Natural Science Foundation of China (Nos.12271158 and 11871121). Maoning Tang was supported by the Natural Science Foundation of Zhejiang Province of China (No. LY21A010001).

Abstract

Abstract: In this paper, the discrete-time linear quadratic (LQ) optimal control problem for a stochastic system with random coefficients is studied. Unlike the classical LQ optimal control problem, there exists a great difficulty in the LQ optimal control problem when the coefficient matrices of the stochastic system and weighting matrices in the cost functional are not assumed to be deterministic. Therefore, two innovative points are mentioned. First, we mainly consider structural changes and innovations in discrete-time LQ optimal control problems once the coefficients are randomized. Second, the stochastic system of this article includes nonhomogeneous terms. Interestingly, we show that the maximum principle leads to a Riccati equation. Specifically speaking, the fully coupled forward-backward stochastic difference equations (FBSDEs) are used to characterize the optimal control. Through decoupling the FBSDEs, we derive the expression corresponding to the Riccati equation with nonhomogeneous terms and get a state feedback representation of the optimal control. Finally, we construct the expression of the value function.

Key words: Linear quadratic (LQ) optimal control, Feedback representation, Random coefficients, Riccati equation, Value function

CLC Number:

Yiwei Wu, Maoning Tang, Qingxin Meng. Discrete-Time Stochastic LQ Optimal Control Problem with Random Coefficients[J]. Communications on Applied Mathematics and Computation, 2026, 8(2): 640-663.

TrendMD

References

[1] Bellman, R., Glicksberg, I., Gross, O.: On the “bang-bang’’ control problem. Q. Appl. Math. 14(1), 11–18 (1956)
[2] Bevilacqua, R., Lehmann, T., Romano, M.: Development and experimentation of LQR/APF guidance and control for autonomous proximity maneuvers of multiple spacecraft. Acta Astronaut. 68(7/8), 1260–1275 (2011)
[3] Bismut, J.-M.: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control. Optim. 14(3), 419–444 (1976)
[4] Carnevale, G., Notarstefano, G.: Timescale separation for discrete-time nonlinear stochastic systems. IEEE Control Syst. Lett. 8, 2133–2138 (2024)
[5] Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. Elsevier, Amsterdam (1976)
[6] Guatteri, G., Masiero, F.: Infinite horizon and ergodic optimal quadratic control for an affine equation with stochastic coefficients. SIAM J. Control. Optim. 48(3), 1600–1631 (2009)
[7] Emil-Kalman, R., et al.: Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5(2), 102–119 (1960)
[8] Kushner, H.: Optimal stochastic control. IRE Trans. Autom. Control. 7(5), 120–122 (1962)
[9] Li, H., Xu, J., Zhang, H.: Linear quadratic regulation for discrete-time systems with input delay and colored multiplicative noise. Syst. Control Lett. 143, 104740 (2020)
[10] Li, Y., Tan, X., Tang, S.: Discrete-time approximation of stochastic optimal control with partial observation. SIAM J. Control. Optim. 62(1), 326–350 (2024)
[11] Liang, X., Xu, J.: Control for networked control systems with remote and local controllers over unreliable communication channel. Automatica 98, 86–94 (2018)
[12] Ni, Y.-H., Li, X., Zhang, J.-F.: Indefinite mean-field stochastic linear-quadratic optimal control: from finite horizon to infinite horizon. IEEE Trans. Autom. Control 61(11), 3269–3284 (2015)
[13] Pu, J., Zhang, Q.: Constrained stochastic LQ optimal control problem with random coefficients on infinite time horizon. Appl. Math. Optim. 83(2), 1005–1023 (2021)
[14] Qi, Q., Xie, L., Zhang, H.: Linear-quadratic optimal control for discrete-time mean-field systems with input delay. IEEE Trans. Autom. Control 67(8), 3806–3821 (2021)
[15] Song, T., Liu, B.: Discrete-time mean-field stochastic linear-quadratic optimal control problem with finite horizon. Asian J. Control 23(2), 979–989 (2021)
[16] Song, T., Liu, B.: Open-loop and closed-loop solvabilities for discrete-time mean-field stochastic linear quadratic optimal control problems. arXiv:2306.14496 (2023)
[17] Sun, J.: Mean-field stochastic linear quadratic optimal control problems: open-loop solvabilities. ESAIM: Control Optim. Calc. Var. 23(3), 1099–1127 (2017)
[18] Sun, Y., Li, H., Zhang, H.: Mixed optimal control for discrete-time stochastic systems with random coefficients. Syst. Control Lett. 169, 105383 (2022)
[19] Tang, S.: General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control. Optim. 42(1), 53–75 (2003)
[20] Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM Journal on Control 6(4), 681–697 (1968)
[21] Yang, X., Yu, Z.: FBSDEs involving time delays and advancements on infinite horizon and IQ problems with delays. Syst. Control Lett. 161, 105149 (2022)
[22] Yang, Y., Tang, M., Qingxin, M.: A mean-field stochastic linear-quadratic optimal control problem with jumps under partial information. ESAIM: Control Optim. Calc. Var. 28, 53 (2022)
[23] Zhang, F., Dong, Y., Meng, Q.: Backward stochastic Riccati equation with jumps associated with stochastic linear quadratic optimal control with jumps and random coefficients. SIAM J. Control. Optim. 58(1), 393–424 (2020)
[24] Zhang, H., Li, L., Xu, J., Fu, M.: Linear quadratic regulation and stabilization of discrete-time systems with delay and multiplicative noise. IEEE Trans. Autom. Control 60(10), 2599–2613 (2015)
[25] Zong, X., Wang, M., Zhang, J.-F.: Optimal consensus control of discrete-time stochastic multi-agent systems. Numer. Algebra Control Optim. 15(1), 155–172 (2024)

Discrete-Time Stochastic LQ Optimal Control Problem with Random Coefficients

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 3

Recommended Articles

Metrics

Comments

[1]	Li Wang, Yi Xiao, Yu-Li Zhu, Yi-Bo Wang. Modified Alternately Linearized Implicit Iteration Methods for Nonsymmetric Coupled Algebraic Riccati Equation [J]. Communications on Applied Mathematics and Computation, 2025, 7(5): 1923-1939.
[2]	Liqun Qi, Chen Ling, Hong Yan. Dual Quaternions and Dual Quaternion Vectors [J]. Communications on Applied Mathematics and Computation, 2022, 4(4): 1494-1508.
[3]	Hong Xiong, Maoning Tang, Qingxin Meng. Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equation with Lévy Process [J]. Communications on Applied Mathematics and Computation, 2022, 4(4): 1386-1415.