In this paper, the discrete-time linear quadratic (LQ) optimal control problem for a stochastic system with random coefficients is studied. Unlike the classical LQ optimal control problem, there exists a great difficulty in the LQ optimal control problem when the coefficient matrices of the stochastic system and weighting matrices in the cost functional are not assumed to be deterministic. Therefore, two innovative points are mentioned. First, we mainly consider structural changes and innovations in discrete-time LQ optimal control problems once the coefficients are randomized. Second, the stochastic system of this article includes nonhomogeneous terms. Interestingly, we show that the maximum principle leads to a Riccati equation. Specifically speaking, the fully coupled forward-backward stochastic difference equations (FBSDEs) are used to characterize the optimal control. Through decoupling the FBSDEs, we derive the expression corresponding to the Riccati equation with nonhomogeneous terms and get a state feedback representation of the optimal control. Finally, we construct the expression of the value function.
Yiwei Wu
,
Maoning Tang
,
Qingxin Meng
. Discrete-Time Stochastic LQ Optimal Control Problem with Random Coefficients[J]. Communications on Applied Mathematics and Computation, 2026
, 8(2)
: 640
-663
.
DOI: 10.1007/s42967-024-00462-2
[1] Bellman, R., Glicksberg, I., Gross, O.: On the “bang-bang’’ control problem. Q. Appl. Math. 14(1), 11–18 (1956)
[2] Bevilacqua, R., Lehmann, T., Romano, M.: Development and experimentation of LQR/APF guidance and control for autonomous proximity maneuvers of multiple spacecraft. Acta Astronaut. 68(7/8), 1260–1275 (2011)
[3] Bismut, J.-M.: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control. Optim. 14(3), 419–444 (1976)
[4] Carnevale, G., Notarstefano, G.: Timescale separation for discrete-time nonlinear stochastic systems. IEEE Control Syst. Lett. 8, 2133–2138 (2024)
[5] Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. Elsevier, Amsterdam (1976)
[6] Guatteri, G., Masiero, F.: Infinite horizon and ergodic optimal quadratic control for an affine equation with stochastic coefficients. SIAM J. Control. Optim. 48(3), 1600–1631 (2009)
[7] Emil-Kalman, R., et al.: Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5(2), 102–119 (1960)
[8] Kushner, H.: Optimal stochastic control. IRE Trans. Autom. Control. 7(5), 120–122 (1962)
[9] Li, H., Xu, J., Zhang, H.: Linear quadratic regulation for discrete-time systems with input delay and colored multiplicative noise. Syst. Control Lett. 143, 104740 (2020)
[10] Li, Y., Tan, X., Tang, S.: Discrete-time approximation of stochastic optimal control with partial observation. SIAM J. Control. Optim. 62(1), 326–350 (2024)
[11] Liang, X., Xu, J.: Control for networked control systems with remote and local controllers over unreliable communication channel. Automatica 98, 86–94 (2018)
[12] Ni, Y.-H., Li, X., Zhang, J.-F.: Indefinite mean-field stochastic linear-quadratic optimal control: from finite horizon to infinite horizon. IEEE Trans. Autom. Control 61(11), 3269–3284 (2015)
[13] Pu, J., Zhang, Q.: Constrained stochastic LQ optimal control problem with random coefficients on infinite time horizon. Appl. Math. Optim. 83(2), 1005–1023 (2021)
[14] Qi, Q., Xie, L., Zhang, H.: Linear-quadratic optimal control for discrete-time mean-field systems with input delay. IEEE Trans. Autom. Control 67(8), 3806–3821 (2021)
[15] Song, T., Liu, B.: Discrete-time mean-field stochastic linear-quadratic optimal control problem with finite horizon. Asian J. Control 23(2), 979–989 (2021)
[16] Song, T., Liu, B.: Open-loop and closed-loop solvabilities for discrete-time mean-field stochastic linear quadratic optimal control problems. arXiv:2306.14496 (2023)
[17] Sun, J.: Mean-field stochastic linear quadratic optimal control problems: open-loop solvabilities. ESAIM: Control Optim. Calc. Var. 23(3), 1099–1127 (2017)
[18] Sun, Y., Li, H., Zhang, H.: Mixed optimal control for discrete-time stochastic systems with random coefficients. Syst. Control Lett. 169, 105383 (2022)
[19] Tang, S.: General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control. Optim. 42(1), 53–75 (2003)
[20] Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM Journal on Control 6(4), 681–697 (1968)
[21] Yang, X., Yu, Z.: FBSDEs involving time delays and advancements on infinite horizon and IQ problems with delays. Syst. Control Lett. 161, 105149 (2022)
[22] Yang, Y., Tang, M., Qingxin, M.: A mean-field stochastic linear-quadratic optimal control problem with jumps under partial information. ESAIM: Control Optim. Calc. Var. 28, 53 (2022)
[23] Zhang, F., Dong, Y., Meng, Q.: Backward stochastic Riccati equation with jumps associated with stochastic linear quadratic optimal control with jumps and random coefficients. SIAM J. Control. Optim. 58(1), 393–424 (2020)
[24] Zhang, H., Li, L., Xu, J., Fu, M.: Linear quadratic regulation and stabilization of discrete-time systems with delay and multiplicative noise. IEEE Trans. Autom. Control 60(10), 2599–2613 (2015)
[25] Zong, X., Wang, M., Zhang, J.-F.: Optimal consensus control of discrete-time stochastic multi-agent systems. Numer. Algebra Control Optim. 15(1), 155–172 (2024)
