Optimal control for continuous-time linear quadratic problems with infinite Markov jump parameters

The subject matter of this paper is the optimal control problem for continuous-time linear systems subject to Markovian jumps in the parameters and the usual infinite-time horizon quadratic cost. What essentially distinguishes our problem from previous ones, inter alia, is that the Markov chain takes values on a countably infinite set. To tackle our problem, we make use of powerful tools from semigroup theory in Banach space and a decomplexification technique. The solution for the problem relies, in part, on the study of a countably infinite set of coupled algebraic Riccati equations (ICARE). Conditions for existence and uniqueness of a positive semidefinite solution of the ICARE are obtained via the extended concepts of stochastic stabilizability (SS) and stochastic detectability (SD). These concepts are couched into the theory of operators in Banach space and parallel to the classical linear quadratic (LQ) case, bound up with the spectrum of a certain infinite dimensional linear operator.