Positive operator based iterative algorithms for solving Lyapunov equations for Itô stochastic systems with Markovian jumps

This paper studies the iterative solutions of Lyapunov matrix equations associated with Itô stochastic systems having Markovian jump parameters. For the discrete‐time case, when the associated stochastic system is mean square stable, two iterative algorithms with one in direct form and the other one in implicit form are established. The convergence of the implicit iteration is proved by the properties of some positive operators associated with the stochastic system. For the continuous‐time case, a transformation is first performed so that it is transformed into an equivalent discrete‐time Lyapunov equation. Then the iterative solution can be obtained by applying the iterative algorithm developed for discrete‐time Lyapunov equation. Similar to the discrete‐time case, an implicit iteration is also proposed for the continuous case. For both discrete‐time and continuous‐time Lyapunov equations, the convergence rates of the established algorithms are analyzed and compared. Numerical examples are worked out to validate the effectiveness of the proposed algorithms.