Stable recursive procedures for numerical computations in Markov models

Recursive methods have been proposed for the numerical solution of equations arising in the analysis and control of Markov processes. Two examples are (i) solving for the equilibrium probabilities, and (ii) solving for the minimal expected cost or time to reach state zero, in Markov process with left-skip-free transition structure. The paper discusses conditions under which recursive techniques are numerically stable and efficient, giving applications to descriptive and control models for queues.