Transient analysis of cumulative measures of Markov model behavior

Markov chains and Markov reward models provide are useful for modeling fault-tolerant, distributed and multi-processor systems. In this paper, the authors consider the transient analysis of ‘cumulative’ or ‘integral’ measures of Markov and Markov reward model behavior. These measures include ‘interval availability’ and ‘expected accumulated reward’ over a finite horizon. The authors consider two methods for numerical model evaluation: Uniformization and differential equation solution. They use a numerical experiment to compare the algorithms’ performance as a function of model size, accuracy, and stiffness. Contrary to ‘folk wisdom’, the authors observe that cumulative measure solver behavior is usually similar to that seen in instantaneous measure analysis. However, for large time values, cumulative measures do not converge to steady-state values, leading to numerical difficulties like overflow and slow convergence. These problems can be avoided by directly solving time-averaged equations.