Performance Analysis for Controlled Semi‐Markov Systems with Application to Maintenance

This paper concerns with the performance analysis for controlled semi‐Markov systems in Borel state and action spaces. The performability of the system is defined as the probability that the system reaches a prescribed reward level during a first passage time to some target set. Under mild conditions, we develop a value iteration algorithm for computing the optimal value, and establish the existence of optimal policies with the maximal performability. Our main results are applied to a maintenance problem.