Performance optimization of continuous-time Markov control processes based on performance potentials

Average-cost optimization problems for a class of continuous-time Markov control processes with a compact action set have been studied. The definition of a generalized average-cost Poisson equation, which can be viewed as an extension to the standard one is first given. Markov performance potentials are defined as its unique solution. Based on the formula of performance potentials, an average-cost optimality equation is derived and the existence of its solution is established. Then, policy iteration and value iteration algorithms are proposed and their convergence is discussed. A numerical example for controlled closed queuing networks illustrates the application of the proposed value iteration algorithm.