Methods to design optimal control of Markov process with finite state set in the presence of constraints

The problem of optimal control of a nonuniform Markov process with a finite state set over a fixed interval in the presence of inequality‐like constraints was considered. The design of control relies on the principle of dynamic programming in combination with the methods of convex programming and the duality theory. Two types of conditions under which it is possible to select a Markov optimal control were proposed.