Nonhomogeneous Markov decision processes with Borel state space – The average criterion with nonuniformly bounded rewards

This paper deals with nonhomogeneous Markov decision processes with Borel state space and nonuniformly bounded rewards under the average criterion. First, under the minorant-type assumption we prove the existence of an apppropriate solution to the optimality equations. Second, from the optimality equations we also establish the existence of epsilon (greater than or equal to 0)-optimal Markov policies under the additional conditions. Third, some sufficient conditions for the validity of the assumptions in this paper and several examples such as inventory/production systems are provided. Finally, as an application of the optimality equations, a rolling horizon algorithm is given.