Semi-infinite discounted Markov decision processes: Policy improvement and singular perturbations

In this paper, Discounted Markov Decision Processes with finite state and countable action set (semi-infinite DMDP for short) are considered. A policy improvement finite algorithm which finds a nearly optimal deterministic strategy is presented. The steps of the algorithm are based on the classical policy improvement algorithm for finite DMDPs. Singularly perturbed semi-infinite DMDPs are investigated. In case of perturbations, some sufficient condition is given to guarantee that there exists a nearly optimal deterministic strategy which can approximate nearly optimal strategies for a whole family of singularly perturbed semi-infinite DMDP.