Recurrence conditions for Markov decision processes with Borel state space: A survey

This paper describes virtually all the recurrence conditions used heretofore for Markov decision processes with Borel state and action spaces, which include some forms of mixing and contraction properties, Doeblin’s condition, Harris recurrence, strong ergodicity, and the existence of bounded solutions to the optimality equation for average reward processes. The aim is to establish (when possible) implications and equivalences between these conditions.