Article ID: | iaor20002335 |
Country: | Germany |
Volume: | 49 |
Issue: | 3 |
Start Page Number: | 435 |
End Page Number: | 439 |
Publication Date: | Jan 1999 |
Journal: | Mathematical Methods of Operations Research (Heidelberg) |
Authors: | Nowak A.S., Vega-Amaya O. |
Keywords: | programming: dynamic |
Brown constructed an aperiodic Markov decision chain in which no overtaking policy (stationary or nonstationary) exists. However, in his example a strong overtaking optimal policy exists in the class of all stationary policies. We provide another example of an aperiodic and geometric ergodic Markov decision chain which shows that the existence of a strong overtaking optimal stationary policy may fail inclusively in the class of stationary policies. We also give a brief survey of the existing literature on overtaking optimality in Markov decision chains.