| Article ID: | iaor20119266 |
| Volume: | 39 |
| Issue: | 5 |
| Start Page Number: | 369 |
| End Page Number: | 374 |
| Publication Date: | Sep 2011 |
| Journal: | Operations Research Letters |
| Authors: | Guo Xianping, Wei Qingda |
| Keywords: | programming: dynamic |
This paper studies Markov decision processes with state‐dependent discount factors. Suitable conditions are given for the existence of optimal stationary policies. Then, an iteration algorithm for computing the optimal value function is developed. Finally, a cash balance model is used to illustrate the main results in this paper.