It is known that the value function in an unconstrained Markov decision process with finitely many states and actions is a piecewise rational function in the discount factor α, and that the value function can be expressed as a Laurent series expansion about α=1 for α close enough to 1. The authors show in this paper that this property also holds for the value function of Markov decision processes with additional constraints. More precisely, they show by a constructive proof that there are numbers 0=α0<α1<ëëë<αmÅ-1<αm=1 such that for every j=1,2,...,m-1 either the problem is not feasible for all discount factors α in the open interval (αÅjÅ+1,αj) or the value function is a rational function in α in the closed interval [αÅjÅ-1,αj]. µAs a consequence, if the constrained problem is feasible in the neighborhood of α=1, then the value function has a Laurent series expansion about α=1. The present proof technique for the constrained case provides also a new proof for the unconstrained case.
