Asymptotic properties of constrained Markov decision processes

Asymptotic properties of constrained Markov decision processes

0.00 Avg rating0 Votes
Article ID: iaor1994267
Country: Germany
Volume: 37
Start Page Number: 151
End Page Number: 170
Publication Date: Dec 1993
Journal: Mathematical Methods of Operations Research (Heidelberg)
Authors:
Abstract:

We present in this paper several asymptotic properties of constrained Markov Decision Processes (MDPs) with a countable state space. We treat both the discounted and the expected average cost, with unbounded cost. We are interested in (1) the convergence of finite horizon MDPs to the infinite horizon MDP, (2) convergence of MDPs with a truncated state space to the problem with infinite state space, (3) convergence of MDPs as the discount factor goes to a limit. In all these cases we establish the convergence of optimal values and policies. Moreover, based on the optimal policy for the limiting problem, we construct policies which are almost optimal for the other (approximating) problems. Based on the convergence of MDPs with a truncated state space to the problem with infinite state space, we show that an optimal stationary policy exists such that the number of randomisations it uses is less or equal to the number of constraints plus one. We finally apply the results to a dynamic scheduling problem.

Reviews

Required fields are marked *. Your email address will not be published.