Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions

Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions

0.00 Avg rating0 Votes
Article ID: iaor20001739
Country: Germany
Volume: 49
Issue: 2
Start Page Number: 299
End Page Number: 324
Publication Date: Jan 1999
Journal: Mathematical Methods of Operations Research (Heidelberg)
Authors: ,
Abstract:

We study controlled Markov chains with denumerable state space and bounded costs per stage. A (long-run) risk-sensitive average cost criterion, associated with an exponential utility function with a constant risk sensitivity coefficient, is used as a performance measure. The main assumption on the probabilistic structure of the model is that the transition law satisfies a simultaneous Doeblin condition. Working within this framwork, the main results obtained can be summarized as follows: If the constant risk-sensitivity coefficient is small enough, then an associated optimality equation has a bounded solution with a constant value for the optimal risk-sensitive average cost; in addition, under further standard continuity-compactness assumptions, optimal stationary policies are obtained. However, it is also shown that the above conclusions fail to hold, in general, for large enough values of the risk-sensitivity coefficient. Our results therefore disprove previous claims on this topic. Also of importance is the fact that our developments are very much self-contained and employ only basic probabilistic and analysis principles.

Reviews

Required fields are marked *. Your email address will not be published.