Structural properties of stochastic dynamic programs

Structural properties of stochastic dynamic programs

0.00 Avg rating0 Votes
Article ID: iaor20032054
Country: United States
Volume: 50
Issue: 5
Start Page Number: 796
End Page Number: 809
Publication Date: Sep 2002
Journal: Operations Research
Authors: ,
Keywords: decision theory
Abstract:

In Markov models of sequential decision processes, one is often interested in showing that the value function is monotonic, convex, and/or supermodular in the state variables. These kinds of results can be used to develop a qualitative understanding of the model and characterize how the results will change with changes in model parameters. In this paper we present several fundamental results for establishing these kinds of properties. The results are, in essence, ‘metatheorems’ showing that the value functions satisfy property P if the reward functions satisfy property P and the transition probabilities satisfy a stochastic version of this property. We focus our attention on closed convex cone properties, a large class of properties that includes monotonicity, convexity, and supermodularity, as well as combinations of these and many other properties of interest.

Reviews

Required fields are marked *. Your email address will not be published.