Survey of linear programming for standard and nonstandard Markovian control problems. Part II: Applications

This paper deals with some applications of Markov decision models for which the linear programming method is efficient. These models are replacement models (with the optimal stopping problem as special case), separable models (including the inventory model as special case) and the multi-armed bandit model. In the companion paper ‘Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory’, general linear programming methods are discussed. These linear programming formulations are the starting point for the efficient methods that will be derived for the special models.