Some properties of the sequential Markovian allocation problem

This paper deals with a stochastic allocation problem associated with the Markov chain. The object is to maximize the expected present value of the sum of the values of the random variables sampled from the process. The maximum number of the allocation units is fixed and the decision to accept or reject the current observation is based on the known and unknown transition probabilities. The problem is formulated and the optimal decision rule derived by the technique of dynamic programming. Certain computational aspects of the optimal decision rule with some numerical results are also presented, and finally, some promising real-life applications and extensions of the models are discussed.