Article ID: | iaor20103205 |
Volume: | 176 |
Issue: | 1 |
Start Page Number: | 7 |
End Page Number: | 39 |
Publication Date: | Apr 2010 |
Journal: | Annals of Operations Research |
Authors: | Bayraktar Erhan, Ludkovski Michael |
We consider a continuous-time model for inventory management with Markov modulated non-stationary demands. We introduce active learning by assuming that the state of the world is unobserved and must be inferred by the manager. We also assume that demands are observed only when they are completely met. We first derive the explicit filtering equations and pass to an equivalent fully observed impulse control problem in terms of the sufficient statistics, the a posteriori probability process and the current inventory level. We then solve this equivalent formulation and directly characterize an optimal inventory policy. We also describe a computational procedure to calculate the value function and the optimal policy and present two numerical illustrations.