Article ID: | iaor19972262 |
Country: | United Kingdom |
Volume: | 24 |
Issue: | 42 |
Start Page Number: | 117 |
End Page Number: | 126 |
Publication Date: | Feb 1997 |
Journal: | Computers and Operations Research |
Authors: | Sinuany-Stern Zilla, David Israel, Biran Sigal |
Keywords: | markov processes, heuristics |
There is, so far, only limited practical experience applying solution schemes for real-life partially observable Markov decision processes (POMDP’s). In this work the authors address the special-case POMDP associated with the famous machine-replacement problem. The machine deteriorates down a series of states according to known transition probabilities. A state is identified by a probability of producing a defective item. Only a sample of the produced items is observable at each stage, in which it is to be decided whether to replace the machine or not. The authors suggest a very simple heuristic decision-rule that can easily handle replacement-type problems of large size and which is based on the Howard solution of the fully observable version of the problem. By a simulation experimental design they compare the performance of this heuristic relative to the generic POMDP solution algorithm which has been proposed by Lovejoy.