Replacement policy under partially observed Markov process

This paper deals with a production system (machine) which deteriorates over time. The system deterioration over time is Markovian. The time scale is discrete and the ‘true’ state of the system (excellent, medium and bad) is not directly observable. What is observed is the performance of the system measured in terms of ‘number of defectives’ per time period. At the end of each period a decision is to be made: whether to replace the system or not. The objective is to minimize the total cost in the long run. Partially Observed Markovian Decision Process (POMDP) approach is used to solve this problem. Various methods to solve this problem are discussed and a simpler approximated solution is presented which utilizes some known properties of this problem.