A Markovian model for process setup and improvement

We examine the setup and improvement policies for a production process with multiple performance states. It is assumed that the production process deteriorates randomly over time, following a Markovian process with known transition probabilities. In order to reduce the production cost incurred because of process deterioration, the process is inspected at the end of each period. Then one of three actions may be taken: do nothing, perform routine process setup, or perform routine setup and process improvement. The routine setup operation returns the process to its best performance state, whereas the process improvement action may reduce future production and setup costs and improve the process-state transition probabilities. A discounted Markovian model is formulated to find the strategy that minimizes the total cost of operating the production process.