Three related stochastic control problems from reliability theory referred to as the optimal replacement, the optimal maintenance-replacement and the optimal maintenance-repair problems are considered in the context of a nondecreasing Markov wear process x(t):t≥0 with an independent random threshold failure mechanism. Each is considered in continuous time and in the discounted case. Let Y be the random threshold and σ=inf{x(t)≥Y} be the time of failure. For the replacement problem, at any time before failure a decision can be made to replace the current device by a new one at cost c(x(t)). At failure the failed device is immediately replaced by a new one at cost d(x(σ)). The maintenance-replacement problem has the same constraints at failure as the replacement problem, but at any time before failure a decision can be made to maintain the current device to any lower wear state y at cost c(x(t),y). For the maintenance-repair problem, before failure a decision to perform maintenance as in the maintenance-replacement problem can be made, while at failure instead of replacing immediately by a new device, repair to any lower wear state y is considered at cost d(x(σ),y). Additionally for each of the problems a cost per unit time f(x(t)) is also considered. The methods of quasi-variational inequalities and impulse control are used to analyze the problems. The main technical point is to generalize the usual variational formulation of the value function of an optimal stopping problem so as to allow consideration of x(σ) dependent cost structures.
