Approximating the Markov property in Markov decision processes

In many real life situations, random events in one period are correlated with random events in earlier periods. In the application of Markov decision process theory to find otpimal control policies for such situations, it is usual to ignore this correlation in the anticipation that the correlation is small enough not to render the resulting policies of dubious value. This paper examines a framework for determining error bounds arising from commonly used approximation models, and a procedure for successively enriching the state of the system until such a point is reached where these errors are acceptable.