Improved dynamic programming methods for optimal control of lumped-parameter stochastic systems

New dynamic programming methods are developed to solve stochastic control problems with a larger number of state variables than previously possible. These methods apply accurate interpolation to numerical approximation of continuous cost-to-go functions, greatly reducing the number of discrete states that must be evaluated. By efficiently incorporating information on first and second derivatives, the approximation reduces computational effort by several orders of magnitude over traditional methods. Consequently, it is practical to apply dynamic programming to complex stochastic problems with a larger number of state variables than traditionally possible. Results are presented for hypothetical reservoir control problems with up to seven state variables and two random inputs.