Application of iterative dynamic-programming to very high-dimensional systems

When iterative dynamic programming is used for optimal control of very high dimensional systems, we seek a piecewise linear continuous control policy and use only a single grid point for the state vector. For convergence, a multi-pass method is used where at the beginning of each pass the initial region over which the control is taken is reduced by a factor. Convergence of such a procedure is systematic and reliable in the optimal control of a system described by 130 differential equations and having 130 control variables. The use of only 10 stages with piecewise linear continuous control yields excellent results. The use of 42 stages caused no computational difficulties. The attractive feature of the method is that the optimization can be performed easily on a personal computer, without requiring much memory.