Application of iterative dynamic programming to optimal control of nonseparable problems

Convergence properties of iterative dynamic programming (IDP) are examined with respect to solving non-separable optimal control problems. As suggested by Luus and Tassone, the best values available from the previous iteration are used for those variables which are required from upstream. As iterations continue, the values tend to converge to the optimal values. Although the convergence is not monotonic, it is nevertheless fast. Two non-separable optimisation problems are used to test the viability of this approach. It is found that iterative dynamic programming is considerably more efficient than direct search optimisation when the number of stages is large. To solve a 100-stage non-separable optimisation problem with 3 state variables and 3 control variables requires less than one minute of computation time with IDP.