Iterative dynamic programming for minimum energy control problems with time delay

This paper presents the use of iterative dynamic programming employing exact penalty functions for minimum energy control problems. It shows that exact continuously non-differentiable penalty functions are superior to continuously differentiable penalty functions in terms of satisfying final state constraints. It also demonstrates that the choice of an appropriate penalty function factor depends on the relative size of the time delay with respect to the final time and on the expected value of the energy consumption. A quadratic approximation (QA) of the delayed variables is much better than a linear approximation (LA) of the same for relatively large time delays. The QA improves the rate of convergence and avoids the formation of ‘kinks’. A more general way of selecting appropriate penalty function factors is given and the results obtained using four illustrative examples of varying complexity corroborate the efficacy of the method.