A forward iterative dynamic programming technique for optimal control of nonlinear dynamic systems

The use of iterative dynamic programming (IDP) for optimal control of nonlinear dynamic systems is based on treating the control as a stage-wise operation, gridding the state and control variables, and contracting the grid size iteratively. In applying the backward IDP to obtain the optimal control, it is required to integrate dynamic system equations over the interval [t(i), t(f)] with each allowable control, where t(i) is the beginning time of a stage, and t(f) is the final time. This will result in wasting considerable computation time in those state grids that are inaccessible from the specified initial condition by using allowable controls. To overcome such a drawback of the backward IDP, a forward IDP technique is proposed in this paper. Due to its forward operation, the proposed IDP provides substantial savings in computation time by eliminating the time integrations for inaccessible state grids and avoiding integrations up to the final time in evaluating the optimal control for each state grid point.