Optimal control of time-delay systems by dynamic programming

The use of iterative dynamic programming employing systematic region contraction and accessible grid points is investigated for the optimal control of time-delay systems. At the time of generating the grid points for the state variables, the corresponding delayed variables at each time stage are also generated and stored in memory. Then, when applying dynamic programming, a linear approximation is used to obtain the initial profile for the delayed variables during integration. This procedure was tested with four problems of different complexity. In each case the optimal policy is easily obtained and the results compare very favourably with those reported in the literature using other computational procedures.