Obtaining smoother singular arc policies using a modified iterative dynamic programming algorithm

Dynamic programming is a very powerful technique for the optimization of dynamic systems. With the ready availability of high-speed computers and the development of the iterative dynamic programming (IDP) algorithm, a feasible alternative to the calculus of variations approach to the optimal control problem is now available. Inherent in the IDP algorithm is the application of piecewise constant discretized controls. This often leads to singular optimal control policies that are highly active. The aim of this research is to modify the IDP algorithm to reduce the control energy variations that are typical of IDP while at the same time developing policies that are very close to the true optimal control. This is achieved by including a filter within the IDP procedure. Two types of filtering schemes are considered: a median filter and a first-order filter. Application of this modified algorithm to two bioreactor systems that yield singular optimal control profiles is presented and the usefulness of this scheme is demonstrated.