Optimal control without solving the Bellman equation

This paper recommends an alternative to solving the Bellman partial differential equation for the value function in optimal control problems involving stochastic differential or difference equations. It recommends solving for the vector Lagrange multiplier associated with a first-order condition for maximum. The method is preferable to Bellman’s in exploiting this first-order condition and in solving only algebraic equations in the control variable and Lagrange multiplier and its derivatives rather than a functional equation. The solution requires no global approximation of the value function and is likely to be more accurate than methods which are based on global approximations.