Reminiscences on the development of the variational approach to Davidon's variable-metric method

In the mid-1960s, Davidon's method was brought to the author's attention by M.J.D. Powell, one of its earliest proponents. Its great efficacy in solving a rather difficult computational problem in which the author was involved led to an attempt to find a ‘best’ updating formula. ‘Best’ seemed to suggest ‘least’ in the sense of some norm, to further the stability of the method. This led to the idea of minimizing a generalized quadratic (Frobenius) norm with the quasi-Newton and symmetry constraints on the updates. Several interesting formulas were derived, including the Davidon–Fletcher–Powell formula (as shown by Goldfarb). This approach was extended to the derivation of updates requiring no derivatives, and to Broyden-like updates for the solution of simultaneous nonlinear equations. Attempts were made to derive minimum-norm corrections in product-form updates, with an eye to preserving positive-definiteness. In the course of this attempt, it was discovered that the DFP formula could be written as a product, leading to some interesting theoretical developments. Finally, a linearized product-form update was developed which was competitive with the best update of that time.