On the limited memory BFGS method for large scale optimization

The authors study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which they call the L-BFGS method. Its performance is compared with that of the method developed by Buckley and LeNir, which combines cycles of BFGS steps and conjugate direction steps. The present numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. The authors show that the L-BFGS method can be greatly accelerated by means of a simple scaling. They then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint. The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However the authors find that for other problems the L-BFGS method is very competitive due to its low iteration cost. They also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.