Nonlinear least squares methods and convergence

The nonlinear least squares problem is of practical importance specially in curve fitting and parameter estimation. This problem can be solved either by general optimization methods or by specialized algorithms which take into account its structure. If this is explored more efficient algorithms can be developed. The purpose of this paper is to survey for methods to be used for the solution of nonlinear least squares problems and to study their local convergence rate. The iterative solution of the normal equations defines the Newton's method. To save derivative and arithmetic calculations Gauss–Newton, Levenberg–Marquardt and Quasi-Newton structured algorithms have been proposed. In order to simplify the calculus, to improve the conditioning and to maintain good local convergence properties, as the Newton's method, a variety of implementations are available at present.