Statistical estimation from an optimization viewpoint

Statistics and Optimization have been closely linked from the very outset. The search for a ‘best’ estimator (least squares, maximum likelihood, etc.) certainly relies on optimization tools. On the other hand, Statistics has often provided the motivation for the development of algorithmic procedures for certain classes of optimization problems. However, it is only relatively recently, more specifically in connection with the development of an approximation and sampling theory for stochastic programming problems, that the full connection has come to light. This in turn suggests a more comprehensive approach to the formulation of statistical estimation questions. This expository paper reviews some of the features of this approach.