Neural networks and flexible approximations

This paper supports the view that neural networks are best seen as devices that can approximate a wide range of functions. The authors argue the need to consider the precise details of how the approximations operate in practice. The paper shows how standard network models can be regarded as polynomial functions, obtained from expanding exponential terms. Multivariate Taylor-series expansions, obtained through the Maple software package, are used for this purpose. The expansions serve to cast light on the role of the hidden nodes. Also considered is the relative difficulty of fitting different types of function. Quadratic functions are compared with Gaussian shapes.