Model identification of Gaussian neural networks in forecasting

Supervised artificial neural networks are, essentially, nonlinear regression parametric models that can be used in time series forecasting, either as causal models or as autoregressive ones. The estimation of the parameters (called weights) is accomplished using sample data (training patterns), namely through iterative methods for numeric learning. We focus our study on so-called radial basis function (RBF) networks, as these models are, in a sense, linear in the parameters, and therefore are easier to estimate. In this paper, we report a number of experiments aiming to compare different heuristic methods for the identification of Gaussian RBF networks. This identification includes determining optimal, or at least adequate values for the number of component neurons (i.e., the dimension of the model), the number of inputs (the order of a nonlinear autoregression), and the values of the different hyperparameters that one needs to specify before the actual estimation process is carried out. We are also especially interested in adaptive approaches, which continually revise the model on the basis of newly observed data. The experiments were based on carefully designed examples, as well as on real and simulated time series.