Nonlinear regression: A hybrid model

Using traditional parametric methods of regression analysis, one must make assumption(s) about the form of the regression equation which may not be valid. In high-dimensional cases of a reasonable sample size, nonparametric techniques of regression analysis (kernel, nearest neighbor, and spline smoothing) do not perform well due to the ‘curse of dimensionality’. This paper proposes a nontraditional nonlinear regression model for cases in which the sample space is high dimensional and the relationship between the independent variables and dependent variable is arbitrary. This research suggests the combination of the linear regression analysis method with the self-organizing feature maps, algorithm for high-dimensional convex polytopes, and back-propagation neural networks. When the sample set is pre-processed by a linear regression function, the self-organizing feature maps can be used to detect clusters of misrepresented sample points when they exist. Using the algorithm for high-dimensional convex polytopes, the sample data points in each of these clusters are sorted into two classes, each of which is supposed to distribute on one of the two sides of the pursued regression function. These groups of data points are then used to train the back-propagation neural network. The function represented by the trained neural network, which represents the boundary between the two groups, is the nonlinear regression function for the original data set.