Improvement of Successive Projection Method for multilayer neural networks

Error back-propagation (BP) is one of the most popular ideas used in learning algorithms for multilayer neural networks. In particular, the on-line BP has been applied to various problems in practice because of its simplicity of implementation. However, an efficient implementation of the on-line BP usually requires an ad hoc rule for determining the learning rate of the algorithm. Recently, we proposed a new learning algorithm called the successive projection method (SPM) that was based on an iteration method for solving a system of nonlinear inequalities. In this paper, we improve the SPM by modifying the sub-problem for each input pattern and the sigmoid output function of the hidden layers. The improved SPM (ISPM) updates the weights associated with the arcs in the network adaptively by solving a quadratic programming sub-problem for each input pattern. Some simulation results on pattern classification problems indicate that the proposed algorithm is more effective and robuster than the standard on-line BP and the original SPM.