A review of combinatorial problems arising in feedforward neural network design

This paper is primarily oriented towards discrete mathematics and emphasizes the occurrence of combinatorial problems in the area of artificial neural networks. The focus is on feedforward networks of binary units and their use as associative memories. Exact and heuristic algorithms for designing networks with single or multiple layers are discussed and complexity results related to the learning problems are reviewed. Several methods do only vary the parameters of networks whose topology has been chosen a priori while others build the networks during the training process. Valiant’s learning from examples model which formalizes the problem of generalization is presented and open questions are mentioned.