A general framework for functional networks

In this paper, we introduce functional networks as a generalization and extension of the standard neural networks in the sense that every problem that can be solved by a neural network can also be formulated by a functional network. But, more importantly, we give examples of problems that cannot be solved using neural networks but can be naturally formulated using functional networks. Functional networks are defined as a collection of connected functional units on a set of nodes. A functional unit or neuron connects input nodes to output nodes. The values of the output nodes are calculated from the values of the input nodes by given functions of one or several arguments. The main differences with neural networks are that (a) the neural functions can be multivariate and can be different from neuron to neuron (in which case, no weights are necessary, because they subsume by the different functions) and (b) the neuron outputs can be coupled, that is, coincident. This mathematical model of functional networks parallels printed circuit boards with electronic components, thus giving an intuitive interpretation to functional networks and an interesting and natural additional application. The existence of functional units with common outputs leads to functional equations whose solution can lead to substantial simplification of the initial topology of the network and the neural functions involved. Two types of functional networks (the one-layer and serial functional networks) are discussed in detail. For the one-layer functional networks, a very simple simplification algorithm is given. For the serial functional networks, systems of functional equations are obtained. The methods are illustrated by several examples of applications.