Effective full connection neural network updating using a quantized full FORCE algorithm.

This paper presents a new training algorithm that can update the situation of layers' network, and therefore, connections, neurons, and firing rate of neurons based on FORCE (first-order reduced and controlled error) training algorithm. The Quantized Full FORCE algorithm (QFF) also updates the number of neurons and connections between different layers in the network per iteration in a way that the whole firing rate of each layer is updated via selecting the best neurons and combining strong features. The update method is sequential, so that with each instance passing through the network, the network structure is updated with the Full FORCE algorithm. The algorithm updates the structure of networks with a multiple/single middle layer of the supervised version of feed forward networks such as Multilayer perceptron (MLP), changing them into partially-connected networks. A combination of principal component analysis PCA and Linear Discriminant Analysis (LDA) algorithms has been used to cluster the network input features. The paper focuses on the deep supervised MLP network with backpropagation (BP) and various datasets and its comparison with other MLP based stat of art methods and hybrid evolutionary algorithms. We achieved 98.15 percent accuracy for facial expression 98.6 and 97.7 percent for Wisconsin breast Cancer and Iris Flower in respectively. The training algorithm employed in the study enjoys a lower computational complexity while yielding faster and more accurate convergence, starting with a very low level of errors of 0.009 in comparison with the full connection network and it solves the challenge of getting stuck in local minima and poor convergence of Gradient Decent with BP. • Providing the new training algorithm to update the connections and firing rate of neurons in each iteration. • The proposed training model is based on Quantized Full FORCE (first-order reduced and controlled error) training. • Providing an accurate framework to update the number of neurons, converting the network to a partial connection network. • In the new training model, the best features have the highest number of connections and neurons within the middle layers. • Being the decision speed of the network in the proposed method much faster than a fully connected network. • Cost-effectiveness and much lower resource requirements for implementation of deep networks on hardware such as FPGAs. [ABSTRACT FROM AUTHOR]