Search

Abstract: This paper presents new experimental results on the variadic neural network (VNN) [McGregor, S. (2007). Neural network processing for multiset data. In Proceedings: Vol. 4668. Artificial neural networks — ICANN 2007, 17th international conference (pp. 460–470). Springer]. The inputs to a variadic network are an arbitrary-length list of -tuples of real numbers, where is fixed, and the function computed by the network is unaffected by permutation of the inputs. This paper describes improvements in the training algorithm for the variadic perceptron, based on a constructive cascade topology, and performance of the improved networks on geometric problems inspired by vector graphics. Further development may allow practical application of these or similar networks to vector graphics processing and statistical analysis. [Copyright &y& Elsevier]

Abstract: This paper proposes the combination of two state-of-the-art algorithms for processing graph input data, viz., the probabilistic mapping graph self organizing map, an unsupervised learning approach, and the graph neural network, a supervised learning approach. We organize these two algorithms in a cascade architecture containing a probabilistic mapping graph self organizing map, and a graph neural network. We show that this combined approach helps us to limit the long-term dependency problem that exists when training the graph neural network resulting in an overall improvement in performance. This is demonstrated in an application to a benchmark problem requiring the detection of spam in a relatively large set of web sites. It is found that the proposed method produces results which reach the state of the art when compared with some of the best results obtained by others using quite different approaches. A particular strength of our method is its applicability towards any input domain which can be represented as a graph. [Copyright &y& Elsevier]

Abstract: Selection of the optimal neural architecture to solve a pattern classification problem entails to choose the relevant input units, the number of hidden neurons and its corresponding interconnection weights. This problem has been widely studied in many research works but their solutions usually involve excessive computational cost in most of the problems and they do not provide a unique solution. This paper proposes a new technique to efficiently design the MultiLayer Perceptron (MLP) architecture for classification using the Extreme Learning Machine (ELM) algorithm. The proposed method provides a high generalization capability and a unique solution for the architecture design. Moreover, the selected final network only retains those input connections that are relevant for the classification task. Experimental results show these advantages. [Copyright &y& Elsevier]

Abstract: This paper introduces an approach called Clustering and Co-evolution to Construct Neural Network Ensembles (CONE). This approach creates neural network ensembles in an innovative way, by explicitly partitioning the input space through a clustering method. The clustering method allows a reduction in the number of nodes of the neural networks that compose the ensemble, thus reducing the execution time of the learning process. This is an important characteristic especially when evolutionary algorithms are used. The clustering method also ensures that different neural networks specialize in different regions of the input space, working in a divide-and-conquer way, to maintain and improve the accuracy. Besides, the clustering method facilitates the understanding of the system and makes a straightforward distributed implementation possible. The experiments performed with seven classification databases and three different co-evolutionary algorithms show that CONE considerably reduces the execution time without prejudicing (and even improving) the accuracy, even when a distributed implementation is not used. [Copyright &y& Elsevier]

Abstract: This paper presents a new online clustering algorithm called SAKM (Self-Adaptive Kernel Machine) which is developed to learn continuously evolving clusters from non-stationary data. Based on SVM and kernel methods, the SAKM algorithm uses a fast adaptive learning procedure to take into account variations over time. Dedicated to online clustering in a multi-class environment, the algorithm designs an unsupervised neural architecture with self-adaptive abilities. Based on a specific kernel-induced similarity measure, the SAKM learning procedures consist of four main stages: Creation, Adaptation, Fusion and Elimination. In addition to these properties, the SAKM algorithm is attractive to be computationally efficient in online learning of real-drifting targets. After a theoretical study of the error convergence bound of the SAKM local learning, a comparison with NORMA and ALMA algorithms is made. In the end, some experiments conducted on simulation data, UCI benchmarks and real data are given to illustrate the capacities of the SAKM algorithm for online clustering in non-stationary and multi-class environment. [Copyright &y& Elsevier]

Abstract: An important drawback of many artificial neural networks (ANN) is their lack of explanation capability [Andrews, R., Diederich, J., & Tickle, A. B. (1996). A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8, 373–389]. This paper starts with a survey of algorithms which attempt to explain the ANN output. We then present HYPINV, 1 [1] HYPINV stands for an algorithm which extracts HYPerplanes using INVersion. a new explanation algorithm which relies on network inversion; i.e. calculating the ANN input which produces a desired output. HYPINV is a pedagogical algorithm, that extracts rules, in the form of hyperplanes. It is able to generate rules with arbitrarily desired fidelity, maintaining a fidelity–complexity tradeoff. To our knowledge, HYPINV is the only pedagogical rule extraction method, which extracts hyperplane rules from continuous or binary attribute neural networks. Different network inversion techniques, involving gradient descent as well as an evolutionary algorithm, are presented. An information theoretic treatment of rule extraction is presented. HYPINV is applied to example synthetic problems, to a real aerospace problem, and compared with similar algorithms using benchmark problems. [Copyright &y& Elsevier]

In this paper, we propose a new network growth model and its learning algorithm to more precisely model such a real-world growing network as the Web. Unlike the conventional models, we have incorporated directional attachment and community structure for this purpose. We show that the proposed model exhibits a degree distribution with a power-law tail, which is an important characteristic of many large-scale real-world networks including the Web. Using real Web data, we experimentally show that predictive ability can be improved by incorporating directional attachment and community structure. Also, using synthetic data, we experimentally show that predictive ability can definitely be improved by incorporating community structure. [Copyright &y& Elsevier]

Abstract: This letter points out that the main ideas and conclusions of the “The No-Prop algorithm” paper which has recently appeared in this journal were proposed earlier by G.-B. Huang et al. 10 years ago and intensively discussed and applied by other authors in the past 10 years. [Copyright &y& Elsevier]