Search

Evolutionary neural networks (ENNs) are an adaptive approach that combines the adaptive mechanism of Evolutionary algorithms (EAs) with the learning mechanism of Artificial Neural Network (ANNs). In view of the difficulties in design and development of DNNs, ENNs can optimize and supplement deep learning algorithm, and the more powerful neural network systems are hopefully built. Many valuable conclusions and results have been obtained in this field, especially in the construction of automated deep learning systems. This study conducted a systematic review of the literature on ENNs by using the PRISMA protocol. In literature analysis, the basic principles and development background of ENNs are firstly introduced. Secondly, the main research techniques are introduced in terms of connection weights, architecture design and learning rules, and the existing research results are summarized and the advantages and disadvantages of different research methods are analyzed. Then, the key technologies and related research progress of ENNs are summarized. Finally, the applications of ENNs are summarized and the direction of future work is proposed. [ABSTRACT FROM AUTHOR]

The accurate identification of moving target types in alert areas is a fundamental task for unattended ground sensors. Considering that the seismic and sound signals generated by ground moving targets in urban areas are easily affected by environmental noise and the power consumption of unattended ground sensors needs to be reduced to achieve low-power consumption, this paper proposes a ground moving target detection method based on evolutionary neural networks. The technique achieves the selection of feature extraction methods and the design of evolving neural network structures. The experimental results show that the improved model can achieve high recognition accuracy with a smaller feature vector and lower network complexity. [ABSTRACT FROM AUTHOR]

The paper presents a neuro-evolutionary algorithm called Hill Climb Assembler Encoding (HCAE) which is a light variant of Hill Climb Modular Assembler Encoding (HCMAE). While HCMAE, as the name implies, is dedicated to modular neural networks, the target application of HCAE is to evolve small/mid-scale monolithic neural networks which, in spite of the great success of deep architectures, are still in use, for example, in robotic systems. The paper analyses the influence of different mechanisms incorporated into HCAE on the effectiveness of evolved neural networks and compares it with a number of rival algorithms. In order to verify the ability of HCAE to evolve effective small/mid-scale neural networks, both feed forward and recurrent, it was tested on fourteen identification problems including the two-spiral problem, which is a well-known binary classification benchmark, and on two control problems, i.e., the inverted-pendulum problem, which is a classical control benchmark, and the trajectory-following problem, which is a real problem in underwater robotics. Four other neuro-evolutionary algorithms, four particle swarm optimization methods, differential evolution, and a well-known back-propagation algorithm, were applied as a point of reference for HCAE. The experiments reported in the paper revealed that the evolutionary approach applied in the proposed algorithm makes it a more effective tool for solving the test problems than all the rivals. [ABSTRACT FROM AUTHOR]

Clustering is a widely used unsupervised learning technique across data mining and machine learning applications and finds frequent use in diverse fields ranging from astronomy, medical imaging, search and optimization, geology, geophysics, and sentiment analysis, to name a few. It is therefore important to verify the effectiveness of the clustering algorithm in question and to make reasonably strong arguments for the acceptance of the end results generated by the validity indices that measure the compactness and separability of clusters. This work aims to explore the successes and limitations of two popular clustering mechanisms by comparing their performance over publicly available benchmarking data sets that capture a variety of data point distributions as well as the number of attributes, especially from a computational point of view by incorporating techniques that alleviate some of the issues that plague these algorithms. Sensitivity to initialization conditions and stagnation to local minima are explored. Further, an implementation of a feedforward neural network utilizing a fully connected topology in particle swarm optimization is introduced. This serves to be a guided random search technique for the neural network weight optimization. The algorithms utilized here are studied and compared, from which their applications are explored. The study aims to provide a handy reference for practitioners to both learn about and verify benchmarking results on commonly used real-world data sets from both a supervised and unsupervised point of view before application in more tailored, complex problems.

Load forecasting/prediction is a challenging task in the energy markets that require adequate attention in generating stable and reliable load demand to deal with energy management and planning strategies. Accurate load prediction is critical for electrical power systems operations, but nonlinear loads involve high volatility. Forecasting these kinds of complex load characteristics requires highly accurate forecasting devices. It is generally accomplished by constructing models on relative details, including weather and previous data on load demand. This study proposes six decomposition-based evolutionary neural networks for city-scale and building energy forecasting. These evolutionary neural networks have also been trained using historical load data, and weather records are already considered to have a significant effect on energy usage (for example, wind speed, precipitation, dry-dew point temperature, relative humidity, clouds fraction and mean sea level perception). The network structure of the model is built and based on error estimation, trend map, and appropriate method of measuring evolutionary neural networks efficiency. The model's hidden layers and neurons network structure is selected based on recent smart models to enhance its accuracy. Several measures were used to improve performance, such as seasonal smoothing changes, coarse-graining of the humidity ratio, load-oriented day-type classification monitoring, outlier removal, and complex climate information. Results show that used models have a very high fitting accuracy and low error rate for short, medium, and long-term forecasting/planning tasks. These models have the potential to reduce overfitting issues, thereby improving load forecasting efficiency. Further, proposed models may also use for forecasting solar and wind demand. The efficiency of the model compared to the existing two models. Forecast results show superiority and efficiency in short, medium and long-term energy forecasting and offer the ability to manage unbalanced utility loads.

Evolutionary Algorithms (EA) are widely applied in Neural Architecture Search (NAS) and have achieved appealing results. Different EA-based NAS algorithms may utilize different encoding schemes for network representation, while they have the same workflow. Specifically, the first step is the initialization of the population with different encoding schemes, and the second step is the evaluation of the individuals by the fitness function. Then, the EA-based NAS algorithm executes evolution operations, e.g., selection, mutation, and crossover, to eliminate weak individuals and generate more competitive ones. Lastly, evolution continues until the max generation and the best neural architectures will be chosen. Because each individual needs complete training and validation on the target dataset, the EA-based NAS always consumes significant computation and time inevitably, which results in the bottleneck of this approach. To ameliorate this issue, this article proposes a distributed framework to boost the computation of the EA-based NAS algorithm. This framework is a server/worker model where the server distributes individuals requested by the computing nodes and collects the validated individuals and hosts the evolution operations. Meanwhile, the most time-consuming phase (i.e., individual evaluation) of the EA-based NAS is allocated to the computing nodes, which send requests asynchronously to the server and evaluate the fitness values of the individuals. Additionally, a new packet structure of the message delivered in the cluster is designed to encapsulate various network representations and support different EA-based NAS algorithms. We design an EA-based NAS algorithm as a case to investigate the efficiency of the proposed framework. Extensive experiments are performed on an illustrative cluster with different scales, and the results reveal that the framework can achieve a nearly linear reduction of the search time with the increase of the computational nodes. Furthermore, the length of the exchanged messages among the cluster is tiny, which benefits the framework expansion. [ABSTRACT FROM AUTHOR]

The accurate identification of moving target types in alert areas is a fundamental task for unattended ground sensors. Considering that the seismic and sound signals generated by ground moving targets in urban areas are easily affected by environmental noise and the power consumption of unattended ground sensors needs to be reduced to achieve low-power consumption, this paper proposes a ground moving target detection method based on evolutionary neural networks. The technique achieves the selection of feature extraction methods and the design of evolving neural network structures. The experimental results show that the improved model can achieve high recognition accuracy with a smaller feature vector and lower network complexity.

This paper is concerned with the problem of improving the convergence properties of evolutionary neural networks, particularly in the context of hybrid neural networks that adopt a diversity of transfer functions at their nodes, i.e.: neural diversity machines. The paper explores the potential of solution complementarity, in the context of pattern recognition problems, and focuses on its incorporation in the selection process of recombination heuristics. In a pattern recognition context, complementarity is defined as the ability of different solutions to correctly classify complementary subsets of patterns. A broad set of experiments was conducted demonstrating that solution selection based on complementarity is statistically significantly better than random selection, in a wide range of conditions, e.g.: different datasets, recombination heuristics and architectural constraints. Although the experiments demonstrated the statistical significance and robustness of the effect, they also indicated that more work is required to increase the degree of the effect and to scale-up to larger and more complex datasets. [ABSTRACT FROM AUTHOR]

In essence, back analysis is a process of system identification. Therefore, artificial neural networks represent a suitable solution methodology for this problem. To overcome the shortcomings of the neural networks and evolutionary neural networks, based on immunized evolutionary programming, a new evolutionary neural network whose architecture and connection weights simultaneously evolve is proposed. Using this new evolutionary neural network, a novel inverse back analysis for underground engineering is studied. Using a numerical example and a real engineering example, namely, an underground roadway of the Huainan coal mine in China, the accuracy of this inverse back analysis is verified. Moreover, the non-uniqueness of the solution generated by the inverse back analysis is analyzed. The results show that, using the back-calculated parameters, the computed displacements agree with the measured ones. Thus, the new inverse back analysis method is demonstrated to be a high-performance method for usage in underground engineering. Moreover, various other conclusions can be drawn: the training samples of the neural network should be collected from the results of the positive analysis by the finite element method and selected based on the orthogonal experimental design, and the precision of the back analysis using multiple parameters is worse than that using a single parameter. [ABSTRACT FROM AUTHOR]

Mobile robot's navigation and obstacle avoidance in an unknown and static environment is analyzed in this paper. From the guidance of position sensors, artificial neural network (ANN) based controllers settle the desired trajectory between current and a target point. Evolutionary algorithms were used to choose the best controller. This approach, known as Evolutionary Robotics (ER), commonly resorts to very simple ANN architectures. Although they include temporal processing, most of them do not consider the learned experience in the controller's evolution. Thus, the ER research presented in this article, focuses on the specification and testing of the ANN based controllers implemented when genetic mutations are performed from one generation to another. Discrete-Time Recurrent Neural Networks based controllers were tested, with two variants: plastic neural networks (PNN) and standard feedforward (FFNN) networks. Also the way in which evolution was performed was also analyzed. As a result, controlled mutation do not exhibit major advantages against over the non controlled one, showing that diversity is more powerful than controlled adaptation.

The paper compares a neuro-evolutionary method called Assembler Encoding with two other methods from the area of neuro-evolution. As a testbed for the methods a variant of the predator-prey problem with Autonomous Underwater Vehicles (AUV) operating in an environment with the sea current was used. In the experiments, the task of vehicles-predators controlled with evolu-tionary neural networks was to capture a vehicle-prey behaving ac-cording to a simple deterministic strategy. All the experiments were carried out in simulation, and in order to simplify calculations in the two-dimensional environment - AUVs moved on a horizontal surface under the water. [ABSTRACT FROM AUTHOR]

Abstract: A new method is proposed to deal with the dual-axis control of a multi-variables system with two induction motors. Investigation of resolving the cross-coupling problem of dual-axis platform is addressed by a neural net-based decoupling compensator and a sufficient condition ensuring closed-loop stability is derived. An evolutionary algorithm processing the universal seeking capability is proposed for finding the optimal connecting weights of the neural decoupling compensator and the gains of PID controllers. Extensive numerical studies verify the performance and applicability of the proposed design under a variety of operating conditions. [Copyright &y& Elsevier]

Abstract: Olive (Olea europaea L.) is the main perennial Spanish crop. Soil management in olive orchards is mainly based on intensive and tillage operations, which have a great relevancy in terms of negative environmental impacts. Due to this reason, the European Union (EU) only subsidizes cropping systems which require the implementation of conservation agro-environmental techniques such as cover crops between the rows. Remotely sensed data could offer the possibility of a precise follow-up of presence of cover crops to control these agrarian policy actions, but firstly, it is crucial to explore the potential for classifying variations in spectral signatures of olive trees, bare soil and cover crops using field spectroscopy. In this paper, we used hyperspectral signatures of bare soil, olive trees, and sown and dead cover crops taken in spring and summer in two locations to evaluate the potential of two methods (MultiLogistic regression with Initial and Radial Basis Function covariates, MLIRBF; and SimpleLogistic regression with Initial and Radial Basis Function covariates, SLIRBF) for classifying them in the 400–900nm spectrum. These methods are based on a MultiLogistic regression model formed by a combination of linear and radial basis function neural network models. The estimation of the coefficients of the model is carried out basically in two phases. First, the number of radial basis functions and the radii and centres’ vector are determined by means of an evolutionary neural network algorithm. A maximum likelihood optimization method determines the rest of the coefficients of a MultiLogistic regression with a set of covariates that include the initial variables and the radial basis functions previously estimated. Finally, we apply forward stepwise techniques of structural simplification. We compare the performance of these methods with robust classification methods: Logistic Regression without covariate selection, MLogistic; Logistic Regression with covariate selection, SLogistic; Logistic Model Trees algorithm (LMT); the C4.5 induction tree; Naïve Bayesian tree algorithm (NBTree); and boosted C4.5 trees using AdaBoost.M1 with 10 and 100 boosting iterations. MLIRBF and SLIRBF models were the best discriminant functions in classifying sown or dead cover crops from olive trees and bare soil in both locations and seasons by using a seven-dimensional vector with green (575nm), red (600, 625, 650 and 675nm), and near-infrared (700 and 725nm) wavelengths as input variables. These models showed a correct classification rate between 95.56% and 100% in both locations and seasons. These results suggest that mapping covers crops in olive trees could be feasible by the analysis of high resolution airborne imagery acquired in spring or summer for monitoring the presence or absence of cover crops by the EU or local administrations in order to make the decision on conceding or not the subsidy. [Copyright &y& Elsevier]

Assembler Encoding is Artificial Neural Network encoding method. To date, Assembler Encoding has been tested in two problems, i.e. in an optimization problem in which a solution is in the form of a matrix and in the so-called predator-prey problem in which the task of ANN is to control agent-predators whose common goal is to capture a fast moving agent-prey. The next problem in which Assembler Encoding was tested is the inverted pendulum problem. In the experiments Assembler Encoding was compared to several evolutionary and reinforcement learning methods. The results of the tests are presented at the end of the paper. [ABSTRACT FROM AUTHOR]

This paper presents a generic and patient-specific classification system designed for robust and accurate detection of ECG heartbeat patterns. The proposed feature extraction process utilizes morphological wavelet transform features, which are projected onto a lower dimensional feature space using principal component analysis, and temporal features from the ECG data. For the pattern recognition unit, feedforward and fully connected artificial neural networks, which are optimally designed for each patient by the proposed multidimensional particle swarm optimization technique, are employed. By using relatively small common and patient-specific training data, the proposed classification system can adapt to significant interpatient variations in ECG patterns by training the optimal network structure, and thus, achieves higher accuracy over larger datasets. The classification experiments over a benchmark database demonstrate that the proposed system achieves such average accuracies and sensitivities better than most of the current state-of-the-art algorithms for detection of ventricular ectopic beats (VEBs) and supra-VEBs (SVEBs). Over the entire database, the average accuracy-sensitivity performances of the proposed system for VEB and SVEB detections are 983%-84.6% and 97.4%-63.5%, respectively. Finally, due to its parameter-invariant nature, the proposed system is highly generic, and thus, applicable to any ECG dataset. [ABSTRACT FROM AUTHOR]

In wireless sensor networks, sensor readings are gathered at a gateway for processing and forwarding to a remote command center. The potential closeness of the gateway to dangerous events, e.g. fires, exposes it to damage and thus risks making the network dysfunctional. Therefore, protecting the gateway by repositioning it away from safety- hazardous spots is critical for the operation of the network. However, moving the gateway too far from the sensors that report on active events would have negative effect on the network performance, e.g. throughput and energy consumption. Therefore, balancing the gateway safety and network performance goals will be necessary. In this paper, we present GRISP, a novel Gateway Relocation algorithm for Improved Safety and Performance. GRISP employs an evolutionary neural network model to assess the safety of the gateway at the various locations. The model is then used to direct the search in an area of interest for a safer position that would enhance or at least maintain an acceptable level of network performance. In addition, GRISP guides the gateway during the move by finding safe paths leading to the new location. Our experimental validation results demonstrate the effectiveness of GRISP. [ABSTRACT FROM AUTHOR]

Abstract: Evolutionary and neural computation has been used widely in solving various problems in biological ecosystems. This paper reviews some of the recent work in evolutionary computation and neural network ensembles that could be explored further in the context of ecoinformatics. Although these bio-inspired techniques were not developed specifically for ecoinformatics, their successes in solving complex problems in other fields demonstrate how these techniques could be adapted and used for tackling difficult problems in ecoinformatics. Firstly, we will review our work in modelling and model calibration, which is an important topic in ecoinformatics. Secondly one example will be given to illustrate how coevolutionary algorithms could be used in problem-solving. Thirdly, we will describe our work on neural network ensembles, which can be used for various classification and prediction problems in ecoinformatics. Finally, we will discuss ecosystem-inspired computational models and algorithms that could be explored as directions of future research. [Copyright &y& Elsevier]

Capability of evolutionary neural network (ENN) based QSAR approach to direct the descriptor selection process towards stable descriptor subset (DS) composition characterized by acceptable generalization, as well as the influence of description stability on QSAR model interpretation have been examined. In order to analyze the DS stability and QSAR model generalization properties multiple random dataset partitions into training and test set were made. Acceptability criteria proposed by Golbraikh et al. [J. Comput.-Aided Mol. Des., 17 (2003) 241] have been chosen for selection of highly predictive QSAR models from a set of all models produced by ENN for each dataset splitting. All QSAR models that pass Golbraikh’s filter generated by ENN for each dataset partition were collected. Two final DS forming principles were compared. Standard principle is based on selection of descriptors characterized by highest frequencies among all descriptors that appear in the pool [J. Chem. Inf. Comput. Sci., 43 (2003) 949]. Search across the model pool for DS that are stable against multiple dataset subsampling i.e. universal DS solutions is the basis of novel approach. Based on described principles benzodiazepine QSAR has been proposed and evaluated against results reported by others in terms of final DS composition and model predictive performance. [ABSTRACT FROM AUTHOR]