Search

With the limitation of air traffic and the rapid increase in the number of flights, flight delay is becoming more frequent. Flight delay leads to financial and time losses for passengers and increases operating costs for airlines. Therefore, the establishment of an accurate prediction model for flight delay becomes vital to build an efficient airline transportation system. The air transportation system has a huge amount of data and complex operation modes, which is suitable for analysis by using machine learning methods. This paper discusses the factors that may affect the flight delay, and presents a new flight delay prediction model. The five warning levels are defined based on flight delay database by using K-means clustering algorithm. After extracting the key factors related to flight operation by the grey relational analysis (GRA) algorithm, an improved machine learning algorithm called GRA — Genetic algorithm (GA) — back propagation neural network, GRA-GA-BP, is introduced, which is optimized by GA. The calculation results show that, compared with models before optimization and other two algorithms in previous papers, the proposed prediction model based on GRA-GA-BP algorithm shows a higher prediction accuracy and more stability. In terms of operation efficiency and memory consumption, it also has good performance. The analysis presented in this paper indicates that this model can provide effective early warnings for flight delay, and can help airlines to intervene in flights with abnormal trend in advance. [ABSTRACT FROM AUTHOR]

Vocal fold, a significant body structure, is accountable for phonation, which regulates air motion within and out of the lungs. The disorders in the vocal fold influence the quality of life. Thus, diagnosis of vocal fold disorders has a significant need, and CT of the neck is employed for an effective imaging scheme. Accordingly, this paper proposes an advanced multi-resolution algorithm (MRA) that optimally identifies and classifies pathologies. The vocal regions are acquired using the genetic k-means algorithm. The pathology features are generated using the local directional pattern (LDP) fed to pathology classification using moth search-rider optimization-based deep convolutional neural networks (MRA-based DCNN). The hybrid optimization (MRA), integrates the standard rider optimization algorithm (ROA) and moth search algorithm (MS) that trains deep learning classifier (DCNN). The analysis using the real databases regarding the performance metrics divulge that the proposed pathology detection module obtained the accuracy, specificity, and sensitivity of 97.020%, 91.698%, and 96.624%. [ABSTRACT FROM AUTHOR]

Siamese network-based object tracking algorithms have recently gained popularity due to continuous improvement in tracking accuracy and robustness. However, these trackers are often limited by inefficient initialization and unstable online update. The model drift accumulates as more frames are processed. To address the above issues, this paper introduced a method that combines both deep learning and conventional regression algorithms (SVR, Support Vector Regression). Specifically, we propose a feature projection algorithm, which can effectively reduce the dimension of the features extracted from deep neural networks and improve the distinguishability of them at the same time. To make the features more robust for SVR training, we further propose two feature aggregation methods at both the channel level and the spatial level. We update the SVR model online instead of the deep neural network to make the tracking process more robust. Extensive experiments on four challenging benchmarks indicate that our proposed tracker is superior to baseline methods both qualitatively and quantitatively. [ABSTRACT FROM AUTHOR]

With the increasing demand for security in many fastest growing applications, biometric recognition is the most prominent authentication system. User authentication through speech and face recognition is the important biometric technique to enhance the security. This paper proposes a speech and facial feature-based multi-modal biometric recognition technique to improve the authentication of any system. Mel Frequency Cepstral Coefficients (MFCC) is extracted from audio as speech features. In visual recognition, this paper proposes cascade hybrid facial (visual) feature extraction method based on static, dynamic and key-point salient features of the face and it proves that the proposed feature extraction method is more efficient than the existing method. In this proposed method, Viola–Jones algorithm is used to detect static and dynamic features of eye, nose, lip, Scale Invariant Feature Transform (SIFT) algorithm is used to detect some stable key-point features of face. In this paper, a research on the audio-visual integration method using AND logic is also made. Furthermore, all the experiments are carried out using Artificial Neural Network (ANN) and Support Vector Machine (SVM). An accuracy of 94.90% is achieved using proposed feature extraction method. The main objective of this work is to improve the authenticity of any application using multi-modal biometric features. Adding facial features to the speech recognition improve system security because biometric features are unique and combining evidence from two modalities increases the authenticity as well as integrity of the system. [ABSTRACT FROM AUTHOR]

Time series change point detection can identify the locations of abrupt points in many dynamic processes. It can help us to find anomaly data in an early stage. At the same time, detecting change points for long, periodic, and multiple input series data has received a lot of attention recently, and is universally applicable in many fields including power, environment, finance, and medicine. However, the performance of classical methods typically scales poorly for such time series. In this paper, we propose CPMAN, a novel prediction-based change point detection approach via multi-stage attention networks. Our model consists of two key modules. First, in the time series prediction module, we employ the multi-stage attention-based networks and integrate the multi-series fusion mechanism. This module can adaptively extract features from the relevant input series and capture the long-term temporal dependencies. Secondly, in the change point detection module, we use the wavelet analysis-based algorithm to detect change points efficiently and identify the change points and outliers. Extensive experiments are conducted on various real-world datasets and synthetic datasets, proving the superiority and effectiveness of CPMAN. Our approach outperforms the state-of-the-art methods by up to 12.1% on the F1 Score. [ABSTRACT FROM AUTHOR]

The 3D measurement system based on line-structured light uses a camera to capture laser stripes due to changing in the shape of an object, and uses the acquired pixel coordinates for 3D reconstruction. System calibration is an important step in 3D measurement. The current camera calibration algorithm research mainly focuses on improving the algorithm itself, and there is less research on the influence of external factors. This paper proposes a coplanar hybrid calibration algorithm based on the error screening model by combining the error screening model, mathematical model and neural network model. It is mainly divided into two steps. The first step is to use the radial array constraint calibration algorithm based on the error screening model to solve the camera's internal and external parameters. The second step uses the camera internal and external parameters obtained in the first step to convert the pixel coordinates into real three-dimensional coordinates, and compares the calculated three-dimensional coordinates with the actual coordinates. Using machine learning to establish a compensation network, get a compensation function, and use the resulting 3D world coordinates to perform point cloud stitching. Experiments show that compared with the traditional calibration algorithm, the calibration algorithm has a small error and reduces the calibration error by about 6.5%. [ABSTRACT FROM AUTHOR]

Homology Continuity is a fundamental property of the nature, but few of the traditional pattern recognition algorithms were aware of it. Firstly, this paper gives a brief description to the Principle of Homology Continuity (PHC), and tries to mathematically redefine it. Then, we introduce a PHC-based pattern learning method — Geometrical Covering Learning (GCL), following the Hyper sausage neural network as an instance of GCL. Lastly, we propose a GCL solution to the “two-spirals” pattern recognition problem. The final experimental results show that the new method is feasible and efficient. [ABSTRACT FROM AUTHOR]

This paper intends to propose an integrated hybrid algorithm for training radial basis function neural network (RBFNN) learning. The proposed integrated of particle swarm and genetic algorithm based optimization (IPGO) algorithm is composed of two approaches based on particle swarm optimization (PSO) and genetic algorithm (GA) for gathering both their virtues to improve the learning performance of RBFNN. The diversity of individuals results in higher chance to search in the direction of global optimal instead of being confined to local optimal particularly in problem with higher complexity. The IPGO algorithm with PSO-based and GA-based approaches has shown promising results in some benchmark problems with three continuous test functions. After proposing the algorithm for these problems with result providing its outperforming performance, this paper supplements a practical application case for the papaya milk sales forecasting to expound the superiority of the IPGO algorithm. In addition, model evaluation results of the case have showed that the IPGO algorithm outperforms other algorithms and auto-regressive moving average (ARMA) models in terms of forecasting accuracy and execution time. [ABSTRACT FROM AUTHOR]

One of the greatest challenges in data mining is related to processing and analysis of massive data streams. Contrary to traditional static data mining problems, data streams require that each element is processed only once, the amount of allocated memory is constant and the models incorporate changes of investigated streams. A vast majority of available methods have been developed for data stream classification and only a few of them attempted to solve regression problems, using various heuristic approaches. In this paper, we develop mathematically justified regression models working in a time-varying environment. More specifically, we study incremental versions of generalized regression neural networks, called IGRNNs, and we prove their tracking properties - weak (in probability) and strong (with probability one) convergence assuming various concept drift scenarios. First, we present the IGRNNs, based on the Parzen kernels, for modeling stationary systems under nonstationary noise. Next, we extend our approach to modeling time-varying systems under nonstationary noise. We present several types of concept drifts to be handled by our approach in such a way that weak and strong convergence holds under certain conditions. Finally, in the series of simulations, we compare our method with commonly used heuristic approaches, based on forgetting mechanism or sliding windows, to deal with concept drift. Finally, we apply our concept in a real life scenario solving the problem of currency exchange rates prediction. [ABSTRACT FROM AUTHOR]

The construction of a Spiking Neural Network (SNN), i.e. the choice of an appropriate topology and the configuration of its internal parameters, represents a great challenge for SNN based applications. Evolutionary Algorithms (EAs) offer an elegant solution for these challenges and methods capable of exploring both types of search spaces simultaneously appear to be the most promising ones. A variety of such heterogeneous optimization algorithms have emerged recently, in particular in the field of probabilistic optimization. In this paper, a literature review on heterogeneous optimization algorithms is presented and an example of probabilistic optimization of SNN is discussed in detail. The paper provides an experimental analysis of a novel Heterogeneous Multi-Model Estimation of Distribution Algorithm (hMM-EDA). First, practical guidelines for configuring the method are derived and then the performance of hMM-EDA is compared to state-of-the-art optimization algorithms. Results show hMM-EDA as a light-weight, fast and reliable optimization method that requires the configuration of only very few parameters. Its performance on a synthetic heterogeneous benchmark problem is highly competitive and suggests its suitability for the optimization of SNN. [ABSTRACT FROM AUTHOR]

A Centroid Neural Network with Weighted Features (CNN-WF) is proposed and presented in this paper. The proposed CNN-WF is based on a Centroid Neural Network (CNN), an effective clustering tool that has been successfully applied to various problems. In order to evaluate the importance of each feature in a set of data, a feature weighting concept is introduced to the Centroid Neural Network in the proposed algorithm. The weight update equations for CNN-WF are derived by applying the Lagrange multiplier procedure to the objective function constructed for CNN-WF in this paper. The use of weighted features makes it possible to assess the importance of each feature and to reject features that can be considered as noise in data. Experiments on a synthetic data set and a typical image compression problem show that the proposed CNN-WF can assess the importance of each feature and the proposed CNN-WF outperforms conventional algorithms including the Self-Organizing Map (SOM) and CNN in terms of clustering accuracy. [ABSTRACT FROM AUTHOR]

This paper investigates the convergence theorems that are associated with a Discrete Hopfield Neural Network (DHNN) with delay. We present two updating rules, one for serial mode and the other for parallel mode. The speed of convergence of these proposed updating rules is faster than all of the existing updating rules. It has been proved in this paper that a DHNN with delay will converge to a stable state when operating in a serial mode if the matrix of weights of the no-delay term is symmetric. In addition, it has been proved that they will converge to a stable state when operating in a parallel mode if the matrix of weights of the no-delay term is a symmetric and non-negative definite matrix. The condition for convergence of a DHNN without delay can been relaxed from the need to have a symmetric matrix to an even weaker condition of having a quasi-symmetric matrix. The results in this paper extend both the existing results concerning the convergence of a DHNN without delay and our previous findings. By means of the new network structure and its convergence theorems, we propose a local searching algorithm for combinatorial optimization. We also relate the maximum value of a bivariate energy function to the stable states of a DHNN with delay, which generalizes Hopfield's energy function. Moreover, for the serial model we give the relationship between the convergence of the energy function and the convergence of the corresponding network. One application is presented to demonstrate the higher rate of convergence and the accuracy of the classification using our algorithm. [ABSTRACT FROM AUTHOR]

This paper describes the application of two evolutionary algorithms to the feedforward neural networks used in classification problems. Besides of a simple backpropagation feedforward algorithm, the paper considers the genetic algorithm and random search algorithm. The objective is to analyze the performance of GAs over the simple backpropagation feedforward in terms of accuracy or speed in this problem. The experiments considered a feedforward neural network trained with genetic algorithm/random search algorithm and 39 types of network structures and artificial data sets. In most cases, the evolutionary feedforward neural networks seemed to have better of equal accuracy than the original backpropagation feedforward neural network. We found few differences in the accuracy of the networks solved by applying the EAs, but found ample differences in the execution time. The results suggest that the evolutionary feedforward neural network with random search algorithm might be the best algorithm on the data sets we tested. [ABSTRACT FROM AUTHOR]

This paper deals with the application of neural networks to design intelligent nonlinear predictive controllers. Predictive controllers are now widely used in many industrial applications. They have been used for linear systems in early applications and then some methods based on predictive control theory were proposed to govern the dynamics of nonlinear systems. In this paper, we will make use of multi-layer perceptron neurofuzzy models with Locally Linear Model Tree (LoLiMoT) learning algorithm as a part of intelligent predictive control system, which has shown excellent performance in identifying of nonlinear systems. The nonlinear dynamics of the system is identified using the neural network based method and then the identified model is used as a part of predictive control algorithm. The proposed method is used to solve the control problems in some benchmark systems. As a first study, the viscosity control in a Continuous Stirred Tank Reactor (CSTR) plant is considered. The mathematical model of the plant is used to generate the input output data set and then the dynamic behavior of the system is identified using a proper multi-layer perceptron neural network, which is used in the predictive control loop. Also, the predictive control based on the locally linear neurofuzzy model is applied to temperature control of an electrically heated micro heat exchanger. The dynamic behavior of the heat exchanger is identified based on some experimental data of the real plant. Comparing the identification results obtained by the neurofuzzy model with those of some linear models such as ARX and BJ, confirms the superior performance for the locally linear neurofuzzy model. Then, the predictive control is applied to the identified model to obtain a satisfactory performance in the output temperature that should track a desired reference signal. As another application, the algorithm is applied to temperature control of a solution polymerization methyl methacrylate in a batch reactor. The results show also somehow satisfactory performance for this highly nonlinear system. All the simulation results reveal the effectiveness of the proposed intelligent control strategy. [ABSTRACT FROM AUTHOR]

In this paper we investigate a hybrid model based on the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. Also we discuss different variants for hybrid models using the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. The Discrete Gradient method has the advantage of being able to jump over many local minima and find very deep local minima. However, earlier research has shown that a good starting point for the discrete gradient method can improve the quality of the solution point. Evolutionary algorithms are best suited for global optimisation problems. Nevertheless they are cursed with longer training times and often unsuitable for real world application. For optimisation problems such as weight optimisation for ANNs in real world applications the dimensions are large and time complexity is critical. Hence the idea of a hybrid model can be a suitable option. In this paper we propose different fusion strategies for hybrid models combining the evolutionary strategy with the discrete gradient method to obtain an optimal solution much quicker. Three different fusion strategies are discussed: a linear hybrid model, an iterative hybrid model and a restricted local search hybrid model. Comparative results on a range of standard datasets are provided for different fusion hybrid models. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel approach of implementing a combination methodology to find appropriate neural network architecture and weights using an evolutionary least square based algorithm (GALS). This paper focuses on aspects such as the heuristics of updating -weights using an evolutionary least square based algorithm, finding the number of hidden neurons for a two layer feed forward neural network, the stopping criterion for the algorithm and finally some comparisons of the results with other existing methods for searching optimal or near optimal solution in the multidimensional complex search space comprising the architecture and the weight variables. We explain how the weight updating algorithm using evolutionary least square based approach can be combined with the growing architecture model to find the optimum number of hidden neurons. We also discuss the issues of finding a probabilistic solution space as a starting point for the least square method and address the problems involving fitness breaking. We apply the proposed approach to XOR problem, 10 bit odd parity problem and many realworld benchmark data sets such as handwriting data set from CEDAR, breast cancer and heart disease data sets from UCI ML repository. The comparative results based on classification accuracy and the time complexity are discussed. [ABSTRACT FROM AUTHOR]

Differing from the conventional back-propagation (BP) neural networks, a novel multi-input Euler polynomial neural network, in short, MIEPNN (specifically, 4-input Euler polynomial neural network, 4IEPNN) is established and investigated in this paper. In order to achieve satisfactory performance of the established MIEPNN, a weights and structure determination (WASD) algorithm with pruning-while-growing (PWG) and twice-pruning (TP) techniques is built up for the established MIEPNN. By employing the weights direct determination (WDD) method, the WASD algorithm not only determines the optimal connecting weights between hidden layer and output layer directly, but also obtains the optimal number of hidden-layer neurons. Specifically, a sub-optimal structure is obtained via the PWG technique, then the redundant hidden-layer neurons are further pruned via the TP technique. Consequently, the optimal structure of the MIEPNN is obtained. To provide a reasonable choice in practice, several different MATLAB computing routines related to the WDD method are studied. Comparative numerical-experiment results of the 4IEPNN using these different MATLAB computing routines and the standard multi-layer perceptron (MLP) neural network further verify the superior performance and efficacy of the proposed MIEPNN equipped with the WASD algorithm including PWG and TP techniques in terms of training, testing and predicting. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is an efficient and practical learning algorithm used for training single hidden layer feed-forward neural networks (SLFNs). ELM can provide good generalization performance at extremely fast learning speed. However, ELM suffers from instability and over-fitting, especially on relatively large datasets. Based on probabilistic SLFNs, an approach of fusion of extreme learning machine (F-ELM) with fuzzy integral is proposed in this paper. The proposed algorithm consists of three stages. Firstly, the bootstrap technique is employed to generate several subsets of original dataset. Secondly, probabilistic SLFNs are trained with ELM algorithm on each subset. Finally, the trained probabilistic SLFNs are fused with fuzzy integral. The experimental results show that the proposed approach can alleviate to some extent the problems mentioned above, and can increase the prediction accuracy. [ABSTRACT FROM AUTHOR]

Machine learning methods, such as neural network (NN) and support vector machine, assume that the training data and the test data are drawn from the same distribution. This assumption may not be satisfied in many real world applications, like long-term financial failure prediction, because the training and test data may each come from different time periods or domains. This paper proposes a novel algorithm known as fuzzy bridged refinement-based domain adaptation to solve the problem of long-term prediction. The algorithm utilizes the fuzzy system and similarity concepts to modify the target instances' labels which were initially predicted by a shift-unaware prediction model. The experiments are performed using three shift-unaware prediction models based on nine different settings including two main situations: (1) there is no labeled instance in the target domain; (2) there are a few labeled instances in the target domain. In these experiments bank failure financial data is used to validate the algorithm. The results demonstrate a significant improvement in the predictive accuracy, particularly in the second situation identified above. [ABSTRACT FROM AUTHOR]

A preprocessing stage in every speech/music applications including audio/speech separation, speech/speaker recognition and audio/genre transcription task is inevitable. The importance of such pre-processing stage is originated from the requisite of determining each frame of the given signal is belonged to which classes, namely: speech only, music only or speech/music mixture. Such classification can significantly decrease the computational burden due to exhaustive search commonly introduced as a problem in model-based speech recognition or separation as well as music transcription scenarios. In this paper, we present a new method to separate mixed type audio frames based on support vector machine (SVM) and neural network. We present a feature type selection algorithm which seeks for the most appropriate features to discriminate possible classes (hypotheses) on the mixed signal. We also propose features based on eigen-decomposition on the mixed frame. Experimental results demonstrate that the proposed features together with the selected audio classifiers achieve acceptable classification results. From the experimental results, it is observed that the proposed system outperforms other classification systems including k-nearest neighbor (k-NN) and multi-layer perceptron (MLP). [ABSTRACT FROM AUTHOR]

Conjugate gradient methods constitute excellent neural network training methods which are characterized by their simplicity and their very low memory requirements. In this paper, we propose a new spectral conjugate gradient method which guarantees sufficient descent using any line search, avoiding thereby the usually inefficient restarts. Moreover, we establish the global convergence of our proposed method under some assumptions. Experimental results provide evidence that our proposed method is preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness. [ABSTRACT FROM AUTHOR]

This paper presents a novel model for performing classification and visualization of high-dimensional data by means of combining two enhancing techniques. The first is a semi-supervised learning, an extension of the supervised learning used to incorporate unlabeled information to the learning process. The second is an ensemble learning to replicate the analysis performed, followed by a fusion mechanism that yields as a combined result of previously performed analysis in order to improve the result of a single model. The proposed learning schema, termed S2-Ensemble, is applied to several unsupervised learning algorithms within the family of topology maps, such as the Self-Organizing Maps and the Neural Gas. This study also includes a thorough research of the characteristics of these novel schemes, by means quality measures, which allow a complete analysis of the resultant classifiers from the viewpoint of various perspectives over the different ways that these classifiers are used. The study conducts empirical evaluations and comparisons on various real-world datasets from the UCI repository, which exhibit different characteristics, so to enable an extensive selection of situations where the presented new algorithms can be applied. [ABSTRACT FROM AUTHOR]

The assessment of the risk of default on credit is important for financial institutions. Different Artificial Neural Networks (ANN) have been suggested to tackle the credit scoring problem, however, the obtained error rates are often high. In the search for the best ANN algorithm for credit scoring, this paper contributes with the application of an ANN Training Algorithm inspired by the neurons' biological property of metaplasticity. This algorithm is especially efficient when few patterns of a class are available, or when information inherent to low probability events is crucial for a successful application, as weight updating is overemphasized in the less frequent activations than in the more frequent ones. Two well-known and readily available such as: Australia and German data sets has been used to test the algorithm. The results obtained by AMMLP shown have been superior to state-of-the-art classification algorithms in credit scoring. [ABSTRACT FROM AUTHOR]

A new approach for nonlinear system identification and control based on modular neural networks (MNN) is proposed in this paper. The computational complexity of neural identification can be greatly reduced if the whole system is decomposed into several subsystems. This is obtained using a partitioning algorithm. Each local nonlinear model is associated with a nonlinear controller. These are also implemented by neural networks. The switching between the neural controllers is done by a dynamical switcher, also implemented by neural networks, that tracks the different operating points. The proposed multiple modelling and control strategy has been successfully tested on simulated laboratory scale liquid-level system. [ABSTRACT FROM AUTHOR]

Resistance spot welding is a major metal connecting method in vehicle and other domestic electronic domains. Among all the welding techniques, the spot welding inverter is an important direction at the present time. The high nonlinearity and strongly coupled multiple parameters in the resistance spot welding process challenge the classical control theory based on some specific conditions and ideal assumptions, which in real practice obstacle the high-quality welding. This paper put the fuzzy neural network into a constant-current-controlled spot welding inverter, where the welding current peak and its variation are adopted as the input parameters and the duty ratio of the switches is regarded as the output. Eventually a five-layer feed-forward network was constructed, back propagation (BP) algorithm was applied to revise the adjustable parameters in the network, and a mathematical model was established to obtain the training samples serving for the network. The ultimate precision could reach 1.75%, the relative control error is 2.28% with strong external disturbances, the overmodulation is 3.35%, and the total modulating period is seven switching period, which indicated that the proposed algorithm has good performance. [ABSTRACT FROM AUTHOR]

Occlusion handling is an old but important problem for the computer vision and pattern recognition community. Features from different objects may twist with each other, and any matched feature points may belong to different objects for many traditional object recognition algorithms. To recognize occlusions, we should not only match objects from different view points but also match features extracted from the same object. In this paper, we propose a method to consider these two perspectives simultaneously by encoding various types of features, such as geometry, color and texture relationships among feature points into a matrix and find the best quadratic feature correlation model to fit them. Experiments on our own built dataset and the publicly available PASCAL VOC dataset shows that, our method can robustly classify objects and handle occluded objects under large occlusions, and the performance is among the state-of-the-art. [ABSTRACT FROM AUTHOR]

Taking advantage of both the scaling property of wavelets and the high learning ability of neural networks, wavelet networks have recently emerged as a powerful tool in many applications in the field of signal processing such as data compression, function approximation as well as image recognition and classification. A novel wavelet network-based method for image classification is presented in this paper. The method combines the Orthogonal Least Squares algorithm (OLS) with the Pyramidal Beta Wavelet Network architecture (PBWN). First, the structure of the Pyramidal Beta Wavelet Network is proposed and the OLS method is used to design it by presetting the widths of the hidden units in PBWN. Then, to enhance the performance of the obtained PBWN, a novel learning algorithm based on orthogonal least squares and frames theory is proposed, in which we use OLS to select the hidden nodes. In the simulation part, the proposed method is employed to classify colour images. Comparisons with some typical wavelet networks are presented and discussed. Simulations also show that the PBWN-orthogonal least squares (PBWN-OLS) algorithm, which combines PBWN with the OLS algorithm, results in better performance for colour image classification. [ABSTRACT FROM AUTHOR]

The development of satellite image acquisition tools helped improving the extraction of information about natural scenes. In the proposed approach, we try to minimize imperfections accompanying the image interpretation process and to maximize useful information extracted from these images through the use of blind source separation (BSS) and fusion methods. In order to extract maximum information from multi-sensor images, we propose to use three algorithms of BSS that are FAST- ICA2D, JADE2D, and SOBI2D. Then by employing various fusion methods such as the probability, possibility, and evidence methods we can minimize both imprecision and uncertainty. In this paper, we propose a hybrid approach based on five main steps. The first step is to apply the three BSS algorithms to the satellites images; it results in obtaining a set of image sources representing each a facet of the land cover. A second step is to choose the image having the maximum of kurtosis and negentropy. After the BSS evaluation, we proceed to the training step using neural networks. The goal of this step is to provide learning regions which are useful for the fusion step. The next step consists in choosing the best adapted fusion method for the selected source images through a case-based reasoning (CBR) module. If the CBR module does not contain a case similar to the one we are seeking, we proceed to apply the three fusion methods. The evaluation of fusion methods is a necessary step for the learning process of our CBR. [ABSTRACT FROM AUTHOR]

Revealing the corresponding identities of the same individual in different systems is a common task in various areas, e.g., criminals inter-network tracking, homologous proteins revealing, ancient words translating, and so on. With the reason that, recently, more and more complex systems are described by networks, this task can also be accomplished by solving a node matching problem among these networks. Revealing one-to-one matching between networks is for sure the best if we can, however, when the target networks are highly symmetric, or an individual has different identities (corresponds to several nodes) in the same network, the exact one-to-one node matching algorithms always lose their effects to obtain acceptable results. In such situations, one-to-many (or many-to-many) node matching algorithms may be more useful. In this paper, we propose two one-to-many node matching algorithms based on local mapping and ensembling, respectively. Although such algorithms may not tell us the exact correspondence of the identities in different systems, they can indeed help us to narrow down the inter-network searching range, and thus are of significance in practical applications. These results have been verified by the matching experiments on pairwise artificial networks and real-world networks. [ABSTRACT FROM AUTHOR]

This paper proposes a supervised training algorithm for Spiking Neural Networks (SNNs) which modifies the Spike Timing Dependent Plasticity (STDP)learning rule to support both local and network level training with multiple synaptic connections and axonal delays. The training algorithm applies the rule to two and three layer SNNs, and is benchmarked using the Iris and Wisconsin Breast Cancer (WBC) data sets. The effectiveness of hidden layer dynamic threshold neurons is also investigated and results are presented. [ABSTRACT FROM AUTHOR]

This paper demonstrates how knowledge can be extracted from evolving spiking neural networks with rank order population coding. Knowledge discovery is a very important feature of intelligent systems. Yet, a disproportionally small amount of research is centered on the issue of knowledge extraction from spiking neural networks which are considered to be the third generation of artificial neural networks. The lack of knowledge representation compatibility is becoming a major detriment to end users of these networks. We show that a high-level knowledge can be obtained from evolving spiking neural networks. More specifically, we propose a method for fuzzy rule extraction from an evolving spiking network with rank order population coding. The proposed method was used for knowledge discovery on two benchmark taste recognition problems where the knowledge learnt by an evolving spiking neural network was extracted in the form of zero-order Takagi-Sugeno fuzzy IF-THEN rules. [ABSTRACT FROM AUTHOR]

This paper presents a study on wavelets and their characteristics for the specific purpose of serving as a feature extraction tool for speaker verification (SV), considering a Radial Basis Function (RBF) classifier, which is a particular type of Artificial Neural Network (ANN). Examining characteristics such as support-size, frequency and phase responses, amongst others, we show how Discrete Wavelet Transforms (DWTs), particularly the ones which derive from Finite Impulse Response (FIR) filters, can be used to extract important features from a speech signal which are useful for SV. Lastly, an SV algorithm based on the concepts presented is described. [ABSTRACT FROM AUTHOR]

Motion blur is one of the most common causes of image corruptions caused by blurring. Several methods have been presented up to now, which precisely identify linear motion blur parameters, but most of them possessed low precision in the presence of the noise. The present paper is aimed to introduce an algorithm for estimating linear motion blur parameters in noisy images. This study presents a method to estimate motion direction by using Radon transform, which is followed by the application of two other different methods to estimate motion length; the first of which is based on one-dimensional power spectrum to estimate parameters of noise free images and the second uses bispectrum modeling in noisy images. A Feed-Forward Back Propagation neural network has been designed on the basis of Weierstrass approximation theorem to model bispectrum and the Delta rule as the network learning rule. The methods were tested on several standard images like Camera man, Lena, Lake, etc. that were degraded by linear motion blur and additive noise. The experimental results have been satisfactory. The proposed method, compared to other related methods, suggests an improvement in the supported lowest SNR and precision of estimation. [ABSTRACT FROM AUTHOR]

This paper presents a repair algorithm for the design of a Radial Basis Function (RBF) neural network. The proposed repair RBF (RRBF) algorithm starts from a single prototype randomly initialized in the feature space. The algorithm has two main phases: an architecture learning phase and a parameter adjustment phase. The architecture learning phase uses a repair strategy based on a sensitivity analysis (SA) of the network's output to judge when and where hidden nodes should be added to the network. New nodes are added to repair the architecture when the prototype does not meet the requirements. The parameter adjustment phase uses an adjustment strategy where the capabilities of the network are improved by modifying all the weights. The algorithm is applied to two application areas: approximating a non-linear function, and modeling the key parameter, chemical oxygen demand (COD) used in the waste water treatment process. The results of simulation show that the algorithm provides an efficient solution to both problems. [ABSTRACT FROM AUTHOR]

A prediction scheme for Ethernet traffic data using a Multiscale-Bilinear Recurrent Neural Network with Adaptive Learning (M-BRNN-AL) is proposed and presented in this paper. The proposed predictor integrates an M-BRNN and an AL algorithm. In M-BRNN, the wavelet transform is employed to decompose the original traffic signals into several simple traffic signals. A BRNN is then used to predict each decomposed traffic signal. An AL algorithm is also applied in order to improve the learning process at each resolution level in M-BRNN-AL. Experiments and results on a set of Ethernet network traffic predictions show that the proposed scheme converges faster and archives better prediction performance than the other conventional models such as the Multi-layer Perception Type Neural Network, BRNN, and the original M-BRNN in terms of the normalized mean square error. [ABSTRACT FROM AUTHOR]

In this paper, two different methodologies respectively based on an unsupervised self-organizing (SOM) neural network and on a graph matching are shown and discussed to validate the performance of a new 3D facial feature identification and localization algorithm. Experiments are performed on a dataset of 23 3D faces acquired by a 3D laser camera at eBIS lab with pose and expression variations. In particular results referred to five nose landmarks are encouraging and reveal the validity of this approach that although low computational complexity and the small number of landmarks guarantees an average face recognition performance greater than 80%. [ABSTRACT FROM AUTHOR]

Unlike feedforward neural networks (FFNN) which can act as universal function approximators, recursive, or recurrent, neural networks can act as universal approximators for multi-valued functions. In this paper, a real time recursive backpropagation (RTRBP) algorithm in a vector matrix form is developed for a two-layer globally recursive neural network that has multiple delays in its feedback path. This algorithm has been evaluated on two GRNNs that approximate both an analytic and nonanalytic periodic multi-valued function that a feedforward neural network is not capable of approximating. [ABSTRACT FROM AUTHOR]

This paper presents the graffiti interpretation of one-stroke handwritten digits (0 to 9) and commands (backspace, carriage return and space) using support vector machines (SVMs). A number of SVM-based graffiti interpreters are proposed for the recognition of graffiti. The performance of the proposed SVM-based graffiti interpreters subject to various kernel functions and parameters are investigated. Simulation and experimental results are presented to show the applicability and the merits of various graffiti interpreters. [ABSTRACT FROM AUTHOR]

Neural network was introduced to sneak circuit analysis (SCA) in previous works. However, it may generate suspect results which were hard to explain. To overcome the shortcomings, this paper proposed a novel neural network model based on circuit architecture, named CArNN, which is used as an individual of an ensemble. In CArNN, neurons represented system components, and weights represented the joints between components. Models of neurons are sigmoid functions. Clone selection algorithm was used to train CArNNs population. The trained antibodies were used as individuals of an ensemble. The inputs of CArNN are states of switches, and the outputs are states of functional components. Ensemble predicted all possible functions of circuit. The sneak circuits can be discovered by comparing the predicted and designed functions. The results revealed that CArNNs can exactly discover sneak circuits. [ABSTRACT FROM AUTHOR]

In this paper, we design a cellular automaton and a discrete-time cellular neural network (DTCNN) using nonlinear passive memristors. They can perform a number of applications, such as logical operations, image processing operations, complex behaviors, higher brain functions, RSA algorithm, etc. By modifying the characteristics of nonlinear memristors, the memristor DTCNN can perform almost all functions of memristor cellular automaton. Furthermore, it can perform more than one function at the same time, that is, it allows multitasking. [ABSTRACT FROM AUTHOR]

This paper investigates robust exponential stability for discrete-time recurrent neural networks with both time-varying delay (0 ≤ τm ≤ τ(k) ≤ τM) and distributed one. Through partitioning delay intervals [0,τm] and [τm,τM], respectively, and choosing an augmented Lyapunov-Krasovskii functional, the delay-dependent sufficient conditions are obtained by using free-weighting matrix and convex combination methods. These criteria are presented in terms of linear matrix inequalities (LMIs) and their feasibility can be easily checked by resorting to LMI in Matlab Toolbox in Ref. 1. The activation functions are not required to be differentiable or strictly monotonic, which generalizes those earlier forms. As an extension, we further consider the robust stability of discrete-time delayed Cohen-Grossberg neural networks. Finally, the effectiveness of the proposed results is further illustrated by three numerical examples in comparison with the reported ones. [ABSTRACT FROM AUTHOR]

In this paper, a new machine vision algorithm for close-range position sensing and bin picking is presented where a Hopfield Neural Network (HNN) is used for the stereo matching process. Stereo Matching is formulated as an energy minimization task and this minimization is accomplished using the HNNs. Various other important aspects of this Vision System are discussed including camera calibration and objects localization using a clustering algorithm. [ABSTRACT FROM AUTHOR]

This paper presents a new approach, known as symmetry axis based feature extraction and recognition (SAFER), for recognizing objects under translation, rotation and scaling. Unlike most previous invariant object recognition (IOR) systems, SAFER puts emphasis on both simplicity and accuracy of the recognition system. To achieve simplicity, it uses simple formulae for extracting invariant features from an object. The scheme used in feature extraction is based on the axis of symmetry and angles of concentric circles drawn around the object. SAFER divides the extracted features into a number of groups based on their similarity. To improve the recognition performance, SAFER uses a number of neural networks (NNs) instead of single NN are used for training and recognition of extracted features. The new approach, SAFER, has been tested on two of real world problems i.e., English characters with two different fonts and images of different shapes. The experimental results show that SAFER can produce good recognition performance in comparison with other algorithms. [ABSTRACT FROM AUTHOR]

Recently, a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden-layer feedforward neural networks (SLFNs). It was much faster than the traditional gradient-descent-based learning algorithms due to the analytical determination of output weights with the random choice of input weights and hidden layer biases. However, this algorithm often requires a large number of hidden units and thus slowly responds to new observations. Evolutionary extreme learning machine (E-ELM) was proposed to overcome this problem; it used the differential evolution algorithm to select the input weights and hidden layer biases. However, this algorithm required much time for searching optimal parameters with iterative processes and was not suitable for data sets with a large number of input features. In this paper, a new approach for training SLFNs is proposed, in which the input weights and biases of hidden units are determined based on a fast regularized least-squares scheme. Experimental results for many real applications with both small and large number of input features show that our proposed approach can achieve good generalization performance with much more compact networks and extremely high speed for both learning and testing. [ABSTRACT FROM AUTHOR]

Some neural network related methods have been applied to nonlinear fuzzy regression analysis by several investigators. The performance of these methods will significantly worsen when the outliers exist in the training data set. In this paper, we propose a training algorithm for fuzzy neural networks with general fuzzy number weights, biases, inputs and outputs for computation of nonlinear fuzzy regression models. First, we define a cost function that is based on the concept of possibility of fuzzy equality between the fuzzy output of fuzzy neural network and the corresponding fuzzy target. Next, a training algorithm is derived from the cost function in a similar manner as the back-propagation algorithm. Last, we examine the ability of our approach by computer simulations on numerical examples. Simulation results show that the proposed algorithm is able to reduce the outlier effects. [ABSTRACT FROM AUTHOR]

Having more hidden units than necessary can produce a neural network that has a poor generalization. This paper proposes a new algorithm for pruning unnecessary hidden units away from the single-hidden layer feedforward neural networks, resulting in a Spartan network. Our approach is simple and easy to implement, yet produces a very good result. The idea is to train the network until it begins to lose its generalization. Then the algorithm measures the sensitivity and automatically prunes away the most irrelevant unit. We define this sensitivity as the absolute difference between the desirable output and the output of the pruned network. Unlike other pruning methods, our algorithm is distinct in calculating the sensitivity from the validation set, instead of the training set, without increasing the asymptotic time complexity of the back-propagation algorithm. In addition, for a classification problem, we raise a point that the sensitivities of some well-known pruning algorithms may still underestimate the irrelevance of hidden unit even though the validation set is used in measuring the sensitivity. We resolve this problem by considering the number of misclassified patterns as the main concern. The Spartan simplicity algorithm is applied to three artificial and seven standard benchmarks. In most problems, the algorithm can produce a compact-sized network with high generalization ability in comparison with other pruning algorithms. [ABSTRACT FROM AUTHOR]

HAM (Hopfield Associative Memory) and BAM (Bidirectinal Associative Memory) are representative associative memories by neural networks. The storage capacity by the Hebb rule, which is often used, is extremely low. In order to improve it, some learning methods, for example, pseudo-inverse matrix learning and gradient descent learning, have been introduced. Oh introduced pseudo-relaxation learning algorithm to HAM and BAM. In order to accelerate it, Hattori proposed quick learning. Noest proposed CAM (Complex-valued Associative Memory), which is complex-valued HAM. The storage capacity of CAM by the Hebb rule is also extremely low. Pseudo-inverse matrix learning and gradient descent learning have already been generalized to CAM. In this paper, we apply pseudo-relaxation learning algorithm to CAM in order to improve the capacity. [ABSTRACT FROM AUTHOR]

The present manuscript aims at illustrating fundamental challenges and solutions arising in the design of learning theories by optimization on manifolds in the context of complex-valued neural systems. The special case of a unitary unimodular group of matrices is dealt with. The unitary unimodular group under analysis is a low dimensional and easy-to-handle matrix group. Notwithstanding, it exhibits a rich geometrical structure and gives rise to interesting speculations about methods to solve optimization problems on manifolds. Also, its low dimension allows us to treat most of the quantities involved in computation in closed form as well as to render them in graphical format. Some numerical experiments are presented and discussed within the paper, which deal with complex-valued independent component analysis. [ABSTRACT FROM AUTHOR]

This paper introduces an Independent Component Analysis (ICA) approach to the separation of nonlinear mixtures in the complex domain. Source separation is performed by a complex INFOMAX approach. The neural network which realizes the separation employs the so called "Mirror Model" and is based on adaptive activation functions, whose shape is properly modified during learning. Nonlinear functions involved in the processing of complex signals are realized by pairs of spline neurons called "splitting functions", working on the real and the imaginary part of the signal respectively. Theoretical proof of existence and uniqueness of the solution under proper assumptions is also provided. In particular a simple adaptation algorithm is derived and some experimental results that demonstrate the effectiveness of the proposed solution are shown. [ABSTRACT FROM AUTHOR]

Formula extraction is the first stage of optical formula recognition which converts printed scientific documents into their corresponding electronic format. So far, little research has been done in this area. In this paper, an improved method using fuzzy classification and irregularity rate feature is proposed to separate formulas from texts in the printed documents. Firstly, according to a statistical threshold of distance, connected components are extracted and merged to form the areas of characters and lines. Secondly, the isolated formulas are extracted based on the line features. Finally, the formula symbols in the rest lines are labeled using irregularity degree, and the embedded formulas are located by extending kernel symbols using the propagation of context. In these steps, fuzzy classification algorithm and irregularity degree feature are introduced to solve the problems existing in traditional methods and improve the extracting accuracy. The experimental results show that the method is of great significance in both theory and practice. [ABSTRACT FROM AUTHOR]