Search

This paper presents two algorithms, both based on the time of flight (ToF) of scattered waves, to locate defect in an anisotropic woven-fabric carbon fiber reinforced polymer (CFRP) plate. The first algorithm uses a probabilistic approach by constructing a probability matrix. Each element of this matrix is associated with a location and represents the probability of existing a defect at the corresponding spatial coordinates of the plate. For the probability matrix method, localization results are influenced by manually chosen parameters and by the anisotropy of the CFRP plate. The second algorithm, based on artificial neural networks (ANNs), enables us to improve the accuracy of locating defects. With this ANN method, the anisotropic feature can be “learned” by the neural network from training data. The ToF of scattered waves obtained from three sensor pairs were used directly as inputs of the neural network. The spatial coordinates of the defect are the ANN outputs. The difficulty of obtaining sufficient experimental data for ANN training was surpassed by using added blocks on the surface of the plate under test to simulate defects. This scheme was validated and proved to be effective by comparing the scattered waves from a delamination and from an added block. Conclusions have been drawn by comparing the two methods. The localization results obtained by the ANN algorithm are proved to be better. [ABSTRACT FROM AUTHOR]

Semantic segmentation tasks can be well modeled by Markov Random Field (MRF). This paper addresses semantic segmentation by incorporating high-order relations and mixture of label contexts into MRF. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms. It has several appealing properties. First, different from the recent works that required many iterations of MF during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing models as its special cases. Furthermore, pairwise terms in DPN provide a unified framework to encode rich contextual information in high-dimensional data, such as images and videos. Third, DPN makes MF easier to be parallelized and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated on standard semantic image/video segmentation benchmarks, where a single DPN model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012, Cityscapes dataset and CamVid dataset. [ABSTRACT FROM AUTHOR]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures. [ABSTRACT FROM PUBLISHER]

This paper presents a neural network-based method to generate multiple images with different exposures from a single input low dynamic range (LDR) image for improved high dynamic range (HDR) imaging. The proposed algorithm consists of three steps: 1) 2-D histogram estimation; 2) neural network-based LDR images estimation; and 3) generation of an optimal set of differently exposed images. The proposed method first generates image features by estimating a patched-based 2-D histogram. The extracted features are used in an input layer of the neural network, which plays a role to select an optimal set of LDR images. A set of LDR images is generated using a curvature-based contrast enhancement method. Experimental results show that the proposed method can generate an optimal set of LDR images using neural network and provide improved HDR images. In addition, the proposed method can be implemented as a preprocessing step in most existing HDR frameworks. The proposed HDR approach is considered as a single-input method that gives almost the same performance to multiple image-based HDR method. [ABSTRACT FROM PUBLISHER]

Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy. [ABSTRACT FROM PUBLISHER]

Gramophone records were the main recording medium for more than seven decades and regained widespread popularity over the past several years. Being an analog storage medium, gramophone records are subject to distortions caused by scratches, dust particles, degradation, and other means of improper handling. The observed noise often leads to an unpleasant listening experience and requires a filtering process to remove the unwanted disruptions and improve the audio quality. This paper proposes a novel approach that employs various feed forward time delay artificial neural networks to detect and reconstruct noise in musical sound waves. A set of 800 songs from eight different genres were used to validate the performance of the neural networks. The performance was analyzed according to the outlier detection and interpolation accuracy, the computational time and the tradeoff between the accuracy and the time. The empirical results of both detection and reconstruction neural networks were compared to a number of other algorithms, including various statistical measurements, duplication approaches, trigonometric processes, polynomials, and time series models. It was found that the neural networks’ outlier detection accuracy was slightly lower than some of the other noise identification algorithms, but achieved a more efficient tradeoff by detecting most of the noise in real time. The reconstruction process favored neural networks with an increase in the interpolation accuracy compared to other widely used time series models. It was also found that certain genres such as classical, country, and jazz music were interpolated more accurately. Volatile signals, such as electronic, metal, and pop music were more challenging to reconstruct and were substantially better interpolated using neural networks than the other examined algorithms. [ABSTRACT FROM PUBLISHER]

Style transfer is a process of migrating a style from a given image to the content of another, synthesizing a new image, which is an artistic mixture of the two. Recent work on this problem adopting convolutional neural-networks (CNN) ignited a renewed interest in this field, due to the very impressive results obtained. There exists an alternative path toward handling the style transfer task, via the generalization of texture synthesis algorithms. This approach has been proposed over the years, but its results are typically less impressive compared with the CNN ones. In this paper, we propose a novel style transfer algorithm that extends the texture synthesis work of Kwatra et al. (2005), while aiming to get stylized images that are closer in quality to the CNN ones. We modify Kwatra’s algorithm in several key ways in order to achieve the desired transfer, with emphasis on a consistent way for keeping the content intact in selected regions, while producing hallucinated and rich style in others. The results obtained are visually pleasing and diverse, shown to be competitive with the recent CNN style transfer algorithms. The proposed algorithm is fast and flexible, being able to process any pair of content + style images. [ABSTRACT FROM PUBLISHER]

In this paper, we focus on tackling the problem of automatic accurate localization of detected objects in high-resolution remote sensing images. The two major problems for object localization in remote sensing images caused by the complex context information such images contain are achieving generalizability of the features used to describe objects and achieving accurate object locations. To address these challenges, we propose a new object localization framework, which can be divided into three processes: region proposal, classification, and accurate object localization process. First, a region proposal method is used to generate candidate regions with the aim of detecting all objects of interest within these images. Then, generic image features from a local image corresponding to each region proposal are extracted by a combination model of 2-D reduction convolutional neural networks (CNNs). Finally, to improve the location accuracy, we propose an unsupervised score-based bounding box regression (USB-BBR) algorithm, combined with a nonmaximum suppression algorithm to optimize the bounding boxes of regions that detected as objects. Experiments show that the dimension-reduction model performs better than the retrained and fine-tuned models and the detection precision of the combined CNN model is much higher than that of any single model. Also our proposed USB-BBR algorithm can more accurately locate objects within an image. Compared with traditional features extraction methods, such as elliptic Fourier transform-based histogram of oriented gradients and local binary pattern histogram Fourier, our proposed localization framework shows robustness when dealing with different complex backgrounds. [ABSTRACT FROM PUBLISHER]

Knowledge-based neural network modeling techniques using space-mapping concept have been demonstrated in the existing literature as efficient methods to overcome the accuracy limitations of empirical/equivalent circuit models when matching new electromagnetic data. For different modeling problems, the mapping structures can be different. In this paper, we propose a unified automated model generation algorithm that uses \boldsymbol l\mathbf {1} optimization to automatically determine the type and the topology of the mapping structure in a knowledge-based neural network model. By encompassing various types of mappings of the knowledge-based neural network model in the existing literature, we present a new unified model structure and derive new sensitivity formulas for the training of the unified model. The proposed \boldsymbol l\mathbf {1} formulation of modeling can force some weights of the mapping neural networks to zeros while leaving other weights as nonzeros. We utilize this feature to allow \boldsymbol l\mathbf {1} optimization to automatically determine which mapping is necessary and which mapping is unnecessary. Using the proposed \boldsymbol l\mathbf {1} optimization method, the mapping structure can be determined to address different needs of different modeling problems. The structure of the final knowledge-based model can be flexible combinations of some or all of linear mapping, nonlinear mapping, input mapping, frequency mapping, and output mapping. In this way, the proposed algorithm is more systematic and can further speed up the knowledge-based modeling process than existing knowledge-based modeling algorithms. The proposed method is illustrated by three microwave filter modeling examples. [ABSTRACT FROM PUBLISHER]

Neural networks (NNs) have numerous applications to online processes, but the problem of stability is rarely discussed. This is an extremely important issue because, if the stability of a solution is not guaranteed, the equipment that is being used can be damaged, which can also cause serious accidents. It is true that in some research papers this problem has been considered, but this concerns continuous-time NN only. At the same time, there are many systems that are better described in the discrete time domain such as population of animals, the annual expenses in an industry, the interest earned by a bank, or the prediction of the distribution of loads stored every hour in a warehouse. Therefore, it is of paramount importance to consider the stability of the discrete-time NN. This paper makes several important contributions. 1) A theorem is stated and proven which guarantees uniform stability of a general discrete-time system. 2) It is proven that the backpropagation (BP) algorithm with a new time-varying rate is uniformly stable for online identification and the identification error converges to a small zone bounded by the uncertainty. 3) It is proven that the weights' error is bounded by the initial weights' error, i.e., overfitting is eliminated in the proposed algorithm. 4) The BP algorithm is applied to predict the distribution of loads that a transelevator receives from a trailer and places in the deposits in a warehouse every hour, so that the deposits in the warehouse are reserved in advance using the prediction results. 5) The BP algorithm is compared with the recursive least square (RLS) algorithm and with the Takagi–Sugeno type fuzzy inference system in the problem of predicting the distribution of loads in a warehouse, giving that the first and the second are stable and the third is unstable. 6) The BP algorithm is compared with the RLS algorithm and with the Kalman filter algorithm in a synthetic example. [ABSTRACT FROM AUTHOR]

Negative correlation learning (NCL) is a neural network ensemble learning algorithm that introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean square error/MSEI together with the correlation of the ensemble. This paper analyzes NCL and reveals that the training of NCL (when λ = 1) corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This analysis explains the reason why NCL is prone to overtitting the noise in the training set. This paper also demonstrates that tuning the correlation parameter λ in NCL by cross validation cannot overcome the overfitting problem. The paper analyzes this problem and proposes the regularized negative correlation learning (RNCL) algorithm which incorporates an additional regularization term for the whole ensemble. RNCL decomposes the ensemble's training objectives, including MSE and regularization, into a set of sub-objectives, and each sub-objective is implemented by an individual neural network. In this paper, we also provide a Bayesian interpretation for RNCL and provide an automatic algorithm to optimize regularization parameters based on Bayesian inference. The RNCL formulation is applicable to any nonlinear estimator minimizing the MSE. The experiments on synthetic as well as real-world data sets demonstrate that RNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. [ABSTRACT FROM AUTHOR]

Customer loads connected to power distribution net- work may be broadly categorized as either linear or nonlinear. Nonlinear loads inject harmonics into the power network. Harmonics in a power system are classified as either load harmonics or as supply harmonics depending on their origin. The source impedance also impacts the harmonic current flowing in the network. Hence, any change in the source impedance is reflected in the harmonic spectrum of the current. This paper proposes a novel method based on artificial neural networks to isolate and evaluate the impact of the source impedance change without disrupting the operation of any load, by using actual field data. The test site chosen for this paper has a significant amount of triplen harmonics in the current. By processing the acquired data with the proposed algorithm, the actual load harmonic contribution of the customer is predicted. Experimental results confirm that attempting to predict the total harmonic distortion of a customer by simply measuring the customer's current may not be accurate. The main advantage of this method is that only waveforms of voltages and currents at the point of common coupling have to be measured. This method is applicable for both single- and three-phase loads. [ABSTRACT FROM AUTHOR]

Classification of power-quality (PQ)-related voltage and current waveform disturbances is a key task for power system monitoring. A new method based on the optimized time-frequency representation (TFR) has been proposed in the first paper [1] of this two-paper series. This paper (the second paper) presents a case study of PQ event classification with the proposed method. The classification algorithm has been successfully tested with $60 real world PQ events that cover five classes, achieving a recognition rate of 98%. The algorithm is implemented on a digital signal processor (DSP) based hardware system and optimized according to the DSP architecture to meet the hard real-time constraints. The DSP-based system is capable of processing a five-cycle (83.3 ms) PQ waveform within 11.2 ms. The real-time computing capability of the algorithm has been verified with this result. The scalability of this method is also discussed. [ABSTRACT FROM AUTHOR]

As TV viewers inherently tend to avoid contents they might dislike, they do not provide equal amounts of positive and negative feedback on their viewing preferences. At the same time, mobile devices are becoming target platforms for multimedia content delivery. Personalized program guides need to cope with these challenges. They must not only properly recognize the undesired content, despite the lack of the learning data, but also provide valuable recommendations without compromising the user's privacy. Moreover, the complexity of the applied algorithms has to be low enough to match the limited hardware resources of the mobile terminals. In this paper, the design of such program guide is described. Several system architectures are developed and compared. The best performance is achieved for a single hidden layer autoencoder neural network trained with the FISTA algorithm1. [ABSTRACT FROM PUBLISHER]

In this paper, a novel method to make smart recommendations to the user is proposed using Hybrid Broadcast Broadband Television (HbbTV) standard and artificial intelligence. In order to prove this, an HbbTV application is developed to collect real television watching data and a multilayer perceptron model is trained and tested with the collected data to create user profiles. The constructed model is then used to provide recommendations to the users again using the same HbbTV application. Since HbbTV is a widely accepted public consumer electronics standard for digital receivers and most of the recent devices support it, the proposed model is device-agnostic and can reach many people using different consumer devices, which is the main motivation of this study1. [ABSTRACT FROM PUBLISHER]

This paper presents a controller integrity monitoring method for a class of second-order nonlinear uncertain systems incorporating neural network-based adaptive control algorithms. The adaptive neural network model is employed to ensure robust tracking performance in the presence of certain modeling uncertainty under consideration during the offline controller design process. Based on Lyapunov stability analysis, an online controller integrity monitoring scheme is developed to detect the occurrence of controller software/algorithm faults and unanticipated physical component faults, which may lead to unstable learning behaviors and malfunctions of the adaptive controller. Adaptive thresholds for detecting controller malfunctions are derived, ensuring the robustness with respect to modeling uncertainty and neural network approximation error. Additionally, the detectability conditions are investigated, characterizing the class of detectable software faults and unanticipated physical faults. An upper bound on the fault detection time is also established. Simulation results are shown to illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

To improve the accuracy of learning result, in practice multiple parties may collaborate through conducting joint Back-Propagation neural network learning on the union of their respective data sets. During this process no party wants to disclose her/his private data to others. Existing schemes supporting this kind of collaborative learning are either limited in the way of data partition or just consider two parties. There lacks a solution that allows two or more parties, each with an arbitrarily partitioned data set, to collaboratively conduct the learning. This paper solves this open problem by utilizing the power of cloud computing. In our proposed scheme, each party encrypts his/her private data locally and uploads the ciphertexts into the cloud. The cloud then executes most of the operations pertaining to the learning algorithms over ciphertexts without knowing the original private data. By securely offloading the expensive operations to the cloud, we keep the computation and communication costs on each party minimal and independent to the number of participants. To support flexible operations over ciphertexts, we adopt and tailor the BGN "doubly homomorphic" encryption algorithm for the multiparty setting. Numerical analysis and experiments on commodity cloud show that our scheme is secure, efficient, and accurate. [ABSTRACT FROM AUTHOR]

The Levenberg–Marquardt (LM) algorithm is a nonlinear parameter learning algorithm that converges accurately and quickly. This paper demonstrates for the first time to our knowledge, a real-time implementation of the LM algorithm on field programmable gate arrays (FPGAs). It was used to train neural networks to solve the eXclusive Or function (XOR), and for 3D-to-2D camera calibration parameter estimation. A Xilinx Virtex-5 ML506 was used to implement the LMA as a hardware-in-the-loop system. The XOR function was approximated in only 13 iterations from zero initial conditions, usually the same function is approximated in thousands of iterations using the error backpropagation algorithm. Also, this type of training not only reduced the number of iterations but also achieved a speed up in excess of 3 \times 10^6 when compared to the software implementation. A real-time camera calibration and parameter estimation was performed successfully on FPGAs. Compared to the software implementation the FPGA implementation led to an increase in the mean squared error and standard deviation by only 17.94% and 8.04% respectively. The FPGA increased the calibration speed by a factor of 1.41 \times 10^6. There are a wide range of systems problems solved via nonlinear parameter optimization, this study demonstrated that a hardware solution for systems such as automated optical inspection systems or systems dealing with projective geometry estimation and motion compensation systems in robotic vision systems is possible in real time. [ABSTRACT FROM PUBLISHER]

Very short-term load forecasting predicts the loads 1 h into the future in 5-min steps in a moving window manner based on real-time data collected. Effective forecasting is important in area generation control and resource dispatch. It is however difficult in view of the noisy data collection process and complicated load features. This paper presents a method of wavelet neural networks with data pre-filtering. The key idea is to use a spike filtering technique to detect spikes in load data and correct them. Wavelet decomposition is then used to decompose the filtered loads into multiple components at different frequencies, separate neural networks are applied to capture the features of individual components, and results of neural networks are then combined to form the final forecasts. To perform moving forecasts, 12 dedicated wavelet neural networks are used based on test results. Numerical testing demonstrates the effects of data pre-filtering and the accuracy of wavelet neural networks based on a data set from ISO New England. [ABSTRACT FROM PUBLISHER]

When number of available items surpasses the users? ability to browse them in a reasonable time, personalized recommender systems are used to assist the users to find the items that would match their interests. In this paper, the design of a context-aware recommender system for digital TV is described. Available programs are represented by their features in the adjacent vector space and genre transform is applied to reduce its dimension. Single-hidden layer feedforward neural network is used as a classifier tool which estimates whether certain program is of substantial interest to the observed user; this network is fed with the data on both program genre and the temporal context related to the user's watching habits. Special attention is paid to choosing an efficient network training algorithm and unobtrusive user feedback scheme. Good performances of the proposed system are verified through a series of experiments. It is shown that context information speeds the learning process up and reduces the so-called cold start phase without compromising the accuracy of the delivered recommendations. [ABSTRACT FROM AUTHOR]

This paper presents a new approach to incipient fault diagnosis in power transformers, based on the results of dissolved gas analysis. A set of autoassociative neural networks or autoencoders is trained, so that each becomes tuned with a particular fault mode or no fault condition. The scarce data available forms clusters that are densified using an Information Theoretic Mean Shift algorithm, allowing all real data to be used in the validation process. Then, a parallel model is built where the autoencoders compete with one another when a new input vector is entered and the closest recognition is taken as the diagnosis sought. A remarkable accuracy of 100% is achieved with this architecture, in a validation data set using all real information available. [ABSTRACT FROM AUTHOR]

In recent years, many alternative methodologies and techniques have been proposed to perform nondestructive inspection and maintenance operations of moving structures. In particular, ultrasonic techniques have shown to be very promising for automatic inspection systems. From the literature, it is evident that the neural paradigms are considered, by now, the best choice to automatically classify ultrasound data. At the same time, the most appropriate preprocessing technique is still undecided. The aim of this paper is to propose a new and innovative data preprocessing technique that converts real-valued ultrasonic data into complex-valued signals. This allows analysis using phase synchrony, a robust tool that has been previously employed in brain science for establishing robust features in noisy data. Synchrony estimation is achieved using complex extensions of empirical mode decomposition, a data-driven algorithm for detecting temporal scales, thus facilitating the modeling of nonlinear and nonstationary signal dynamics. Experimental tests aiming to detect defective areas in composite materials are reported, and the effectiveness of the proposed methodology is illustrated. [ABSTRACT FROM PUBLISHER]

Radial basis function (RBF) networks have advantages of easy design, good generalization, strong tolerance to input noise, and online learning ability. The properties of RBF networks make it very suitable to design flexible control systems. This paper presents a review on different approaches of designing and training RBF networks. The recently developed algorithm is introduced for designing compact RBF networks and performing efficient training process. At last, several problems are applied to test the main properties of RBF networks, including their generalization ability, tolerance to input noise, and online learning ability. RBF networks are also compared with traditional neural networks and fuzzy inference systems. [ABSTRACT FROM AUTHOR]

In the compressed sensing framework, different algorithms have been proposed for sparse signal recovery from an incomplete set of linear measurements. The most known can be classified into two categories: \ell1 norm minimization-based algorithms and \ell0 pseudo-norm minimization with greedy matching pursuit algorithms. In this paper, we propose a modified matching pursuit algorithm based on the orthogonal matching pursuit (OMP). The idea is to replace the correlation step of the OMP, with a neural network. Simulation results show that in the case of random sparse signal reconstruction, the proposed method performs as well as the OMP. Complexity overhead, for training and then integrating the network in the sparse signal recovery is thus not justified in this case. However, if the signal has an added structure, it is learned and incorporated in the proposed new OMP. We consider three structures: first, the sparse signal is positive, second the positions of the non zero coefficients of the sparse signal follow a certain spatial probability density function, the third case is a combination of both. Simulation results show that, for these signals of interest, the probability of exact recovery with our modified OMP increases significantly. Comparisons with \ell1 based reconstructions are also performed. We thus present a framework to reconstruct sparse signals with added structure by embedding, through neural network training, additional knowledge to the decoding process in order to have better performance in the recovery of sparse signals of interest. [ABSTRACT FROM AUTHOR]

This paper proposes an optimal feature selection approach, namely, probabilistic neural network-based feature selection (PFS), for power-quality disturbances classification. The PFS combines a global optimization algorithm with an adaptive probabilistic neural network (APNN) to gradually remove redundant and irrelevant features in noisy environments. To validate the practicability of the features selected by the proposed PFS approach, we employed three common classifiers: multilayer perceptron, k-nearest neighbor and APNN. The results indicate that this PFS approach is capable of efficiently eliminating nonessential features to improve the performance of classifiers, even in environments with noise interference. [ABSTRACT FROM AUTHOR]

In this paper, we propose an algorithm based on the central limit theorem to compute the sensitivity of the multilayer perceptron (MLP) due to the errors of the inputs and weights. For simplicity and practicality, all inputs and weights studied here are independently identically distributed (i.i.d.). The theoretical results derived from the proposed algorithm show that the sensitivity of the MLP is affected by the number of layers and the number of neurons adopted in each layer. To prove the reliability of the proposed algorithm, some experimental results of the sensitivity are also presented, and they match the theoretical ones. The good agreement between the theoretical results and the experimental results verifies the reliability and feasibility of the proposed algorithm. Furthermore, the proposed algorithm can also be applied to compute precisely the sensitivity of the MLP with any available activation functions and any types of i.i.d. inputs and weights. [ABSTRACT FROM AUTHOR]

A dual neural-network method for the coordination of kinematically redundant robots is proposed in this paper. The performance criteria for single robots provided by Nedungadi and Kazerounian are generalized to a multicriteria form for the coordinated-manipulation system composed of multiple serial manipulators. By optimizing the local joint torques and generalized forces applied on the object/workpiece using a designed weighting matrix, the proposed method achieves the global stability during the coordinated-manipulation process. Moreover, the proposed algorithm has an explicit physical meaning, i.e., both the global kinetic energy of the coordination system and the two-norm of the generalized forces applied on the object are minimized simultaneously. In addition, the physical limits of both joint torques and the generalized forces applied on the object are considered, which makes the original coordination problem become a complicated optimization problem subject to both equality and inequality constraints. Compared with numerical optimization algorithms used in existing literatures, the dual neural-network method has better computational capability to deal with the complicated optimization problem. Finally, illustrative examples are given to show that the proposed method is effective and efficient for the multirobot coordinated-manipulation system. [ABSTRACT FROM AUTHOR]

This paper presents a novel vehicle-classification algorithm that uses the time-variable signal generated by a single inductive loop detector. In earlier studies, the noisy raw signal was fed into the algorithm by reducing its size with rough sampling. However, this approach loses the original signal form and cannot be the best exemplar vector. The developed algorithm suggests three contributions to cope with these problems. The first contribution is to clear the noise with discrete Fourier transform (DFT). The second contribution is to transfer the noiseless pattern into the Principal Component Analysis (PCA) domain. PCA is exploited not only for decorrelation but for explicit dimensionality reduction as well. This goal cannot be achieved by simple raw data sampling. The last contribution is to expand the principal components with a local maximum (Lmax) parameter. It strengthens the classification accuracy by emphasizing the undercarriage height variation of the vehicle. These parameters are fed into the three-layered backpropagation neural network (BPNN). BPNN classifies the vehicles into five groups, and the recognition rate is 94.21%. This recognition rate has performed best, compared with the methods presented in published works. [ABSTRACT FROM AUTHOR]

This paper proposes a multiclassification algorithm using multilayer perceptron neural network models. It tries to boost two conflicting main objectives of multiclassifiers: a high correct classification rate level and a high classification rate for each class. This last objective is not usually optimized in classification, but is considered here given the need to obtain high precision in each class in real problems. To solve this machine learning problem, we use a Pareto-based multiobjective optimization methodology based on a memetic evolutionary algorithm. We consider a memetic Pareto evolutionary approach based on the NSGA2 evolutionary algorithm (MPENSGA2). Once the Pareto front is built, two strategies or automatic individual selection are used: the best model in accuracy and the best model in sensitivity (extremes in the Pareto front). These methodologies are applied to solve 17 classification benchmark problems obtained from the University of California at Irvine (UCI) repository and one complex real classification problem. The models obtained show high accuracy and a high classification rate for each class. [ABSTRACT FROM AUTHOR]

Generally, training neural networks with the global extended Kalman filter (GEKF) technique exhibits excellent performance at the expense of a large increase in computational costs which can become prohibitive even for networks of moderate size. This drawback was previously addressed by heuristically decoupling some of the weights of the networks. Inevitably, ad hoc decoupling leads to a degradation in the quality (accuracy) of the resultant neural networks. In this paper, we present an algorithm that emulates the accuracy of GEKF, but avoids the construction of the state covariance matrix--the source of the computational bottleneck in GEKF. In the proposed algorithm, all the synaptic weights remain connected while the amount of computer memory required is similar to (or cheaper than) the memory requirements in the decoupling schemes. We also point out that the new method can be extended to derivative-free nonlinear Kalman filters, such as the unscented Kalman filter and ensemble Kalman filters. [ABSTRACT FROM AUTHOR]

Detection of fluid leaks in pipe-type cable installations is important for both environmental and operational reasons. A new application of modern numerical algorithms, such as neural and probabilistic networks, for monitoring pressure system installations has been recently presented by the authors. This approach led to the development of a complete system for monitoring high-pressure, fluid-filled cables. During the initial system implementation, a need arose for the detection of very small leaks. To meet this need, the detection algorithms were revised and new features were implemented. This paper describes the new algorithms and discusses their implementation. [ABSTRACT FROM AUTHOR]

In this paper, the near-optimal control problem for class of nonlinear discrete-time systems with control constraints is solved by iterative adaptive dynamic programming algorithm. First, a novel nonquadratic performance functional is introduced to overcome the control constraints, and then an iterative adaptive dynamic programming algorithm is developed to solve the optimal feedback control problem of the original constrained system with convergence analysis. In the present control scheme, there are three neural networks used as parametric structures for facilitating the implementation of the iterative algorithm. Two examples are given to demonstrate the convergence and feasibility of the proposed optimal control scheme. [ABSTRACT FROM AUTHOR]

The collection of the rich flow of information provided by the current generation of fast vision sensing systems brings new challenges in the selection of only relevant features out of the avalanche of data generated by those sensors. This paper discusses some aspects of intelligent sensing for advanced robotic applications, with the main objective of designing innovative approaches for automatic selection of regions of observation for fixed and mobile sensors to collect only relevant measurements without human guidance. The proposed neural-gas-network solution selects regions of interest for further sampling from a cloud of sparsely collected 3-D measurements. The technique automatically determines bounded areas where sensing is required at a higher resolution to accurately map 3-D surfaces. Therefore, it provides significant benefits over brute-force strategies as scanning time is reduced and the size of the data set is kept manageable. Experimental evaluation of this technology is presented for 3-D surface measurement and modeling. [ABSTRACT FROM AUTHOR]

In this paper, a modular system for medical image archiving, management, diagnosis support, and telematic cooperation is presented. The system provides digital imaging and communications in medicine (DICOM)-compatible tools for digital image processing and database management of medical images. The software features algorithms for preprocessing, manual or semi-automatic segmentation, automatic calculation of geometrical/size characteristics, and 3-D visualization of organs or selected regions of interest. Additionally, the system incorporates a database where patient data and information can be stored and retrieved. Access to the database is only permitted to authorized users. The user-friendly interface makes the software handy and accessible to clinicians, whereas the telematic components allow collaboration with remote experts. The pilot system incorporates a computer-aided diagnosis module aiming at providing support in the diagnosis of focal liver lesions from computed tomography images. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of predicting hourly load demand using adaptive artificial neural networks (ANNs). A particle swarm optimization (PSO) algorithm is employed to adjust the network's weights in the training phase of the ANNs. The advantage of using a PSO algorithm over other conventional training algorithms such as the back-propagation (BP) is that potential solutions will be flown through the problem hyperspace with accelerated movement towards the best solution. Thus the training phase should result in obtaining the weights configuration associated with the minimum output error. Data are wavelet transformed during the preprocessing stage and then inserted into the neural network to extract redundant information from the load curve. This results in better load characterization which creates a more reliable forecasting model. The transformed data of historical load and weather information were trained and tested over various periods of time. The generalized error estimation is done by using the reverse part of the data as a "test" set. The results were compared with traditional BP algorithm and offered a high forecasting precision. [ABSTRACT FROM AUTHOR]

This paper presents the practical implementation of a novel fault diagnostic and protection scheme for the interior permanent-magnet (IPM) synchronous motors using wavelet packet transform (WPT) and artificial neural network. In the proposed technique, the line currents of different faulted and normal conditions of the IPM motor are preprocessed by the WPT. The second level WPT coefficients of line currents are used as inputs of a three-layer feedforward neural network. The proposed protection technique is successfully simulated and experimentally tested on the line-fed and inverter-fed IPM motors. The Texas Instrument 32-bit floating-point digital signal processor TMS320C31 is used for the real-time implementation of the proposed protection algorithm. The offline and online test results of both line-fed and inverter-fed IPM motors are given. These test results showed satisfactory performances of the proposed diagnostic and protection technique in terms of speed, accuracy, and reliability. [ABSTRACT FROM AUTHOR]

This paper presents system modeling, analysis, and simulation of an electric vehicle (EV) with two independent rear wheel drives. The traction control system is designed to guarantee the EV dynamics and stability when there are no differential gears. Using two in-wheel electric motors makes it possible to have torque and speed control in each wheel. This control level improves EV stability and safety. The proposed traction control system uses the vehicle speed, which is different from wheel speed characterized by a slip in the driving mode, as an input. In this case, a generalized neural network algorithm is proposed to estimate the vehicle speed. The analysis and simulations lead to the conclusion that the proposed system is feasible. Simulation results on a test vehicle propelled by two 37-kW induction motors showed that the proposed control approach operates satisfactorily. [ABSTRACT FROM AUTHOR]

In this paper, systematic neural network modeling techniques are presented for microwave modeling and design using the concept of inverse modeling where the inputs to the inverse model are electrical parameters and outputs are geometrical parameters. Training the neural network inverse model directly may become difficult due to the nonuniqueness of the input-output relationship in the inverse model. We propose a new method to solve such a problem by detecting multivalued solutions in training data. The data containing multivalued solutions are divided into groups according to derivative information using a neural network forward model such that individual groups do not have the problem of multivalued solutions. Multiple inverse models are built based on divided data groups, and are then combined to form a complete model. A comprehensive modeling methodology is proposed, which includes direct inverse modeling, segmentation, derivative division, and model combining techniques. The methodology is applied to waveguide filter modeling and more accurate results are achieved compared to the direct neural network inverse modeling method. Full electromagnetic simulation and measurement results of Ku-band circular waveguide dual-mode pseudoelliptic bandpass filters are presented to demonstrate the efficiency of the proposed neural network inverse modeling methodology. [ABSTRACT FROM AUTHOR]

This paper presents flaws identification algorithm based on artificial neural networks and dedicated approximation functions. An eddy-current differential transducer was used to detect the flaws in thin conducting plates. The measured signals were approximated and utilized for flaws identification. Various experiments with the flaws having rectangular and nonrectangular profiles were carried out in order to verify usability of the proposed technique. [ABSTRACT FROM AUTHOR]

In this paper, an automatic algorithm based an unsupervised neural network for an on-line diagnostics of three-phase induction motor stator fault is presented. This algorithm uses the alfa-beta stator currents as input variables. Then, a fully automatic unsupervised method is applied in which a Hebbian-based unsupervised neural network is used to extract the principal components of the stator current data. These main directions are used to decide where the fault occurs and a relationship between the current components is calculated to verify the severity of the fault. One of the characteristics of this method, given its unsupervised nature, is that it does not need a prior identification of the system. The proposed methodology has been experimentally tested on a 1 kW induction motor. The obtained experimental results show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., Backpropagation for Multilabel Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms. [ABSTRACT FROM AUTHOR]

Discriminative training refers to an approach to pattern recognition based on direct minimization of a cost function commensurate with the performance of the recognition system. This is in contrast to the procedure of probability distribution estimation as conventionally required in Bayes' formulation of the statistical pattern recognition problem. Currently, most discriminative training algorithms for nonlinear classifier designs are based on gradient-descent (GD) methods for cost minimization. These algorithms are easy to derive and effective in practice, but are slow in training speed and have difficulty selecting the learning rates. To address the problem, we present our study on a fast discriminative training algorithm. The algorithm initializes the parameters by the expectation-maximization (EM) algorithm, and then uses a set of closed-form formulas derived in this paper to further optimize a proposed objective of minimizing error rate. Experiments in speech applications show that the algorithm provides better recognition accuracy in a fewer iterations than the EM algorithm and a neural network trained by hundreds of GD iterations. Although some convergent properties need further research, the proposed objective and derived formulas can benefit further study of the problem. [ABSTRACT FROM AUTHOR]

This paper presents a new application of modern numerical algorithms, such as neural and probabilistic networks, for monitoring pressure system installations. Theoretical background as well as custom implementation of the algorithms is presented, and a complete system for monitoring high-pressure, fluid-filled (HPFF) cables is described. [ABSTRACT FROM AUTHOR]

The process of learning the behavior of a given pro- gram by using machine-learning techniques (based on system-call audit data) is effective to detect intrusions. Rule learning, neural networks, statistics, and hidden Markov models (HMMs) are some of the kinds of representative methods for intrusion detection. Among them, neural networks are known for good performance in learning system-call sequences. In order to apply this knowledge to real-world problems successfully, it is important to determine the structures and weights of these call sequences. However, finding the appropriate structures requires very long time periods be- cause there are no suitable analytical solutions. In this paper, a novel intrusion-detection technique based on evolutionary neural networks (ENNs) is proposed. One advantage of using ENNs is that it takes less time to obtain superior neural networks than when using conventional approaches. This is because they discover the structures and weights of the neural networks simultaneously. Experimental results with the 1999 Defense Advanced Research Projects Agency (DARPA) Intrusion Detection Evaluation (IDEVAL) data confirm that ENNs are promising tools for intrusion detection. [ABSTRACT FROM AUTHOR]

An NNC-Tree is a decision tree (DT) with each nonterminal node containing a nearest neighbor classifier (NNC). Compared with the conventional axis-parallel DTs (APDTs), the NNC-Trees can be more efficient, because the decision boundary made by an NNC is more complex than an axis- parallel hyperplane. Compared with single-layer NNCs, the NNC-Trees can classify given data in a hierarchical structure that is often useful for many applications. This paper proposes an algorithm for inducing NNC-Trees based on the R4-rule, which was proposed by the author for finding the smallest nearest neighbor based multilayer perceptrons (NN-MLPs). There are mainly two contributions here. 1) A heuristic but effective method is given to define the teacher signals (group labels) for the data assigned to each nonterminal node. 2) The R4-rule is modified so that an NNC with proper size can be designed automatically in each nonterminal node. Experiments with several public databases show that the proposed algorithm can produce NNC-Trees effectively and efficiently. [ABSTRACT FROM AUTHOR]

Vector hysteresis models are regarded as helpful tools that can be utilized in the simulation of multidimensional field-media interactions. Recently, substantial efforts have been focused on the refinement of vector Preisach-type models of hysteresis. The purpose of this paper is to present a computationally efficient vector Preisach-type hysteresis model constructed from only two scalar models having orthogonally inter-related elementary operators. Such a model is implemented via a linear neural network (LNN) fed from the outputs of discrete Hopfield neural network (DHNN) blocks having step activation functions. With this DHNN-LNN configuration, it is possible to carry out the identification process using well-established widely available algorithms. Details of the model, its identification, and experimental testing are presented. [ABSTRACT FROM AUTHOR]

This paper presents analysis of the recently proposed modulated Hebb-Oja (MHO) method that performs linear mapping to a lower-dimensional subspace. Principal component subspace is the method that will be analyzed. Comparing to some other well-known methods for yielding principal component subspace (e.g., Oja's Subspace Learning Algorithm), the proposed method has one feature that could be seen as desirable from the biological point of view—synaptic efficacy learning rule does not need the explicit information about the value of the other efficacies to make individual efficacy modification. Also, the simplicity of the "neural circuits" that perform global computations and a fact that their number does not depend on the number of input and output neurons, could be seen as good features of the proposed method. [ABSTRACT FROM AUTHOR]

The main problem in radiation pyrometry is the large error arising from the unknown or varying emissivity of the target surface. In this paper, a combined neural network emissivity model, which allows calculating the true temperature and emissivity of any measurand from the measured data of continuous spectral emissivity, is established by the combined neural network. The proposed single parameter dynamic search algorithm using the hybrid steepest descent and Newton's method is proposed to optimize the training algorithm of the model. This optimization algorithm can quicken the convergence speed of the combined neural network emissivity model (CNNE model). Through theoretical derivation and simulation experiments, we can see that this model is theoretically useful for any target, which calculates the function relationships between wavelength and emissivity. Simulation and experimental results have shown the high accuracy in measurements by using this method. [ABSTRACT FROM AUTHOR]