Search

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

The abnormal rise in contact temperature in switchgear may lead to overheating or even explosion. It is necessary to monitor the rise in real contact temperature to ensure the safety and stability of switchgear. The accurate measurement of real contact temperature is still a problem, and the continuous state of switchgear cannot be assessed accurately by the temperature monitor. In this paper, a particular installation scheme of temperature sensor is put forward and a calculation algorithm for temperature rise of tulip contacts by the measured temperature of bus bar in real time based on artificial neural network (ANN) is established. An electromagnetic-thermal-fluid coupled simulation model for 10-kV/4000-A switchgears was built, and the thermal characteristic was studied taking the influential factors such as load current, ambient temperature, and contact resistance into consideration. The effectiveness of simulation model is discussed by the comparison to temperature rise experiment. Then, an ANN with three layers of structure is established to calculate the temperature of tulip contacts. The sensitivity and accuracy of network are calculated and verified. The results of this paper may provide a novel method for the monitor of switchgear temperature and assess continuous condition of high-current switchgear. [ABSTRACT FROM AUTHOR]

This paper proposes a virtual 3D eyeglasses try-on system driven by a 2D Internet image of a human face wearing with a pair of eyeglasses. The main technical challenge of this system is the automatic 3D eyeglasses model reconstruction from the 2D glasses on a frontal human face. Against this challenge, this paper first proposes an eyeglasses segmentation method using a convolutional neural network-based parsing algorithm to label the glasses pixels, followed with a proposed symmetry-based level-set optimization algorithm to refine the contour of the eyeglasses. With the precisely extracted silhouette image, we take advantages of the smoothness and the symmetry priors of the eyeglasses, and propose a silhouette-based 3D deformation method to deform a 3D eyeglasses model selected from a predefined 3D model database. The obtained model is plausibly approximated to the input 2D eyeglasses after a texture mapping step. Finally, we develop a virtual try-on system to interactively synthesize the reconstructed eyeglasses on a moving target face in real time. The experimental results demonstrate the efficient and convincing virtual try-on performance of our approach and the commercial potential of our proposed system. [ABSTRACT FROM PUBLISHER]

For a class of stochastic nonlinear systems in pure-feedback form with dead-zone input and multiple time-varying delays, a novel neural network (NN)-based adaptive control approach is presented in this paper through the use of backstepping approach and dynamic surface technique. By choosing proper Lyapunov–Krasovskii functionals, utilizing the characteristic of hyperbolic tangent functions and adopting the function separation technique, difficulties of controller design that introduced by the time-varying delays can be dealt with properly. Moreover, all unknown nonlinear functions are lumped together and approximated by the NN. Additionally, any information over the boundedness of dead-zone parameters is not needed in the process of controller design. The control scheme proposed in this paper ensures the boundedness in probability of all signals in the closed-loop system, besides, excellent performance of arbitrarily small tracking error will be achieved by selecting control parameters appropriately. At last, two numerical simulation examples are provided to verify the validity of the designed algorithm. [ABSTRACT FROM AUTHOR]

This paper establishes an approximate optimal critic learning algorithm based on single neural network (NN) policy iteration (PI) aiming at solving for continuous-time (CT) 2-player zero-sum games (ZSGs). In fact, we have to face the problem that the errors will disturb the dynamics and in turn identifying dynamics will generate errors. In order to prevent the effect of errors, in this paper, a single NN-based online PI algorithm is developed for the CT system, which is disturbed nonlinear ZSG. With plenty of online data, the Hamilton–Jacobi–Isaacs equation can be solved without complete dynamics. Then by the least-squares method, we can obtain the NN weights. Moreover, in the process of dealing with the undisturbed system, we find the way that obtains NN weights in this paper is equal to the way that obtains the optimal solution by the Gauss–Newton method. Based on the convergence of the Gauss–Newton method, we can efficiently obtain the optimal controller for the undisturbed system by utilizing online data. After getting the controller of the undisturbed system, it is time to take disturbance into consideration, so that we design a robust control pair to overcome the disturbance. In order to demonstrate the effectiveness of this algorithm, we design a set of simulations. The results verify that we can solve the disturbed nonlinear ZSG by this algorithm. [ABSTRACT FROM AUTHOR]

Compared with other applications in computer vision, convolutional neural networks (CNNs) have underperformed on pedestrian detection. A breakthrough was made very recently using sophisticated deep CNN (DCNN) models, with a number of handcrafted features or explicit occlusion handling mechanism. In this paper, we show that by reusing the convolutional feature maps of a DCNN model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms. We empirically identify and disclose important implementation details. We also show that pixel labeling may be simply combined with a detector to boost the detection performance. By adding complementary handcrafted features such as optical flow, the DCNN-based detector can be further improved. We advance the state-of-the-art results by lowering the log-average miss rate from 11.7% to 8.9% on the Caltech data set and from 11.2% to 8.6% on the Inria data set. We also achieve a comparable result to state-of-the-art approaches on the KITTI data set. [ABSTRACT FROM AUTHOR]

The papers in this special section are focused on one aspect of the current technological wave. As we all know, nature-inspired (NI) computing and algorithms play an ever increasing role in problem solving by way of optimization, machine intelligence, data mining, and resource management. Nature has evolved over millions of years under a variety of harsh and severe environments and, thus, provides a rich source of inspiration for designing algorithms and approaches to tackle real-world challenging EMC and Signal & Power Integrity (SI/PI) problems. [ABSTRACT FROM AUTHOR]

This paper deals with a modified technique for the recognition of single stage and multiple power quality (PQ) disturbances. An algorithm based on Stockwell's transform and artificial neural network-based classifier and a rule-based decision tree is proposed in this paper. The analysis and classification of single stage PQ disturbances consisting of both events and variations such as sag, swell, interruption, harmonics, transients, notch, spike, and flicker are presented. Moreover, the proposed algorithm is also applied on multiple PQ disturbances such as harmonics with sag, swell, flicker, and interruption. A database of these PQ disturbances based on IEEE-1159 standard is generated in MATLAB for simulation studies. The proposed algorithm extracts significant features of various PQ disturbances using S-transform, which are used as input to this hybrid classifier for the classification of PQ disturbances. Satisfactory results of effective recognition and classification of PQ disturbances are obtained with the proposed algorithm. Finally, the proposed method is also implemented on real-time PQ events acquired in a laboratory to confirm the validity of this algorithm in practical conditions. [ABSTRACT FROM PUBLISHER]

This paper addresses the challenge of learning to play many different video games with little domain-specific knowledge. Specifically, it introduces a neuroevolution approach to general Atari 2600 game playing. Four neuroevolution algorithms were paired with three different state representations and evaluated on a set of 61 Atari games. The neuroevolution agents represent different points along the spectrum of algorithmic sophistication—including weight evolution on topologically fixed neural networks (conventional neuroevolution), covariance matrix adaptation evolution strategy (CMA–ES), neuroevolution of augmenting topologies (NEAT), and indirect network encoding (HyperNEAT). State representations include an object representation of the game screen, the raw pixels of the game screen, and seeded noise (a comparative baseline). Results indicate that direct-encoding methods work best on compact state representations while indirect-encoding methods (i.e., HyperNEAT) allow scaling to higher dimensional representations (i.e., the raw game screen). Previous approaches based on temporal-difference (TD) learning had trouble dealing with the large state spaces and sparse reward gradients often found in Atari games. Neuroevolution ameliorates these problems and evolved policies achieve state-of-the-art results, even surpassing human high scores on three games. These results suggest that neuroevolution is a promising approach to general video game playing (GVGP). [ABSTRACT FROM PUBLISHER]

The multimode power-split architecture for hybrid electric vehicle (HEV) powertrains is generally known for the complexity of its operation. This paper first addresses the challenge of developing an automated on-line near-optimal control strategy for these systems. Particularly, a machine learning logic based on supervised learning is developed for on-line selection of the HEV operating mode, thus the particular set of clutches to be engaged. An efficiency-based approach is then adopted to determine the optimal power-split between powertrain components. Later, the developed strategy finds integration in an optimal design methodology for multimode power-split HEVs considering the effectiveness and ease of on-line controllability. The obtained results are compared with the ones by the traditional HEV design methodology that only considers off-line energy management. The illustrated design methodology with on-line control reveals efficient at identifying the multimode HEV design that demonstrates optimal predicted fuel economy values and ease of on-line controllability simultaneously. Results suggest that the HEV design optimization procedure may produce different outcomes and demonstrate more effective when the evaluation of the on-line controlled operation is integrated. [ABSTRACT FROM AUTHOR]

This paper proposes the energy-based controller (EC) incorporated with radical basis function (RBF) neural-network compensation (ECRBFC), which is used to swing up the Pendubot and raise it to its uppermost unstable equilibrium position. First, for the known dynamics model of the two-link arm, the EC is designed. In the EC, the singularity is successfully avoided by constructing an appropriate energy evaluation function. Second, as for the friction of the Pendubot, because of the time-varying characteristics, an accurate friction dynamics model cannot be known absolutely; thus, the RBF neural network is introduced to offset the bad effect of friction. Finally, in order to evaluate the performance of ECRBFC, the numerical simulations and the experimental results are given, and by comparing the results with that of other algorithms, it is found that ECRBFC proposed in this paper has better performance under the same conditions. [ABSTRACT FROM PUBLISHER]

This paper investigates the problem of quasi-synchronization (QS) for the periodic neural networks (NNs). In order to address more general NNs, the parameter and the period mismatches are both considered, that is, excluding parameters, periods of the target dynamic and the followers are also different. In addition, the constrainted target information is studied, where the logarithmic quantizer is used to overcome the limited communication capacity and Bernoulli processes are employed to model cases of the information loss. A new period is established based on the lowest common multiple period of the target dynamic and the followers to obtain an augmented synchronization error system (ASES). Further, a suboptimal iterative algorithm is proposed to cut down the QS range of the ASES and the corresponding controllers are designed. At last, the controller design method is illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

This paper presents a controller integrity monitoring method for a class of second-order nonlinear uncertain systems incorporating neural network-based adaptive control algorithms. The adaptive neural network model is employed to ensure robust tracking performance in the presence of certain modeling uncertainty under consideration during the offline controller design process. Based on Lyapunov stability analysis, an online controller integrity monitoring scheme is developed to detect the occurrence of controller software/algorithm faults and unanticipated physical component faults, which may lead to unstable learning behaviors and malfunctions of the adaptive controller. Adaptive thresholds for detecting controller malfunctions are derived, ensuring the robustness with respect to modeling uncertainty and neural network approximation error. Additionally, the detectability conditions are investigated, characterizing the class of detectable software faults and unanticipated physical faults. An upper bound on the fault detection time is also established. Simulation results are shown to illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The aim of this paper is to investigate the estimation of the variation of ground resistance throughout the year by using artificial neural networks (ANNs). An ANN was trained, validated, and tested with different training algorithms by using experimental data of soil resistivity, ground resistance, and rainfall in order to select the optimum training algorithm and the respective parameters and predict the behavior of the ground resistance of a single rod. Moreover, a sensitivity analysis of the proposed ANN was carried out in order to determine the impact of certain factors on the efficiency of the ANN. The high value of the correlation index between estimated and experimental values demonstrates the high efficiency of the ANN. The proposed methodology based on ANN is a useful tool for the estimation of the grounding resistance during the year in case of difficulties in measuring its value. [ABSTRACT FROM AUTHOR]

This paper presents a new neural network (NN) algorithm for real-time retrievals of low amounts of precipitable water vapor (PWV) and integrated liquid water from millimeter-wave ground-based observations. Measurements are collected by the 183.3-GHz G-band vapor radiometer (GVR) operating at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility, Barrow, AK. The NN provides the means to explore the nonlinear regime of the measurements and investigate the physical boundaries of the operability of the instrument. A methodology to compute individual error bars associated with the NN output is developed, and a detailed error analysis of the network output is provided. Through the error analysis, it is possible to isolate several components contributing to the overall retrieval errors and to analyze the dependence of the errors on the inputs. The network outputs and associated errors are then compared with results from a physical retrieval and with the ARM two-channel microwave radiometer (MWR) statistical retrieval. When the NN is trained with a seasonal training data set, the retrievals of water vapor yield results that are comparable to those obtained from a traditional physical retrieval, with a retrieval error percentage of ~5% when the PWV is between 2 and l0 mm, but with the advantages that the NN algorithm does not require vertical profiles of temperature and humidity as input and is significantly faster computationally. Liquid water path (LWP) retrievals from the NN have a significantly improved clear-sky bias (mean of ~2.4 g/m²) and a retrieval error varying from 1 to about 10 g/ms when the PWV amount is between 1 and 10 mm. As an independent validation of the LWP retrieval, the Iongwave downwelling surface flux was computed and compared with observations. The comparison shows a significant improvement with respect to the MWR statistical retrievals, particularly for LWP amounts of less than 60 g/m². This paper shows that the GVR alone can provide overall improved PWV and LWP retrievals when the PWV amount is less than 10 mm, and, when combined with the MWR, can provide improved retrievals over the whole water-vapor range. [ABSTRACT FROM AUTHOR]

This paper presents a subgradient-based neural network to solve a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of affine equality constraints, and a class of nonsmooth convex inequality constraints. The proposed neural network is modeled with a differential inclusion. Under a suitable assumption on the constraint set and a proper assumption on the objective function, it is proved that for a sufficiently large penalty parameter, there exists a unique global solution to the neural network and the trajectory of the network can reach the feasible region in finite time and stay there thereafter. It is proved that the trajectory of the neural network converges to the set which consists of the equilibrium points of the neural network, and coincides with the set which consists of the critical points of the objective function in the feasible region. A condition is given to ensure the convergence to the equilibrium point set in finite time. Moreover, under suitable assumptions, the coincidence between the solution to the differential inclusion and the "slow solution" of it is also proved. Furthermore, three typical examples are given to present the effectiveness of the theoretic results obtained in this paper and the good performance of the proposed neural network. [ABSTRACT FROM AUTHOR]

Classification of power-quality (PQ)-related voltage and current waveform disturbances is a key task for power system monitoring. A new method based on the optimized time-frequency representation (TFR) has been proposed in the first paper [1] of this two-paper series. This paper (the second paper) presents a case study of PQ event classification with the proposed method. The classification algorithm has been successfully tested with $60 real world PQ events that cover five classes, achieving a recognition rate of 98%. The algorithm is implemented on a digital signal processor (DSP) based hardware system and optimized according to the DSP architecture to meet the hard real-time constraints. The DSP-based system is capable of processing a five-cycle (83.3 ms) PQ waveform within 11.2 ms. The real-time computing capability of the algorithm has been verified with this result. The scalability of this method is also discussed. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive dynamic programming (ADP) algorithm is developed for the infinite-horizon ($H_{\infty}$) optimal control problems with unknown continuous-time (CT) nonlinear systems subject to external disturbances. To facilitate the implementation of the algorithm, generalized fuzzy hyperbolic models (GFHMs) are utilized to establish an identifier–critic architecture, where the identifier is designed to reconstruct the unknown system dynamics, and the GFHM-based critic network is employed to approximate the value functions. The CT $H_{\infty}$ optimal control issue is converted into a two-player zero-sum game and the corresponding Hamilton–Jacobi–Isaacs equation is derived. The learning procedure of the critic design is adaptively implemented with the help of the reconstructed model, thus the requirement of the complete system dynamics is relaxed. Furthermore, by the means of Lyapunov direct method, the uniform ultimate boundedness stability analysis of the closed-loop control system is explicitly provided. Finally, to compare the control performances and disturbance attenuation properties of the proposed method and the existing ADP algorithms, two numerical examples are given. [ABSTRACT FROM AUTHOR]

A power transformer is one of the most crucial items of equipment in the electricity supply chain. The reliability of this valuable asset is strongly dependent on the condition of its subsystems such as insulation, core, windings, bushings and tap changer. Integration of various measured parameters of these subsystems makes it possible to evaluate the overall health condition of an in-service transformer. This paper develops an artificially intelligent algorithm based on multiple general regression neural networks to combine the operating condition of various subsystems of a transformer to form a quantitative health index. The model is developed using a training set derived from four conditional boundaries based on IEEE standards, the literature and the knowledge of transformer experts. Performance of the proposed method is compared with expert classifications using a database of 345 power transformers. This shows that the proposed method is reliable and effective for condition assessment and is sensitive to poor condition of any single subsystem. [ABSTRACT FROM PUBLISHER]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

This paper presents a practical discrete-time fractional order terminal sliding mode (DFOTSM) control strategy for high-precision tracking tasks based on a linear motor. In particular, the practical parametric uncertainties involving sliding friction, uncertain payload, and disturbance in tracking tasks are considered in this paper. Combining Grünwald–Letnikov fractional order definition and terminal sliding mode technique, the proposed method synthesizes a novel DFOTSM control law to drive the sliding mode dynamics into the stable region in finite steps theoretically, even though the system is suffering from uncertainties and disturbances, and the motion on the surface can guarantee higher tracking precision than the conventional discrete-time terminal sliding surface by selecting suitable controller parameters. The theoretical analyses give out the guideline of parameter selection, and experiments are carried out on the linear-motor-based test platform to demonstrate that the proposed controller is easily implemented and can achieve high-precision tracking, fast response, and considerable robustness to uncertainties. [ABSTRACT FROM AUTHOR]

In this paper, a novel event-sampled robust optimal controller is proposed for a class of continuous-time constrained-input nonlinear systems with unknown dynamics. In order to solve the robust optimal control problem, an online data-driven identifier is established to construct the system dynamics, and an event-sampled critic-only adaptive dynamic programming method is developed to replace the conventional time-driven actor–critic structure. The designed online identification method runs during the solving process and is not applied as a priori part for the solutions, which simplifies the architecture and reduces computational load. The proposed robust optimal control algorithm tunes the parameters of critic-only neural network (NN) by event-triggering condition and runs in a plug-n-play framework without system functions, where fewer transmissions and less computation are required as all the measurements received simultaneously. Based on the novel design, the stability of system and the convergence of critic NN are demonstrated by Lyapunov theory, where the state is asymptotically stable and weight error is guaranteed to be uniformly ultimately bounded. Finally, the applications in a basic nonlinear system and the complex rotational–translational actuator problem demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In recent times, with the incremental demand for fully autonomous systems, research interests are observed in learning machine-based intelligent, self-organizing, and evolving controllers. In this paper, a new evolving and self-organizing controller, namely generic-controller (G-controller), is proposed. The G-controller works in a fully online mode with minor expert domain knowledge. It is developed by incorporating the sliding mode control (SMC) theory with an advanced incremental learning machine, namely generic evolving neuro-fuzzy inference system. The controller starts operating from scratch with an empty set of fuzzy rule, and therefore, no offline training is required. To cope with the changing dynamic characteristics of the plant, the controller can add or prune the rules on demand. Control law and adaptation laws for the consequent parameters are derived from the SMC algorithm to establish a stable closed-loop system, where the stability of the G-controller is guaranteed by using the Lyapunov function. The uniform asymptotic convergence of tracking error to zero is witnessed through the implication of an auxiliary robustifying control term. In addition, the implementation of the multivariate Gaussian function helps the controller to handle the nonaxis parallel data from the plant and consequently, enhances the robustness against uncertainties and environmental perturbations. Finally, the controller's performance has been evaluated by observing the tracking performance in controlling simulated plants of unmanned aerial vehicle, namely bio-inspired flapping wing micro air vehicle and hexacopter for a variety of trajectories. [ABSTRACT FROM AUTHOR]

Gramophone records were the main recording medium for more than seven decades and regained widespread popularity over the past several years. Being an analog storage medium, gramophone records are subject to distortions caused by scratches, dust particles, degradation, and other means of improper handling. The observed noise often leads to an unpleasant listening experience and requires a filtering process to remove the unwanted disruptions and improve the audio quality. This paper proposes a novel approach that employs various feed forward time delay artificial neural networks to detect and reconstruct noise in musical sound waves. A set of 800 songs from eight different genres were used to validate the performance of the neural networks. The performance was analyzed according to the outlier detection and interpolation accuracy, the computational time and the tradeoff between the accuracy and the time. The empirical results of both detection and reconstruction neural networks were compared to a number of other algorithms, including various statistical measurements, duplication approaches, trigonometric processes, polynomials, and time series models. It was found that the neural networks’ outlier detection accuracy was slightly lower than some of the other noise identification algorithms, but achieved a more efficient tradeoff by detecting most of the noise in real time. The reconstruction process favored neural networks with an increase in the interpolation accuracy compared to other widely used time series models. It was also found that certain genres such as classical, country, and jazz music were interpolated more accurately. Volatile signals, such as electronic, metal, and pop music were more challenging to reconstruct and were substantially better interpolated using neural networks than the other examined algorithms. [ABSTRACT FROM PUBLISHER]

An algorithm called adaptive-neural-intention estimator (ANIE) is presented to infer the intent of a human operator’s arm movements based on the observations from a 3-D camera sensor (Microsoft Kinect). Intentions are modeled as the goal locations of reaching motions in 3-D space. Human arm’s nonlinear motion dynamics are modeled using an unknown nonlinear function with intentions represented as parameters. The unknown model is learned by using a neural network. Based on the learned model, an approximate expectation-maximization algorithm is developed to infer human intentions. Furthermore, an identifier-based online model learning algorithm is developed to adapt to the variations in the arm motion dynamics, the motion trajectory, the goal locations, and the initial conditions of different human subjects. The results of experiments conducted on data obtained from different users performing a variety of reaching motions are presented. The ANIE algorithm is compared with an unsupervised Gaussian mixture model algorithm and an Euclidean distance-based approach by using Cornell’s CAD-120 data set and data collected in the Robotics and Controls Laboratoy at UConn. The ANIE algorithm is compared with the inverse LQR and ATCRF algorithms using a labeling task carried out on the CAD-120 data set. [ABSTRACT FROM AUTHOR]

Knowledge-based neural network modeling techniques using space-mapping concept have been demonstrated in the existing literature as efficient methods to overcome the accuracy limitations of empirical/equivalent circuit models when matching new electromagnetic data. For different modeling problems, the mapping structures can be different. In this paper, we propose a unified automated model generation algorithm that uses \boldsymbol l\mathbf {1} optimization to automatically determine the type and the topology of the mapping structure in a knowledge-based neural network model. By encompassing various types of mappings of the knowledge-based neural network model in the existing literature, we present a new unified model structure and derive new sensitivity formulas for the training of the unified model. The proposed \boldsymbol l\mathbf {1} formulation of modeling can force some weights of the mapping neural networks to zeros while leaving other weights as nonzeros. We utilize this feature to allow \boldsymbol l\mathbf {1} optimization to automatically determine which mapping is necessary and which mapping is unnecessary. Using the proposed \boldsymbol l\mathbf {1} optimization method, the mapping structure can be determined to address different needs of different modeling problems. The structure of the final knowledge-based model can be flexible combinations of some or all of linear mapping, nonlinear mapping, input mapping, frequency mapping, and output mapping. In this way, the proposed algorithm is more systematic and can further speed up the knowledge-based modeling process than existing knowledge-based modeling algorithms. The proposed method is illustrated by three microwave filter modeling examples. [ABSTRACT FROM PUBLISHER]

This is the second paper in a series of two that describe our research in intelligent energy management in a hybrid electric vehicle (HEV). In the first paper, we presented the machine-learning framework ML_EMO_HEV, which was developed for learning the knowledge about energy optimization in an HEV. The framework consists of machine-learning algorithms for predicting driving environments and generating the optimal power split of the HEV system for a given driving environment. In this paper, we present the following three online intelligent energy controllers: 1) IEC_HEV_SISE; 2) IEC_HEV_MISE ; and 3) IEC_HEV_MIME. All three online intelligent energy controllers were trained within the machine-learning framework ML_EMO_HEV to generate the best combination of engine power and battery power in real time such that the total fuel consumption over the whole driving cycle is minimized while still meeting the driver's demand and the system constraints, including engine, motor, battery, and generator operation limits. The three online controllers were integrated into the Ford Escape hybrid vehicle model for online performance evaluation. Based on their performances on ten test drive cycles provided by the Powertrain Systems Analysis Toolkit library, we can conclude that the roadway type and traffic congestion level specific machine learning of optimal energy management is effective for in-vehicle energy control. The best controller, IEC_HEV_MISE, trained with the optimal power split generated by the DP optimization algorithm with multiple initial SOC points and single ending point, can provide fuel savings ranging from 5% to 19%. Together, these two papers cover the innovative technologies for modeling power flow, mathematical background of optimization in energy management, and machine-learning algorithms for generating intelligent energy controllers for quasioptimal energy flow in a power-split HEV. [ABSTRACT FROM PUBLISHER]

This paper presents two algorithms, both based on the time of flight (ToF) of scattered waves, to locate defect in an anisotropic woven-fabric carbon fiber reinforced polymer (CFRP) plate. The first algorithm uses a probabilistic approach by constructing a probability matrix. Each element of this matrix is associated with a location and represents the probability of existing a defect at the corresponding spatial coordinates of the plate. For the probability matrix method, localization results are influenced by manually chosen parameters and by the anisotropy of the CFRP plate. The second algorithm, based on artificial neural networks (ANNs), enables us to improve the accuracy of locating defects. With this ANN method, the anisotropic feature can be “learned” by the neural network from training data. The ToF of scattered waves obtained from three sensor pairs were used directly as inputs of the neural network. The spatial coordinates of the defect are the ANN outputs. The difficulty of obtaining sufficient experimental data for ANN training was surpassed by using added blocks on the surface of the plate under test to simulate defects. This scheme was validated and proved to be effective by comparing the scattered waves from a delamination and from an added block. Conclusions have been drawn by comparing the two methods. The localization results obtained by the ANN algorithm are proved to be better. [ABSTRACT FROM AUTHOR]

It is necessary for cost consideration to minimize the number of the used layers in a high-speed printed circuit board (PCB) design. In this paper, any independent net cannot be treated as a bus-oriented net in a high-speed PCB design because of no timing-matching constraint on an independent net. Any independent net in a high-speed PCB design can be modeled to obey the via-count constraint as the maximum number of the permitted vias for signal integrity. Clearly, the introduction of the permitted vias onto the independent nets can lead to the reduction on the number of the used layers in a high-speed PCB design. Given a set of bus-oriented nets and a set of independent nets with a via-count constraint in a high-speed PCB design, by introducing virtual vias onto independent nets and eliminating redundant vias on any used layer, a generalized algorithm can be proposed to minimize the number of the used layers with satisfying the via-count constraint on any independent net and assign the given bus-oriented nets and the separated segments inside the given independent nets onto the used layers. Compared with Yan’s algorithm with no via introduction on independent nets, the experimental results show that our proposed algorithm with ${c_{\max } = 1}$ , ${c_{\max } = 2}$ , ${c_{\max } = 3}$ , ${c_{\max } = 4}$ , and ${c_{\max } = 5}$ use reasonable CPU time to insert permitted vias to reduce 2.1, 2.8, 3.8, 4.4, and 4.6 used layers on the average for ten tested examples, respectively. Compared with a two-phase algorithm with via introduction on independent nets, the experimental results show that our proposed algorithm with ${c_{\max } = 1}$ , ${c_{\max } = 2}$ , ${c_{\max } = 3}$ , ${c_{\max } = 4}$ , and ${c_{\max } = 5}$ use less CPU time to reduce 1.6, 1.7, 1.7, 1.6, and 1.5 used layers on the average with increasing 13.3%, 16.3%, 10.6%, 4.2%, and 2.6% of the total used vias on the independent nets for ten tested examples, respectively. [ABSTRACT FROM AUTHOR]

We study a multihop “virtual” full-duplex relay channel as a special case of a general multiple multicast relay network. For such a channel, quantize-map-and-forward (QMF) [and its generalization of noisy network coding (NNC) and short message NNC] achieves the cut-set upper bound within a constant additive gap, where the gap grows linearly with the number of relay stages $K$. This gap, however, may not be acceptable for practical communication systems with multihop transmissions (e.g., a wireless backhaul operating at high frequencies). Recently, we improved the capacity scaling by using a forward sliding-window (SW) decoding and by optimizing the quantization level at each relay, obtaining the gap that grows logarithmically as $\log {K}$. Furthermore, the improved scheme has lower decoding complexity and delay than the general QMF and NNC approaches. In this paper, we further improve the performance by presenting a mixed scheme in which each relay can perform either decode-and-forward (DF) or the improved QMF (with SW decoding) and can choose to perform rate-splitting to enable partial interference cancellation. In general, the optimization of the relay DF/QMF configuration is combinatorial. Nevertheless, we provide that a simple greedy algorithm finds an optimal configuration under some practically reasonable assumptions. We derive an achievable rate that is easily computable and show that the proposed mixed scheme outperforms the QMF-only schemes. We demonstrate that the performance improvement increases with $K$ , which indicates that the mixed scheme is indeed beneficial for multihop transmission. [ABSTRACT FROM AUTHOR]

The tensor completion issues have obtained a great deal of attention in the past few years. However, the data fidelity part minimizes a squared loss function, which may be inappropriate for the case of noisy one-bit observations. In this paper, we alleviate the mentioned difficulty by drawing on the experience of matrix scenarios. Based on the convex relation to $\ell _{1}$ norm of the tensor multi-rank, we propose a novel optimization model trying to recover the underlying tensor in case of one-bit observations. The feasibility of this model is proved by theoretical derivations. Furthermore, an alternating direction method of multipliers based algorithm is designed to find the solution. The numerical experiments demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

The performance of an adaptive neurofuzzy inference system (ANFIS) significantly drops when uncertainty exists in the data or system operation. Prediction intervals (PIs) can quantify the uncertainty associated with ANFIS point predictions. This paper first presents a methodology to adapt the delta technique for the construction of PIs for outcomes of the ANFIS models. As the ANFIS models are linear in their consequent part, the ANFIS-based PIs are computationally less expensive than neural network (NN)-based PIs. Second, this paper proposes a method to optimize ANFIS-based PIs. A new PI-based cost function is developed for the training of the ANFIS models. A simulated annealing-based algorithm is applied to minimize the new nonlinear cost function and adjust the premise and consequent parameters of the ANFIS model. Using three real-world case studies, it is shown that ANFIS-based PIs are computationally less expensive than NN-based PIs. The application of the proposed optimization algorithm leads to better quality PIs than optimized NN-based PIs. [ABSTRACT FROM AUTHOR]

Neuro-inspired architectures based on synaptic memory arrays have been proposed for on-chip acceleration of weighted sum and weight update in machine/deep learning algorithms. In this paper, we developed NeuroSim, a circuit-level macro model that estimates the area, latency, dynamic energy, and leakage power to facilitate the design space exploration of neuro-inspired architectures with mainstream and emerging device technologies. NeuroSim provides flexible interface and a wide variety of design options at the circuit and device level. Therefore, NeuroSim can be used by neural networks (NNs) as a supporting tool to provide circuit-level performance evaluation. With NeuroSim, an integrated framework can be built with hierarchical organization from the device level (synaptic device properties) to the circuit level (array architectures) and then to the algorithm level (NN topology), enabling instruction-accurate evaluation on the learning accuracy as well as the circuit-level performance metrics at the run-time of online learning. Using multilayer perceptron as a case-study algorithm, we investigated the impact of the “analog” emerging nonvolatile memory (eNVM)’s “nonideal” device properties and benchmarked the tradeoffs between SRAM, digital, and analog eNVM-based architectures for online learning and offline classification. [ABSTRACT FROM AUTHOR]

This paper investigates the application of support vector machine to the modeling of high-speed interconnects with largely varying and/or highly uncertain design parameters. The proposed method relies on a robust and well-established mathematical framework, yielding accurate surrogates of complex dynamical systems. An identification procedure based on the observation of a small set of system responses allows generating compact parametric relations, which can be used for design optimization and/or stochastic analysis. The feasibility and strength of the method are demonstrated based on a benchmark function and on the statistical assessment of a realistic printed circuit board interconnect, highlighting the main features and benefits of this technique over state-of-the-art solutions. Emphasis is given to the effects of the initial sample size and of input noise on the model estimation. [ABSTRACT FROM AUTHOR]

For the first time, this paper addresses the problem of adaptive output-feedback control for a class of uncertain stochastic nonlinear strict-feedback systems with time-varying delays using neural networks (NNs). The circle criterion is applied to designing a nonlinear observer, and no linear growth condition is imposed on nonlinear functions depending on system states. Under the assumption that time-varying delays exist in the system output, only an NN is employed to compensate for all unknown nonlinear terms depending on the delayed output, and thus, the proposed control algorithm is more simple even than the existing NN backstepping control schemes for uncertain systems described by ordinary differential equations. Three examples are given to demonstrate the effectiveness of the control scheme proposed in this paper. [ABSTRACT FROM AUTHOR]

This paper proposes a new recurrent model, known as the locally recurrent fuzzy neural network with support vector regression (LRFNN-SVR), that handles problems with temporal properties. Structurally, an LRFNN-SVR is a five-layered recurrent network. The recurrent structure in an LRFNN-SVR comes from locally feeding the firing strength of each fuzzy rule back to itself. The consequent layer in an LRFNN-SVR is a Takagi-Sugeno-Kang (T-S-K)-type consequent, which is a linear function of current states, regardless of system input and output delays. For the structure learning, a one-pass clustering algorithm clusters the input-training data and determines the number of network nodes in hidden layers. For the parameter learning, an iterative linear SVR algorithm is proposed to tune free parameters in the rule consequent part and feedback loops. The motivation for using SVR for parameter learning is to improve the LRFNN-SVR generalization ability. This paper demonstrates LRFNN-SVR capabilities by conducting simulations in dynamic system prediction and identification problems with noiseless and noisy data. In addition, this paper compares simulation results from the LRFNN-SVR with other recurrent fuzzy models. [ABSTRACT FROM AUTHOR]

Negative correlation learning (NCL) is a neural network ensemble learning algorithm that introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean square error/MSEI together with the correlation of the ensemble. This paper analyzes NCL and reveals that the training of NCL (when λ = 1) corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This analysis explains the reason why NCL is prone to overtitting the noise in the training set. This paper also demonstrates that tuning the correlation parameter λ in NCL by cross validation cannot overcome the overfitting problem. The paper analyzes this problem and proposes the regularized negative correlation learning (RNCL) algorithm which incorporates an additional regularization term for the whole ensemble. RNCL decomposes the ensemble's training objectives, including MSE and regularization, into a set of sub-objectives, and each sub-objective is implemented by an individual neural network. In this paper, we also provide a Bayesian interpretation for RNCL and provide an automatic algorithm to optimize regularization parameters based on Bayesian inference. The RNCL formulation is applicable to any nonlinear estimator minimizing the MSE. The experiments on synthetic as well as real-world data sets demonstrate that RNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. [ABSTRACT FROM AUTHOR]

Original Hebbian encoding scheme of bidirectional associative memory (BAM) provides a poor pattern capacity and recall performance. Based on Rosenblatt's perceptron learning algorithm, the pattern capacity of BAM is enlarged, and perfect recall of all training pattern pairs is guaranteed. However, these methods put their emphases on pattern capacity, rather than error correction capability which is another critical point of BAM. This paper analyzes the sensitivity to noise in RAM and obtains an interesting idea to improve noise immunity of BAM. Some researchers have found that the noise sensitivity of BAM relates to the minimum absolute value of net inputs (MAV). However, in this paper, the analysis on failure association shows that it is related not only to MAV but also to the variance of weights associated with synapse connections. In fact, it is a positive monotone increasing function of the quotient of MAV divided by the variance of weights. This idea provides an useful principle of improving error correction capability of RAM. Some revised encoding schemes, such as small variance learning for RAM (SVBAM), evolutionary pseudorelaxation learning for BAM (EPRLAB) and evolutionary bidirectional learning (EBL), have been introduced to illustrate the performance of this principle. All these methods perform better than their original versions in noise immunity. Moreover, these methods have no negative effect on the pattern capacity of BAM. The convergence of these methods is also discussed in this paper. If there exist solutions, EPRLAB and EBL always converge to a global optimal solution in the senses of both, pattern capacity and noise immunity. However, the convergence of SVBAM may be affected by a preset function. [ABSTRACT FROM AUTHOR]

Under-complete models, which derive lower dimensional representations of input data, are valuable in domains in which the number of input dimensions is very large, such as data consisting of a temporal sequence of images. This paper presents the under-complete product of experts (UPoE), where each expert models a one-dimensional projection of the data. Maximum-likelihood learning rules for this model constitute a tractable and exact algorithm for learning under-complete independent components. The learning rules for this model coincide with approximate learning rules proposed earlier for under-complete independent component analysis (UICA) models. This paper also derives an efficient sequential learning algorithm from this model and discusses its relationship to sequential independent component analysis (ICA), projection pursuit density estimation, and feature induction algorithms for additive random field models. This paper demonstrates the efficacy of these novel algorithms on high-dimensional continuous datasets. [ABSTRACT FROM AUTHOR]

For the first time, a deep learning (DL) algorithm is presented to study the effect of the source of variability on the performance of semiconductor nanodevice. This paper reports the possibility of an alternative solution of device simulation in order to optimize the source variation. It is based on the statistical distribution of work function fluctuation (WKF) on the metal gate depending on the orientation and location of metal grains. It has been revealed that the WKF of a metal gate can lead to different fluctuations in electrical characteristics. Therefore, an emerging DL algorithm, artificial neural network (ANN) is utilized to identify the appropriate WKF patterns on the metal gate that can reduce the impact of characteristic fluctuation, i.e., $\sigma V_{TH}, \sigma I_{ON}$ and $\sigma I_{OFF}$ , simultaneously. The application of the DL-ANN algorithm to multichannel gate-all-around silicon nanosheet MOSFETs is explored to suppress the effect of WKF on the characteristic fluctuation. Consequently, it can be further utilized to investigate the implication of WKF for the process variation, modeling the nanodevices and analysis of circuit design. Notably, this technique can be extended to study the diverse random sources and process variation effects for emerging nano-CMOS devices and can effectively accelerate the device simulation and optimization. [ABSTRACT FROM AUTHOR]

Prediction and shaping of dynamic response of the demand will enable advanced control algorithms for active demand management as well as improved stability assessment of the power system. Based on previously developed demand disaggregation approach, this paper develops a methodology for day-ahead prediction and shaping of dynamic response of the demand at bulk supply points without having to perform field measurements. The methodology is broadly based on application of artificial neural network and Monte Carlo simulations and incorporates multiple approaches namely, load forecasting, load disaggregation and the component-based load modelling approach. The input data include standard rms measurements at bulk supply points and actual and day-ahead forecasted weather data and does not rely on having access to detailed customer surveys or high-resolution load signatures. Measured dynamic responses of the demand from the substations of the local utility are used for validation. Load shifting and shaping of dynamic response of the demand are also illustrated. [ABSTRACT FROM PUBLISHER]

A deep learning assisted sum-product detection algorithm (DL-SPDA) for faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed detection algorithm works on a modified factor graph which concatenates a neural network function node to the variable nodes of the conventional FTN factor graph to approach the maximum a posterior probabilities (MAP) error performance. In specific, the neural network performs as a function node in the modified factor graph to deal with the residual intersymbol interference (ISI) that is not considered by the conventional detector with a limited complexity. We modify the updating rule in the conventional sum-product algorithm so that the neural network assisted detector can be complemented to a turbo equalization receiver. Furthermore, we propose a compatible training technique to improve the detection performance of the proposed DL-SPDA with turbo equalization. In particular, the neural network is optimized in terms of the mutual information between the transmitted sequence and the extrinsic information. We also investigate the maximum-likelihood bit error rate (BER) performance of a finite length coded FTN system. Simulation results show that the error performance of the proposed algorithm approaches the MAP performance, which is consistent with the analytical BER. [ABSTRACT FROM AUTHOR]

In order to enable voltage ride-through operation, fast and reliable voltage dips detection is necessary. It has been noticed by the analysis of a huge number of measurement results recorded in-field, in laboratory, and obtained by simulation that the voltage dips may be characterized by specific low-order voltage harmonics pattern, which appears at the beginning and at the end of the voltage dips. This feature is named harmonic footprint. This paper proposes a new method of voltage dips detection based on harmonic footprint as part of smart algorithm for detection and classification of power quality events. The aim is to achieve reliable smart voltage dips detection in 1 ms range. Several methods for harmonic estimation in real time were compared in order to choose the optimal one. The footprint is modeled by mathematical function and applied for voltage dips smart detection using recurrent neural network. The proposed method was tested for voltage dips detection analyzing signals from measurement in the real distribution grid (680 records), computer simulations using IEEE-13 bus test grid, and testing in the laboratory. The reliability (detection rate) and speed of voltage dips detection are tested and verified. Furthermore, the future research directions for presented method are outlined. [ABSTRACT FROM AUTHOR]

Stochastic computation has recently been proposed for implementing artificial neural networks with reduced hardware and power consumption, but at a decreased accuracy and processing speed. Most existing implementations are based on pre-training such that the weights are predetermined for neurons at different layers, thus these implementations lack the ability to update the values of the network parameters. In this paper, a stochastic computational multi-layer perceptron (SC-MLP) is proposed by implementing the backward propagation algorithm for updating the layer weights. Using extended stochastic logic (ESL), a reconfigurable stochastic computational activation unit (SCAU) is designed to implement different types of activation functions such as the $tanh$ and the rectifier function. A triple modular redundancy (TMR) technique is employed for reducing the random fluctuations in stochastic computation. A probability estimator (PE) and a divider based on the TMR and a binary search algorithm are further proposed with progressive precision for reducing the required stochastic sequence length. Therefore, the latency and energy consumption of the SC-MLP are significantly reduced. The simulation results show that the proposed design is capable of implementing both the training and inference processes. For the classification of nonlinearly separable patterns, at a slight loss of accuracy by 1.32-1.34 percent, the proposed design requires only 28.5-30.1 percent of the area and 18.9-23.9 percent of the energy consumption incurred by a design using floating point arithmetic. Compared to a fixed-point implementation, the SC-MLP consumes a smaller area (40.7-45.5 percent) and a lower energy consumption (38.0-51.0 percent) with a similar processing speed and a slight drop of accuracy by 0.15-0.33 percent. The area and the energy consumption of the proposed design is from 80.7-87.1 percent and from 71.9-93.1 percent, respectively, of a binarized neural network (BNN), with a similar accuracy. [ABSTRACT FROM AUTHOR]