Search

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

With the advancing growth of the World Wide Web (WWW) and the expanding availability of electronic text documents, the automatic assignment of text classification (ATC) has become more important in sorting out information and knowledge. One of the most crucial tasks that should be carried out is document representation using word embedding and Rule-Based methodologies. As a result, this, along with their modeling methods, has become an essential step to improve neural language processing for text classification. In this paper, a systematic mapping study is a way to survey all the primary studies on word embedding to rule-based and machine learning of automatic text classification. The search procedure identifies 20 articles as relevant to answer our research questions. This study maps what is currently known about word embedding in rule-based text classification (TC). The result shows that the research is concentrated on some main areas, mainly in social sciences, shopping products classification, digital libraries, and spam filtering. The present paper contributes to the available literature by summarizing all research in the field of TC and it can be beneficial to other researchers and specialists in order to sort information. [ABSTRACT FROM AUTHOR]

The paper proposes the concept of inverse P-augmented matrix, and gives its structure; the inverse P-augmented matrix is obtained by improving the common augmented matrix; under certain condition, the inverse P-augmented matrix is reduced to the common augmented matrix. Inverse P-sets is the theoretical basis of proposing inverse P-augmented matrix, which is a new set model with dynamic feature. The paper gives the introduction of inverse-P set, the generation method of inverse P-augmented matrix and the relation between inverse P-augmented matrix and common augmented matrix, and gives the application of inverse P-augmented matrix in the dynamic findings of unknown information by virtue of these studies. [ABSTRACT FROM AUTHOR]

Analyzing and predicting the trend of price fluctuation has been receiving more and more attention, as price risk has become the focus of risk control research in heating oil futures market. A novel time series prediction model combined with the complex network method is put forward in the paper. First of all, this paper counts the cumulative time interval of different nodes in the network, and fits its growth trend with the Fourier model. Then a novel price fluctuation prediction model is established based on the effective information such as some topology properties extracted from the network. The results show that the Fourier model can predict the emergence time of new nodes in the next stage, and the established price fluctuation prediction model can infer the names of nodes in the prediction interval, so as to determine the forward-looking behavior of price evolution. Besides, liken to the NAR neural network, the prediction results obtained by the proposed method also show superiority, which has important theoretical value and academic significance for early warning and prediction of price behavior in the heating oil futures market. [ABSTRACT FROM AUTHOR]

Due to the huge amount of data being generating from different sources, the analyzing and extracting of useful information from these data becomes a very complex task. The difficulty of dealing with big data optimization problems comes from many factors such as the high number of features, and the existing of lost data. The feature selection process becomes an important step in many data mining and machine learning algorithms to reduce the dimensionality of the optimization problems and increase the performance of the classification or clustering algorithms. In this paper, a set of hybrid and efficient genetic algorithms are proposed to solve feature selection problem, when the handled data has a large feature size. The proposed algorithms use a new gene-weighted mechanism that can adaptively classify the features into strong relative features, weak or redundant features, and unstable features during the evolution of the algorithm. Based on this classification, the proposed algorithm gives the strong features high priority and the weak features less priority when generating new candidate solutions. In the same time, the proposed algorithm tries to more concentrate on unstable features that sometimes appear and sometimes disappear from the best solutions of the population. The performance of proposed algorithms is investigated by using different datasets and feature selection algorithms. The results show that our proposed algorithms can outperform the other feature selection algorithms and effectively enhance the classification performance over the tested datasets. [ABSTRACT FROM AUTHOR]

The 3D measurement system based on line-structured light uses a camera to capture laser stripes due to changing in the shape of an object, and uses the acquired pixel coordinates for 3D reconstruction. System calibration is an important step in 3D measurement. The current camera calibration algorithm research mainly focuses on improving the algorithm itself, and there is less research on the influence of external factors. This paper proposes a coplanar hybrid calibration algorithm based on the error screening model by combining the error screening model, mathematical model and neural network model. It is mainly divided into two steps. The first step is to use the radial array constraint calibration algorithm based on the error screening model to solve the camera's internal and external parameters. The second step uses the camera internal and external parameters obtained in the first step to convert the pixel coordinates into real three-dimensional coordinates, and compares the calculated three-dimensional coordinates with the actual coordinates. Using machine learning to establish a compensation network, get a compensation function, and use the resulting 3D world coordinates to perform point cloud stitching. Experiments show that compared with the traditional calibration algorithm, the calibration algorithm has a small error and reduces the calibration error by about 6.5%. [ABSTRACT FROM AUTHOR]

Deep learning is a branch of neural network which has been intensively developed in the last decade. Due to the high-accuracy classification ability, the deep learning algorithms have been widely used in many fields, such as speech recognition, image recognition, and natural speech processing. However, they also show some shortcomings especially on the selection of some parameters in the network, including hyper-parameters, which is still treated as a time consuming task. In this paper, a modified ABC (ABC-ISB) optimization algorithm is proposed to automatically train the parameters of Feed-Forward Artificial Neural Networks, which is a typical a neural network. In the proposed ABC algorithm, we utilize the information of neighbors with better performance to accelerate the convergence of employed and onlooker bees respectively. In addition, a new selection strategy and a gbest-guided strategy are introduced to enhance the global search capability and balance the exploration and exploitation of the algorithm separately. The experimental results show our ABC-ISB is generally leading and competitive. [ABSTRACT FROM AUTHOR]

Along with potential benefits to healthcare delivery, machine learning healthcare applications (ML-HCAs) raise a number of ethical concerns. Ethical evaluations of ML-HCAs will need to structure the overall problem of evaluating these technologies, especially for a diverse group of stakeholders. This paper outlines a systematic approach to identifying ML-HCA ethical concerns, starting with a conceptual model of the pipeline of the conception, development, implementation of ML-HCAs, and the parallel pipeline of evaluation and oversight tasks at each stage. Over this model, we layer key questions that raise value-based issues, along with ethical considerations identified in large part by a literature review, but also identifying some ethical considerations that have yet to receive attention. This pipeline model framework will be useful for systematic ethical appraisals of ML-HCA from development through implementation, and for interdisciplinary collaboration of diverse stakeholders that will be required to understand and subsequently manage the ethical implications of ML-HCAs. [ABSTRACT FROM AUTHOR]

Analyzing geological drilling hole images acquired by Axial View Panoramic Borehole Televiewer (APBT) is a key step to explore the geological structure in a geological exploration. Conventionally, the borehole images are examined by technicians, which is inefficient and subjective. In this paper, three dominant types of borehole-wall images on coal-rock mass structure, namely, border images, fracture images and intact rock mass images are mainly studied. The traditional image classification methods based on unified feature extraction algorithm and single classifier is not effect for the borehole images. Therefore, this paper proposes a novel two-stage classification approach to improve the classification performance of borehole images. In the first-stage classification, the border images are identified from three kinds of images based on texture features and gray-scale histograms features. For the remaining two types of images, in the second-stage classification, Gabor filter is first applied to segment the region of interest (ROI) (such as microfracture, absciss layer and horizontal cracks, etc.) and the central interference region. Then, using the same feature vector after eliminating the central interference region, fracture images are separated from intact rock mass images. We test our two-stage classification system with real borehole images. The results of experimental show that the two-stage classification method can effectively classify three major borehole-wall images with the correction rate of 95.55% in the first stage and 95% in the second stage. [ABSTRACT FROM AUTHOR]

In this paper, we propose a hybrid method to forecast the stock prices called High-order-fuzzy-fluctuation-Trends-based Back Propagation(HTBP)Neural Network model. First, we compare each value of the historical training data with the previous day's value to obtain a fluctuation trend time series (FTTS). On this basis, the FTTS blur into fuzzy time series (FFTS) based on the fluctuation of the increasing, equality, decreasing amplitude and direction. Since the relationship between FFTS and future wave trends is nonlinear, the HTBP neural network algorithm is used to find the mapping rules in the form of self-learning. Finally, the results of the algorithm output are used to predict future fluctuations. The proposed model provides some innovative features:(1)It combines fuzzy set theory and neural network algorithm to avoid overfitting problems existed in traditional models. (2)BP neural network algorithm can intelligently explore the internal rules of the actual existence of sequential data, without the need to analyze the influence factors of specific rules and the path of action. (3)The hybrid modal can reasonably remove noises from the internal rules by proper fuzzy treatment. This paper takes the TAIEX data set of Taiwan stock exchange as an example, and compares and analyzes the prediction performance of the model. The experimental results show that this method can predict the stock market in a very simple way. At the same time, we use this method to predict the Shanghai stock exchange composite index, and further verify the effectiveness and universality of the method. [ABSTRACT FROM AUTHOR]

The primary purpose of this paper is to realize robust place recognition algorithms towards simultaneous viewpoint and condition changes, and provide satisfactory computational efficiency. In this paper, we significantly improve the viewpoint invariance of the SeqSLAM algorithm by using state-of-the-art deep learning techniques to generate robust feature representations of images and develop the SeqCNNSLAM . Experimental results show that SeqCNNSLAM outperforms state-of-the-art place recognition systems in most cases, such as, when precision is maintained at 100%, the maximum recall obtained by SeqCNNSLAM is 50% higher than SeqSLAM on the Norland dataset with simultaneous condition change and 12.5% viewpoint change. Besides, we develop an acceleration method called A-SeqCNNSLAM , which exploits the location relationship between the matching images of adjacent images to reduce the matching range of the current image. Experimental results demonstrate that an acceleration of  ∼  5 times is achieved with minimal accuracy degradation of  ∼  5%. Finally, to enable A-SeqCNNSLAM adaptability in new environments, O-SeqCNNSLAM is devised for the online parameter adjustment in A-SeqCNNSLAM. [ABSTRACT FROM AUTHOR]

An algorithm that generates an exponential number of stable states for the very well-known Hopfield Neural Network (HNN) is introduced in this paper. We show that the quantity of stable states depends on the dimension and number of components of the input pattern supporting noise. Extensive tests verify that the states generated by our algorithm are stable states and show the exponential storage capacity of a HNN. This paper opens the possibility of designing improved HNNs able to achieve exponential storage, and thus find their applicability in complex real-world problems. [ABSTRACT FROM AUTHOR]

Recently, Extreme learning machine (ELM), an efficient training algorithm for single-hidden-layer feedforward neural networks (SLFN), has gained increasing popularity in machine learning communities. In this paper the ELM based Area Under the ROC Curve (AUC) optimization algorithms are studied so as to further improve the performance of ELM for imbalanced datasets. For binary class problems, a novel ELM algorithm is proposed based on an efficient least square method. For multi-class problems, the following works are done in this paper: First of all, theoretical comparison analysis is proposed for the potential multi-class extensions of AUC; Secondly, a unified objective function for multi-class AUC optimization is proposed following the theoretical analysis; Subsequently, two ELM based multi-class AUC optimization algorithms called E L M M A U C and E L M m a c r o A U C respectively are proposed followed with complexity analyses; Finally, the generalization analysis is established for E L M M A U C in search of theoretical supports. Empirical study on a variety of real-world datasets show the effectiveness of our proposed algorithms. [ABSTRACT FROM AUTHOR]

Nature-inspired algorithms have been developed with biological mimicking. Machine learning algorithms from artificial neurons and artificial neural networks have been developed to mimic the human brain with synthetic neurons. This research can be traced back to the 1940s and has been expanded to agri-food problem solving in the last three decades. Now, the research and applications have entered the stage of deep learning with more layers and neurons that have complex connections to extract deep features of the target. In this paper, the developments of artificial neural networks and deep learning algorithms are presented and discussed in conjunction with their biological connections for agri-food applications. The related independent studies previously conducted by the author are summarized with the newly conducted being presented. At the same time, the algorithms motivated by recent bionics studies are compared and discussed for their potentials for agri-food production. [ABSTRACT FROM AUTHOR]

The Industry 4.0 revolution is insisting strongly for use of machine learning-based processes and condition monitoring. In this paper, emphasis is given on machine learning-based approach for condition monitoring of shaft misalignment. This work highlights combined approach of artificial neural network and support vector machine for identification and measure of shaft misalignment. The measure of misalignment requires more features to be extracted under variable load conditions. Hence, primary objective is to measure misalignment with a minimum number of extracted features. This is achieved through normalization of vibration signal. An experimental setup is prepared to collect the required vibration signals. The normalized time domain nonstationary signals are given to discrete wavelet transform for features extraction. The extracted features such as detailed coefficient is considered for feature selection viz. Skewness, Kurtosis, Max, Min, Root mean square, and Entropy. The ReliefF algorithm is used to decide best feature on rank basis. The ratio of maximum energy to Shannon entropy is used in wavelet selection. The best feature is used to train machine learning algorithm. The rank-based feature selection has improved classification accuracy of support vector machine. The result obtained with the combined approach are discussed for different misalignment conditions. [ABSTRACT FROM AUTHOR]

Introduction/purpose: The anomaly-based intrusion detection system detects intrusions based on a reference model which identifies the normal behavior of a computer network and flags an anomaly. Machine-learning models classify intrusions or misuse as either normal or anomaly. In complex computer networks, the number of training records is large, which makes the evaluation of the classifiers computationally expensive. Methods: A feature selection algorithm that reduces the dataset size is presented in this paper. Results: The experiments are conducted on the Kyoto 2006+ dataset and four classifier models: feedforward neural network, k-nearest neighbor, weighted k-nearest neighbor, and medium decision tree. The results show high accuracy of the models, as well as low false positive and false negative rates. Conclusion: The three-step pre-processing algorithm for feature selection and instance normalization resulted in improving performances of four binary classifiers and in decreasing processing time. [ABSTRACT FROM AUTHOR]

With age, our blood vessels are prone to aging, which induces cardiovascular disease. As an important basis for diagnosing heart disease and evaluating heart function, the electrocardiogram (ECG) records cardiac physiological electrical activity. Abnormalities in cardiac physiological activity are directly reflected in the ECG. Thus, ECG research is conducive to heart disease diagnosis. Considering the complexity of arrhythmia detection, we present an improved convolutional neural network (CNN) model for accurate classification. Compared with the traditional machine learning methods, CNN requires no additional feature extraction steps due to the automatic feature processing layers. In this paper, an improved CNN is proposed to automatically classify the heartbeat of arrhythmia. Firstly, all the heartbeats are divided from the original signals. After segmentation, the ECG heartbeats can be inputted into the first convolutional layers. In the proposed structure, kernels with different sizes are used in each convolution layer, which takes full advantage of the features in different scales. Then a max-pooling layer followed. The outputs of the last pooling layer are merged and as the input to fully-connected layers. Our experiment is in accordance with the AAMI inter-patient standard, which included normal beats (N), supraventricular ectopic beats (S), ventricular ectopic beats (V), fusion beats (F), and unknown beats (Q). For verification, the MIT arrhythmia database is introduced to confirm the accuracy of the proposed method, then, comparative experiments are conducted. The experiment demonstrates that our proposed method has high performance for arrhythmia detection, the accuracy is 99.06%. When properly trained, the proposed improved CNN model can be employed as a tool to automatically detect different kinds of arrhythmia from ECG. [ABSTRACT FROM AUTHOR]

Computational drug repositioning, designed to identify new indications for existing drugs, significantly reduced the cost and time involved in drug development. Prediction of drug-disease associations is promising for drug repositioning. Recent years have witnessed an increasing number of machine learning-based methods for calculating drug repositioning. In this paper, a novel feature learning method based on Gaussian interaction profile kernel and autoencoder (GIPAE) is proposed for drug-disease association. In order to further reduce the computation cost, both batch normalization layer and the full-connected layer are introduced to reduce training complexity. The experimental results of 10-fold cross validation indicate that the proposed method achieves superior performance on Fdataset and Cdataset with the AUCs of 93.30% and 96.03%, respectively, which were higher than many previous computational models. To further assess the accuracy of GIPAE, we conducted case studies on two complex human diseases. The top 20 drugs predicted, 14 obesity-related drugs, and 11 drugs related to Alzheimer's disease were validated in the CTD database. The results of cross validation and case studies indicated that GIPAE is a reliable model for predicting drug-disease associations. [ABSTRACT FROM AUTHOR]

Leveraging smart metering solutions to support energy efficiency on the individual household level poses novel research challenges in monitoring usage and providing accurate load forecasting. Forecasting electricity usage is an especially important component that can provide intelligence to smart meters. In this paper, we propose an enhanced approach for load forecasting at the household level. The impacts of residents’ daily activities and appliance usages on the power consumption of the entire household are incorporated to improve the accuracy of the forecasting model. The contributions of this paper are threefold: (1) we addressed short-term electricity load forecasting for 24 hours ahead, not on the aggregate but on the individual household level, which fits into the Residential Power Load Forecasting (RPLF) methods; (2) for the forecasting, we utilized a household specific dataset of behaviors that influence power consumption, which was derived using segmentation and sequence mining algorithms; and (3) an extensive load forecasting study using different forecasting algorithms enhanced by the household activity patterns was undertaken. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) is a common base signal used to monitor brain activities and diagnose sleep disorders. Manual sleep stage scoring is a time-consuming task for sleep experts and is limited by inter-rater reliability. In this paper, we propose an automatic sleep stage annotation method called SleepEEGNet using a single-channel EEG signal. The SleepEEGNet is composed of deep convolutional neural networks (CNNs) to extract time-invariant features, frequency information, and a sequence to sequence model to capture the complex and long short-term context dependencies between sleep epochs and scores. In addition, to reduce the effect of the class imbalance problem presented in the available sleep datasets, we applied novel loss functions to have an equal misclassified error for each sleep stage while training the network. We evaluated the performance of the proposed method on different single-EEG channels (i.e., Fpz-Cz and Pz-Oz EEG channels) from the Physionet Sleep-EDF datasets published in 2013 and 2018. The evaluation results demonstrate that the proposed method achieved the best annotation performance compared to current literature, with an overall accuracy of 84.26%, a macro F1-score of 79.66% and κ = 0.79. Our developed model can be applied to other sleep EEG signals and aid the sleep specialists to arrive at an accurate diagnosis. The source code is available at . [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have revolutionized the computer vision research and have seen unprecedented adoption for multiple tasks, such as classification, detection, and caption generation. However, they offer little transparency into their inner workings and are often treated as black boxes that deliver excellent performance. In this paper, we aim at alleviating this opaqueness of CNNs by providing visual explanations for the network’s predictions. Our approach can analyze a variety of CNN-based models trained for computer vision applications, such as object recognition and caption generation. Unlike the existing methods, we achieve this via unraveling the forward pass operation. The proposed method exploits feature dependencies across the layer hierarchy and uncovers the discriminative image locations that guide the network’s predictions. We name these locations CNN fixations, loosely analogous to human eye fixations. Our approach is a generic method that requires no architectural changes, additional training, or gradient computation, and computes the important image locations (CNN fixations). We demonstrate through a variety of applications that our approach is able to localize the discriminative image locations across different network architectures, diverse vision tasks, and data modalities. [ABSTRACT FROM AUTHOR]

Antenna selection in Multiple-Input Multiple-Output (MIMO) systems has attracted increasing attention due to the challenge of keeping a balance between communication performance and computational complexity. Recently, deep learning based methods have achieved promising performance in many application fields. This paper proposed a deep learning (DL) based antenna selection technique. First, we generated the label of training antenna systems by maximizing the channel capacity. Then, we adopted the deep convolutional neural network (CNN) on the channel matrices to explicitly exploit the massive latent cues of attenuation coefficients. Finally, we used the adopted CNN to assign the class label and then select the optimal antenna subset. Experimental results demonstrate that our method can achieve better performance than the state-of-the-art baselines for data-driven based antenna selection. [ABSTRACT FROM AUTHOR]

Abstract In streaming data classification, most of the existing methods assume that all arrived evolving data are completely labeled. One challenge is that some applications where only small amount of labeled examples are available for training. Incremental semi-supervised learning algorithms have been proposed for regularizing neural networks by incorporating various side information, such as pairwise constraints or user-provided labels. However, it is hard to put them into practice, especially for non-stationary environments due to the effectiveness and parameter sensitivity of such algorithms. In this paper, we propose a novel incremental semi-supervised learning framework on streaming data. Each layer of model is comprised of a generative network, a discriminant structure and the bridge. The generative network uses dynamic feature learning based on autoencoders to learn generative features from streaming data which has been demonstrated its potential in learning latent feature representations. In addition, the discriminant structure regularizes the network construction via building pairwise similarity and dissimilarity constraints. It is also used for facilitating the parameter learning of the generative network. The network and structure are integrated into a joint learning framework and bridged by enforcing the correlation of their parameters, which balances the flexible incorporation of supervision information and numerical tractability for non-stationary environments as well as explores the intrinsic data structure. Moreover, an efficient algorithm is designed to solve the proposed optimization problem and we also give an ensemble method. Particularly, when multiple layers of model are stacked, the performance is significantly boosted. Finally, to validate the effectiveness of the proposed method, extensive experiments are conducted on synthetic and real-life datasets. The experimental results demonstrate that the performance of the proposed algorithms is superior to some state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The multimode power-split architecture for hybrid electric vehicle (HEV) powertrains is generally known for the complexity of its operation. This paper first addresses the challenge of developing an automated on-line near-optimal control strategy for these systems. Particularly, a machine learning logic based on supervised learning is developed for on-line selection of the HEV operating mode, thus the particular set of clutches to be engaged. An efficiency-based approach is then adopted to determine the optimal power-split between powertrain components. Later, the developed strategy finds integration in an optimal design methodology for multimode power-split HEVs considering the effectiveness and ease of on-line controllability. The obtained results are compared with the ones by the traditional HEV design methodology that only considers off-line energy management. The illustrated design methodology with on-line control reveals efficient at identifying the multimode HEV design that demonstrates optimal predicted fuel economy values and ease of on-line controllability simultaneously. Results suggest that the HEV design optimization procedure may produce different outcomes and demonstrate more effective when the evaluation of the on-line controlled operation is integrated. [ABSTRACT FROM AUTHOR]

In general, the stock prices of the same industry have a similar trend, but those of different industries do not. When investing in stocks of different industries, one should select the optimal model from lots of trading models for each industry because any model may not be suitable for capturing the stock trends of all industries. However, the study has not been carried out at present. In this paper, firstly we select 424 S&P 500 index component stocks (SPICS) and 185 CSI 300 index component stocks (CSICS) as the research objects from 2010 to 2017, divide them into 9 industries such as finance and energy respectively. Secondly, we apply 12 widely used machine learning algorithms to generate stock trading signals in different industries and execute the back-testing based on the trading signals. Thirdly, we use a non-parametric statistical test to evaluate whether there are significant differences among the trading performance evaluation indicators (PEI) of different models in the same industry. Finally, we propose a series of rules to select the optimal models for stock investment of every industry. The analytical results on SPICS and CSICS show that we can find the optimal trading models for each industry based on the statistical tests and the rules. Most importantly, the PEI of the best algorithms can be significantly better than that of the benchmark index and “Buy and Hold” strategy. Therefore, the algorithms can be used for making profits from industry stock trading. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we propose the use of data symmetries, in the sense of equivalences under signal transformations, as priors for learning symmetry-adapted data representations, i.e., representations that are equivariant to these transformations. We rely on a group-theoretic definition of equivariance and provide conditions for enforcing a learned representation, for example the weights in a neural network layer or the atoms in a dictionary, to have the structure of a group and specifically the group structure in the distribution of the input. By reducing the analysis of generic group symmetries to permutation symmetries, we devise a regularization scheme for representation learning algorithm, using an unlabeled training set. The proposed regularization is aimed to be a conceptual, theoretical and computational proof of concept for symmetry-adapted representation learning, where the learned data representations are equivariant or invariant to transformations, without explicit knowledge of the underlying symmetries in the data. [ABSTRACT FROM AUTHOR]

Abstract This paper presents a machine learning-based approach to power transformer fault diagnosis based on dissolved gas analysis (DGA), a bat algorithm (BA), optimizing the probabilistic neural network (PNN). PNN is a radial basis function feedforward neural network based on Bayesian decision theory, which has a strong fault tolerance and significant advantages in pattern classification. However, one challenge still remains: the performance of PNN is greatly affected by its hidden layer element smooth factor which impacts the classification performance. The proposed approach addresses this challenge by deploying the BA algorithm, a kind of bio-inspired algorithm to optimize PNN. Using the real data collected from a transformer system, we conducted the experiments for validating the performance of the developed method. The experimental results demonstrated that BA is an effective algorithm for optimizing PNN smooth factor and BA-PNN can improve the fault diagnosis performance; in turn, and the machine learning-based model (BA-PNN) can significantly enhance the accuracies of power transformer fault diagnosis. [ABSTRACT FROM AUTHOR]

Liao et al. (Neurocomputing 128:81-87, 2014) proposed a meta-learning approach to extreme learning machine (Meta-ELM), which can obtain good generalization performance by training multiple ELMs. However, one of its open problems is overfitting when minimizing training error. In this paper, we propose an improved meta-learning model of ELM (improved Meta-ELM) to handle the problem. The improved Meta-ELM architecture is composed of some base ELMs which are error feedback incremental extreme learning machine (EFI-ELM) and the top ELM. The improved Meta-ELM includes two stages. First, each base ELM with EFI-ELM is trained on a subset of training data. Then, the top ELM learns with the base ELMs as hidden nodes. Simulation results on some artificial and benchmark datasets show that the proposed improved Meta-ELM model is more feasible and effective than Meta-ELM. [ABSTRACT FROM AUTHOR]

Object tracking still remains challenging in computer vision because of the severe object variation, e.g., deformation, occlusion, and rotation. To handle the object variation and achieve robust object tracking performance, we propose a novel relationship-based tracking algorithm using neural networks in this paper. Compared with existing approaches in the literature, our method assumes the targeted object to be consisted of several parts and considers the evolution of the topology structure among these parts. After training a candidate neural network for predicting the probable areas each part may locate at in the successive frame, we then design a novel collaboration neural network to determine the precise area each part will locate at with account for the topology structure among these individual parts, which is learned from their historical physical locations during online tracking process. Experimental results show that the proposed method outperforms state-of-the-art trackers on a benchmark dataset, yielding the significant improvements in accuracy on high-distorted sequences. [ABSTRACT FROM AUTHOR]

Neuro-inspired architectures based on synaptic memory arrays have been proposed for on-chip acceleration of weighted sum and weight update in machine/deep learning algorithms. In this paper, we developed NeuroSim, a circuit-level macro model that estimates the area, latency, dynamic energy, and leakage power to facilitate the design space exploration of neuro-inspired architectures with mainstream and emerging device technologies. NeuroSim provides flexible interface and a wide variety of design options at the circuit and device level. Therefore, NeuroSim can be used by neural networks (NNs) as a supporting tool to provide circuit-level performance evaluation. With NeuroSim, an integrated framework can be built with hierarchical organization from the device level (synaptic device properties) to the circuit level (array architectures) and then to the algorithm level (NN topology), enabling instruction-accurate evaluation on the learning accuracy as well as the circuit-level performance metrics at the run-time of online learning. Using multilayer perceptron as a case-study algorithm, we investigated the impact of the “analog” emerging nonvolatile memory (eNVM)’s “nonideal” device properties and benchmarked the tradeoffs between SRAM, digital, and analog eNVM-based architectures for online learning and offline classification. [ABSTRACT FROM AUTHOR]

This paper investigates the application of support vector machine to the modeling of high-speed interconnects with largely varying and/or highly uncertain design parameters. The proposed method relies on a robust and well-established mathematical framework, yielding accurate surrogates of complex dynamical systems. An identification procedure based on the observation of a small set of system responses allows generating compact parametric relations, which can be used for design optimization and/or stochastic analysis. The feasibility and strength of the method are demonstrated based on a benchmark function and on the statistical assessment of a realistic printed circuit board interconnect, highlighting the main features and benefits of this technique over state-of-the-art solutions. Emphasis is given to the effects of the initial sample size and of input noise on the model estimation. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure between two arbitrary variables, and it has been applied in a variety of learning algorithms to improve noise insensitivity. In this paper, based on the correntropy, a non-convex and bounded loss function is obtained which contains second and higher order moments of the classification margin. And the novel loss function is robust to noises and close to the 0–1 loss function. Then we introduce it into extreme learning machine (ELM), and propose a correntropy-based robust ELM framework for classification, trained by half quadratic optimization to cope with non-convexity of the algorithm. To evaluate robustness, feature noise and label noise are simulated to provide noisy environments. Experimental results on benchmark datasets demonstrate that the proposed algorithm is better than original algorithms and robust algorithms. Moreover, the superiority of proposed algorithm in noisy environment is more evident, which further proves its robustness to noises. [ABSTRACT FROM AUTHOR]

In palynology, the visual classification of pollen grains from different species is a hard task which is usually tackled by human operators using microscopes. Its complete automatization would save a high quantity of resources and provide valuable improvements especially for allergy-related information systems, but also for other application fields as paleoclimate reconstruction, quality control of honey based products, collection of evidences in criminal investigations or fabric dating and tracking. This paper presents three state-of-the-art deep learning classification methods applied to the recently published POLEN23E image dataset. The three methods make use of convolutional neural networks: the first one is strictly based on the idea of transfer learning, the second one is based on feature extraction and the third one represents a hybrid approach, combining transfer learning and feature extraction. The results from the three methods are indeed very good, reaching over 97% correct classification rates in images not previously seen by the models, where other authors reported around 70. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) is one of the most popular machine learning methods and is widely used in many modern applications. The training process of DNNs is a time-consuming process. Accelerating the training of DNNs has been the focus of many research works. In this paper, we speed up the training of DNNs applied for automatic speech recognition and the target architecture is heterogeneous (CPU + MIC). We apply asynchronous methods for I/O and communication operations and propose an adaptive load balancing method. Besides, we also employ a momentum idea to speed up the convergence of the gradient descent algorithm. Experimental results show that our optimized algorithm is able to acquire a 20-fold speedup on a CPU + MIC platform compared with the original sequential algorithm on a single-core CPU. [ABSTRACT FROM AUTHOR]

This paper intends to propose an integrated hybrid algorithm for training radial basis function neural network (RBFNN) learning. The proposed integrated of particle swarm and genetic algorithm based optimization (IPGO) algorithm is composed of two approaches based on particle swarm optimization (PSO) and genetic algorithm (GA) for gathering both their virtues to improve the learning performance of RBFNN. The diversity of individuals results in higher chance to search in the direction of global optimal instead of being confined to local optimal particularly in problem with higher complexity. The IPGO algorithm with PSO-based and GA-based approaches has shown promising results in some benchmark problems with three continuous test functions. After proposing the algorithm for these problems with result providing its outperforming performance, this paper supplements a practical application case for the papaya milk sales forecasting to expound the superiority of the IPGO algorithm. In addition, model evaluation results of the case have showed that the IPGO algorithm outperforms other algorithms and auto-regressive moving average (ARMA) models in terms of forecasting accuracy and execution time. [ABSTRACT FROM AUTHOR]

Digital holographic microscopy (DHM) is a rapidly developing imaging technique that allows for the reconstruction of three-dimensional images from holographic data. In this review paper, we present an overview of the state-of-the-art in on-axis DHM reconstruction methods. We classify these methods into three categories: direct, iterative, and machine learning based algorithms. In the direct category, we summarize methods that directly apply the holographic data to reconstruct the sample without any iterative optimization. These methods are fast and efficient, but they often suffer from noise and low contrast. The iterative category contains methods that use an iterative optimization process to improve the quality of the reconstructed image. This can lead to higher accuracy and noise reduction, but at the cost of increased computational complexity. Within the iterative category, we also discuss methods that assume sparsity in the reconstruction and other domains, which can further improve the convergence of the reconstruction algorithm. Finally, we describe machine learning methods for DHM reconstruction. These methods use deep neural networks trained on large datasets to learn the relationship between 2D holographic data and their reconstructed fields. Machine learning based methods can achieve similar results to the iterative methods at a fraction of the time, however, they require large amounts of computational resources and training data. We provide a summary of these methods in a table (Table B.1) and a map (Fig. 1), and unify the naming convention of all methods, allowing readers to easily compare and contrast the different approaches. • Review of current State-of-the-Art in on-axis digital holographic microscopy. • Compares methods, experimental setups, pros, and cons of each method. • Unifies all methods under single notation. • Provides an easy-to-read map of methods for easy comparison. [ABSTRACT FROM AUTHOR]

• Rethinking the PASCAL-VOC and MS-COCO dataset for small object detection. • There are different labeling errors in these two benchmarks by visual analysis. • Building several specific datasets to analyze small object detection. • Data quality is another factor that affects the performance of small object detection. The data and the algorithm are critical to deep learning-based small object detectors. In this paper, we rethink the PASCAL-VOC and MS-COCO dataset for small object detection. By visual analysis of the original annotations, we find that there are different labeling errors in these two datasets. To solve these problems, we build specific datasets, including SDOD, Mini6K, Mini2022 and Mini6KClean. The experimental results of several typical algorithms (e.g. SSD, YOLOv5, Faster RCNN and Deformable DETR) on the datasets show that data labeling errors (such as missing labels, category label errors, inappropriate labels) are another factor that affects the detection performance of small objects. [ABSTRACT FROM AUTHOR]

Partial discharge (PD) recognition is an important tool for online monitoring of gas insulated switchgear (GIS) and diagnosing existing defects. At present, there are two different types of data patterns used for analysis and evaluation of PD signals: phase resolved partial discharge (PRPD) mode and time resolved partial discharge (TRPD) mode. Using different types of data patterns separately can lead to inconsistent or even conflicted recognition results, but the two types of data patterns having complementary information between each other can be used together for data fusion. Dempster-Shafer (DS) evidence theory is introduced to address the problem of evidence conflict and low fusion efficiency. First, two sub-networks for PD recognition are established and compared employing the back propagation neural network (BPNN) learning algorithm under the two modes respectively. Secondly, to minimize (with possible elimination of) the possibility of misclassification, a new fusion decision-making system for PD recognition is proposed on the basis of fusing the results from the sub-networks. Finally, extensive field experiments on two artificial defects are conducted in order to evaluate the performance of the proposed method in this paper. The classification results reveal that the proposed method significantly outperforms the ways of using only single type of data patterns. The method proposed in this paper reduces the uncertainty and effectively improves the credibility of diagnosis. [ABSTRACT FROM AUTHOR]

Twin support vector machines (TWSVM) is a new machine learning method based on the theory of Support Vector Machine (SVM). Unlike SVM, TWSVM would generate two non-parallel planes, such that each plane is closer to one of the two classes and is as far as possible from the other. In TWSVM, a pair of smaller sized quadratic programming problems (QPPs) is solved, instead of solving a single large one in SVM, making the computational speed of TWSVM approximately 4 times faster than the standard SVM. At present, TWSVM has become one of the popular methods because of its excellent learning performance. In this paper, the research progress of TWSVM is reviewed. Firstly, it analyzes the basic theory of TWSVM, then tracking describes the research progress of TWSVM including the learning model and specific applications in recent years, finally points out the research and development prospects. This helps researchers to effectively use TWSVM as an emerging research approach, encouraging them to work further on performance improvement. [ABSTRACT FROM AUTHOR]

Deep learning techniques, e.g., Convolutional Neural Networks (CNNs), have been explosively applied to the research in the fields of information retrieval and natural language processing. However, few research efforts have addressed semantic indexing with deep learning. The use of semantic indexing in the biomedical literature has been limited for several reasons. For instance, MEDLINE citations contain a large number of semantic labels from automatically annotated MeSH terms, and for a great deal of the literature, only the information of the title and the abstract is readily available. In this paper, we propose a Boltzmann Convolutional neural network framework (B-CNN) for biomedicine semantic indexing. In our hybrid learning framework, the CNN can adaptively deal with features of documents that have sequence relationships, and can capture context information accordingly; the Deep Boltzmann Machine (DBM) merges global (the entity in each document) and local information through its training with undirected connections. Additionally, we have designed a hierarchical coarse to fine style indexing structure for learning and classifying documents, and a novel feature extension approach with word sequence embedding and Wikipedia categorization. Comparative experiments were conducted for semantic indexing of biomedical abstract documents; these experiments verified the encouraged performance of our B-CNN model. [ABSTRACT FROM AUTHOR]

This paper presents a new method to reduce the computational cost when using Neural Networks as Language Models, during recognition, in some particular scenarios. It is based on a Neural Network that considers input contexts of different length in order to ease the use of a fallback mechanism together with the precomputation of softmax normalization constants for these inputs. The proposed approach is empirically validated, showing their capability to emulate lower order N-grams with a single Neural Network. A machine translation task shows that the proposed model constitutes a good solution to the normalization cost of the output softmax layer of Neural Networks, for some practical cases, without a significant impact in performance while improving the system speed. [ABSTRACT FROM AUTHOR]

Background and objective Relative location prediction in computed tomography (CT) scan images is a challenging problem. Many traditional machine learning methods have been applied in attempts to alleviate this problem. However, the accuracy and speed of these methods cannot meet the requirement of medical scenario. In this paper, we propose a regression model based on one-dimensional convolutional neural networks (CNN) to determine the relative location of a CT scan image both quickly and precisely. Methods In contrast to other common CNN models that use a two-dimensional image as an input, the input of this CNN model is a feature vector extracted by a shape context algorithm with spatial correlation. Normalization via z-score is first applied as a pre-processing step. Then, in order to prevent overfitting and improve model's performance, 20% of the elements of the feature vectors are randomly set to zero. This CNN model consists primarily of three one-dimensional convolutional layers, three dropout layers and two fully-connected layers with appropriate loss functions. Results A public dataset is employed to validate the performance of the proposed model using a 5-fold cross validation. Experimental results demonstrate an excellent performance of the proposed model when compared with contemporary techniques, achieving a median absolute error of 1.04 cm and mean absolute error of 1.69 cm. The time taken for each relative location prediction is approximately 2 ms. Conclusion Results indicate that the proposed CNN method can contribute to a quick and accurate relative location prediction in CT scan images, which can improve efficiency of the medical picture archiving and communication system in the future. [ABSTRACT FROM AUTHOR]

In this paper, a new machine learning algorithm for multi-agent systems is introduced. The algorithm is based on associative arrays, thus it becomes less complex and more efficient substitute of artificial neural networks and Bayesian networks, which is confirmed by performance measurements. Implementation of machine learning algorithm in multi-agent system for aided design of selected control systems allowed to improve the performance by reducing time of processing requests, that were previously acknowledged and stored in learning module. This article contains an insight into different machine learning algorithms and includes the classification of learning techniques regarding the criteria depicted by multi-agent systems. The publication is also an attempt to provide the answer for a question posted by Shoham, Powers and Grenager: “If multi-agent learning is the answer, what is the question?” [ABSTRACT FROM AUTHOR]

This paper addresses a chaos kernel function for the relevance vector machine (RVM) in EEG signal classification, which is an important component of Brain-Computer Interface (BCI). The novel kernel function has evolved from a chaotic system, which is inspired by the fact that human brain signals depict some chaotic characteristics and behaviors. By introducing the chaotic dynamics to the kernel function, the RVM will be enabled for higher classification capacity. The proposed method is validated within the framework of one versus one common spatial pattern (OVO-CSP) classifier to classify motor imagination (MI) of four movements in a public accessible dataset. To illustrate the performance of the proposed kernel function, Gaussian and Polynomial kernel functions are considered for comparison. Experimental results show that the proposed kernel function achieved higher accuracy than Gaussian and Polynomial kernel functions, which shows that the chaotic behavior consideration is helpful in the EEG signal classification. [ABSTRACT FROM AUTHOR]

With the rapid development of machine learning, artificial intelligence (AI) has drawn much more attention. Under this circumstances, abstract strategy games, such as chess, checkers and Go, have been a fascinating problem of AI research. Most of the existing state-of-the-art Go programs used deep neural network technology, like convolutional neural networks (CNNs). However, CNNs require multiple iterations to optimize weights and spend a lot of training time. Therefore, in this paper, in order to solve the above shortcomings, we propose a new learning algorithm ECNN, which integrates CNNs with extreme learning machine (ELM). We remove pooling layers of CNNs and insert ELM layers between convolutional layers. The newly added ELM layers will be updated in back-propagation process, and they accelerate the convergence of weights in CNNs. Therefore, our ECNN can reduce the training time of CNNs. Further, we propose ECNN-Go algorithm, which applies ECNN to Go game. Because of the advantage of ECNN, ECNN-Go algorithm has the fast learning speed to make move prediction in Go game. Finally, the experimental results show the efficiency and accuracy of ECNN algorithm and demonstrate the strength of ECNN-Go. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is a non-iterative algorithm for training single-hidden layer feed-forward networks, whose training speed is much faster than those of conventional neural networks. However, its objective is only to minimize the empirical risk, which may cause overfitting easily. To overcome this defect, this paper proposes a novel algorithm named LGE2LM based on the localized generalization error model which provides an upper bound of generalization error through adopting the stochastic sensitivity. LGE2LM aims to improve the generalization capability of ELM by using the regularization technique which makes a trade-off between the empirical risk and the stochastic sensitive measure. The essence of LGE2LM is a quadratic problem without iterative process. Similar to ELM, all the parameters of this new algorithm are obtained without tuning, which makes the efficiency of LGE2LM much higher than those of traditional neural networks. Several experiments conducted on both artificial and real-world datasets show that LGE2LM has much better generalization capability and stronger robustness in comparison with ELM. [ABSTRACT FROM AUTHOR]