Search

To provide a new numerical algorithm for solving elliptic partial differential equations (PDEs), the Legendre neural network (LNN) and improved extreme learning machine (IELM) algorithm are introduced to propose a Legendre improved extreme learning machine (L-IELM) method, which is applied to solving elliptic PDEs in this paper. The product of two Legendre polynomials is chosen as basis functions of hidden neurons. Single hidden layer LNN is used to construct approximate solutions and its derivatives of differential equations. IELM algorithm is used for network weights training, and the algorithm steps of the proposed L-IELM method are summarized. Finally, in order to evaluate the present algorithm, various test examples are selected and solved by the proposed approach to validate the calculation accuracy. Comparative study with the earlier methods in literature is described to verify the superiority of the presented L-IELM method. Experiment results show that the proposed L-IELM algorithm can perform well in terms of accuracy and execution time, which in addition provides a new algorithm for solving elliptic PDEs. [ABSTRACT FROM AUTHOR]

In the context of computer experiments, metamodels are largely used to represent the output of computer codes. Among these models, Gaussian process regression (kriging) is very efficient see e.g Snelson (Flexible and efficient Gaussian process models for machine learning. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–University of London, University College London, London, 2008). In high dimension that is with a large number of input variables, but with few observations, the estimation of the parameters with a classical anisotropic kriging can be completely inaccurate. Because there are equal numbers of ranges and input variables the optimization space becomes too large compared to available information. One way to overcome this drawback is to use an isotropic kernel that only depends on one parameter. However this model is too restrictive. The aim of this paper is twofold. Our first objective is to propose a smooth kernel with as few parameters as warranted. We introduce a kernel which is a tensor product of few isotropic kernels built on well-chosen subgroup of variables. The main difficulty is to find the number and the composition of the groups. Our second objective is to propose algorithmic strategies to overcome this difficulty. Four forward strategies are proposed. They all start with the simplest isotropic kernel and stop when the best model according to BIC criterion is found. They all show very good accuracy results on simulation test cases. But one of them is more efficient. Tested on a real data set, our kernel shows very good prediction results. [ABSTRACT FROM AUTHOR]

The traditional way of lexical interpretation is to use some words to explain other words. These semantic information cannot be used to solve the problem of natural language processing oriented to the physical situation. Based on this, this paper proposes a new method based on the feature vector Word semantic representation model and its corresponding learning algorithm Sal-F. This algorithm applies the idea of cross-scenario learning to the alignment of "word-shape features" and uses the result of feature selection under MSAV to construct a visual semantic model Lsm-G, A semantic dictionary based on graph features is constructed. Using machine-oriented evaluation method, the average selection accuracy of this algorithm is about 70%, and the accuracy of sentence selection is up to 16%. [ABSTRACT FROM AUTHOR]

Very large-scale social networks are typically sparse and dynamic and often have millions of nodes and billions of links. Link prediction in very large-scale networks is a challenging task for most existing methods. This paper investigates the link prediction problem in very large-scale networks. We propose a model, namely, a very large-scale network co-occurrence embedding algorithm (Hsem), which learns the co-occurrence features of node pairs to embed nodes into a vector with a lower and fixed dimension. A damping-based random walk algorithm is also developed to extract the hierarchical structural information of node pairs. Moreover, we design an incremental learning algorithm based on Hsem to meet the requirements of highly dynamic evolution in very large-scale networks, which significantly improves learning speed while maintaining accuracy. Finally, we present controlled experiments employing the proposed and baseline methods on eight challenging real-world large-scale datasets with a fixed dropout percentage and calculate the time overhead and receiver operating characteristics. The results show that Hsem performs comparably to current state-of-the-art methods and consistently outperforms the baselines in different experimental settings. [ABSTRACT FROM AUTHOR]

This paper proposes a hybridized machine-learning framework called Extreme Learning Machine using self-adaptive multi-population-based Jaya algorithm for forecasting the currency exchange value. This learning technique attempts to take the advantages of generalization ability of Extreme Learning Machines (ELMs) along with the multi-population search scheme of Jaya optimization technique. This model can very well forecast the exchange price of USD–INR and USD–EURO based on statistical measures, technical indicators and combination of both measures over a time frame varying from 1 day to 1 month ahead. Proposed model has been compared with original ELM and ELM-Jaya along with technical analysis method such as discrete wavelet neural network optimized with self-adaptive multi-population-based Jaya and the comparison of different performance measures like MAPE, Theil's U, ARV and MAE reveal that ELM using self-adaptive multi-population-based Jaya hybrid models possesses superior compared to the rest predictive models. Comparison of different features demonstrates technical indicators outperform other two features such as statistical measures and combination of both technical indicators and statistical measures. [ABSTRACT FROM AUTHOR]

In situ monitoring and accurate detecting of welding quality have been one of the common challenges of automatic welding process. This paper contributes an intelligent decision-making framework for the weld penetration prediction from the keyhole dynamic behavior under time-varying VPPAW pools. Initially, a series of dynamic experiments under different welding conditions were conducted to acquire the backside images of keyhole and corresponding backside bead width. Then, the geometry appearance of keyhole was described by the supervised descent method (SDM)–based image processing algorithm. Subsequently, the internal correlation between the keyhole characteristics and the backside width was further derived to help understand the nonlinear and time-varying VPPAW process. Finally, a novel dynamic model based on an online sequential extreme learning machine (OS-ELM) was designed to predict the weld penetration as measured by the backside bead width in real time. Extensive experiment results further verify and validate that the proposed dynamic OS-ELM model is significantly better than other state-of-the-art algorithms in terms of predicting accuracy, efficiency, and robustness. [ABSTRACT FROM AUTHOR]

Movie box-office research is an important work for the rapid development of the film industry, and it is also a challenging task. Our study focuses on finding the regular box-office revenue patterns. Clustering algorithm is unsupervised machine learning algorithm which classifies the data in the absence of early knowledge of the classes. Unlike static data, the time series data vary with time. The work focused on time series clustering analysis is relatively less than those focused on static data. In this paper, the sparse subspace clustering (SSC) algorithm is introduced to analyze the time series data. The SSC algorithm has a better performance both on the artificial data set and the daily box-office data than recently developed well-known clustering algorithm such as K-means and spectral clustering algorithm. On the artificial data set, SSC is more suitable for time series, whether from the angle of clustering error or visualization. On the actual data, movies are divided into five clusters by SSC algorithm, and each cluster represents a distinct type of distribution pattern. And these patterns can be used in movie recommendation, film evaluation and can guide theater exhibitors and distributors. In addition, this is the first time to apply SSC to deal with time series clustering problem and get a pleasant effect. [ABSTRACT FROM AUTHOR]

The aim of this study is to propose a high-accuracy and high-efficiency sleep staging algorithm using single-channel electroencephalograms (EEGs). The process consists four parts: signal preprocessing, feature extraction, feature selection, and classification algorithms. In the preconditioning of EEG, wavelet function and IIR filter are used for noise reduction. In feature selection, 15 feature algorithms in time domain, time-frequency domain, and nonlinearity are selected to obtain 30 feature parameters. Feature selection is very important for eliminating irrelevant and redundant features. Feature selection algorithms as Fisher score, Sequential Forward Selection (SFS), Sequential Floating Forward Selection (SFFS), and Fast Correlation-Based Filter Solution (FCBF) were used. The paper establishes a new ensemble learning algorithm based on stacking model. The basic layers are k-Nearest Neighbor (KNN), Random Forest (RF), Extremely Randomized Trees (ERT), Multi-layer Perceptron (MLP), and Extreme Gradient Boosting (XGBoost) and the second layer is a Logistic regression. Comparing classification of RF, Gradient Boosting Decision Tree (GBDT), and XGBoost, the accuracies and kappa coefficients are 96.67% and 0.96 using the proposed method. It is higher than other classification algorithms.The results show that the proposed method can accurately sleep staging using single-channel EEG and has a high ability to predict sleep staging. Graphical abstract. [ABSTRACT FROM AUTHOR]

Linear equations with infinite or no solutions play key roles in machine learning and optimization. However, the existed quantum algorithms cannot be applied directly for these classes of equations. In this paper, based on the modification of algorithm (Wossnig et al., Phys. Rev. Lett. 120, 050502 2017), we describe a quantum algorithm to compute the minimal norm solution or minimum norm least-squares solution for equations with infinite or no solutions, respectively. It can be shown that the presented algorithm can achieve an exponential speedup over the best classical algorithm. Furthermore, the corresponding quantum circuit is designed on a quantum computer. [ABSTRACT FROM AUTHOR]

Aiming at the shortage of classic ID3 decision tree and C4.5 decision tree algorithm in ability of time series data mining, this paper increases Bayesian classification algorithm in the node of the decision tree algorithm as a preprocessing Bayesian node to form incremental learning decision tree. In incremental learning, the decision tree is more powerful in the mining of time series data, and the result is more robust and accurate. In order to verify the effectiveness of the algorithm, time series data mining simulation based on UCR time series data sets have been conducted, and the performance and efficiency of ID3 decision tree, Bayesian algorithm and incremental decision tree algorithm have been contrasted respectively in non-incremental learning and incremental learning. The experimental results show that in both the incremental and non-incremental time series data mining, the incremental decision tree algorithm based on Bayesian nodes optimization which can improve the classification accuracy. When optimizing the parameters of Bayesian nodes, the consumption of pruning time will be decreased, and the efficiency of the data mining is greatly improved. [ABSTRACT FROM AUTHOR]

Latest studies done on huge data collected from aging features proved that the performance of facial image based age estimation is low and need to be improved. One of the significant biometric traits for human recognition or search is Human age. Age assessment is very much exigent over other pattern recognition problems since the aging differs from person to person. This paper proposes a new framework that uses periocular region for age feature extraction and application of hybrid algorithm for age recognition. Firstly, preprocessing and periocular region normalization is done to acquire age invariant features. Secondly, the periocular region that underwent preprocessing is analyzed using hybrid approach, a novel machine algorithm that combines both SVM and kNN. The proposed technique generates the best recognition outputs. [ABSTRACT FROM AUTHOR]

Consider the classical problem of solving a general linear system of equations A x = b . It is well known that the (successively over relaxed) Gauss–Seidel scheme and many of its variants may not converge when A is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G–S type algorithm that works for anyA? In this paper we answer this question affirmatively by proposing a doubly stochastic G–S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a nonuniform double stochastic scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem A x ≤ b with an arbitrary A, as well as high-dimensional minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G–S algorithm converge. [ABSTRACT FROM AUTHOR]

In this paper, we propose a perturbed proximal primal–dual algorithm (PProx-PDA) for an important class of linearly constrained optimization problems, whose objective is the sum of smooth (possibly nonconvex) and convex (possibly nonsmooth) functions. This family of problems can be used to model many statistical and engineering applications, such as high-dimensional subspace estimation and the distributed machine learning. The proposed method is of the Uzawa type, in which a primal gradient descent step is performed followed by an (approximate) dual gradient ascent step. One distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results (to first-order stationary solutions) can be obtained. Finally, we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper considers nonconvex distributed constrained optimization over networks, modeled as directed (possibly time-varying) graphs. We introduce the first algorithmic framework for the minimization of the sum of a smooth nonconvex (nonseparable) function—the agent's sum-utility—plus a difference-of-convex function (with nonsmooth convex part). This general formulation arises in many applications, from statistical machine learning to engineering. The proposed distributed method combines successive convex approximation techniques with a judiciously designed perturbed push-sum consensus mechanism that aims to track locally the gradient of the (smooth part of the) sum-utility. Sublinear convergence rate is proved when a fixed step-size (possibly different among the agents) is employed whereas asymptotic convergence to stationary solutions is proved using a diminishing step-size. Numerical results show that our algorithms compare favorably with current schemes on both convex and nonconvex problems. [ABSTRACT FROM AUTHOR]

Since the emergence of the innovative field of artificial intelligence (AI) in the 1960s, the late Hubert Dreyfus insisted on the ontological distinction between man and machine, human and artificial intelligence. In the different editions of his classic and influential book What computers can't do (1972), he posits that an algorithmic machine can never fully simulate the complex functioning of the human mind—not now, nor in the future. Dreyfus' categorical distinctions between man and machine are still relevant today, but their relation has become more complex in our increasingly data-driven society. We, humans, are continuously immersed within a technological universe, while at the same time ubiquitous computing, in the words of computer scientist Mark Weiser, "forces computers to live out here in the world with people" (De Souza e Silva in Interfaces of hybrid spaces. In: Kavoori AP, Arceneaux N (eds) The cell phone reader. Peter Lang Publishing, New York, 2006, p 20). Dreyfus' ideas are therefore challenged by thinkers such as Weiser, Kevin Kelly, Bruno Latour, Philip Agre, and Peter Paul Verbeek, who all argue that humans are much more intrinsically linked to machines than the original dichotomy suggests—they have evolved in concert. Through a discussion of the classical concepts of individuum and 'authenticity' within Western civilization, this paper argues that within the ever-expanding data-sphere of the twenty-first century, a new concept of man as 'aggregate of data' has emerged, which further erodes and undermines the categorical distinction between man and machine. This raises political and ethical questions beyond the confines of technology and artificial intelligence. Moreover, this seemingly never-ending debate on what computers should (or should not) do provokes the philosophical necessity to once again define the concept of what it is to be 'human.' [ABSTRACT FROM AUTHOR]

The aim of this paper is to present some philosophical considerations about the supposed AI emergence in the future. However, the predicted timeline of this process is uncertain. To avoid any kind of speculations on the proposed analysis from a scientific point of view, a metaphysical approach is undertaken as a modal context of the discussion. I argue that modal claim about possible AI emergence at a certain point of time in the future is justified from a temporal perspective. Therefore, worldwide society must be prepared for possible AI emergence and the expected profound impact of such an event on the existential status of humanity. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks, (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, and three publicly available face databases demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. We also evaluate the proposed approaches on four existing quasi-imperceptible distortions: DeepFool, Universal adversarial perturbations, l 2 , and Elastic-Net (EAD). The proposed method is able to detect both types of attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition. [ABSTRACT FROM AUTHOR]

Evolutionary algorithms are widely used for solving multiobjective optimization problems but are often criticized because of a large number of function evaluations needed. Approximations, especially function approximations, also referred to as surrogates or metamodels are commonly used in the literature to reduce the computation time. This paper presents a survey of 45 different recent algorithms proposed in the literature between 2008 and 2016 to handle computationally expensive multiobjective optimization problems. Several algorithms are discussed based on what kind of an approximation such as problem, function or fitness approximation they use. Most emphasis is given to function approximation-based algorithms. We also compare these algorithms based on different criteria such as metamodeling technique and evolutionary algorithm used, type and dimensions of the problem solved, handling constraints, training time and the type of evolution control. Furthermore, we identify and discuss some promising elements and major issues among algorithms in the literature related to using an approximation and numerical settings used. In addition, we discuss selecting an algorithm to solve a given computationally expensive multiobjective optimization problem based on the dimensions in both objective and decision spaces and the computation budget available. [ABSTRACT FROM AUTHOR]

A major factor blocking the practical application of brain-computer interfaces (BCI) is the long calibration time. To obtain enough training trials, participants must spend a long time in the calibration stage. In this paper, we propose a new framework to reduce the calibration time through knowledge transferred from the electroencephalogram (EEG) of other subjects. We trained the motor recognition model for the target subject using both the target's EEG signal and the EEG signals of other subjects. To reduce the individual variation of different datasets, we proposed two data mapping methods. These two methods separately diminished the variation caused by dissimilarities in the brain activation region and the strength of the brain activation in different subjects. After these data mapping stages, we adopted an ensemble method to aggregate the EEG signals from all subjects into a final model. We compared our method with other methods that reduce the calibration time. The results showed that our method achieves a satisfactory recognition accuracy using very few training trials (32 samples). Compared with existing methods using few training trials, our method achieved much greater accuracy. Graphical abstract The framework of the proposed method. The workflow of the framework have three steps: 1, process each subjects EEG signals according to the target subject's EEG signal. 2, generate models from each subjects' processed signals. 3, ensemble these models to a final model, the final model is a model for the target subject. [ABSTRACT FROM AUTHOR]

This paper presents a method for continuous indoor-outdoor environment detection on mobile devices based solely on Wi-Fi fingerprints. Detection of indoor-outdoor switching is an important part of identifying a user's context, and it provides important information for upper layer context aware mobile applications such as recommender systems, navigation tools, etc. Moreover, future indoor positioning systems are likely to use Wi-Fi fingerprints, and therefore Wi-Fi receivers will be on most of the time. In contrast to existing research, we believe that these fingerprints should be leveraged, and they serve as the basis of the proposed method. Using various machine learning algorithms, we train a supervised classifier based on features extracted from the raw fingerprints, clusters, and cluster transition graph. The contribution of each of the features to the method is assessed. Our method assumes no prior knowledge of the environment, and a training set consisting of the data collected for just a few hours on a single device is sufficient in order to provide indoor-outdoor classification, even in an unknown location or when using new devices. We evaluate our method in an experiment involving 12 participants during their daily routine, with a total of 828 h' worth of data collected by the participants. We report a predictive performance of the AUC (area under the curve) of 0.94 using the gradient boosting machine ensemble learning method. We show that our method can be used for other context detection tasks such as learning and recognizing a given building or room. [ABSTRACT FROM AUTHOR]

Advantages of machine learning algorithms are compared in terms of "generalization," which means how accurate they perform upon unseen data. There are many applications in which test data are available (obviously without target). A central question with regard to "generalization" is as follows: Would it be possible to deploy test feature vectors in the training phase? For test data, nonetheless, there is no target—this is the paradox. Transductive learning is a form of learning that uses test data in addition to train data in the training phase. Using decision tree as the base model reveals many errors occur on those data, the feature values of which, are very close to splitting values of decision tree's nodes. This observation is the essence of answering the aforementioned paradox in this study. This paper proposes a combination of decision tree and a novel Fisher discriminant analysis, in which the proposed modified FDA has an embedded mechanism using test data as a shortcut to generalization. This combination further arranged in a boosting system with weights on both train and test data. Experiments are evaluated on 21 datasets from various fields. The results decisively demonstrate the promising performance of the proposed method. [ABSTRACT FROM AUTHOR]

Feature extraction and classification is a vital part in motor imagery-based brain-computer interface (BCI) system. Traditional deep learning (DL) methods usually perform better with more labeled training samples. Unfortunately, the labeled samples are usually scarce for electroencephalography (EEG) data, while unlabeled samples are available in large quantity and easy to collect. In addition, traditional DL algorithms are notoriously time-consuming for the training process. To address these issues, a novel method of hierarchical semi-supervised extreme learning machine (HSS-ELM) is proposed in this paper and applied for motor imagery (MI) task classification. Firstly, the deep architecture of hierarchical ELM (H-ELM) approach is employed for feature learning automatically, and then these new high-level features are classified using the semi-supervised ELM (SS-ELM) algorithm which can exploit the information from both labeled and unlabeled data. Extensive experiments were conducted on some benchmark datasets and EEG datasets to evaluate the effectiveness of the proposed method. Compared with several state-of-the-art methods, including SVM, ELM, SAE, H-ELM, and SS-ELM, our HSS-ELM method can achieve better classification accuracy, a mean kappa value of 0.7945 and 0.5701 across all subjects in the training and evaluation sessions of BCI Competition IV Dataset 2a, respectively. Finally, it comes to the conclusion that the proposed method has achieved superior performance for feature extraction and classification of EEG signals. Graphical abstract The schematic of the proposed HSS-ELM algorithm. [ABSTRACT FROM AUTHOR]

This paper investigates the channel selection problem for anti-jamming defense in an adversarial environment. In our work, we simultaneously consider malicious jamming and co-channel interference among users, and formulate this anti-jamming defense problem as a Stackelberg game with one leader and multiple followers. Specifically, the users and jammer independently and selfishly select their respective optimal strategies and obtain the optimal channels based on their own utilities. To derive the Stackelberg Equilibrium, a hierarchical learning framework is formulated, and a hierarchical learning algorithm (HLA) is proposed. In addition, the convergence performance of the proposed HLA algorithm is analyzed. Finally, we present simulation results to validate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Orthogonal frequency-division multiplexing (OFDM) is one of the most successful digital communication techniques. Nevertheless, the decrease in inter-symbol interference in quadrature amplitude modulation (QAM) over dispersive channels is still challenging. Different researches recently proposed the idea of using unsupervised learning as an alternative to the classic approaches to equalization of OFDM channels. In those purposes, the identification of a received QAM symbol is possible by the comparison of its position on the in-phase/quadrature (IQ) plane relative to the positions of previously arrived symbols, generally processed by the Kohonen’s Self-Organizing Map (SOM) algorithm. This work presents the SOM unsupervised learning method executed on an embedded system applied to QAM symbols identification. The system is implemented on an FPGA, a configurable digital circuit able to meet the low power and parallel process requirements of mobile applications. Also, in order to extend the classical set of experiments to evaluate our system, this paper proposes a theoretical model of the time-varying scheme representing the transition between different channel characteristics, obtained from real measurements available on a public repository. The model is employed to verify our purpose under dynamically both changing and realistic conditions. On the assumption that it is provided enough IQ symbols for the initial training process, the hardware implementation of SOM is able to track and identify the time-varying distorted QAM constellation. No knowledge of channel characteristics is necessary. The system spends only some microseconds at start-up to reach about 100% performance, and no dedicated training phase is needed afterward. [ABSTRACT FROM AUTHOR]

Liao et al. (Neurocomputing 128:81-87, 2014) proposed a meta-learning approach to extreme learning machine (Meta-ELM), which can obtain good generalization performance by training multiple ELMs. However, one of its open problems is overfitting when minimizing training error. In this paper, we propose an improved meta-learning model of ELM (improved Meta-ELM) to handle the problem. The improved Meta-ELM architecture is composed of some base ELMs which are error feedback incremental extreme learning machine (EFI-ELM) and the top ELM. The improved Meta-ELM includes two stages. First, each base ELM with EFI-ELM is trained on a subset of training data. Then, the top ELM learns with the base ELMs as hidden nodes. Simulation results on some artificial and benchmark datasets show that the proposed improved Meta-ELM model is more feasible and effective than Meta-ELM. [ABSTRACT FROM AUTHOR]

Object tracking still remains challenging in computer vision because of the severe object variation, e.g., deformation, occlusion, and rotation. To handle the object variation and achieve robust object tracking performance, we propose a novel relationship-based tracking algorithm using neural networks in this paper. Compared with existing approaches in the literature, our method assumes the targeted object to be consisted of several parts and considers the evolution of the topology structure among these parts. After training a candidate neural network for predicting the probable areas each part may locate at in the successive frame, we then design a novel collaboration neural network to determine the precise area each part will locate at with account for the topology structure among these individual parts, which is learned from their historical physical locations during online tracking process. Experimental results show that the proposed method outperforms state-of-the-art trackers on a benchmark dataset, yielding the significant improvements in accuracy on high-distorted sequences. [ABSTRACT FROM AUTHOR]

Machine learning techniques are widely used for network intrusion detection (NID). However, it has to face the unbalance of training samples between classes as it is hard to collect samples of some intrusion classes. This would produce false positives for these intrusion classes. Meanwhile, since there are various types of intrusions, classification boundaries between different classes are seriously nonlinear. Due to the huge amount of training data, computational efficiency is also required. This paper therefore proposes an efficient cascaded classifier for NID. This classifier consists of a collection of binary base classifiers which are serially connected. Each base classifier corresponds to a type of intrusion. The order of these base classifiers is automatically determined based on the number of false positives to cope with the unbalance of training samples. Extreme learning machine algorithm, which has low computational cost, is used to train these base classifiers to delineate the nonlinear boundaries between classes. This proposed NID method is evaluated on the KDD99 data set. Experimental results have shown that this proposed method outperforms other state-of-the-art methods including decision tree, back-propagation neural network and support vector machines. [ABSTRACT FROM AUTHOR]

In Homo Deus, Yuval Noah Harari argues that technological advances of the twenty-first century will usher in a significant shift in how humans make important life decisions. Instead of turning to the Bible or the Quran, to the heart or to our therapists, parents, and mentors, people will turn to Big Data recommendation algorithms to make these choices for them. Much as we rely on Spotify to recommend music to us, we will soon rely on algorithms to decide our careers, spouses, and commitments. Harari also predicts that next, the state will take away individuals’ rights to make their own choices about their lives. If Google knows where your children would flourish best in school, why should the state allow a fallible human parent to decide? Liberalism—which, as Harari uses this term, refers to a state of society in which human freedom to choose is respected and championed—will collapse. In this paper, I argue that Harari’s conception of the future implications of recommendation algorithms is deeply flawed, for two reasons. First, users will not rely on algorithms to make decisions for them because they have no reason to trust algorithms, which are developed by companies with their own incentives, such as profit. Second, for most of our life decisions, algorithms will not be able to be developed, because the factors relevant to the decisions we face are unique to our situation. I present an alternative depiction of the future: instead of relying on algorithms to make decisions for us, humans will use algorithms to enhance our decision-making by helping us consider the most relevant choices first and notice information we might not otherwise. Finally, I will also argue that even if computers could make many of our decisions for us, liberalism as a political system would emerge unscathed. [ABSTRACT FROM AUTHOR]

Generalizing both mixed-integer linear optimization and convex optimization, mixed-integer convex optimization possesses broad modeling power but has seen relatively few advances in general-purpose solvers in recent years. In this paper, we intend to provide a broadly accessible introduction to our recent work in developing algorithms and software for this problem class. Our approach is based on constructing polyhedral outer approximations of the convex constraints, resulting in a global solution by solving a finite number of mixed-integer linear and continuous convex subproblems. The key advance we present is to strengthen the polyhedral approximations by constructing them in a higher-dimensional space. In order to automate this extended formulation we rely on the algebraic modeling technique of disciplined convex programming (DCP), and for generality and ease of implementation we use conic representations of the convex constraints. Although our framework requires a manual translation of existing models into DCP form, after performing this transformation on the MINLPLIB2 benchmark library we were able to solve a number of unsolved instances and on many other instances achieve superior performance compared with state-of-the-art solvers like Bonmin, SCIP, and Artelys Knitro. [ABSTRACT FROM AUTHOR]

The global k-means algorithm is an incremental approach to clustering that dynamically adds one cluster center at a time through a deterministic global search procedure from suitable initial positions, and employs k-means to minimize the sum of the intra-cluster variances. However the global k-means algorithm sometimes results singleton clusters and the initial positions sometimes are bad, after a bad initialization, poor local optimal can be easily obtained by k-means algorithm. In this paper, we modified the global k-means algorithm to eliminate the singleton clusters at first, and then we apply MinMax k-means clustering error method to global k-means algorithm to overcome the effect of bad initialization, proposed the global Minmax k-means algorithm. The proposed clustering method is tested on some popular data sets and compared to the k-means algorithm, the global k-means algorithm and the MinMax k-means algorithm. The experiment results show our proposed algorithm outperforms other algorithms mentioned in the paper. [ABSTRACT FROM AUTHOR]

Imbalanced problems are quite pervasive in many real-world applications. In imbalanced distributions, a class or some classes of data, called minority class(es), is/are under-represented compared to other classes. This skewness in the data underlying distribution causes many difficulties for typical machine learning algorithms. The notion becomes even more complicated when machine learning algorithms are to combat multi-class imbalanced problems. The presented solutions for tackling the issues arising from imbalanced distributions, generally fall into two main categories: data-oriented methods and model-based algorithms. Focusing on the latter, this paper suggests an elegant blend of boosting and over-sampling paradigms, which is called MDOBoost, to bring considerable benefits to the learning ability of multi-class imbalanced data sets. The over-sampling technique introduced and adopted in this paper, Mahalanobis distance-based over-sampling technique (MDO in short), is delicately incorporated into boosting algorithm. In fact, the minority classes are over-sampled via MDO technique in such a way that they almost preserve the original minority class characteristics. MDO, in comparison with the popular method in this field, SMOTE, generates more similar minority class examples to original class samples. Moreover, the broader representation of minority class examples is provided via MDO, and this, in turn, causes the classifier to build larger decision regions. MDOBoost increases the generalization ability of a classifier, since it indicates better results with pruned version of C4.5 classifier; unlike other over-sampling/boosting procedures, which have difficulties with pruned version of C4.5. MDOBoost is applied to real-world multi-class imbalanced benchmarks and its performance is then compared with several data-level and model-based algorithms. The empirical results and theoretical analyses reveal that MDOBoost offers superior advantages compared to popular class decomposition and over-sampling techniques in terms of MAUC, G-mean, and minority class recall. [ABSTRACT FROM AUTHOR]

This paper presents a new method based on fuzzy cognitive map (FCM) and possibilistic fuzzy c-means (PFCM) clustering algorithm for categorizing celiac disease (CD). CD is a complex disorder whose development is affected by genetics (HLA alleles) and gluten ingestion. The celiac patients who are not treated are at a high risk of cancer, malignant lymphoma, and small bowel neoplasia. Therefore, CD diagnosis and grading are of paramount importance. The proposed FCM models human thinking for the purpose of classifying patients suffering from CD. We used the latest grading method where three grades A, B1, and B2 are used. To improve FCM efficiency and classification capability, a nonlinear Hebbian learning algorithm is applied for adjusting the FCM weights. To this end, 89 cases are studied. Three experts extracted seven main determinant characteristics of CD which were considered as FCM concepts. The mutual effects of these concepts on one another and on the final concept were expressed in the form of fuzzy rules and linguistic variables. Using the center of gravity defuzzifier, we obtained the numerical values of these weights and obtained the total weight matrix. Ultimately, combining the FCM model with PFCM algorithm, we obtained the grades A, B1, and B2 accuracies as 88, 90, and 91%, respectively. The main advantage of the proposed FCM is the good transparency and interpretability in the decision-making procedure, which make it a suitable tool for daily usage in the clinical practice. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) is one of the most popular machine learning methods and is widely used in many modern applications. The training process of DNNs is a time-consuming process. Accelerating the training of DNNs has been the focus of many research works. In this paper, we speed up the training of DNNs applied for automatic speech recognition and the target architecture is heterogeneous (CPU + MIC). We apply asynchronous methods for I/O and communication operations and propose an adaptive load balancing method. Besides, we also employ a momentum idea to speed up the convergence of the gradient descent algorithm. Experimental results show that our optimized algorithm is able to acquire a 20-fold speedup on a CPU + MIC platform compared with the original sequential algorithm on a single-core CPU. [ABSTRACT FROM AUTHOR]

In the fields of pattern recognition and machine learning, the use of data preprocessing algorithms has been increasing in recent years to achieve high classification performance. In particular, it has become inevitable to use the data preprocessing method prior to classification algorithms in classifying medical datasets with the nonlinear and imbalanced data distribution. In this study, a new data preprocessing method has been proposed for the classification of Parkinson, hepatitis, Pima Indians, single proton emission computed tomography (SPECT) heart, and thoracic surgery medical datasets with the nonlinear and imbalanced data distribution. These datasets were taken from UCI machine learning repository. The proposed data preprocessing method consists of three steps. In the first step, the cluster centers of each attribute were calculated using k-means, fuzzy c-means, and mean shift clustering algorithms in medical datasets including Parkinson, hepatitis, Pima Indians, SPECT heart, and thoracic surgery medical datasets. In the second step, the absolute differences between the data in each attribute and the cluster centers are calculated, and then, the average of these differences is calculated for each attribute. In the final step, the weighting coefficients are calculated by dividing the mean value of the difference to the cluster centers, and then, weighting is performed by multiplying the obtained weight coefficients by the attribute values in the dataset. Three different attribute weighting methods have been proposed: (1) similarity-based attribute weighting in k-means clustering, (2) similarity-based attribute weighting in fuzzy c-means clustering, and (3) similarity-based attribute weighting in mean shift clustering. In this paper, we aimed to aggregate the data in each class together with the proposed attribute weighting methods and to reduce the variance value within the class. Thus, by reducing the value of variance in each class, we have put together the data in each class and at the same time, we have further increased the discrimination between the classes. To compare with other methods in the literature, the random subsampling has been used to handle the imbalanced dataset classification. After attribute weighting process, four classification algorithms including linear discriminant analysis, k-nearest neighbor classifier, support vector machine, and random forest classifier have been used to classify imbalanced medical datasets. To evaluate the performance of the proposed models, the classification accuracy, precision, recall, area under the ROC curve, κ value, and F-measure have been used. In the training and testing of the classifier models, three different methods including the 50-50% train-test holdout, the 60-40% train-test holdout, and tenfold cross-validation have been used. The experimental results have shown that the proposed attribute weighting methods have obtained higher classification performance than random subsampling method in the handling of classifying of the imbalanced medical datasets. [ABSTRACT FROM AUTHOR]

Clustering algorithms (i.e., Gaussian mixture models, k-means) tackle the problem of grouping a set of elements in such a way that elements from the same group (or cluster) have more similar properties to each other than to those elements in other clusters. This simple concept turns out to be the basis in complex algorithms from many application areas, including sequence analysis and genotyping in bioinformatics, medical imaging, antimicrobial activity, market research, social networking, etc. However, as the data volume continues to increase, the performance of clustering algorithms is heavily influenced by the memory subsystem. In this paper, we propose a novel and efficient implementation of Lloyd’s k-means clustering algorithm to substantially reduce data movement along the memory hierarchy. Our contributions are based on the fact that the vast majority of processors are equipped with powerful Single Instruction Multiple Data (SIMD) instructions that are, in most cases, underused. SIMD improves the CPU computational power and, if used wisely, can be seen as an opportunity to improve on the application data transfers by compressing/decompressing the data, specially for memory-bound applications. Our contributions include a SIMD-friendly data layout organization, in-register implementation of key functions and SIMD-based compression. We demonstrate that using our optimized SIMD-based compression method, it is possible to improve the performance and energy of k-means by a factor of 4.5x and 8.7x, respectively, for a i7 Haswell machine, and 22x and 22.2x for Xeon Phi: KNL, running a single thread. [ABSTRACT FROM AUTHOR]

With the rapid development of machine learning, artificial intelligence (AI) has drawn much more attention. Under this circumstances, abstract strategy games, such as chess, checkers and Go, have been a fascinating problem of AI research. Most of the existing state-of-the-art Go programs used deep neural network technology, like convolutional neural networks (CNNs). However, CNNs require multiple iterations to optimize weights and spend a lot of training time. Therefore, in this paper, in order to solve the above shortcomings, we propose a new learning algorithm ECNN, which integrates CNNs with extreme learning machine (ELM). We remove pooling layers of CNNs and insert ELM layers between convolutional layers. The newly added ELM layers will be updated in back-propagation process, and they accelerate the convergence of weights in CNNs. Therefore, our ECNN can reduce the training time of CNNs. Further, we propose ECNN-Go algorithm, which applies ECNN to Go game. Because of the advantage of ECNN, ECNN-Go algorithm has the fast learning speed to make move prediction in Go game. Finally, the experimental results show the efficiency and accuracy of ECNN algorithm and demonstrate the strength of ECNN-Go. [ABSTRACT FROM AUTHOR]

Focusing on non-iterative approaches in training feed-forward neural networks, this special issue includes 12 papers to share the latest progress, current challenges, and potential applications of this topic. This editorial presents a background of the special issue and a brief introduction to the 12 contributions. [ABSTRACT FROM AUTHOR]

Learning from categorical data is a critical yet challenging task. Current research focuses on either leveraging the complex interaction between and within categorical values to generate a numerical representation, or designing a model that can tackle this types of data directly. However, both of these paradigms overlook the relation between the data characteristics and learning model hypothesis. In this paper, we propose a model-aware categorical data embedding framework that jointly reveals the intrinsic categorical data characteristics and optimizes the fitness of the representation for the follow-up learning model. An ELM-aware and a SVM-aware representation methods have been instantiated under this framework. Extensive experiments of classification with the embedded representation on 17 data sets demonstrate that the proposed framework can significantly improve the categorical data representation performance compared with state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

Reasonable burden distribution matrix is one of important requirements that can realize low consumption, high efficiency, high quality and long campaign life of the blast furnace. This paper proposes a data-driven prediction model of adjusting the burden distribution matrix based on the improved multilayer extreme learning machine (ML-ELM) algorithm. The improved ML-ELM algorithm is based on our previously modified ML-ELM algorithm (named as PLS-ML-ELM) and the ensemble model. It is named as EPLS-ML-ELM. The PLS-ML-ELM algorithm uses the partial least square (PLS) method to improve the algebraic property of the last hidden layer output matrix for the ML-ELM algorithm. However, the PLS-ML-ELM algorithm may have different results in different trails of simulations. The ensemble model can overcome this problem. Moreover, it can improve the generalization performance. Hence, the EPLS-ML-ELM algorithm is consisted of several PLS-ML-ELMs. The real blast furnace data are used to testify the data-driven prediction model. Compared with other prediction models which are based on the SVM algorithm, the ELM algorithm, the ML-ELM algorithm and the PLS-ML-ELM algorithm, the simulation results demonstrate that the data-driven prediction model based on the EPLS-ML-ELM algorithm has better prediction accuracy and generalization performance. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is a non-iterative algorithm for training single-hidden layer feed-forward networks, whose training speed is much faster than those of conventional neural networks. However, its objective is only to minimize the empirical risk, which may cause overfitting easily. To overcome this defect, this paper proposes a novel algorithm named LGE2LM based on the localized generalization error model which provides an upper bound of generalization error through adopting the stochastic sensitivity. LGE2LM aims to improve the generalization capability of ELM by using the regularization technique which makes a trade-off between the empirical risk and the stochastic sensitive measure. The essence of LGE2LM is a quadratic problem without iterative process. Similar to ELM, all the parameters of this new algorithm are obtained without tuning, which makes the efficiency of LGE2LM much higher than those of traditional neural networks. Several experiments conducted on both artificial and real-world datasets show that LGE2LM has much better generalization capability and stronger robustness in comparison with ELM. [ABSTRACT FROM AUTHOR]

Stable augmented reality applications consist of an accurate registration supported by a robust tracking module. In outdoor locations, the changing environmental and light conditions compromise this tracking. Reliable descriptors under unsettled conditions are essential for this process. The most used descriptors have this distinctive capacity, but computers and mobile devices process them in a long time frame. This paper investigates a new lightweight environment dedicated descriptor (EDD) trained with a machine-learning algorithm. The descriptor analyzes the scene characteristics with elements that can be computed fast and that have distinctive information about the selected area. The complete descriptor is used for semantic feature extraction with the aid of a trained random forest classifier. The descriptor is compared with the most popular descriptors—with respect to speed, accuracy, and invariance to illumination changes, scale, affine transformation, and rotation—and the results show that it is faster and in most cases equally reliable. [ABSTRACT FROM AUTHOR]

Density peaks clustering (DPC) algorithm is able to get a satisfactory result with the help of artificial selecting the clustering centers, but such selection can be hard for a large amount of clustering tasks or the data set with a complex decision diagram. The purpose of this paper is to propose an automatic clustering approach without human intervention. Inspired by the visual selection rule of DPC, the judgment index which equals the lower value within density and distance (after normalization) is proposed for selecting the clustering centers. The judgment index approximately follows the generalized extreme value (GEV) distribution, and each clustering center’s judgment index is much higher. Hence, it is reasonable that the points are selected as clustering centers if their judgment indices are larger than the upper quantile of GEV. This proposed method is called density peaks clustering based on generalized extreme value distribution (DPC-GEV). Furthermore, taking the computational complexity into account, an alternative method based on density peak detection using Chebyshev inequality (DPC-CI) is also given. Experiments on both synthetic and real-world data sets show that DPC-GEV and DPC-CI can achieve the same accuracy as DPC on most data sets but consume much less time. [ABSTRACT FROM AUTHOR]

This paper presents a robust point-pattern matching (PPM) algorithm, in which the invariant feature and probabilistic relaxation labeling are combined to improve the assignment accuracy and efficiency. A local feature descriptor, namely, point pair local topology (PPLT), is proposed first. The feature descriptor is defined by histogram which is constructed using the weighting of distance measures and angle measures based on local point pair. We use the matching scores of point pair local topology descriptor's statistic test to define new compatibility coefficients. Then, the robust support functions are constructed based on the obtained compatibility coefficients. Finally, according to the relaxed iterations of matching probability matrix and the mapping constraints required by the bijective correspondence, the correct matching results are obtained. A number of comparison and evaluation experiments on both synthetic point sets and real-world data demonstrate that the proposed algorithm performs better in the presence of outliers and positional jitter. In addition, it achieves the superior performance under similarity and even nonrigid transformation among point sets in the meantime compared with state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Single-hidden-layer feedforward neural network (SLFN) is an effective model for data classification and regression. However, it has a very important defect that it is rather time-consuming to explore the training algorithm of SLFN. In order to shorten the learning time, a special non-iterative learning algorithm was proposed, named as extreme learning machine (ELM). The main idea is that the input weights and bias are chosen randomly and the output weights are calculated by a pseudo-inverse matrix. However, ELM also has a very important drawback that it cannot achieve stable solution for different runs because of randomness. In this paper, we propose a stabilized learning algorithm based on iteration correction. The convergence analysis shows that the proposed algorithm can finish the learning process in fewer steps than the number of neurons. Three theorems and their proofs can prove that the proposed algorithm is stable. Several data sets are selected from UCI databases, and the experimental results demonstrate that the proposed algorithm is effective. [ABSTRACT FROM AUTHOR]

The diffusion of Device-to-Device (D2D) communications opens the door to exploit the contributions of multiple Mobile Devices (MDs) to accomplish collaborative tasks. In this paper a speaker recognition algorithm for MDs based on a multiple-observations approach is presented. We propose various fusion and clustering algorithms aimed at efficiently exploiting data coming from MDs. Numerical results show that in many cases our multiple-observation approach is able to significantly improve the accuracy of the considered speaker recognition algorithm. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is an unsupervised learning method for decomposing high-dimensional nonnegative data matrices and extracting basic and intrinsic features. Since image data are described and stored as nonnegative matrices, the mining and analysis process usually involves the use of various NMF strategies. NMF methods have well-known applications in face recognition, image reconstruction, handwritten digit recognition, image denoising and feature extraction. Recently, several projective NMF (P-NMF) methods based on positively constrained projections have been proposed and were found to perform better than the standard NMF approach in some aspects. However, some drawbacks still affect the existing NMF and P-NMF algorithms; these include dense factors, slow convergence, learning poor local features, and low reconstruction accuracy. The aim of this paper is to design algorithms that address the aforementioned issues. In particular, we propose two embedded P-NMF algorithms: the first method combines the alternating least squares (ALS) algorithm with the P-NMF update rules of the Frobenius norm and the second one embeds ALS with the P-NMF update rule of the Kullback-Leibler divergence. To assess the performances of the proposed methods, we conducted various experiments on four well-known data sets of faces. The experimental results reveal that the proposed algorithms outperform other related methods by providing very sparse factors and extracting better localized features. In addition, the empirical studies show that the new methods provide highly orthogonal factors that possess small entropy values. [ABSTRACT FROM AUTHOR]

Aiming at the problem of the absence of effective coal-rock recognition methods during top coal caving, we propose a new multi-class feature selection approach to detecting full coal falling/30% rock mixture falling/50% rock mixture falling/full rock falling recognition during caving. The method is based on vibration and acoustic sensors fixed under the tail beam of the hydraulic as well as signal processing techniques. Distinctive vibration and acoustic signals generate during caving that depend on the state of coal-rock mixed. Via a contribution threshold $$\lambda_{0}$$ and classification accuracy P, the optimal combination of feature attributes can be automatically and sequentially decided by using the multi-class F-score (MF-Score) feature reduction approach proposed in this paper. The main idea of the MF-Score is to construct the within-class scatter matrix with a membership function based on the aggregation degree within the sample. The support vector machine classifier has been employed to test the proposed algorithm and identified the state of the coal-rock mixed. The contribution threshold is readjusted according to the calculated feature selection criterion J( k) and user requirement, and then a feedback loop is developed to keep the dynamic of the samples. Experimental results show the good generality and effectiveness of this new approach. [ABSTRACT FROM AUTHOR]

Decision tree learning algorithms are known to be unstable, such that small changes in the training data can result in highly different output models. Instability is an important issue in the context of machine learning which is usually overlooked. In this paper, we illustrate and discuss the problem of instability of decision tree induction algorithms and propose a framework to induce more stable decision trees. In the proposed framework, the split test encompasses two advantageous properties: First, it is able to contribute multiple attributes. Second, it has a polylithic structure. The first property alleviates the race between the competing attributes to be installed at an internal node, which is the major cause of instability. The second property has the potential of improving the stability by providing the locality of the effect of the instances on the split test. We illustrate the effectiveness of the proposed framework by providing a complying decision tree learning algorithm and conducting several experiments. We have evaluated the structural stability of the algorithms by employing three measures. The experimental results reveal that the decision trees induced by the proposed framework exhibit great stability and competitive accuracy in comparison with several well-known decision tree learning algorithms. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and theory of TWSVM, including the theory of proximal support vector machine, generalized eigenvalue proximal support vector machine, and TWSVM. Then, we focus on the various improvements made to TWSVM, mainly including least squares twin support vector machine, smooth twin support vector machine, regularized twin support vector machine, projection twin support vector machine, and modified TWSVM on the model selection problem. These newly emerging algorithms greatly expand the applications of TWSVM. In recent years, there is a lot of research on application of TWSVM. Next, we list some research on application of TWSVM in detail. Finally, we try to provide a comprehensive view of these advances in TWSVM and discuss the direction of future research. [ABSTRACT FROM AUTHOR]

K-nearest neighbors (KNN) algorithm is a common algorithm used for classification, and also a sub-routine in various complicated machine learning tasks. In this paper, we presented a quantum algorithm (QKNN) for implementing this algorithm based on the metric of Hamming distance. We put forward a quantum circuit for computing Hamming distance between testing sample and each feature vector in the training set. Taking advantage of this method, we realized a good analog for classical KNN algorithm by setting a distance threshold value t to select k − n e a r e s t neighbors. As a result, QKNN achieves O( n ) performance which is only relevant to the dimension of feature vectors and high classification accuracy, outperforms Llyod's algorithm (Lloyd et al. 2013) and Wiebe's algorithm (Wiebe et al. 2014). [ABSTRACT FROM AUTHOR]