Search

• Pinch force strength is accurately predicted using sEMG. • Different pinch movements are accurately classified using sEMG. • Similarity of EMG features is evaluated. • An EMG feature-to-image method is proposed. The movement of intelligent EMG-driven prosthesis mainly relies on the synergy of different fingers to achieve function of grasping objects. The paper proposes a novel scheme for force prediction and movement classification about pinch between different fingers based on surface electromyography (sEMG) using machine learning. The pinch force and sEMG signals are recorded synchronously by a data acquisition device. Eight features are extracted, which are proven to have better performance in the estimation of sEMG-to-force. We present a novel feature selection method that uses the one-dimensional time series similarity assessment based on Manhattan distance to eliminate the repetitive information between features. Three optimal features carrying less repetitive information are retained. Seven machine learning algorithms are used to predict force strength. The results show that the Long Short Term Memory (LSTM) has the best performance of force prediction, achieving a R 2 of 0.9517 and RMSE of 3.2723. The paper proposes a novel method of converting EMG feature sequences to the normalized gray image in order to classify the finger movement. Five classifiers based on image feature extraction and the Convolutional Neural Network (CNN) are developed respectively. The experimental results indicate that the CNN performs best, achieving an accuracy of 97.66%. In this way, it not only realizes the accurate force prediction, but also realizes the movement classification between different fingers. The proposed methodology has the potential to realize simultaneous force and movement control of prosthetic hand. [ABSTRACT FROM AUTHOR]

The long-tailed distribution data poses many challenges for machine learning because the tail classes are extremely scarce. Long-tailed data augmentation is a powerful technique for enriching the tail class diversity. However, existing methods often treat each class independently, assuming that classes are isolated from each other. These approaches overlook the presence of easily confused tail classes, making it challenging for models to distinguish between them accurately. In this paper, we propose a long-tailed classification method based on data augmentation, which utilizes multi-granularity knowledge to select and combine easily confused tail samples, thereby enhancing the classification performance of these samples. First, we utilize multi-granularity knowledge and semantic relation trees to build a class relation matrix. This matrix records the relationship between classes and helps the model search for easily confused classes from bilateral branch samplers. Second, we crop and combine the easily confused head and tail class samples in a foreground–background manner to generate new samples, thereby augmenting the model training. The extensive head class knowledge is transferred to the scarce tail class samples through the combination of fore-background, and the discriminative and generalized abilities of the model are improved. The experimental results affirm the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

• This paper proposes the Recursive General Regression Neural Network Oracle. • The algorithm was compared to several other classifiers. • It had the highest performance metrics such as accuracy and area under the curve. • The algorithm had also bested the literature in the performance metrics. This research introduces the Recursive General Regression Neural Network Oracle (R-GRNN Oracle) and is demonstrated on several binary classification datasets. The traditional GRNN Oracle classifier (Masters et al., 1998) combines the predictive powers of several machine learning classifiers by weighing the amount of error each classifier has on the final predictions. Each classifier is assigned a weight based on the percentage of errors it contributes to the final predictions as the classifiers evaluate the dataset. The proposed R-GRNN Oracle is an enhancement to the GRNN Oracle in which the proposed algorithm consists of an oracle within an oracle – where the inner oracle acts as a classifier with its own predictions and error contribution. By combining the inner oracle with other classifiers, the R-GRNN Oracle produces superior results. The classifiers considered in this study are: Support Vector Machine (SVM), Multilayer Perceptron (MLP), Probabilistic Neural Network (PNN), Gaussian Naïve Bayes (GNB), K-Nearest Neighbor (KNN), and Random Forest (RF). To demonstrate the effectiveness of the proposed approach, several datasets were used, with the primary one being the publicly available Spambase dataset. The predictions of SVM, MLP, KNN, and RF were used to create the first GRNN Oracle, which was then enhanced with the high performances of SVM and RF to create the second oracle, the R-GRNN Oracle. The combined recursive model was 93.24% accurate using 10-fold cross validation, higher than the 91.94% of the inner GRNN Oracle and the 91.29% achieved by RF, the highest performance by a stand-alone classifier. The R-GRNN Oracle was not only the most accurate, but it also had the highest AUC, sensitivity, specificity, precision, and F1-score (97.99%, 91.86%, 94.40%, 93.28%, and 92.57%, respectively). The research contribution of this paper is introducing the concept of recursion (a concept not fully explored in machine learning models and applications) and testing this structure's ability on further enhancing the performance of the traditional oracle. The recursive model has also been applied to several other datasets: The Human Resources, Bank Marketing, and Monoclonal Gammopathy of Undetermined Significance (MGUS) datasets. The results of these implementations are summarized in this paper. [ABSTRACT FROM AUTHOR]

The high dimensionality of data hinders the learning ability of machine learning algorithms. Feature selection techniques can be used to reduce dimensionality, which is an important step for processing high-dimensional data. Feature selection solves this problem by removing irrelevant and redundant information, which can improve learning models, reduce calculation time, and improve learning accuracy. In this paper, a novel filter in mixed-attribute datasets for feature selection is proposed. The independent attributes are mixed or heterogeneous in the sense that both numerical and categorical attribute types may appear together in the same dataset. Based on the preordonnances theory, we use a new concept to quantify the relevance and redundancy of features even if there are heterogeneous (mixed-type) data. The technique for order preference by similarity to the ideal solution is one of the well-known multicriteria decision-making methods; it is utilized as a weighting and informative feature selection filter. To assess the effectiveness of the proposed method, several experiments, both simulated and real, are performed, including a comparison to other well-known filter methods. The experimental results show that, in most cases, the method yielded competitive results in comparison to other methods. • Introducing new criterions for relevance and complementarity. • Dealing with mixed-attribute datasets without requiring a preprocessing step. • The multi-criteria decision method, namely TOPSIS, is used to score the explanatory features. • Investigate the performance of the proposed filter in high-dimensional data by using simulated and real data sets. [ABSTRACT FROM AUTHOR]

Due to the enormous amount of user-generated content being generated on the web, labeling such data is a time-consuming and expensive endeavor. As a result, we have limited annotated data and the vast majority of data are unlabeled. Analysis reveals that extracting (external) knowledge from unlabeled data and integrating it with knowledge extracted from labeled data is a beneficial task for text information processing, in particular text classification. In this paper, we present a hybrid approach for classifying sentiments that employs external knowledge, which is categorized as either general-purpose sentiment knowledge or domain-related knowledge. General-purpose sentiment knowledge is extracted from sentiment lexicons, whereas domain-related knowledge is extracted from unlabeled data from the same or related domains. Similar domains for a given domain are identified based on their similarity score in terms of overlapping features. The proposed approach utilizes both forms of external knowledge and combines them with logistic regression to train an improved classification model. The classification model uses the conventional gradient descent algorithm for optimization, and its convergence analysis indicates that it is convex and converges to the global optimum. The proposed approach is empirically evaluated and compared to three baselines and one state-of-the-art method using standard performance evaluation metrics on a multi-domain sentiment dataset. The experiment results are encouraging, demonstrating that the proposed approach considerably outperforms the baseline approaches and outperforms the state-of-the-art approach by up to 2% in terms of both f-score and accuracy. • A hybrid approach for document-level sentiment classification that makes use of external knowledge. • A domain-specific knowledge extraction approach from unlabeled text documents. • A detailed convergence analysis of the proposed optimization model used in the proposed approach. [ABSTRACT FROM AUTHOR]

• Propose an improved random forest based on the improvement of decision trees. • Improve the evaluation mechanism for the classification effect of decision trees. • Propose a method for quantifying the diversity between decision trees. • Multiple tests verify the superiority of the proposed improved random forest. Random forest is one of the most widely used machine learning algorithms. Decision trees used to construct the random forest may have low classification accuracies or high correlations, which affects the comprehensive performance of the random forest. Aiming at these problems, the authors proposed an improved random forest based on the classification accuracy and correlation measurement of decision trees in this paper. Its main idea includes two parts, one is retaining the classification and regression trees (CARTs) with better classification effects, the other is reducing the correlations between the CARTs. Specifically, in the classification effect evaluation part, each CART was applied to make predictions on three reserved data sets, then the average classification accuracies were achieved, respectively. Thus, all the CARTs were sorted in descending order according to their achieved average classification accuracies. In the correlation measurement part, the improved dot product method was proposed to calculate the cosine similarity, i.e., the correlation, between CARTs in the feature space. By using the achieved average classification accuracy as reference, the grid search method was used to find the inner product threshold. On this basis, the CARTs with low average classification accuracy among CART pairs whose inner product values are higher than the inner product threshold were marked as deletable. The achieved average classification accuracies and correlations of CARTs were comprehensively considered, those with high correlation and weak classification effect were deleted, and those with better quality were retained to construct the random forest. Multiple experiments show that, the proposed improved random forest achieved higher average classification accuracy than the five random forests used for comparison, and the lead was stable. The G-means and out-of-bag data (OBD) score obtained by the proposed improved random forest were also higher than the five random forests, and the lead was more obvious. In addition, the test results of three non-parametric tests show that, there were significant diversities between the proposed improved random forest and the other five random forests. This effectively proves the superiority and practicability of the proposed improved random forest. [ABSTRACT FROM AUTHOR]

In real-world scenarios, new audio classes with insufficient samples usually emerge continually, which motivates the study of few-shot class-incremental audio classification (FCAC) in this paper. FCAC aims to enable the model to recognize new audio classes while remembering the base ones continually. To solve the FCAC problem, the discriminability of the prototypes is vital to the model's classification performance. Thus, we proposed a method to learn the discriminative prototypes from two aspects. First, since the generalization ability of the embedding module (EM) significantly affects the discriminability of the prototypes, the proposed method employs a scheme of pseudo-episodic incremental training to train the EM by simulating the test scenario. Second, to enable the model to achieve a balanced classification performance on both base and new audio classes, the proposed method employs a selective-attention module to adjust different prototypes to enhance their discriminability. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in solving the FCAC problem. Notably, the proposed method achieves a comprehensive performance score (CPS) of 87.82% and 59.25% on the Neural Synthesis musical notes of 100 classes (NSynth-100) and Free sound clips of 89 classes (FSC-89) datasets, respectively, which outperforms the comparison methods. Our code is available at https://github.com/chester-w-xie/DPL_FCAC. • The few-shot class-incremental audio classification problem is studied in this paper. • The proposed method expands the model with discriminative prototypes. • A scheme is proposed to enhance the generalization ability of the embedding module. • A selective-attention-based module is proposed for prototype adjustment. • The proposed method achieves state-of-the-art performance in solving the problem. [ABSTRACT FROM AUTHOR]

This paper proposes a novel hybrid classifier method using a mixed-integer linear programming (MILP) model and an improved online learning algorithm for general fuzzy min–max neural networks (IOL_GFMM). IOL_GFMM is used to generate initial hyper-boxes for the MILP model in order to improve its efficiency. This paper is the first attempt that integrates the fuzzy min–max neural networks and the MILP model. The applicability of the proposed hybrid model is proven on both real and artificial datasets. Performance evaluation parameters of the novel hybrid classifier are classification accuracy, training time, number of iterations, and number of misclassified data. Results show that the proposed hybrid method provides satisfying improvements in performance evaluation parameters compared with the pure MILP model. Our proposed approach achieves to speed the training time up to 76 times and decreases the number of misclassified data by up to 17 times. Results of artificial datasets show another contribution that the proposed hybrid approach is very efficient in order to classify data samples not only when they belong to separate regions but also when there exists significant overlap. [ABSTRACT FROM AUTHOR]

Nearest neighbor-based methods are classic techniques that, due to their efficiency, still are widely used today. However, they have not been broadly applied to solve the multi-instance multi-label (MIML) problem, a supervised learning paradigm that combines multi-instance (MI) and multi-label (ML) learning. This work presents new neighbor-based approaches for solving MIML problems. On the one hand, MIML data are transformed into ML data and ML nearest neighbor algorithms are used. On the other hand, algorithms that directly address MIML data and use a bag-based distance are proposed. A comprehensive study and an overall comparison have been conducted to study the performance of these methods using different configurations. Experiments included 16 datasets and 8 performance metrics. The results and statistical tests showed that the problem transformation applied and the distance function used impacted the performance and that the approaches that do not transform the problem obtained the best predictive results. Furthermore, most of the proposed algorithms outperformed the MIMLkNN algorithm, the state-of-art algorithm for MIML learning based on nearest-neighbor. Therefore, the relevance and capabilities of neighbor-based approaches to obtain competitive results in MIML learning are shown. Finally, all the algorithms developed in this paper have been included in the MIML library to facilitate the comparison with other future proposals. • New kNN methods for MIML based on logistic regression, gravitational model, and MAP. • Exhaustive study with 16 datasets and 8 evaluation measures to study the performance. • The transformation and the distance function influenced the predictive performance. • The gravitational model gets the best results with reasonable time complexity. • All the proposals outperform the state-of-art kNN-based algorithm in MIML. [ABSTRACT FROM AUTHOR]

In semiconductor manufacturing, wafer map defect pattern provides critical information for facility maintenance and yield management, so the classification of defect patterns is one of the most important tasks in the manufacturing process. In this paper, we propose a novel way to represent the shape of the defect pattern as a finite-dimensional vector, which will be used as an input for a neural network algorithm for classification. The main idea is to extract the topological features of each pattern by using the theory of persistent homology from topological data analysis (TDA). Through some experiments with a simulated dataset, we show that the proposed method is faster and much more efficient in training with higher accuracy, compared with the method using convolutional neural networks (CNN) which is the most common approach for wafer map defect pattern classification. Moreover, it was shown that our method outperforms the CNN-based method when the number of training data is not enough and is imbalanced. • A novel approach for wafer map defect pattern classification is proposed. • This new method is based on the theory from topological data analysis. • The proposed method is accurate and computationally efficient. • This algorithm is fast enough to provide a real-ti me prediction. • This method is robust against a small and imbalanced dataset. [ABSTRACT FROM AUTHOR]

In machine learning, some models can make uncertain and imprecise predictions, they are called evidential models. These models may also be able to handle imperfect labeling and take into account labels that are richer than the commonly used hard labels, containing uncertainty and imprecision. This paper proposes an Evidential Decision Tree, and an Evidential Random Forest. These two models use a distance and a degree of inclusion to allow the model to group observations whose response elements are included in each other into a single node. Experimental results showed better performance for the presented methods compared to other evidential models and to recent Cautious Random Forests when the data is noisy. The models also offer a better robustness to the overfitting effect when using datasets that are effectively uncertainly and imprecisely labeled by the contributors. The proposed models are also able to predict rich labels, an information that can be used in other approaches, such as active learning. • Source uncertainty and imprecision can be modeled in richer labels. • Richer labels improve classification performance. • Evidential decision tree and evidential random forest are robust to overfitting. [ABSTRACT FROM AUTHOR]

Grouping functions, as one kind of binary continuous aggregation functions, have attracted the continuous attention of scholars since they were proposed. Meanwhile, since Lucca, Sanz and Dimuro et al. proposed the notion of pre-aggregation functions lately, the study on such new "aggregation-like" functions has become a hot topic in the research fields of information fusion. This paper continues to consider the r → -quasi-grouping functions proposed by the author recently based on the grouping functions. At first, we show the relationship among certain classes of r → -quasi-grouping functions. Second, we give the constructions and equivalent characterizations of r → -quasi-grouping functions along with their generalized forms, respectively. Finally, we propose a r → -quasi-grouping functions-based classifier ensemble algorithm (G -CEA), test the performance of G -CEA through numerical experiments, and verify the effectiveness and stability of G -CEA from comparative analysis with three classical machine learning ensemble algorithms and sensitivity analysis of parameter r →. [ABSTRACT FROM AUTHOR]

Contrast patterns, which lie in the core of most understandable classifiers, are frequently evaluated by quality measures. Since many different quality measures are available, they should be compared to select the most appropriate for each applications. This paper introduces a method to compare quality measures, using a set of mined patterns and a collection of objects not used for mining. The comparison is performed by correlating quality values with a quality estimation of the patterns. Additionally, a meta-learning study is performed to show that combining quality measures could be better than using the best single measures in isolation. The results of this paper can help researchers to create new quality measures or to find new combinations of quality measures to create better understandable classification systems. [ABSTRACT FROM AUTHOR]

• Our approach outperforms previous ones in identifying different query types. • Our approach has practical advantages related to the reduced number of features. • RandomForest outperformed other algorithms with 99.6% accuracy. In real-life classification problems, prior information about the problem and expert knowledge about the domain are often used to obtain reliable and consistent solutions. This is especially true in fields where the data is ambiguous, such as text, in which the same words can be used in seemingly similar texts, but have a different meaning. A promising avenue for text classification is machine learning, which has been shown to perform well in a variety of applications including query classification and sentiment analysis. Many of the proposed approaches rely on the bag-of-words representation, which loses the information about the structure of the text. In this paper, we propose a Customised Grammar Framework for text classification, which exploits domain-related information and a new way to represent text as a series of syntactic categories forming syntactic patterns. The framework employs a formal grammar approach for transforming the text into the syntactic patterns representation. We applied the framework for the query classification problem and our results show that our approach outperforms previous ones in terms of classification performance. [ABSTRACT FROM AUTHOR]

• A novel text classifier having the ability to visually explain its rationale. • Domain-independent classification that does not require feature engineering. • Support for incremental learning and text classification over streams. • Efficient framework for addressing early risk detection problems. • State-of-the-art performance on early depression detection task. With the rise of the Internet, there is a growing need to build intelligent systems that are capable of efficiently dealing with early risk detection (ERD) problems on social media, such as early depression detection, early rumor detection or identification of sexual predators. These systems, nowadays mostly based on machine learning techniques, must be able to deal with data streams since users provide their data over time. In addition, these systems must be able to decide when the processed data is sufficient to actually classify users. Moreover, since ERD tasks involve risky decisions by which people's lives could be affected, such systems must also be able to justify their decisions. However, most standard and state-of-the-art supervised machine learning models (such as SVM, MNB, Neural Networks, etc.) are not well suited to deal with this scenario. This is due to the fact that they either act as black boxes or do not support incremental classification/learning. In this paper we introduce SS3, a novel supervised learning model for text classification that naturally supports these aspects. SS3 was designed to be used as a general framework to deal with ERD problems. We evaluated our model on the CLEF's eRisk2017 pilot task on early depression detection. Most of the 30 contributions submitted to this competition used state-of-the-art methods. Experimental results show that our classifier was able to outperform these models and standard classifiers, despite being less computationally expensive and having the ability to explain its rationale. [ABSTRACT FROM AUTHOR]

• We proposed a new ensemble learning framework in the setting of granular computing. • We conducted data transformation by information granulation for in-depth training. • We developed multi-level fusion strategies for in-depth classifiers fusion. • The proposed framework outperforms popular ensemble learning methods. • The proposed framework outperforms popular standard learning methods. Classification is a special type of machine learning tasks, which is essentially achieved by training a classifier that can be used to classify new instances. In order to train a high performance classifier, it is crucial to extract representative features from raw data, such as text and images. In reality, instances could be highly diverse even if they belong to the same class, which indicates different instances of the same class could represent very different characteristics. For example, in a facial expression recognition task, some instances may be better described by Histogram of Oriented Gradients features, while others may be better presented by Local Binary Patterns features. From this point of view, it is necessary to adopt ensemble learning to train different classifiers on different feature sets and to fuse these classifiers towards more accurate classification of each instance. On the other hand, different algorithms are likely to show different suitability for training classifiers on different feature sets. It shows again the necessity to adopt ensemble learning towards advances in the classification performance. Furthermore, a multi-class classification task would become increasingly more complex when the number of classes is increased, i.e. it would lead to the increased difficulty in terms of discriminating different classes. In this paper, we propose an ensemble learning framework that involves transforming a multi-class classification task into a number of binary classification tasks and fusion of classifiers trained on different feature sets by using different learning algorithms. We report experimental studies on a UCI data set on Sonar and the CK+ data set on facial expression recognition. The results show that our proposed ensemble learning approach leads to considerable advances in classification performance, in comparison with popular learning approaches including decision tree ensembles and deep neural networks. In practice, the proposed approach can be used effectively to build an ensemble of ensembles acting as a group of expert systems, which show the capability to achieve more stable performance of pattern recognition, in comparison with building a single classifier that acts as a single expert system. [ABSTRACT FROM AUTHOR]

The application of machine learning to physics problems is widely found in the scientific literature. Both regression and classification problems are addressed by a large array of techniques that involve learning algorithms. Unfortunately, the measurement errors of the data used to train machine learning models are almost always neglected. This leads to estimations of the performance of the models (and thus their generalization power) that is too optimistic since it is always assumed that the target variables (what one wants to predict) are correct. In physics, this is a dramatic deficiency as it can lead to the belief that theories or patterns exist where, in reality, they do not. This paper addresses this deficiency by deriving formulas for commonly used metrics (both for regression and classification problems) that take into account measurement errors of target variables. The new formulas give an estimation of the metrics which is always more pessimistic than what is obtained with the classical ones, not taking into account measurement errors. The formulas given here are of general validity, completely model-independent, and can be applied without limitations. Thus, with statistical confidence, one can analyse the existence of relationships when dealing with measurements with errors of any kind. The formulas have wide applicability outside physics and can be used in all problems where measurement errors are relevant to the conclusions of studies. [ABSTRACT FROM AUTHOR]

Automatic detection of brain tumors in single-spectral magnetic resonance images is a challenging task. Existing techniques suffer from inadequate performance, dependence on initial assumptions, and, sometimes, the need for manual interference. The research reported in this paper seeks to reduce some of these shortcomings, and to remove others, achieving satisfactory performance at reasonable computational costs. The success of the system described here is explained by the synergy of the following aspects: (1) a broad choice of high-level features to characterize the image’s texture, (2) an efficient mechanism to eliminate less useful features (3) a machine-learning technique to induce a classifier that signals the presence of a tumor-affected tissue, and (4) an improved version of the skippy greedy snake algorithm to outline the tumor’s contours. The paper describes the system and reports experiments with synthetic as well as real data. [ABSTRACT FROM AUTHOR]

Highlights • A combination algorithm for ensemble of decision trees is proposed. • RuleCOSI algorithm transforms boosted decision trees into a single ruleset. • The combined rules are simpler than original rules of boosting ensemble. • The performance of the simple production rules was acceptable. Abstract In the field of machine learning, the problem of imbalanced classification arises when the class percentage on the data is unevenly distributed. Different strategies using boosting ensemble algorithms have shown improved results over the imbalanced classification problem by combining weak learners to produce a single strong learner. In particular, decision trees are often used as base learners in ensemble learning for classification or regression. However, boosting ensemble algorithms sometimes generate a large number of decision trees that could grow too large to be understandable and interpretable. Additionally, the use of weights adds more complexity to the final result. For this reason, in this paper, we present RuleCOSI, a novel method for combining and simplifying the output of an ensemble of binary decision trees into a single set of production rules. The proposed method takes into account the weight of each decision tree and using a combination matrix generates a single set of simplified production rules with performance comparable to that of the original boosting ensemble. In order to measure the performance and prove the applicability of the proposed method, we carried out an empirical validation using three different boosting algorithms over several well-known machine learning datasets as well as real-life data collected from a manufacturing company. The results of the algorithm are acceptable in most of the experiments reducing the complexity of the boosting ensemble output while maintaining a similar performance. [ABSTRACT FROM AUTHOR]

Highlights • Here Dispersion Ratio is proposed to identify splitting attribute in Decision Trees. • The proposed method works for any type of variable: categorical, nominal, continuous. • Efficient discretization method is proposed for continuous features. • Has no bias towards features with more distinct values like many other methods. • Extensive evaluation and analysis is performed on a large number of datasets. Abstract In predictive tasks like classification, Information Gain (IG) based Decision Tree is very popularly used. However, IG method has some inherent problems like its preference towards choosing attributes with higher number of distinct values as the splitting attribute in case of nominal attributes and another problem is associated with imbalanced datasets. Most of the real-world datasets have many nominal attributes, and those nominal attributes may have many number of distinct values. In this paper, we have tried to point out these characteristics of the datasets while discussing the performance of our proposed approach. Our approach is a variant of the traditional Decision Tree model and uses a new technique called Dispersion_Ratio, a modification of existing Correlation Ratio (CR) method. The whole approach is divided into two phases - firstly, the dataset is discretised using a discretization module and secondly, the preprocessed dataset is used to build a Dispersion Ratio based Decision Tree model. The proposed method does not prefer the attributes with many unique values and indifferent about class distribution. It performs better than previously proposed CR based Decision Tree (CRDT) Model since an efficient discretization module has been added with it. We have evaluated the performance of our approach on some benchmark datasets from various domains to demonstrate the effectiveness of the proposed technique and also compared our model with Information Gain, Gain Ratio and Gini Index based models. Result shows that the proposed model outperforms other models in majority of the cases that we have considered in our experiment. [ABSTRACT FROM AUTHOR]

Feature selection is very important for pattern recognition to reduce the dimensions of data and to improve the efficiency of learning algorithms. Recent research on new approaches has focused mostly on improving accuracy and reducing computing time. This paper presents a flexible feature-selection method based on an optimized kernel mutual information (OKMI) approach. Mutual information (MI) has been applied successfully in decision trees to rank variables; its aim is to connect class labels with the distribution of experimental data. The use of MI removes irrelevant features and decreases redundant features. However, MI is usually less robust when the data distribution is not centralized. To overcome this problem, we propose to use the OKMI approach, which combines MI and a kernel function. This approach may be used for feature selection with nonlinear models by defining kernels for feature vectors and class-label vectors. By optimizing the objection equations, we develop a new feature-selection algorithm that combines both MI and kernel learning, we discuss the relationship among various kernel-selection methods. Experiments were conducted to compare the new technique applied to various data sets with other methods, and in each case the OKMI approach performs better than the other methods in terms of feature-classification accuracy and computing time. OKMI method solves the problem of computation complexity in the probability of distribution, and avoids this problem by finding the optimal features at very low computational cost. As a result, the OKMI method with the proposed algorithm is effective and robust over a wide range of real applications on expert systems. [ABSTRACT FROM AUTHOR]

In machine learning field, Graph-based Semi-Supervised Learning (GSSL) has recently attracted much attention and many researchers have proposed a number of different methods. GSSL generally constructs a k nearest neighbors graph to explore manifold structure which may improve learning performance of GSSL. If one uses an inappropriate graph to learn a semi-supervised classifier, the performance of the classifier may be worse than that of supervised learning (SL) only trained by labeled samples. Hence, it is worthy to design a safe version to broaden the application area of GSSL. In this paper, we introduce a Safety-aware GSSL (SaGSSL) method which can adaptively select the good graphs and learn a safe semi-supervised classifier simultaneously. The basic assumption is that a graph has a high quality if the sample margin obtained by GSSL with the graph is larger than that obtained by SL. By identifying the high-quality graphs and setting the corresponding weights large, the predictions of our algorithm will approach to those of GSSL with the graphs. Meanwhile, the weights of the low-quality graphs should be small and the predictions of our algorithm will be close to those of SL. Hence the degeneration probability will be reduced and our algorithm is expected to realize the goal of safe exploitation of different graphs. Experimental results on several datasets show that our algorithm can simultaneously implement the graph selection and safely exploit the unlabeled samples. [ABSTRACT FROM AUTHOR]

• Presented an extensive overview on different stages of electrocardiogram analysis. • Summarized the most notable developments of smart health monitoring systems. • Discussed the latest hardware implementations of real-time ECG monitoring systems. • Identified challenges and limitations for various automatic ECG diagnosis systems. • Outlines the future vision of next generation ECG monitoring systems for healthcare. Over the last decade, cardiovascular diseases (CVD's) are the leading cause of death globally. Early prediction of CVD's can help in reducing the complications of high-risk patients. The electrocardiogram (ECG) is an efficient aiding tool, provides accurate information about various cardiac conditions of the human heart. Evaluation and interpretation of ECG signal has become the major goal in current research to identify and mitigate risky cardiovascular conditions. The ECG signal is efficiently analysed and classified using signal processing techniques, ranging from traditional to machine learning approaches and its subbranches, such as deep learning are used for the early detection and diagnosis of cardiac conditions and arrhythmias. The development of novel types of body sensors increases the need for automated, low-cost, real-time, and efficient ECG monitoring systems that can be used at home or in ambulatory settings. IoT in healthcare industry enables the doctors and specialists to diagnose the patient status remotely in a smart and efficient manner. Motivated with all the research done in this area for early diagnosis of cardiac abnormalities and save the life of high-risk patients, we surveyed numerous research works reported in the literature and provide a glance on several aspects of ECG signal monitoring systems. We first introduce the step-by-step framework of ECG signal analysis starting data acquisition to classification, describes each stage analysis of both traditional and advanced machine learning models that have been reported in the literature. We provide a deep discussion on ECG signal acquisition, pre-recorded clinical ECG data taken from the databases, signal processing and denoising, detection of feature based on feature engineering, and signal classification along with the comparative assessment amongst reviewed studies. This work also presents the detailed analysis of smart health care system that employs portable and wearable devices, as well as the Internet of Things (IoT) and wireless technologies, as an innovative medium for data transmission that allows for remote access to a patient's health status at lower costs and with greater efficiency. Moreover, this work provides an extensive knowledge on various hardware platforms that have been adopted for the development of biomedical processors including microcontrollers, FPGA and ASIC. Additionally, the challenges and limitations are discussed in this research field and directions for future work are suggested. This is the first review paper of its kind to present a comprehensive discussion on all aspects of cardiovascular disease monitoring systems in one place. [ABSTRACT FROM AUTHOR]

Expert and intelligent systems understand the underlying information behind the data by relying on a wide range of machine learning techniques. The interpretation of machine learning models is often the key to success in research areas such as business, finance, medical and health science, and bioinformatics; such research areas demand human understanding of the obtained model. The logistic model tree (LMT) algorithm is a popular classification method that combines a decision tree and logistic regression models. The combination of two complementary algorithms produces an accurate and interpretable classifier by combining the advantages of both logistic regression and tree induction. However, LMT has the disadvantage of high computational cost, which makes the algorithm undesirable in practice. In this paper, we propose an efficient method to learn the logistic regression models in the tree. We employ least angle regression to update the regression model in LogitBoost so that the algorithm efficiently learns sparse logistic regression models composed of relevant input variables. We compare the performance of our proposed method with the original LMT algorithm using 14 benchmark datasets and show that the training time dramatically decreases while the accuracy is preserved. Our proposed algorithm is not only accurate and intuitively interpretable but also computationally efficient. It helps users in making the best possible use of the data that are included in expert and intelligent systems. [ABSTRACT FROM AUTHOR]

Radio frequency identification (RFID) has been widely used for the automatic identification, tracking and tracing of goods throughout the supply chain from the manufacturer to the customer. However, one technological problem that impedes the productive and reliable use of RFID is the constraint of false positive readings, which refers to tags that are detected accidentally by the reader but not the ones of interest. This paper focuses on the use of machine learning algorithms to identify such RFID readings. A total of 11 statistical features are extracted from received signal strength (RSS) and phase rotations derived from the raw RFID data. Each of the features is highly statistically different to distinguish the false positive readings, but satisfactory classification cannot be achieved when these features are considered individually. Classifiers based on logistic regression (LR), support vector machine (SVM) and decision tree (DT) are constructed, which combine all of the extracted features to classify the RFID readings more effectively. The performance of the classifiers is evaluated in a real-world factory. Results show that SVM provides the highest accuracy of up to 95.3%. DT shows slightly better accuracy (92.85%) than LR (92.75%), while LR has the larger area under the curve (0.976) than DT (0.949). Overall, machine learning algorithms could achieve accuracy of 93% on average. The proposed methodology provides a much more reliable RFID application as false-positive readings are detected immediately without human intervention, which enables a significant potential of fully automatic identification and tracking of goods throughout the supply chain. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce a novel similarity measure of intuitionistic fuzzy sets (IFSs). The proposed measure is based on the equivalence relation in the IFS-IBA approach. Due to the logic-based background, this measure compares IFS from a different viewpoint than the standard measures, emphasizing comprehension of intuitionism. The IFS-IBA similarity measure has a solid mathematical background and can be combined with various IF aggregation operators. Additionally, we define IFS-IBA distance function as a complement of IFS-IBA similarity. Both IFS-IBA similarity and distance functions may have different realizations that are easy to interpret. Hence, the measures are offering great descriptive power and the ability to model various problems. The benefits of the proposed measure are illustrated on the problem of pattern recognition and classification within k -NN algorithm. Finally, we show that the proposed measure is appropriate for IF hierarchical clustering on the problem of clustering Serbian medium-sized companies according to their financial ratios. Results obtained using the IFS-IBA measure are clear-cut and more meaningful compared to a standard IF distances regardless of the I-fuzzification method used. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new decision forest algorithm that builds a set of highly accurate decision trees by exploiting the strength of all non-class attributes available in a data set, unlike some existing algorithms that use a subset of the non-class attributes. At the same time to promote strong diversity, the proposed algorithm imposes penalties (disadvantageous weights) to those attributes that participated in the latest tree in order to generate the subsequent trees. Besides, some other weight-related concerns are taken into account so that the trees generated by the proposed algorithm remain individually accurate and retain strong diversity. In order to show the worthiness of the proposed algorithm, we carry out experiments on 20 well known data sets that are publicly available from the UCI Machine Learning Repository. The experimental results indicate that the proposed algorithm is effective in generating highly accurate and more balanced decision forests compared to other prominent decision forest algorithms. Accordingly, the proposed algorithm is expected to be very effective in the domain of expert and intelligent systems. [ABSTRACT FROM AUTHOR]

Individuals use Twitter for personal communication, whereas businesses, politicians and celebrities use Twitter for branding purposes. Distinguishing Personal from Branding Twitter accounts is important for Twitter analytics. Existing studies of Twitter account classification apply classical supervised learning, which requires intensive manual annotation for training. In this paper, we propose CDS (Collaborative Distant Supervision), a novel learning scheme for Twitter account classification that does not require intensive manual labelling. Twitter accounts are automatically labelled using heuristics for distant supervision learning. To achieve effective learning from heuristic labels, active learning is applied to identify and correct false positive labels, and semi-supervised learning is applied to further use false negatives missed by labelling heuristics for learning. Extensive experiments on Twitter data showed that CDS achieved high classification accuracy. [ABSTRACT FROM AUTHOR]

The aim of this paper is, first, to recall fuzzy relational compositions (products) and, to introduce an idea, how excluding features could be incorporated into the theoretical background. Apart from definitions, we provide readers with a theoretical investigation. This investigation addresses two natural questions. Firstly, under which conditions (in which underlying algebraic structures) the given three natural approaches to the incorporation of excluding symptoms coincide. And secondly, under which conditions, the proposed incorporation of excluding features preserves the same natural and desirable properties similar to those preserved by fuzzy relational compositions. The positive impact of the incorporation on reducing the suspicions provided by the basic “circlet” composition without losing the possibly correct suspicion is demonstrated on a real taxonomic identification (classification) of Odonata. Here, we demonstrate how the proposed concept may eliminate the weaknesses provided by the classical fuzzy relational compositions and, at the same time, compete with powerful machine learning methods. The aim of the demonstration is not to show that proposed concept outperforms classical approaches, but to show, that its potential is strong enough in order to complete them or in order to be combined with them and to use its different nature. [ABSTRACT FROM AUTHOR]

Breast cancer is one of the leading killers of women around the world. Finding compounds with good bioactivity, metabolic dynamics and safety, including Absorption, Distribution, Metabolism, Excretion and Toxicity (ADMET for short), is a long and challenging task in breast cancer therapy. In the paper, molecular descriptor data of compounds was analyzed by the ensemble learning algorithm, and important features were selected for the development and validation of ADMET classification models. The overall process includes data cleaning, data splitting to training and testing sets, feature selection and classification model evaluation. A Two-Level Stacking Algorithm (TLSA) based on ensemble learning is proposed for ADMET classification. Various performance measures like classification accuracy, precision, recall, confusion matrix, F1-score, Receiver Operating Characteristic (ROC) curves and the Area Under the ROC Curves (AUC) are reported to show the superiority of the proposed method as compared to different classifiers. The experimental results show that the second level algorithm for TLSA utilizes Logistic Regression is better than other classifiers for the properties of Absorption, Distribution and Excretion, with accuracy of 94.6037%, 94.9410% and 88.1956% respectively. For the properties Metabolism and Toxicity, the second level algorithm utilizes Support Vector Machine to achieve the best classification performance, with accuracy of 88.8702% and 96.7960% respectively. The results show that the proposed approach works well with the classification of compound properties and can be a good alternative for the well-known machine learning program. • A novel TLSA model for ADMET classification of compounds is proposed. • The proposed method improves the accuracy of classification results. • Applying a wide range of evaluation metrics to ensure the model's reliability. • Most important molecular descriptors of compounds were selected using RF. [ABSTRACT FROM AUTHOR]

Paroxysmal atrial fibrillation (PAF) is a cardiac arrhythmia which is often challenging to diagnose because patients may be asymptomatic, and episodes are usually intermittent. In this paper we describe a method for identifying patients with a history of PAF using electrocardiogram (ECG) recordings during sinus rhythm. We analyzed, on a beat-to-beat basis, the P-waves in the sinus rhythm ECGs of 69 patients with a history of PAF and 59 healthy individuals. From each subject's P-waves, we calculated key electrocardiographic metrics, including some which are proposed here for the first time. Using means testing and feature selection methods, we discerned the features which were most useful for classification, and trained a Random Forest which identified PAF patients. Our approach achieved a 93.45% accuracy, sensitivity of 95.21%, and specificity of 91.40% using P-wave integral and novel amplitude and slope-based features which ranked highest in importance compared to other metrics from the literature. In particular, descriptive statistics of P-wave amplitudes, slopes, and integrals, were effective for identifying subjects with PAF history vs. healthy individuals. Our method has a high sensitivity and discrimination ability, with an AUC=0.9669 which is superior to others', and can thus be potentially valuable for the early identification of patients who are prone to episodes of PAF, even as part of the standard cardiological checkup that most adults undergo periodically. • Early identification of PAF during sinus rhythm with high accuracy. • P-wave signal metrics on a beat-to-beat basis are proposed. • P-wave amplitude, integral and slope features were ranked as the most important. • A RF classifier distinguishes PAF from healthy subjects with a state-of-the-art AUC. [ABSTRACT FROM AUTHOR]

A smart camera is a vision system capable of extracting application-specific information from the captured images. The paper proposes a decentralized and efficient solution for visual parking lot occupancy detection based on a deep Convolutional Neural Network (CNN) specifically designed for smart cameras. This solution is compared with state-of-the-art approaches using two visual datasets: PKLot, already existing in literature, and CNRPark-EXT. The former is an existing dataset, that allowed us to exhaustively compare with previous works. The latter dataset has been created in the context of this research, accumulating data across various seasons of the year, to test our approach in particularly challenging situations, exhibiting occlusions, and diverse and difficult viewpoints. This dataset is public available to the scientific community and is another contribution of our research. Our experiments show that our solution outperforms and generalizes the best performing approaches on both datasets. The performance of our proposed CNN architecture on the parking lot occupancy detection task, is comparable to the well-known AlexNet, which is three orders of magnitude larger. [ABSTRACT FROM AUTHOR]

Data streams classification is an important approach to get useful knowledge from massive and dynamic data. Because of concept drift, traditional data mining techniques cannot be directly applied in data streams environment. Extreme learning machine (ELM) is a single hidden layer feedforward neural network (SLFN), comparing with the traditional neural network (e.g. BP network), ELM has a faster speed, so it is very suitable for real-time data processing. In order to deal with the challenge in data streams classification, a new approach based on extreme learning machine is proposed in this paper. The approach utilizes ELMs as base classifiers and adaptively decides the number of the neurons in hidden layer, in addition, activation functions are also randomly selected from a series of functions to improve the performance of the approach. Finally, the algorithm trains a series of classifiers and the decision results for unlabeled data are made by weighted voting strategy. When the concept in data streams keeps stable, every classifier is incrementally updated by using new data; if concept drift is detected, the classifiers with weak performance will be cleared away. In the experiment, we used 7 artificial data sets and 9 real data sets from UCI repository to evaluate the performance of the proposed approach. The testing results showed, comparing with the conventional classification methods for data streams such as ELM, BP, AUE2 and Learn++.MF, on most data sets, the new approach could not only be simplest in the structure, but also get a higher and more stable accuracy with lower time consuming. [ABSTRACT FROM AUTHOR]

Although the development of new supervised learning algorithms for machine learning techniques are mostly oriented to improve the predictive power or classification accuracy, the capacity to understand how the classification process is carried out is of great interest for many applications in business and industry. Inductive learning algorithms, like the Rules family, induce semantically interpretable classification rules in the form of if-then rules. Although the effectiveness of the Rules family has been studied thoroughly and new and improved versions are constantly been developed, one important drawback is the effect of the presentation order of the training patterns which has not been studied in depth previously. In this paper this issue is addressed, first by studying empirically the effect of random presentation orders in the number of rules and the generalization power of the resulting classifier. Then a presentation order method for the training examples is proposed which combines a clustering stage with a new density measure developed specifically for this problem. The results using benchmark datasets and a real application of wood defect classification show the effectiveness of the proposed method. Also, since the presentation order method is employed as a preprocessing stage, the simplicity of the Rules family is not affected but instead it enables the generation of fewer and more accurate rules, which can have a direct impact in the performance and usefulness of the Rules family in an expert system context. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

• A novel dataset including combinations of RIR and BAR individuals has been created. • 20 characteristic egg features have been used to classify egg quality and HU. • Eggs have been classified as excellent quality and very good quality by ML models. • HU at the 24th week and the 32nd week have been classified by ML models. With the increase in the world population, the nutritional needs of people have been increased. The demand for eggs which is one of the most important food sources has been increased over years. Therefore, it is very important to inform people about egg quality in order to be prepared for adverse situations such as substitution, mislabeling, and fraud. In this study, it is aimed to specify egg quality without using haugh unit (HU). Besides, another aim is to find how much information HU carries about the specification of egg quality. A dataset including 20 features related to eggs taken from 438 chickens created by Poultry Research Institute (PRI) has been analyzed. An application that can classify egg qualities as very good and excellent using machine learning (ML) models like decision tree (DT), linear discriminant analysis (LDA), logistic regression (LR), naïve bayes (NB), support vector machines (SVM), K-nearest neighboring (KNN), random forest (RF) and artificial neural networks (ANN) has been developed in this study. In addition to that, HU at the 24th week and the 32nd week which are the most important classifier to determine egg qualities have been classified as low informative, medium informative, and high informative by the developed application. Egg quality has the best been classified by LR model based on accuracy and Matthews correlation coefficient (MCC) values as 98.6% and 0.96, respectively. HU at the 24th week has the best been classified by RF based on accuracy and MCC as 96.8% and 0.93, respectively. HU at the 32nd week has the best been classified by RF based on accuracy and MCC as 95.1% and 0.92, respectively. This paper mainly focuses on the classification of egg quality based on not only HU but also egg characteristic features and the importance of the informative feature of HU to classify egg quality. [ABSTRACT FROM AUTHOR]

• First, NOC_SET standard data set is constructed. • Second, some flow features are proposed in this paper. • Third, this paper proposes an improved SVM algorithm, CMSVM. • Finally, this paper gives out Appendix A which is detailed derived for some equals. With the current massive amount of traffic that is going through the internet, internet service providers (ISPs) and networking service providers (NSPs) are looking for various ways to accurately predict the application type of flow that is going through the internet. Such prediction is critical for security and network monitoring applications as they require application type to be known in prior. Traditional ways using port-based or payload-based analysis are not sufficient anymore as many applications start using dynamic unknown port numbers, masquerading, and encryption techniques to avoid being detected. Recently, machine learning has gained significant attention in many prediction applications including traffic classification from flow features or characteristics. However, such algorithms suffer from an imbalanced data problem where some applications have fewer flow data and hence difficult to predict. In this paper, we employ network flow-level characteristics to identify the application type of traffic. Furthermore, we propose the use of an improved support vector machine (SVM) algorithm, named cost-sensitive SVM (CMSVM), to solve the imbalance problem in network traffic identification. CMSVM adopts a multi-class SVM algorithm with active learning which dynamically assigns a weight for applications. We examine the classification accuracy and performance of the CMSVM algorithm using two different datasets, namely MOORE_SET and NOC_SET datasets. Our results show that the CMSVM algorithm can reduce computation cost, improve classification accuracy and solve the imbalance problem when compared to other machine learning techniques. [ABSTRACT FROM AUTHOR]

• Learning style helps to personalize learning experiences. • Attributes identified to deduce the learning style of learners. • Machine learning models trained and validated for accurate results. • Interesting patterns involving learning styles observed and discussed. Learning style plays a vital role in helping students retain learned concepts for a longer time and also improves the understanding of the concepts. Learning styles in offline and online scenarios are recognized using questionnaires. The recent trend is to identify and use attributes to detect the learning style of the learner automatically without disturbing the learner. The paper is an extension of the authors' earlier work with some changes to the methodology. In this paper, the authors have identified new attributes and scaled-down the attributes identified earlier, which would help identify the learner's learning style. The authors implemented classification algorithms and compared the accuracy of the different algorithms on the dataset. Various interesting patterns are observed in learner's behaviour while learning different types of concepts in different situations. [ABSTRACT FROM AUTHOR]

• Systematic literature review based on 136 papers on artificial intelligence applications in supply chain. • The distribution of the different AI techniques used across the SCOR areas is presented. • A classification of the research articles is proposed based on four critical structural dimensions. • Future research perspectives and areas needing further investigations are suggested. Today's supply chains are very different from those of just a few years ago, and they continue to evolve within an extremely competitive economy. Dynamic supply chain processes require a technology that can cope with their increasing complexity. In recent years, several functional supply chain applications based on artificial intelligence (AI) have emerged, yet very few studies have addressed the applications of AI in supply chain processes. Machine learning, natural language processing, and robotics are all potential enablers of supply chain transformation. Aware of the potential advantages of AI implementation in supply chains and of the paucity of work done regarding it, we explore what researchers have done so far with respect to AI and what needs further exploration. We reviewed 136 research papers published between 1996 and 2020 from the Scopus database and provided a classification of the research material according to four critical structural dimensions (level of analytics, AI algorithms or techniques, sector or industry of application, and supply chain processes). This study is the first attempt to study the AI applications in SC from a process perspective and provides a decisional framework for adequate use of AI techniques in the different SC processes. [ABSTRACT FROM AUTHOR]

Multi-label classification (MLC) has recently attracted increasing interest in the machine learning community. Several studies provide surveys of methods and datasets for MLC, and a few provide empirical comparisons of MLC methods. However, they are limited in the number of methods and datasets considered. This paper provides a comprehensive empirical investigation of a wide range of MLC methods on a wealth of datasets from different domains. More specifically, our study evaluates 26 methods on 42 benchmark datasets using 20 evaluation measures. The evaluation methodology used meets the highest literature standards for designing and conducting large-scale, time-limited experimental studies. First, the methods were selected based on their use in the community to ensure a balanced representation of methods across the MLC taxonomy of methods within the study. Second, the datasets cover a wide range of complexity and application domains. The selected evaluation measures assess the predictive performance and efficiency of the methods. The results of the analysis identify RFPCT, RFDTBR, ECCJ48, EBRJ48, and AdaBoost.MH as the best-performing methods across the spectrum of performance measures. Whenever a new method is introduced, it should be compared with different subsets of MLC methods selected according to relevant (and possibly different) evaluation criteria. • A comprehensive empirical investigation of multi-label classification methods. • A strict and robust evaluation methodology for large-scale benchmarking. • Evaluated 26 methods across 42 benchmark datasets using 20 evaluation measures. • Best overall methods are RFPCT, RFDTBR, ECCJ48, EBRJ48, and AdaBoost. MH. [ABSTRACT FROM AUTHOR]

How to design an efficient method to handle mixed-attribute data classification (MADC) problems has become a hot topic in data mining and machine learning. Current MADC methods mostly transform mixed-attribute data into discrete-attribute data or continuous-attribute data before classification algorithms are trained. The discretization of continuous-attribute data usually results in information loss, while the binarization of discrete-attribute data generally yield more discrete-attributes. To address these issues, this paper proposes a novel MADC method abbreviated as DO-RVFL-NBC, which is a Dependency-Oriented aggregation model of random vector functional link (RVFL) network and naive Bayes classifier (NBC). First, the method transforms the original mixed-attribute set into a dependent attribute set and an independent attribute set by considering the variation rates of dependence and independence, respectively. Second, a RVFL network is trained based on the dependent attribute set where each attribute has a weight to represent its dependence importance degree. Third, a weighted NBC is constructed by assigning the independence importance degrees as weights for the calculation of class-conditional probability. Finally, exhaustive experiments are conducted to validate the feasibility, rationality, and effectiveness of the DO-RVFL-NBC method using 22 benchmark mixed-attribute data sets. Experimental results show that (1) dependence and independence exist in the original mixed-attribute set and can be effectively explored; (2) changes of attribute dependences can improve the generalization capabilities of the RVFL network and NBC; and (3) a statistical analysis indicates that DO-RVFL-NBC can obtain considerably better testing accuracies on the benchmark mixed-attribute data sets in comparison with 13 other MADC methods. This demonstrates that DO-RVFL-NBC is a viable approach for MADC problems. • Dependency-Oriented aggregation of RVFL network and NBC was proposed. • The new feature dependence and independence exploration strategies were designed. • The exhaustive experiments were conducted to validate the effectiveness of new model. • The reasons why model can obtain the better prediction performances were revealed. [ABSTRACT FROM AUTHOR]

Over the last few decades, classification applied to numerous applications in science, engineering, business and industries have rapidly been increased, especially for big data. However, classifiers dealing with complicated high dimension problems with non-conforming patterns with high accuracy are rare, especially for bit-level features. It is a challenging research problem. This paper proposed a novel efficient classifier based on cellular automata model, called Cellular Automata-based Classifier (CAC). CAC possesses the promising capability to deal with non-conforming patterns in the bit-level features. It was developed on a new kind of the proposed elementary cellular automata, called Decision Support Elementary Cellular Automata (DS-ECA). The classification capability of DS-ECA is promising since it can describe very complicated decision rule in high dimension problems with less complexity. CAC comprises double rule vectors and a decision function, the structure of which has two layers; the first layer is employed to evolve an input pattern into feature space and the other interprets the patterns in feature space as binary answer through the decision function. It has a time complexity of learning at O( n 2 ), while the classification for one instance is O(1), where n is a number of bit patterns. For classification performance, 12 datasets consisting of binary and non-binary features are empirically implemented in comparison with Support Vector Machines (SVM) using k -fold cross validation. In this respect, CAC outperforms SVM with the best kernel for binary features, and provides the promising results equivalent to SVM on average for non-binary features. [ABSTRACT FROM AUTHOR]

This paper presents a novel ensemble classifier framework for improved classification of mammographic lesions in Computer-aided Detection (CADe) and Diagnosis (CADx) systems. Compared to previously developed classification techniques in mammography, the main novelty of proposed method is twofold: (1) the “combined use” of different feature representations (of the same instance) and data resampling to generate more diverse and accurate base classifiers as ensemble members and (2) the incorporation of a novel “ensemble selection” mechanism to further maximize the overall classification performance. In addition, as opposed to conventional ensemble learning, our proposed ensemble framework has the advantage of working well with both weak and strong classifiers, extensively used in mammography CADe and/or CADx systems. Extensive experiments have been performed using benchmark mammogram dataset to test the proposed method on two classification applications: (1) false-positive (FP) reduction using classification between masses and normal tissues, and (2) diagnosis using classification between malignant and benign masses. Results showed promising results that the proposed method (area under the ROC curve (AUC) of 0.932 and 0.878, each obtained for the aforementioned two classification applications, respectively) impressively outperforms (by an order of magnitude) the most commonly used single neural network (AUC = 0.819 and AUC =0.754) and support vector machine (AUC = 0.849 and AUC = 0.773) based classification approaches. In addition, the feasibility of our method has been successfully demonstrated by comparing other state-of-the-art ensemble classification techniques such as Gentle AdaBoost and Random Forest learning algorithms. [ABSTRACT FROM AUTHOR]

Stream-mining approach is defined as a set of cutting-edge techniques designed to process streams of data in real time, in order to extract knowledge. In the particular case of classification, stream-mining has to adapt its behavior to the volatile underlying data distributions, what has been called concept drift. It is important to note that concept drift may lead to situations where predictive models become invalid and have therefore to be updated to represent the actual concepts that data poses. In this context, there is a specific type of concept drift, known as recurrent concept drift, where the concepts represented by data have already appeared in the past. In those cases the learning process could be saved or at least minimized by applying a previously trained model. To deal with the aforementioned scenario, meta-models can be used in the process of enhancing the drift detection mechanisms used by data stream algorithms, by representing and predicting when the change will occur. There are some real-world situations where a concept reappears, as in the case of intrusion detection systems (IDS), where the same incidents or an adaptation of them usually reappear over time. In these environments the early prediction of drift by means of a better knowledge of past models can help to anticipate to the change, thus improving efficiency of the model regarding the training instances needed. Furthermore, as a complement of meta-models, a mechanism to assess the similarity between classification models is also needed when dealing with recurrent concepts. In this context, when reusing a previously trained model a rough comparison between concepts is usually made, applying boolean logic. The introduction of fuzzy logic comparisons between models could lead to a better efficient reuse of previously seen concepts, by applying not just equal models, but also similar ones. This work faces the aforementioned open issues by means of the MM-PRec system, that integrates a meta-model mechanism and a fuzzy similarity function. The theoretical proposal of MM-PRec is also validated in this paper by means of different experiments using both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

Classification reliability and accuracy are important components for any computer-aided diagnostic system. This paper presents a dermatology CADx system to automatically classify dermatology images into psoriatic lesion and healthy skin using an online system. The novelty of the system is an exploration of the unique and comprehensive feature space combined with classification in support vector machine (SVM) paradigm. The unique feature space consists of grayscale space, color space and aggressiveness of psoriatic disease such as redness and chaoticness. The proposed CADx framework is conventional in paradigm that it has offline and online components. The offline system trains using unique integrated feature space and apriori dermatologist derived ground truth. This training system yields machine learning parameters. The online system is applied on the incoming test images which get transformed by an online classifier utilizing the offline machine learning parameters. The accuracy of the system is evaluated using cross-validation procedure depending upon one of the three partition protocols such as (5-fold, 10-fold and Jack Knife). The proposed CADx system shows the classification accuracy of 99.53%, 99.66% and 99.81% for 5-fold, 10-fold and Jack Knife protocols respectively for 15 optimal features. Further, our results show that, we can demonstrate the reliability and consistency factor by showing the monotonously rising accuracy with increase in data size. Our system is benchmarked against previous reported systems and outstands besides being unique and novel. [ABSTRACT FROM AUTHOR]

• Systematic analysis of overoptimistic results in machine learning fault diagnosis. • Computational framework to test new feature models. • Computational framework to test new classifier architectures • Experimental analysis of more realistic fault diagnosis scenarios. This paper presents a systematic procedure to fairly compare experimental performance scores for machine learning methods for fault diagnosis based on vibration signals. In the vast majority of related scientific publications, the estimated accuracy and similar performance criteria are the sole quality parameter presented. However, the experimental design giving rise to these results is mostly biased, based on unacceptably simple validation methods and on recycling identical patterns in test data sets, previously used for training. Moreover, the methods in general overfit their hyperparameters, introducing additional overoptimistic results. In order to remedy this defect, we critically analyse the usual training-validation-test division and propose an algorithmic guideline in the form of a validation framework. This allows a well defined comparison of experimental results. In order to illustrate the ideas of the paper, the Case Western Reserve University Bearing Data benchmark is used as a case study. Four distinct classifiers are experimentally compared, under gradually more difficult generalization tasks using the proposed evaluation framework: K-Nearest-Neighbor, Support Vector Machine, Random Forest and One-Dimensional Convolutional Neural Network. An extensive literature review suggests that most vibration based research papers, particularly for the Case Western Reserve University Bearing Data, use similar patterns for training and testing, making their classification an easy task. [ABSTRACT FROM AUTHOR]

In this paper, we have proposed a new pipeline for landscape analysis of time-series machine learning datasets that enables us to better understand a benchmarking problem landscape, allows us to select a diverse benchmark datasets portfolio, and reduces the presence of performance assessment bias via bootstrapping evaluation. Combining a large multi-domain representation corpus of time-series specific features and the results of a large empirical study of time-series classification (TSC) benchmark, we showcase the capability of the pipeline to point out issues with non-redundancy and representativeness in the benchmark. By observing discrepancy between the empirical results of the bootstrap evaluation and recently adopted practices in TSC literature when introducing novel methods, we warn on the potentially harmful effects of tuning the methods on certain parts of the landscape (unless this is an explicit and desired goal of the study). Finally, we propose a set of datasets uniformly distributed across the landscape space one should consider when benchmarking novel TSC methods. • Complementary landscape analysis of time-series datasets. • Selecting unbiased benchmark datasets portfolio for comparison study. • Bootstrapping evaluation for reproducible statistical outcomes. [ABSTRACT FROM AUTHOR]

The recent advancements in computational power and machine learning algorithms have led to vast improvements in manifold areas of research. Especially in finance, the application of machine learning enables both researchers and practitioners to gain new insights into financial data and well-studied areas such as company classification. In our paper, we demonstrate that unsupervised machine learning algorithms can visualize and classify company data in an economically meaningful and effective way. In particular, we implement the data-driven dimension reduction and visualization tool t-distributed stochastic neighbor embedding (t-SNE) in combination with spectral clustering. The resulting company groups can then be utilized by experts in the field for empirical analysis and optimal decision making. We are the first to demonstrate how this approach can be implemented in manifold areas of finance by developing a general decision engine. With two exemplary out-of-sample studies on portfolio optimization and company valuation with multiples, we show that the application of t-SNE and spectral clustering improves competitive benchmark models. • Combining t-SNE and spectral clustering achieves compelling company classification. • The decision engine for optimal grouping allows incorporation in various frameworks. • Selecting the group size with cross-validation improves the resulting classification. • Applied to portfolio optimization, our approach results in a Sharpe ratio of 1.5 p.a. • Applied to company valuation, our approach achieves high valuation accuracy. [ABSTRACT FROM AUTHOR]

Statistical methods enable rational and quantitative studies of systems or processes, and support management decisions. Statistical Process Control (SPC) is a standard methodology for measuring, controlling and improving the quality of processes and products through statistics. Presented by Walter A. Shewhart in the 1920's, control charts are among the most popular SPC tools for monitoring processes. However, despite their age, their application in Artificial Intelligence (AI) has only recently begun appearing in the literature. In this paper, we use SPC and AI techniques to present a new process monitoring tool. The proposed classification control chart, which we call class-chart, offers a more robust and flexible alternative to traditional SPC tools. In addition to the ability to recognize patterns and diagnose problems, regardless of the sample scenario, this new approach is capable of performing its monitoring functions on a large scale, predicting market scenarios and processes on large volumes of data. We evaluate the performance of the class-chart by the average run length in extensive numerical simulations. Finally, two real data sets are used to illustrate the applicability of the proposed control chart for classification data. • New monitoring tool based on Statistical Process Control and Artificial Intelligence. • Quality characteristic of interest is multiple categorical and latent. • So-called class-chart is more robust and flexible than usual process monitoring tools. • Ability to take control of future predictions. [ABSTRACT FROM AUTHOR]

Discrete choice models has been for decades the most used technique to model travel mode choice, being the multinomial logit (MNL) the most popular model among them. Several versions of the MNL model have been proposed, such as the mixed multinomial logit (MMNL) model which takes into account unobserved taste heterogeneity. On the other hand, machine learning (ML) methods have begun to gain ground in the transportation field, showing a high predictive power that surpasses logit models. Nowadays, most studies comparing machine learning methods and logit models mainly focus on predictive accuracy, while others – to a lesser extent – focus on post-hoc explanation analysis. In this paper, we compare the predictive performance of five machine learning classifiers and the MNL and MMNL models. Also, we shed light on explanation capability by computing the effect of different variables not only on the overall prediction but also on the prediction of different choice alternatives using an agnostic-model method. The different methods are tested based on synthetic datasets with and without taste heterogeneity between decision-maker, showing a reduction of the accuracy gap between discrete choice models and ML methods when taste heterogeneity is present. We also present an empirical application using four mode choice datasets. Our results show that Neural Networks generally perform better than other models in terms of accuracy and interpretation. Results highlight the importance of analyzing the equivalence between the models in order to complement the explanations obtained through the two approaches. Our analysis can be used to support management decision making and to better understand the factors that determine people's travel behavior. • Systematic comparison between logit models and five machine learning methods. • Discrete choice models for travel mode with response heterogeneity are used. • Prediction accuracy and variable importance for each travel mode is investigated. • XGB, RF and NN outperform traditional logit models in predicting travel mode choices. • The feature importance method recognized the non-informative variables for each mode. [ABSTRACT FROM AUTHOR]