Search

Copyright of Informatica (03505596) is the property of Slovene Society Informatika and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The opportunity for large amounts of open-for-public and available data is one of the main drivers of the development of an information society at the beginning of the 21st century. In this sense, acquiring knowledge from these data using different methods of machine learning is a prerequisite for solving complex problems in many spheres of human activity, starting from medicine to education and the economy, including traffic as today's important economic branch. Having this in mind, this paper deals with the prediction of the risk of traffic incidents using both historical and real-time data for different atmospheric factors. The main goal is to construct an ensemble model based on the use of several machine learning algorithms which has better characteristics of prediction than any of those installed when individually applied. In global, a case-proposed model could be a multi-agent system, but in a considered case study, a two-agent system is used so that one agent solves the prediction task by learning from the historical data, and the other agent uses the real time data. The authors evaluated the obtained model based on a case study and data for the city of Niš from the Republic of Serbia and also described its implementation as a practical web citizen application. [ABSTRACT FROM AUTHOR]

When performing classification tasks on high-dimensional data, traditional machine learning algorithms often fail to filter out valid information in the features adequately, leading to low levels of classification accuracy. Therefore, this paper explores the high-dimensional data from both the data feature dimension and the model ensemble dimension. We propose a high-dimensional ensemble learning classification algorithm focusing on feature space reconstruction and classifier ensemble, called the HDELC algorithm. First, the algorithm considers feature space reconstruction and then generates a feature space reconstruction matrix. It effectively achieves feature selection and reconstruction for high-dimensional data. An optimal feature space is generated for the subsequent ensemble of the classifier, which enhances the representativeness of the feature space. Second, we recursively determine the number of classifiers and the number of feature subspaces in the ensemble model. Different classifiers in the ensemble system are assigned mutually exclusive non-intersecting feature subspaces for model training. The experimental results show that the HDELC algorithm has advantages compared with most high-dimensional datasets due to its more efficient feature space ensemble capability and relatively reliable ensemble operation performance. The HDELC algorithm makes it possible to solve the classification problem for high-dimensional data effectively and has vital research and application value. [ABSTRACT FROM AUTHOR]

In order to categorize feature classes by capturing subtle differences, hyperspectral images (HSIs) have been extensively used due to the rich spectral-spatial information. The 3D convolution-based neural networks (3DCNNs) have been widely used in HSI classification because of their powerful feature extraction capability. However, the 3DCNN-based HSI classification approach could only extract local features, and the feature maps it produces include a lot of spatial information redundancy, which lowers the classification accuracy. To solve the above problems, we proposed a spatial attention network (SATNet) by combining 3D OctConv and ViT. Firstly, 3D OctConv divided the feature maps into high-frequency maps and low-frequency maps to reduce spatial information redundancy. Secondly, the ViT model was used to obtain global features and effectively combine local-global features for classification. To verify the effectiveness of the method in the paper, a comparison with various mainstream methods on three publicly available datasets was performed, and the results showed the superiority of the proposed method in terms of classification evaluation performance. [ABSTRACT FROM AUTHOR]

Predictive maintenance (PdM) combines the Internet of Things (IoT) technologies with machine learning (ML) to predict probable failures, which leads to the necessity of maintenance for manufacturing equipment, providing the opportunity to solve the related problems and thus make adaptive decisions in a timely manner. However, a standard ML algorithm cannot be directly applied to a PdM dataset, which is highly imbalanced since, in most cases, signals correspond to normal rather than critical conditions. To deal with data imbalance, in this paper, a novel explainable ML method entitled "Balanced K-Star" based on the K-Star classification algorithm is proposed for PdM in an IoT-based manufacturing environment. Experiments conducted on a PdM dataset showed that the proposed Balanced K-Star method outperformed the standard K-Star method in terms of classification accuracy. The results also showed that the proposed method (98.75%) achieved higher accuracy than the state-of-the-art methods (91.74%) on the same data. [ABSTRACT FROM AUTHOR]

Dimension reduction is a preprocessing step in machine learning for eliminating undesirable features and increasing learning accuracy. In order to reduce the redundant features, there are data representation methods, each of which has its own advantages. On the other hand, big data with imbalanced classes is one of the most important issues in pattern recognition and machine learning. In this paper, a method is proposed in the form of a cost-sensitive optimization problem which implements the process of selecting and extracting the features simultaneously. The feature extraction phase is based on reducing error and maintaining geometric relationships between data by solving a manifold learning optimization problem. In the feature selection phase, the cost-sensitive optimization problem is adopted based on minimizing the upper limit of the generalization error. Finally, the optimization problem which is constituted from the above two problems is solved by adding a cost-sensitive term to create a balance between classes without manipulating the data. To evaluate the results of the feature reduction, the multi-class linear SVM classifier is used on the reduced data. The proposed method is compared with some other approaches on 21 datasets from the UCI learning repository, microarrays and high-dimensional datasets, as well as imbalanced datasets from the KEEL repository. The results indicate the significant efficiency of the proposed method compared to some similar approaches. [ABSTRACT FROM AUTHOR]

Algorithmic trading is a common topic researched in the neural network due to the abundance of data available. It is a phenomenon where an approximately linear relationship exists between two or more independent variables. It is especially prevalent in financial data due to the interrelated nature of the data. The existing feature selection methods are not efficient enough in solving such a problem due to the potential loss of essential and relevant information. These methods are also not able to consider the interaction between features. Therefore, we proposed two improvements to apply to the Long Short-Term Memory neural network (LSTM) in this study. It is the Multicollinearity Reduction Module (MRM) based on correlation-embedded attention to mitigate multicollinearity without removing features. The motivation of the improvements is to allow the model to predict using the relevance and redundancy within the data. The first contribution of the paper is allowing a neural network to mitigate the effects of multicollinearity without removing any variables. The second contribution is improving trading returns when our proposed mechanisms are applied to an LSTM. This study compared the classification performance between LSTM models with and without the correlation-embedded attention module. The experimental result reveals that a neural network that can learn the relevance and redundancy of the financial data to improve the desired classification performance. Furthermore, the trading returns of our proposed module are 46.82% higher without sacrificing training time. Moreover, the MRM is designed to be a standalone module and is interoperable with existing models. [ABSTRACT FROM AUTHOR]

Feature selection is one of the main steps in preprocessing data in machine learning, and its goal is to reduce features by removing additional and noisy features. Feature selection methods and feature reduction in a dataset must consider the accuracy of the classifying algorithms. Meta-heuristic algorithms serve as the most successful and promising methods to solve this problem. Symbiotic Organisms Search (SOS) is one of the most successful meta-heuristic algorithms inspired by organisms' interaction in nature called mutualism, commensalism, and parasitism. In this paper, three SOS-based binary approaches are offered to solve the feature selection problem. In the first and second approaches, several S-shaped transfer functions and several Chaotic Tent Function-based V-shaped transfer functions called BSOSST and BSOSVT are used to make the binary SOS (BSOS). In the third approach, an advanced BSOS based on changing SOS and the chaotic Tent function operators called EBCSOS is provided. The EBCSOS algorithm uses the chaotic Tent function and the Gaussian mutation to increase usefulness and exploration. Moreover, two new operators, i.e., BMPT and BCPT, are suggested to make the commensalism and mutualism stage binary based on a chaotic function to solve the feature selection problem. Finally, the proposed BSOSST and BSOSVT methods and the advanced version of EBCSOS were implemented on 25 datasets than the basic algorithm's binary meta-heuristic algorithms. Various experiments demonstrated that the proposed EBCSOS algorithm outperformed other methods in terms of several features and accuracy. To further confirm the proposed EBCSOS algorithm, the problem of detecting spam E-mails was applied, with the results of this experiment indicating that the proposed EBCSOS algorithm significantly improved the accuracy and speed of all categories in detecting spam E-mails. [ABSTRACT FROM AUTHOR]

In real-world scenarios, new audio classes with insufficient samples usually emerge continually, which motivates the study of few-shot class-incremental audio classification (FCAC) in this paper. FCAC aims to enable the model to recognize new audio classes while remembering the base ones continually. To solve the FCAC problem, the discriminability of the prototypes is vital to the model's classification performance. Thus, we proposed a method to learn the discriminative prototypes from two aspects. First, since the generalization ability of the embedding module (EM) significantly affects the discriminability of the prototypes, the proposed method employs a scheme of pseudo-episodic incremental training to train the EM by simulating the test scenario. Second, to enable the model to achieve a balanced classification performance on both base and new audio classes, the proposed method employs a selective-attention module to adjust different prototypes to enhance their discriminability. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in solving the FCAC problem. Notably, the proposed method achieves a comprehensive performance score (CPS) of 87.82% and 59.25% on the Neural Synthesis musical notes of 100 classes (NSynth-100) and Free sound clips of 89 classes (FSC-89) datasets, respectively, which outperforms the comparison methods. Our code is available at https://github.com/chester-w-xie/DPL_FCAC. • The few-shot class-incremental audio classification problem is studied in this paper. • The proposed method expands the model with discriminative prototypes. • A scheme is proposed to enhance the generalization ability of the embedding module. • A selective-attention-based module is proposed for prototype adjustment. • The proposed method achieves state-of-the-art performance in solving the problem. [ABSTRACT FROM AUTHOR]

Aiming at solving the problems of high background complexity of some butterfly images and the difficulty in identifying them caused by their small inter-class variance, we propose a new fine-grained butterfly classification architecture, called Network based on Multi-rate Dilated Attention Mechanism and Multi-granularity Feature Sharer (MRDA-MGFSNet). First, in this network, in order to effectively identify similar patterns between butterflies and suppress the information that is similar to the butterfly's features in the background but is invalid, a Multi-rate Dilated Attention Mechanism (MRDA) with a symmetrical structure which assigns different weights to channel and spatial features is designed. Second, fusing the multi-scale receptive field module with the depthwise separable convolution module, a Multi-granularity Feature Sharer (MGFS), which can better solve the recognition problem of a small inter-class variance and reduce the increase in parameters caused by multi-scale receptive fields, is proposed. In order to verify the feasibility and effectiveness of the model in a complex environment, compared with the existing methods, our proposed method obtained a mAP of 96.64%, and an F1 value of 95.44%, which showed that the method proposed in this paper has a good effect on the fine-grained classification of butterflies. [ABSTRACT FROM AUTHOR]

It is an enormous challenge for imbalanced data learning in the field of machine learning. To construct balanced datasets, oversampling techniques have been studied extensively. However, many oversampling methods suffer from introducing noisy samples and blurring classification boundaries, leading to overfitting. To solve this problem, this paper proposes a new oversampling method, namely CS-SMOTE, for synthesizing minority class samples by three-point interpolation. CS-SMOTE is mainly based on the center offset factor and a synthesis strategy. First, the CS-SMOTE method removes noise samples, calculates the center offset factor, and selects sparsely distributed minority class samples by using the K-distance graph technique. Next, new samples are generated based on sparse minority samples, random minority samples, and sub-cluster centers located in the same sub-cluster samples. Finally, multiple comparative experiments on 18 well-known datasets demonstrate the effectiveness and general applicability of the proposed CS-SMOTE method for the imbalanced data classification. The experiments show that CS-SMOTE outperforms other competitors in terms of classification accuracy, while avoiding the issue of overfitting. [ABSTRACT FROM AUTHOR]

Most of the existing graph embedding methods are used to describe the single view network and solve the single relation in the network. However, the real world is made up of networks with multiple views of complex relationships, and the existing methods can no longer meet the needs of people. To solve this problem, we propose a novel multi-view attention network inspired by coupled P system(MVAN-CP) to deal with node classification. More specifically, we design a multi-view attention network to extract abundant information from multiple views in the network and obtain a learning representation for each view. To enable the views to collaborate, we further apply attention mechanism to facilitate the view fusion process. Taking advantage of the maximum parallelism of P system, the process of learning and fusion will be realized in the coupled P system, which greatly improves the computational efficiency. Experiments on real network data sets indicate that our model is effective. [ABSTRACT FROM AUTHOR]

This paper presents a method for adapting the cost function in the Monge-Kantorovich Problem (MKP) to a classification task. More specifically, we introduce a criterion that allows to learn a cost function which tends to produce large distance values for elements belonging to different classes and small distance values for elements belonging to the same class. Under some additional constraints (one of them being the well-known Monge condition), we show that the optimization of this criterion writes as a linear programming problem. Experimental results on synthetic data show that the output optimal cost function provides good retrieval performances in the presence of two types of perturbations commonly found in histograms. When compared to a set of various commonly used cost functions, our optimal cost function performs as good as the best cost function of the set, which shows that it can adapt well to the task. Promising results are also obtained on real data for two-class image retrieval based on grayscale intensity histograms. [ABSTRACT FROM AUTHOR]

At present, the graph neural network has achieved good results in the semisupervised classification of graph structure data. However, the classification effect is greatly limited in those data without graph structure, incomplete graph structure, or noise. It has no high prediction accuracy and cannot solve the problem of the missing graph structure. Therefore, in this paper, we propose a high-order graph learning attention neural network (HGLAT) for semisupervised classification. First, a graph learning module based on the improved variational graph autoencoder is proposed, which can learn and optimize graph structures for data sets without topological graph structure and data sets with missing topological structure and perform regular constraints on the generated graph structure to make the optimized graph structure more reasonable. Then, in view of the shortcomings of graph attention neural network (GAT) that cannot make full use of the graph high-order topology structure for node classification and graph structure learning, we propose a graph classification module that extends the attention mechanism to high-order neighbors, in which attention decays according to the increase of neighbor order. HGLAT performs joint optimization on the two modules of graph learning and graph classification and performs semisupervised node classification while optimizing the graph structure, which improves the classification performance. On 5 real data sets, by comparing 8 classification methods, the experiment shows that HGLAT has achieved good classification results on both a data set with graph structure and a data set without graph structure. [ABSTRACT FROM AUTHOR]

To beat competition and obtain valuable information, decision-makers must conduct in-depth machine learning or data mining for data analytics. Traditionally, clustering and classification are two common methods used in machine mining. For clustering, data are divided into various groups according to the similarity or common features. On the other hand, classification refers to building a model by given training data, where the target class or label is predicted for the test data. In recent years, many researchers focus on the hybrid of clustering and classification. These techniques have admirable achievements, but there is still room to ameliorate performances, such as distributed process. Therefore, we propose clustering and classification based on distributed automatic feature engineering (AFE) for customer segmentation in this paper. In the proposed algorithm, AFE uses artificial bee colony (ABC) to select valuable features of input data, and then RFM provides the basic data analytics. In AFE, it first initializes the number of cluster k. Moreover, the clustering methods of k-means, Wald method, and fuzzy c-means (FCM) are processed to cluster the examples in variant groups. Finally, the classification method of an improved fuzzy decision tree classifies the target data and generates decision rules for explaining the detail situations. AFE also determines the value of the split number in the improved fuzzy decision tree to increase classification accuracy. The proposed clustering and classification based on automatic feature engineering is distributed, performed in Apache Spark platform. The topic of this paper is about solving the problem of clustering and classification for machine learning. From the results, the corresponding classification accuracy outperforms other approaches. Moreover, we also provide useful strategies and decision rules from data analytics for decision-makers. [ABSTRACT FROM AUTHOR]

Machine learning algorithms perform differently in settings with varying levels of training set mislabeling noise. Therefore, the choice of the right algorithm for a particular learning problem is crucial. The contribution of this paper is towards two, dual problems: first, comparing algorithm behavior; and second, choosing learning algorithms for noisy settings. We present the 'sigmoid rule' framework, which can be used to choose the most appropriate learning algorithm depending on the properties of noise in a classification problem. The framework uses an existing model of the expected performance of learning algorithms as a sigmoid function of the signal-to-noise ratio in the training instances. We study the characteristics of the sigmoid function using five representative non-sequential classifiers, namely, Naïve Bayes, kNN, SVM, a decision tree classifier, and a rule-based classifier, and three widely used sequential classifiers based on hidden Markov models, conditional random fields and recursive neural networks. Based on the sigmoid parameters we define a set of intuitive criteria that are useful for comparing the behavior of learning algorithms in the presence of noise. Furthermore, we show that there is a connection between these parameters and the characteristics of the underlying dataset, showing that we can estimate an expected performance over a dataset regardless of the underlying algorithm. The framework is applicable to concept drift scenarios, including modeling user behavior over time, and mining of noisy time series of evolving nature. [ABSTRACT FROM AUTHOR]

Machine learning provides an effective approach to execute big data analysis. As a branch of machine learning, classification has been widely adopted in data processing. However, the sensitivity of data raises the concern of data privacy. How to balance data utility and data privacy is a challenging issue. Privacy-preserving data classification, which supports flexible and privacy-friendly access to datasets and data classification, enables users' data to be collected in an authenticated manner. However, the priva-cy-preserving data classification approach has a limitation in that the correctness of data classification cannot be guaranteed. As a consequence, it is possible for a malicious classifier to manipulate the classification result. To solve these problems, in this article, we propose a transparent and accountable privacy-preserving data classification framework, which involves a tracer to assert the behavior of the classifier and maintains the utility and privacy of data. Specifically, we take advantage of cryptography techniques to balance data privacy and data utility, and use blockchain to achieve transparency and accountability for the behavior of the classifier. To illustrate the practicability of this framework, we implement concrete cryptographic algorithms and develop a prototype system to evaluate and test its performance. [ABSTRACT FROM AUTHOR]

Nearest neighbor-based methods are classic techniques that, due to their efficiency, still are widely used today. However, they have not been broadly applied to solve the multi-instance multi-label (MIML) problem, a supervised learning paradigm that combines multi-instance (MI) and multi-label (ML) learning. This work presents new neighbor-based approaches for solving MIML problems. On the one hand, MIML data are transformed into ML data and ML nearest neighbor algorithms are used. On the other hand, algorithms that directly address MIML data and use a bag-based distance are proposed. A comprehensive study and an overall comparison have been conducted to study the performance of these methods using different configurations. Experiments included 16 datasets and 8 performance metrics. The results and statistical tests showed that the problem transformation applied and the distance function used impacted the performance and that the approaches that do not transform the problem obtained the best predictive results. Furthermore, most of the proposed algorithms outperformed the MIMLkNN algorithm, the state-of-art algorithm for MIML learning based on nearest-neighbor. Therefore, the relevance and capabilities of neighbor-based approaches to obtain competitive results in MIML learning are shown. Finally, all the algorithms developed in this paper have been included in the MIML library to facilitate the comparison with other future proposals. • New kNN methods for MIML based on logistic regression, gravitational model, and MAP. • Exhaustive study with 16 datasets and 8 evaluation measures to study the performance. • The transformation and the distance function influenced the predictive performance. • The gravitational model gets the best results with reasonable time complexity. • All the proposals outperform the state-of-art kNN-based algorithm in MIML. [ABSTRACT FROM AUTHOR]

Abstract In the fields of heuristic optimization and machine learning, experimentation is the way to assess the performance of an algorithm setup and the hardness of problems. Most algorithms in the domain are anytime algorithms, meaning that they can improve their approximation quality over time. This means that one algorithm may initially perform better than another one, but converge to worse solutions in the end. Instead of single final results, the whole runtime behavior of algorithms needs to be compared. Moreover, a researcher does not just want to know which algorithm performs best and which problem is the hardest – she/he wants to know why. In this paper, we introduce a process which can 1) automatically model the progress of algorithm setups on different problem instances based on data collected in experiments, 2) use these models to discover clusters of algorithm (or problem instance) behaviors, and 3) propose causes why a certain algorithm setup (or problem instance) belongs to a certain algorithm (or problem instance) behavior cluster. These high-level conclusions are presented in form of decision trees relating algorithm parameters (or instance features) to cluster ids. We emphasize the duality of analyzing algorithm setups and problem instances. Our process is implemented as open source software and tested in two case studies, on the Maximum Satisfiability Problem and the Traveling Salesman Problem. Besides its basic application to raw experimental data, yielding clusters and explanations of "quantitative" algorithm behavior, our process also allows for "qualitative" conclusions by feeding it with data which is normalized based on problem features or algorithm parameters. It can also be applied recursively, e.g., to further investigate the behavior of the algorithms in the cluster with the best-performing setups on the problem instances belonging to the cluster of hardest instances. Both use cases are investigated in the case studies. We conclude our article by a comprehensive analysis of the drawbacks of our method and with suggestions on how it can be improved. Highlights • Our method finds groups of optimization algorithm and problem instance behaviors. • It can also discover potential reasons for the differences between the group. • It automates a significant part of the research work in the field of optimization. • It applies to many kinds of single-objective optimization problems and algorithms. • We provide it as ready-to-use open source implementation. [ABSTRACT FROM AUTHOR]

Context. Optimization of the method of nearest neighbors k-NN for the classification of text documents by their topics and experimentally solving the problem based on the method. Objective. The study aims to study the method of nearest neighbors k-NN for classifying text documents by their topics. The task of the study is to classify text documents by their topics based on a dataset for the optimal time and with high accuracy. Method. The k-nearest neighbors (k-NN) method is a metric algorithm for automatic object classification or regression. The k-NN algorithm stores all existing data and categorizes the new point based on the distance between the new point and all points in the training set. For this, a certain distance metric, such as Euclidean distance, is used. In the learning process, k-NN stores all the data from the training set, so it belongs to the “lazy” algorithms since learning takes place at the time of classification. The algorithm makes no assumptions about the distribution of data and it is nonparametric. The task of the k-NN algorithm is to assign a certain category to the test document x based on the categories k of the nearest neighbors from the training dataset. The similarity between the test document x and each of the closest neighbors is scored by the category to which the neighbor belongs. If several of k’s closest neighbors belong to the same category, then the similarity score of that category for the test document x is calculated as the sum of the category scores for each of these closest neighbors. After that, the categories are ranked by score, and the test document is assigned to the category with the highest score. Results. The k-NN method for classifying text documents has been successfully implemented. Experiments have been conducted with various methods that affect the efficiency of k-NN, such as the choice of algorithm and metrics. The results of the experiments showed that the use of certain methods can improve the accuracy of classification and the efficiency of the model. Conclusions. Displaying the results on different metrics and algorithms showed that choosing a particular algorithm and metric can have a significant impact on the accuracy of predictions. The application of the ball tree algorithm, as well as the use of different metrics, such as Manhattan or Euclidean distance, can lead to improved results. Using clustering before applying k-NN has been shown to have a positive effect on results and allows for better grouping of data and reduces the impact of noise or misclassified points, which leads to improved accuracy and class distribution. [ABSTRACT FROM AUTHOR]

Radio frequency identification (RFID) has been widely used for the automatic identification, tracking and tracing of goods throughout the supply chain from the manufacturer to the customer. However, one technological problem that impedes the productive and reliable use of RFID is the constraint of false positive readings, which refers to tags that are detected accidentally by the reader but not the ones of interest. This paper focuses on the use of machine learning algorithms to identify such RFID readings. A total of 11 statistical features are extracted from received signal strength (RSS) and phase rotations derived from the raw RFID data. Each of the features is highly statistically different to distinguish the false positive readings, but satisfactory classification cannot be achieved when these features are considered individually. Classifiers based on logistic regression (LR), support vector machine (SVM) and decision tree (DT) are constructed, which combine all of the extracted features to classify the RFID readings more effectively. The performance of the classifiers is evaluated in a real-world factory. Results show that SVM provides the highest accuracy of up to 95.3%. DT shows slightly better accuracy (92.85%) than LR (92.75%), while LR has the larger area under the curve (0.976) than DT (0.949). Overall, machine learning algorithms could achieve accuracy of 93% on average. The proposed methodology provides a much more reliable RFID application as false-positive readings are detected immediately without human intervention, which enables a significant potential of fully automatic identification and tracking of goods throughout the supply chain. [ABSTRACT FROM AUTHOR]

Associative classification (AC) has been shown to outperform other methods of single-label classification for over 20 years. In order to create rules that are both more precise and simpler to grasp, AC combines the rules of mining associations with the task of classification. However, the current state of knowledge and the views of various specialists indicate that the issue of multi-label classification (MLC) cannot be solved by any AC method. Since this is the case, adapting or using an AC algorithm to manage multi-label datasets is one of the most pressing issues. To solve the MLC issue, this research proposes modifying the classification based on associations (msCBA) method by extending its capabilities to consider more than one class label in the consequent of its rules and modifying its rules order procedure to fit the nature of the multi-label dataset. The proposed algorithm outperforms several other MLC algorithms from various learning techniques across a variety of performance measuresand using six datasets with different domains. The main findings of this research are the significance of utilizing the local dependencies among labels compared to global dependencies, and the important rule of AC in solving the problem of MLC. [ABSTRACT FROM AUTHOR]

The zero-shot image classification (ZSIC) is designed to solve the classification problem when the sample is very small, or the category is missing. A common method is to use attribute or word vectors as a priori category features (auxiliary information) and complete the domain transfer from training of seen classes to recognition of unseen classes by building a mapping between image features and a priori category features. However, feature extraction of the whole image lacks discrimination, and the amount of information of single attribute features or word vector features of categories is insufficient, which makes the matching degree between image features and prior class features not high and affects the accuracy of the ZSIC model. To this end, a spatial attention mechanism is designed, and an image feature extraction module based on this attention mechanism is constructed to screen critical features with discrimination. A semantic information fusion method based on matrix decomposition is proposed, which first decomposes the attribute features and then fuses them with the extracted word vector features of a dataset to achieve information expansion. Through the above two improvement measures, the classification accuracy of the ZSIC model for unseen images is improved. The experimental results on public datasets verify the effect and superiority of the proposed methods. [ABSTRACT FROM AUTHOR]

This article is dedicated to solving the problem of an insufficient degree of automation of artificial neural network training. Despite the availability of a large number of libraries for training neural networks, machine learning engineers often have to manually control the training process to detect overfitting or underfitting. This article considers the task of automatically estimating neural network training results through an analysis of learning curves. Such analysis allows one to determine one of three possible states of the training process: overfitting, underfitting, and optimal training. We propose several algorithms for extracting feature descriptions from learning curves using mathematical statistics. Further state classification is performed using classical machine learning models. The proposed automatic estimation model serves to improve the degree of automation of neural network training and interpretation of its results, while also taking a step toward constructing self-training models. In most cases when the training process of neural networks leads to overfitting, the developed model determines its onset ahead of the early stopping method by 3–5 epochs. [ABSTRACT FROM AUTHOR]

The grade-added rough set (GRS) approach is an extension of the rough set theory proposed by Pawlak to deal with numerical data. However, the GRS has problems with overtraining, unclassified and unnatural results. In this study, we propose a new approach called the directional neighborhood rough set (DNRS) approach to solve the problems of the GRS. The information granules in the DNRS are based on reflexive and antisymmetric relations. Following these relations, new lower and upper approximations are defined. Based on these definitions, we developed a classifier with a three-step algorithm, including DN-lower approximation classification, DN-upper approximation classification, and exceptional processing. Three experiments were conducted using the University of California Irvine (UCI)'s machine learning dataset to demonstrate the effect of each step in the DNRS model, overcoming the problems of the GRS, and achieving more accurate classifiers. The results showed that when the number of dimensions is reduced and both the lower and upper approximation algorithms are used, the DNRS model is more efficient than when the number of dimensions is large. Additionally, it was shown that the DNRS solves the problems of the GRS and the DNRS model is as accurate as existing classifiers. [ABSTRACT FROM AUTHOR]

In remote sensing, active learning (AL) is considered to be an effective solution to the problem of producing sufficient classification accuracy with a limited number of training samples. Though this field has been extensively studied, most papers exist in the pixel-based paradigm. In object-based image analysis (OBIA), AL has been comparatively less studied. This paper aims to propose a new AL method for selecting object-based samples. The proposed AL method solves the problem of how to identify the most informative segment-samples so that classification performance can be optimized. The advantage of this algorithm is that informativeness can be estimated by using various object-based features. The new approach has three key steps. First, a series of one-against-one binary random forest (RF) classifiers are initialized by using a small initial training set. This strategy allows for the estimation of the classification uncertainty in great detail. Second, each tested sample is processed by using the binary RFs, and a classification uncertainty value that can reflect informativeness is derived. Third, the samples with high uncertainty values are selected and then labeled by a supervisor. They are subsequently added into the training set, based on which the binary RFs are re-trained for the next iteration. The whole procedure is iterated until a stopping criterion is met. To validate the proposed method, three pairs of multi-spectral remote sensing images with different landscape patterns were used in this experiment. The results indicate that the proposed method can outperform other state-of-the-art AL methods. To be more specific, the highest overall accuracies for the three datasets were all obtained by using the proposed AL method, and the values were 88.32%, 85.77%, and 93.12% for "T1," "T2," and "T3," respectively. Furthermore, since object-based features have a serious impact on the performance of AL, eight combinations of four feature types are investigated. The results show that the best feature combination is different for the three datasets due to the variation of the feature separability. [ABSTRACT FROM AUTHOR]

Using deep learning to classify hyperspectral image(HSI) with only a few labeled samples available is a challenge. Recently, the knowledge distillation method based on soft label generation has been used to solve classification problems with a limited number of samples. Unlike normal labels, soft labels are considered the probability of a sample belonging to a certain category, and are therefore more informative for the sake of classification. The existing soft label generation methods for HSI classification cannot fully exploit the information of existing unlabeled samples. To solve this problem, we propose a novel self-supervised learning method with knowledge distillation for HSI classification, termed SSKD. The main motivation is to exploit more valuable information for classification by adaptively generating soft labels for unlabeled samples. First, similarity discrimination is performed using all unlabeled and labeled samples by considering both spatial distance and spectral distance. Then, an adaptive nearest neighbor matching strategy is performed for the generated data. Finally, probabilistic judgment for the category is performed to generate soft labels. Compared to the state-of-the-art method, our method improves the classification accuracy by 4.88%, 7.09% and 4.96% on three publicly available datasets, respectively. [ABSTRACT FROM AUTHOR]

Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals' CPS competency are necessary, as traditional assessment types such as multiple -choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals' CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students' display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. [ABSTRACT FROM AUTHOR]

While MOOCs offer educational data on a new scale, many educators find great potential of the big data including detailed activity records of every learner. A learner’s behavior such as if a learner will drop out from the course can be predicted. How to provide an effective, economical, and scalable method to detect cheating on tests such as surrogate exam-taker is a challenging problem. In this paper, we present a grade predicting method that uses student activity features to predict whether a learner may get a certification if he/she takes a test. The method consists of two-step classifications: motivation classification (MC) and grade classification (GC). The MC divides all learners into three groups including certification earning, video watching, and course sampling. The GC then predicts a certification earning learner may or may not obtain a certification. Our experiment shows that the proposed method can fit the classification model at a fine scale and it is possible to find a surrogate exam-taker. [ABSTRACT FROM AUTHOR]

In order to effectively solve the problems of high error rate, long time consuming, and low accuracy of feature extraction in current painting image classification methods, a painting image classification method based on transfer learning and feature fusion was proposed. The global characteristics of the painting picture, such as color, texture, and form, are extracted. The SIFT method is used to extract the painting's local features, and the global and local characteristics are normalized and merged. The painting images are preliminarily classified using the result of feature fusion, the deterministic and nondeterministic samples are divided, and the estimated Gaussian model parameters are transferred to the target domain via a transfer learning algorithm to alter the distribution of nondeterministic samples, completing the painting image classification. Experimental results show that the proposed method has a low error rate and low feature extraction time and a high accuracy rate of painting image classification. [ABSTRACT FROM AUTHOR]

Classification is among the core tasks in machine learning. Existing classification algorithms are typically based on the assumption of at least roughly balanced data classes. When performing tasks involving imbalanced data, such classifiers ignore the minority data in consideration of the overall accuracy. The performance of traditional classification algorithms based on the assumption of balanced data distribution is insufficient because the minority-class samples are often more important than others, such as positive samples, in disease diagnosis. In this study, we propose a cost-sensitive variational autoencoding classifier that combines data-level and algorithm-level methods to solve the problem of imbalanced data classification. Cost-sensitive factors are introduced to assign a high cost to the misclassification of minority data, which biases the classifier toward minority data. We also designed misclassification costs closely related to tasks by embedding domain knowledge. Experimental results show that the proposed method performed the classification of bulk amorphous materials well. [ABSTRACT FROM AUTHOR]

[Display omitted] • Two novel DR methods are proposed to extract critical discriminative information. • Neighborhood size parameters of DR models are decided adaptively via data similarity. • Outside data can be projected directly through SD-IsoP that prevents over-fitting. • Margins between classes in reduced space can be maximized by proposed methods. As one of the most popular nonlinear dimensionality reduction methods, Isomap has been widely used in pattern recognition and machine learning. However, Isomap has the following problems: (1) Isomap is an unsupervised dimensionality reduction method, it cannot use class label information to obtain discriminative low dimensional embedding for classification; (2) The embedding performance of Isomap is sensitive to neighborhood size parameter; (3) Isomap cannot deal with outside new data by direct embedding. In this paper, a novel dimensionality reduction method called supervised discriminant Isomap is proposed to solve the first two problems mentioned above. Specifically, first, raw data points are partitioned into different manifolds by using their class label information. Then, supervised discriminant Isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the Isomap criterion, and to enhance the discriminating capability by maximizing the distances between data points of different classes and the maximum margin graph regularization term. Finally, the corresponding optimization problems are solved by using eigen-decomposition algorithm. Further, we extend supervised discriminant Isomap to a linear dimensionality reduction method called supervised discriminant Isomap projection for handling the above three problems. Moreover, our approaches have three important characteristics: (1) Proposed methods adaptively estimate the local neighborhood surrounding each sample based on data density and similarity; (2) The objective functions of proposed methods can maximize margins between the each classes in the dimension-reduced feature space; (3) The objective functions of proposed methods have closed-form solutions. Furthermore, our methods can capture more discriminative information from raw data than other Isomap based methods. Extensive experiments on nine data sets demonstrate that the proposed methods are superior to the related state-of-the-art methods. [ABSTRACT FROM AUTHOR]

For many machine learning problems, there are sufficient data to train a wide range of algorithms. However, many geoscience applications are challenged with limited training data. Seismic petrophysical classification, mapping seismic data to litho-fluid classes, is one of these examples because the training data labels are based on data gathered from wells. Supervised machine learning algorithms are prone to overfitting in scarce training data situations, but semisupervised approaches are designed for these problems because the unlabelled data are also used to inform the learning process. We adopt label propagation (LP) and self-training methods to solve this problem, because they are semisupervised methods that are conceptually simple and easy to implement. The supervised method we consider for comparison is the popular extreme gradient boosting (XGBoost) classifier. The data set we use for our study is one we generate ourselves from the SEG Advanced Modelling (SEAM) Phase 1 model. We first synthesize seismic data from this model and then perform pre-stack seismic inversion to recover seismic attributes. We formulate a classification problem using the seismic attributes as unlabelled data, with training labels from a single well. The benefit of this being a synthetic problem is that we have full control and the ability to quantitatively assess the machine learning predictions. Our initial results reveal that the inherent depth-dependent background trends of the input attributes produce artefacts in each of the machine learning predictions. We address this problem by using a simple median filter to remove these background trends. The predictions using the detrended inputs improve the performance for all three algorithms, in some cases on the order of 10 to 20 per cent. XGBoost and LP perform similarly in some situations, but our results indicate that XGBoost is rather unstable depending on the attributes used. However, LP coupled with self-training outperforms XGBoost by up to 10 per cent in some instances. Through this synthetic study, our results support the premise that semisupervised algorithms can provide more robust, generalized predictions than supervised techniques in minimal training data scenarios. [ABSTRACT FROM AUTHOR]

The detection of coffee bean defects is the most crucial step prior to bean roasting. Existing defect detection methods used in the specialty coffee bean industry entail manual screening and sorting, require substantial human resources, and are not standardized. To solve these problems, this study developed a deep learning algorithm to detect defects in coffee beans. The results reveal that when the pooling layer was used to enhance features and reduce neural dimensionality, some of the coffee been features were lost or misclassified. Therefore, a novel dimensionality reduction method was adopted to increase the ability of feature extraction. The developed model also overcame the drawbacks of padding causing blurred image boundaries and the dead neurons causing impeding feature propagation. Images of eight types of coffee beans were used to train and test the proposed detection model. The proposed method was verified to reduce the bias when classifying defects in coffee beans. The detection accuracy rate of the proposed model was 95.2%. When the model was only used to detect the presence of defects, the accuracy rate increased to 100%. Thus, the proposed model is highly accurate in coffee bean defect detection in the classification of eight types of coffee beans. [ABSTRACT FROM AUTHOR]

Typically, classification is conducted on a dataset that consists of numerical features and target classes. For instance, a grayscale image, which is usually represented as a matrix of integers varying from 0 to 255, enables one to apply various classification algorithms to image classification tasks. However, datasets represented as binary features cannot use many standard machine learning algorithms optimally, yet their amount is not negligible. On the other hand, oversampling algorithms such as synthetic minority oversampling technique (SMOTE) and its variants are often used if the dataset for classification is imbalanced. However, since SMOTE and its variants synthesize new minority samples based on the original samples, the diversity of the samples synthesized from binary features is highly limited due to the poor representation of original features. To solve this problem, a preprocessing approach is studied. By converting binary features into numerical ones using feature extraction methods, succeeding oversampling methods can fully display their potential in improving the classifiers' performances. Through comprehensive experiments using benchmark datasets and real medical datasets, it was observed that a converted dataset consisting of numerical features is better for oversampling methods (maximum improvements of accuracy and F1-score were 35.11% and 42.17%, respectively). In addition, it is confirmed that feature extraction and oversampling synergistically contribute to the improvement of classification performance. [ABSTRACT FROM AUTHOR]

The purpose of aspect-based sentiment classification is to identify the sentiment polarity of each aspect in a sentence. Recently, due to the introduction of Graph Convolutional Networks (GCN), more and more studies have used sentence structure information to establish the connection between aspects and opinion words. However, the accuracy of these methods is limited by noise information and dependency tree parsing performance. To solve this problem, we proposed an attention-enhanced graph convolutional network (AEGCN) for aspect-based sentiment classification with multi-head attention (MHA). Our proposed method can better combine semantic and syntactic information by introducing MHA and GCN. We also added an attention mechanism to GCN to enhance its performance. In order to verify the effectiveness of our proposed method, we conducted a lot of experiments on five benchmark datasets. The experimental results show that our proposed method can make more reasonable use of semantic and syntactic information, and further improve the performance of GCN. [ABSTRACT FROM AUTHOR]

Random forests (RF) is a supervised machine learning algorithm, which has recently started to gain prominence in water resources applications. However, existing applications are generally restricted to the implementation of Breiman's original algorithm for regression and classification problems, while numerous developments could be also useful in solving diverse practical problems in the water sector. Here we popularize RF and their variants for the practicing water scientist, and discuss related concepts and techniques, which have received less attention from the water science and hydrologic communities. In doing so, we review RF applications in water resources, highlight the potential of the original algorithm and its variants, and assess the degree of RF exploitation in a diverse range of applications. Relevant implementations of random forests, as well as related concepts and techniques in the R programming language, are also covered. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) is a recently proposed algorithm, efficient and fast for learning the parameters of single layer neural structures. One of the main problems of this algorithm is to choose the optimal architecture for a given problem solution. To solve this limitation several solutions have been proposed in the literature, including the regularization of the structure. However, to the best of our knowledge, there are no works where such adjustment is applied to classification problems in the presence of a non-linearity in the output; all published works tackle modelling or regression problems. Our proposal has been applied to a series of standard databases for the evaluation of machine learning techniques. Results obtained in terms of classification success rate and training time, are compared to the original ELM, to the well known Least Square Support Vector Machine (LS-SVM) algorithm and with two other methods based on the ELM regularization: Optimally Pruned Extreme Learning Machine (OP-ELM) and Bayesian Extreme Learning Machine (BELM). The obtained results clearly demonstrate the usefulness of the proposed method and its superiority over a classical approach. [ABSTRACT FROM AUTHOR]

Since most classifiers are biased toward the dominant class, class imbalance is a challenging problem in machine learning. The most popular approaches to solving this problem include oversampling minority examples and undersampling majority examples. Oversampling may increase the probability of overfitting, whereas undersampling eliminates examples that may be crucial to the learning process. We present a linear time resampling method based on random data partitioning and a majority voting rule to address both concerns, where an imbalanced dataset is partitioned into a number of small subdatasets, each of which must be class balanced. After that, a specific classifier is trained for each subdataset, and the final classification result is established by applying the majority voting rule to the results of all of the trained models. We compared the performance of the proposed method to some of the most well-known oversampling and undersampling methods, employing a range of classifiers, on 33 benchmark machine learning class-imbalanced datasets. The classification results produced by the classifiers employed on the generated data by the proposed method were comparable to most of the resampling methods tested, with the exception of SMOTEFUNA, which is an oversampling method that increases the probability of overfitting. The proposed method produced results that were comparable to the Easy Ensemble (EE) undersampling method. As a result, for solving the challenge of machine learning from class-imbalanced datasets, we advocate using either EE or our method. [ABSTRACT FROM AUTHOR]

One of the important research problems in the context of financial institutions is the assessment of credit risk and the decision to whether grant or refuse a loan. Recently, machine learning based methods are increasingly employed to solve such problems. However, the selection of appropriate feature selection technique, sampling mechanism, and/or classifiers for credit decision support is very challenging, and can affect the quality of the loan recommendations. To address this challenging task, this article examines the effectiveness of various data science techniques in issue of credit decision support. In particular, processing pipeline was designed, which consists of methods for data resampling, feature discretization, feature selection, and binary classification. We suggest building appropriate decision models leveraging pertinent methods for binary classification, feature selection, as well as data resampling and feature discretization. The selected models' feasibility analysis was performed through rigorous experiments on real data describing the client's ability for loan repayment. During experiments, we analyzed the impact of feature selection on the results of binary classification, and the impact of data resampling with feature discretization on the results of feature selection and binary classification. After experimental evaluation, we found that correlation-based feature selection technique and random forest classifier yield the superior performance in solving underlying problem. [ABSTRACT FROM AUTHOR]

In the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classifiers, the random and two intuitive classifiers which are based on how a non-expert person could realize classification simply by applying a frequentist approach. We show that these three simple methods are closely related to different aspects of the entropy of the dataset. Therefore, these measures account somewhat for entropy in the dataset when evaluating the performance of classifiers. This allows us to measure the improvement in the classification results compared to simple methods, and at the same time how entropy affects classification capacity. To illustrate how these new performance measures can be used to analyze classifiers taking into account the entropy of the dataset, we carry out an intensive experiment in which we use the well-known J48 algorithm, and a UCI repository dataset on which we have previously selected a subset of the most relevant attributes. Then we carry out an extensive experiment in which we consider four heuristic classifiers, and 11 datasets. [ABSTRACT FROM AUTHOR]

Machine learning and deep learning methods have been employed in the hyperspectral image (HSI) classification field. Of deep learning methods, convolution neural network (CNN) has been widely used and achieved promising results. However, CNN has its limitations in modeling sample relations. Graph convolution network (GCN) has been introduced to HSI classification due to its demonstrated ability in processing sample relations. Introducing GCN into HSI classification, the key issue is how to transform HSI, a typical euclidean data, into non-euclidean data. To address this problem, we propose a supervised framework called the Global Random Graph Convolution Network (GR-GCN). A novel method of constructing the graph is adopted for the network, where the graph is built by randomly sampling from the labeled data of each class. Using this technique, the size of the constructed graph is small, which can save computing resources, and we can obtain an enormous quantity of graphs, which also solves the problem of insufficient samples. Besides, the random combination of samples can make the generated graph more diverse and make the network more robust. We also use a neural network with trainable parameters, instead of artificial rules, to determine the adjacency matrix. An adjacency matrix obtained by a neural network is more flexible and stable, and it can better represent the relationship between nodes in a graph. We perform experiments on three benchmark datasets, and the results demonstrate that the GR-GCN performance is competitive with that of current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Genetic programming (GP) is a powerful machine learning (ML) algorithm that can produce readable white-box models. Although successfully used for solving an array of problems in different scientific areas, GP is still not well known in the field of remote sensing. The M3GP algorithm, a variant of the standard GP algorithm, performs feature construction by evolving hyperfeatures from the original ones. In this work, we use the M3GP algorithm on several sets of satellite images over different countries to create hyperfeatures from satellite bands to improve the classification of land cover types. We add the evolved hyperfeatures to the reference datasets and observe a significant improvement of the performance of three state-of-the-art ML algorithms (decision trees, random forests, and XGBoost) on multiclass classifications and no significant effect on the binary classifications. We show that adding the M3GP hyperfeatures to the reference datasets brings better results than adding the well-known spectral indices NDVI, NDWI, and NBR. We also compare the performance of the M3GP hyperfeatures in the binary classification problems with those created by other feature construction methods such as FFX and EFS. [ABSTRACT FROM AUTHOR]

Selecting the proper performance metric constitutes a key issue for most classification problems in the field of machine learning. Although the specialized literature has addressed several topics regarding these metrics, their symmetries have yet to be systematically studied. This research focuses on ten metrics based on a binary confusion matrix and their symmetric behaviour is formally defined under all types of transformations. Through simulated experiments, which cover the full range of datasets and classification results, the symmetric behaviour of these metrics is explored by exposing them to hundreds of simple or combined symmetric transformations. Cross-symmetries among the metrics and statistical symmetries are also explored. The results obtained show that, in all cases, three and only three types of symmetries arise: labelling inversion (between positive and negative classes); scoring inversion (concerning good and bad classifiers); and the combination of these two inversions. Additionally, certain metrics have been shown to be independent of the imbalance in the dataset and two cross-symmetries have been identified. The results regarding their symmetries reveal a deeper insight into the behaviour of various performance metrics and offer an indicator to properly interpret their values and a guide for their selection for certain specific applications. [ABSTRACT FROM AUTHOR]

The rather impressive extension library of medical image-processing platform 3D Slicer lacks a wide range of machine-learning toolboxes. The authors have developed such a toolbox that incorporates commonly used machine-learning libraries. The extension uses a simple graphical user interface that allows the user to preprocess data, train a classifier, and use that classifier in common medical image-classification tasks, such as tumor staging or various anatomical segmentations without a deeper knowledge of the inner workings of the classifiers. A series of experiments were carried out to showcase the capabilities of the extension and quantify the symmetry between the physical characteristics of pathological tissues and the parameters of a classifying model. These experiments also include an analysis of the impact of training vector size and feature selection on the sensitivity and specificity of all included classifiers. The results indicate that training vector size can be minimized for all classifiers. Using the data from the Brain Tumor Segmentation Challenge, Random Forest appears to have the widest range of parameters that produce sufficiently accurate segmentations, while optimal Support Vector Machines' training parameters are concentrated in a narrow feature space. [ABSTRACT FROM AUTHOR]