Search

Abstract This paper addresses the problem of a finite set of entities which are required to achieve a goal within a predefined deadline. For example, a group of students is supposed to submit a homework by a specified cutoff. Further, we are interested in predicting which entities will achieve the goal within the deadline. The predictive models are built based only on the data from that population. The predictions are computed at various time instants by taking into account updated data about the entities. The first contribution of the paper is a formal description of the problem. The important characteristic of the proposed method for model building is the use of the properties of entities that have already achieved the goal. We call such an approach “Self-Learning”. Since typically only a few entities have achieved the goal at the beginning and their number gradually grows, the problem is inherently imbalanced. To mitigate the curse of imbalance, we improved the Self-Learning method by tackling information loss and by several sampling techniques. The original Self-Learning and the modifications have been evaluated in a case study for predicting submission of the first assessment in distance higher education courses. The results show that the proposed improvements outperform the specified two base-line models and the original Self-Learner, and also that the best results are achieved if domain-driven techniques are utilised to tackle the imbalance problem. We also showed that these improvements are statistically significant using Wilcoxon signed rank test. [ABSTRACT FROM AUTHOR]

In this paper a novel approach to reuse units of learning (UoLs) – such as courses, seminars, workshops, and so on – is presented. Virtual learning environments (VLEs) do not usually provide the tools to export in a standardized format the designed UoLs, making thus more challenging their reuse in a different platform. Taking into account that many of these VLEs are legacy or proprietary systems, the implementation of a specific software is usually out of place. However, these systems have in common that they record the events of students and teachers during the learning process. The approach presented in this paper makes use of these logs (i) to extract the learning flow structure using process mining, and (ii) to obtain the underlying rules that control the adaptive learning of students by means of decision tree learning. Finally, (iii) the process structure and the adaptive rules are recompiled in IMS Learning Design (IMS LD) – the de facto educational modeling language standard. The three steps of our approach have been validated with UoLs from different domains. [ABSTRACT FROM AUTHOR]

This paper proposes a novel learning model that introduces the calculation of the pairwise gravitation of the selected patterns into the classical fixed radius nearest neighbor method, in order to overcome the drawback of the original nearest neighbor rule when dealing with imbalanced data. The traditional k nearest neighbor rule is considered to lose power on imbalanced datasets because the final decision might be dominated by the patterns from negative classes in spite of the distance measurements. Differently from the existing modified nearest neighbor learning model, the proposed method named GFRNN has a simple structure and thus becomes easy to work. Moreover, all parameters of GFRNN do not need initializing or coordinating during the whole learning procedure. In practice, GFRNN first selects patterns as candidate s out of the training set under the fixed radius nearest neighbor rule, and then introduces the metric based on the modified law of gravitation in the physical world to measure the distance between the query pattern and each candidate . Finally, GFRNN makes the decision based on the sum of all the corresponding gravitational forces from the candidate s on the query pattern. The experimental comparison validates both the effectiveness and the efficiency of GFRNN on forty imbalanced datasets, comparing to nine typical methods. As a conclusion, the contribution of this paper is constructing a new simple nearest neighbor architecture to deal with imbalanced classification effectively without any manually parameter coordination, and further expanding the family of the nearest neighbor based rules. [ABSTRACT FROM AUTHOR]

As news events on the same subject occur, our knowledge about the subject will accumulate and become more comprehensive. In this paper, we formally define the problem of incremental knowledge learning from similar news events on the same subject, where each event consists of a set of news articles reporting about it. The knowledge is represented by a topic hierarchy presenting topics at different levels of granularity. Though topic (hierarchy) mining from text has been researched a lot, incremental learning from similar events remains under developed. In this paper, we propose a scalable two-phase framework to incrementally learn a topic hierarchy for a subject from events on the subject as the events occur. First, we recursively construct a topic hierarchy for each event based on a novel topic model considering the named entities and entity types in news articles. Second, we incrementally merge the topic hierarchies through top-down hierarchical topic alignment. Extensive experimental results on real datasets demonstrate the effectiveness and efficiency of the proposed framework in terms of both qualitative and quantitative measures. [ABSTRACT FROM AUTHOR]

Class binarization strategies decompose the original multi-class problem into several binary sub-problems. One versus One (OVO) is one of the most popular class binarization techniques, which considers every pair of classes as a different sub-problem. Usually, the same classifier is applied to every sub-problem and then all the outputs are combined by some voting scheme. In this paper we present a novel idea where for each test instance we try to assign the best classifier in each sub-problem of OVO. To do so, we have used two simple Dynamic Classifier Selection (DCS) strategies that have not been yet used in this context. The two DCS strategies use K-NN to obtain the local region of the test-instance, and the classifier that performs the best for those instances in the local region, is selected to classify the new test instance. The difference between the two DCS strategies remains in the weight of the instance. In this paper we have also proposed a novel approach in those DCS strategies. We propose to use the K-Nearest Neighbor Equality (K-NNE) method to obtain the local accuracy. K-NNE is an extension of K-NN in which all the classes are treated independently: the K nearest neighbors belonging to each class are selected. In this way all the classes take part in the final decision. We have carried out an empirical study over several UCI databases, which shows the robustness of our proposal. [ABSTRACT FROM AUTHOR]

Name entity recognition (NER) is one of the most basic tasks for extracting information from Internet text. Chinese NER remains a major challenge due to the language complexity. Although researchers have recently used domain knowledge to embed word-level information into the deep learning models to deal with the Chinese NER, they have not considered the global interdependence between word-level information, i.e., the entities in the same document should be semantically related to each other. In addition, domain knowledge often cannot be used efficiently due to the presence of irregular expressions in the Internet text, such as abbreviations and aliases. In this paper, we propose a referent graph embedding model for the NER, specifically concentrating on the Chinese car review. First, domain knowledge is used to generate character-level candidate entities and model the global interdependence between these entities based on the referent graph model. Second, the latest BERT-based character vectors and the character-level candidate entities are jointly embedded into the deep learning model to perform the NER. Last, Chinese car reviews are collected and labeled for use as the experimental dataset. The experimental results demonstrate the efficiency and effectiveness of the proposed model for the Chinese car NER task compared with the other start-of-the-art models. • This paper proposes an RGE-NER model to solve the NER problem for Chinese car reviews. The model innovatively combines referent graph and deep learning models. • Word-based and pronunciation-based methods are designed to expand the index range of entities in lexicon, which reduces the impact of irregular text expressions. • The latest BERT-based character vectors and the character-level candidate entities are jointly embedded into the deep learning model to perform the NER. • The embedding weights of candidate entities are measured by exploiting the global interdependence between candidate entities instead of algorithm learning, which significantly reduces the time complexity while improving the model performance. [ABSTRACT FROM AUTHOR]

Abstract With the development of Deep Learning (DL) in recent years, integrating traditional machine learning methods with DL has received a lot of attention. One of such representative work is the Principal Component Analysis Network (PCANet), which adopts Principal Component Analysis (PCA) to learn convolutional kernels (or filters) for image classification. Nevertheless, PCANet does not use the discriminative information during learning filters. In this paper, based on PCA in the PCANet, we propose a new model called Fisher PCA (FPCA) which combines Fisher Linear Discriminant Analysis (LDA) with PCA. To facilitate the practical calculation, a approximate model of FPCA is given by introducing a intermediate variable. Theoretically, we analyze the relationship between the original FPCA model and its approximate model, and give a convergence analysis of the approximate model. Additionally, stacking the approximate model of FPCA, we also construct a deep network named FPCA Network (FPCANet). Extensive experiments are conducted to compare FPCANet with other state-of-the-art models for classification problems. The results show that the proposed FPCANet can learn features with more discriminative information, and thus demonstrating its competitive performances on classification tasks. [ABSTRACT FROM AUTHOR]

Abstract: Case-based reasoning systems need to maintain their case base in order to avoid performance degradation. Degradation mainly results from memory swamping or exposure to harmful experiences and so, it becomes vital to keep a compact, competent case base. This paper proposes an adaptive case-based reasoning model that develops the case base during the reasoning cycle by adding and removing cases. The rationale behind this approach is that a case base should develop over time in the same way that a human being evolves her overall knowledge: by incorporating new useful experiences and forgetting invaluable ones. Accordingly, our adaptive case-based reasoning model evolves the case base by using a measure of “case goodness” in different retention and forgetting strategies. This paper presents empirical studies of how the combination of this new goodness measure and our adaptive model improves three different performance measures: classification accuracy, efficiency and case base size. [ABSTRACT FROM AUTHOR]

Highlights • A novel zero-shot learning method is developed to rectify the hubness and domain shift problem. • The proposed method exploits the class level visual samples to learn the projection function. • The unseen visual prototypes are modified by the label correlations and their knns. • The proposed method outperforms existing methods on 5 zero-shot learning datasets. Abstract The number of object classes is increasing rapidly leading to the recognition of new classes difficult. Zero-shot learning aims to predict the labels of the new class samples by using the seen class samples and their semantic representations. In this paper, we propose a simple method to learn the unseen visual prototypes (LUVP) by learning the projection function from semantic space to visual feature space to reduce hubness problem. We exploit the class level samples rather than instance level samples, which can alleviate expensive computational costs. Since the disjointness of seen and unseen classes, directly applying the projection function to unseen samples will cause a domain shift problem. Thus, we preserve the unseen label semantic correlations and then adjust the unseen visual prototypes to minimize the domain shift problem. We demonstrate through extensive experiments that the proposed method (1) alleviates the hubness problem, (2) overcomes the domain shift problem, and (3) significantly outperforms existing methods for zero-shot classification on five benchmark datasets. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach for building transparent knowledge-based systems by generating accurate and interpretable fuzzy rules. The learning mechanism reported here induces fuzzy rules via making use of only predefined fuzzy labels that reflect prescribed notations and domain expertise, thereby ensuring transparency in the knowledge model adopted for problem solving. It works by mapping every coarsely learned crisp production rule in the knowledge base onto a set of potentially useful fuzzy rules, which serves as an initial step towards an intuitive technique for similarity-based rule generalisation. This is followed by a procedure that locally selects a compact subset of the emerging fuzzy rules, so that the resulting subset collectively generalises the underlying original crisp rule. The outcome of this local procedure forms the input to a global genetic search process, which seeks for a trade-off between accuracy and complexity of the eventually induced fuzzy rule base while maintaining transparency. Systematic experimental results are provided to demonstrate that the induced fuzzy knowledge base is of high performance and interpretability. [ABSTRACT FROM AUTHOR]

Many real-world optimization problems involve several conflicting objectives that must be optimized simultaneously. Furthermore, most optimization problems have a dynamic structure and change over time. In addition to trying to establish trade-offs among conflicting objectives and explore a diverse set of solutions on a Pareto-optimal front, a dynamic multi-objective optimization (DMOO) algorithm tries to detect changes and track them, using the knowledge of prior environments to converge to the new Pareto-optimal front more quickly. In this paper, a cellular automata-based approach is first proposed for managing and evaluating solutions during the optimization process. Then, using the above approach and the teaching-learning-based optimization algorithm, two new algorithms are introduced for DMOO problems. The first algorithm works to optimize the objectives all at once in a multi-objective manner, while the second algorithm uses the vector evaluated technique to evolve solutions in collaborative single-objective optimization units, and then analyzes them from a multi-objective perspective. These algorithms have been evaluated and compared with some other DMOO algorithms for some standard benchmark problems. The results indicate their superiority in many of the experiments. [ABSTRACT FROM AUTHOR]

A Siamese Deep Forest (SDF) is proposed in the paper. It is based on the Deep Forest or gcForest proposed by Zhou and Feng and can be viewed as a gcForest modification. It can be also regarded as an alternative to the well-known Siamese neural networks. The SDF uses a modified training set consisting of concatenated pairs of vectors. Moreover, it defines the class distributions in the deep forest as the weighted sum of the tree class probabilities such that the weights are determined in order to reduce distances between similar pairs and to increase them between dissimilar points. We show that the weights can be obtained by solving a quadratic optimization problem. The SDF aims to prevent overfitting which takes place in neural networks when only limited training data are available. The numerical experiments illustrate the proposed distance metric method. [ABSTRACT FROM AUTHOR]

Cellular learning automata (CLAs) are learning models that bring together the computational power of cellular automata and also the learning capability of learning automata in unknown environments. CLAs can be open or closed. In a closed CLA , the action of each learning automaton depends on the neighboring cells, whereas in an open CLA , the action of each learning automaton depends on the neighboring cells, and a global environment. These models can be synchronous or asynchronous . In a synchronous CLA , all cells are activated at the same time, but in an asynchronous CLA , at a given time only some cells are activated. These models can be also static or dynamic . In a dynamic CLA, one of its aspects such as structure, local rule or neighborhood may vary with time . All existing dynamic models of the CLAs are closed. In this paper, an open asynchronous dynamic CLA has been introduced. In order to show the potential of this model, an algorithm based on this model for solving allocation hub location problem with imprecise distances among nodes has been designed. To evaluate the proposed algorithm computer simulations have been conducted. The results of simulations show that the proposed algorithm is more robust to imprecise distances as compared to existing algorithms. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) has proved to be well suited to different kinds of classification and regression problems. However, failing to seek deep representation of raw data completely brought by shallow architecture has made a plenty of research work stagnant, when ELM was chosen as the basic model. Recent years, deep ELM models like Hierarchical ELM (H-ELM), deep representations learning via ELM (Dr-ELM) have been proposed to be applied in multiple applications in machine learning. In this paper, a novel double deep ELMs ensemble system (DD-ELMs-ES) is proposed to focus on the problem of time series forecasting. In the proposed system, besides H-ELM and Dr-ELM are utilized as the basic models, a novel Constrained H-ELM (CH-ELM) is presented and serves as another basic model as well. CH-ELM intends to constrain the hidden neurons’ input connection weights, so that they could be consistent with the directions of sample vectors. Whats more, a self-adaptive ReTSP-Trend pruning technique is proposed to implement ensemble pruning in DD-ELMs-ES. Benefited from the merits of combining deep learning scheme with ensemble pruning paradigm, in the empirical results, DD-ELMs-ES demonstrates better generalization performance than the basic deep ELM models and some other state-of-the-art algorithms in tackling with time series forecasting tasks. [ABSTRACT FROM AUTHOR]

There are situations in which lexicon-based methods for Sentiment Analysis (SA) are not able to generate a classification output for specific instances of a given dataset. Most often, the reason for this situation is the absence of specific terms in the sentiment lexicon required in the classification effort. In such cases, there were only two possible paths to follow: (1) add terms to the lexicon (off-line process) by human intervention to guarantee no noise is introduced into the lexicon, which prevents the classification system to provide an immediate answer; or (2) use the services of a word-frequency dictionary (on-line process), which is computationally costly to build. This paper investigates an alternative approach to compensate for the lack of ability of a lexicon-based method to produce a classification output. The method is based on the combination of the classification outputs of non lexicon-based tools. Specifically, firstly the outcome values of applying two or more non-lexicon classification methods are obtained. Secondly, these non-lexicon outcomes are fused using a uninorm based approach, which has been proved to have desirable compensation properties as required in the SA context, to generate the classification output the lexicon based approach is unable to achieve. Experimental results based on the execution of two well-known supervised machine learning algorithms, namely Naïve Bayes and Maximum Entropy, and the application of a cross-ratio uninorm operator are presented. Performance indices associated to options (1) and (2) above are compared against the results obtained using the proposed approach for two different datasets. Additionally, the performance of the proposed cross-ratio uninorm operator based approach is also compared when the aggregation operator used is the arithmetic mean instead. It is shown that the combination of non lexicon-based classification methods with specific uninorm operators improves the classification performance of lexicon-based methods, and it enables the offering of an alternative solution to the SA classification problem when needed. The proposed aggregation method could be used as well as a replacement of ensemble averaging techniques commonly applied when combining the results of several machine learning classifiers’ outputs. [ABSTRACT FROM AUTHOR]

Contrast pattern-based classifiers are an important family of both understandable and accurate classifiers. Nevertheless, these classifiers do not achieve good performance on class imbalance problems. In this paper, we introduce a new contrast pattern-based classifier for class imbalance problems. Our proposal for solving the class imbalance problem combines the support of the patterns with the class imbalance level at the classification stage of the classifier. From our experimental results, using highly imbalanced databases, we can conclude that our proposed classifier significantly outperforms the current contrast pattern-based classifiers designed for class imbalance problems. Additionally, we show that our classifier significantly outperforms other state-of-the-art classifiers not directly based on contrast patterns, which are also designed to deal with class imbalance problems. [ABSTRACT FROM AUTHOR]

Recommender systems typically store personal preference profiles. Many items in the profiles can be represented by numerical attributes. However, the initial profile of each user is incomplete and imprecise. One important problem in the development of these systems is how to learn user preferences, and how to automatically adapted update the profiles. To address this issue, this paper presents an unsupervised approach for learning user preferences over numeric attributes by analyzing the interactions between users and recommender systems. When a list of recommendations shown to a target user, the favorite item will be selected by him/her, then the selected item and the over-ranked items will be employed as valuable feedback to learn the user profile. Specifically, two contributions are offered: 1), a learning approach to measure the influence of over-ranked items through analysis of user feedbacks and 2), a weighting algorithm to calculate weights of different attributes by analyzing user selections. These two approaches are integrated into a traditional adaption model for updating user preference profile. Extensive simulations and results show that both approaches are more effective than existing approaches. [ABSTRACT FROM AUTHOR]

As a building block of information retrieval, relation extraction aims at predicting the relation type between two given entities in a piece of text. This task becomes challenging when it is confronted with long text that contains many task-unrelated tokens. Recent attempts to solve this problem have resorted to learning the relatedness among tokens. However, how to obtain appropriate graph for better relatedness representation still remains outstanding, while existing methods have room to improve. In this paper, we propose a novel latent graph learning method to enhance the expressivity of contextual information for the entities of interest. In particular, we design a dual-channel attention mechanism for multi-view graph learning and pool the learned multi-views to sift unrelated tokens for latent graph. This process can be repeated many times for refining the latent structure. We show that our method achieves superior performance on several benchmark datasets, compared to strong baseline models and prior multi-view graph learning approach. [ABSTRACT FROM AUTHOR]

Structural risk consists of empirical risk (training error) and Vapnik–Chemonenkis (VC) dimension (complexity). Since minimizing empirical risk makes a classifier have a high classification accuracy on label-known patterns while minimizing complexity makes it have a low prediction error on label-unknown patterns, thus structural risk can be used to measure the effectiveness of a classifier and Structural Risk Minimization (SRM) becomes its design goal. Moreover, since a global space can be divided into several local ones, thus from the view of global space, the structural risk is named global structural risk while the one from the view of local space is named local structural risk. Traditional classifiers are designed based on global or local structural risk and their designs do not take both of them into account simultaneously. So this paper proposes a new design goal of a classifier with both global and local structural risks. Furthermore, a locality-sensitive term is introduced so as to measure the relationship between global structural risk and the local one. The new design goal is named Global and Local Structural Risk Minimization (GLSRM). Contribution of GLSRM is that it makes a classifier have a lower training error on label-known patterns and a lower prediction error on label-unknown ones in both global space and local spaces. Innovation of GLSRM is that it introduces a locality-sensitive term so as to reflect the relationship between global structural risk and the local one and then combines global structural risk, local structural risk, and their relationship together in the procedure of classifier design. Experiments on some real-world data sets show feasibility and effectiveness of GLSRM. [ABSTRACT FROM AUTHOR]

Due to the enormous volume of email data generated each day, email management has become a vital area of research. Among the email management tasks, automatic email categorization is one of the most interesting problems. However, the dynamic nature of email data makes the email categorization problem difficult to address for traditional machine learning approaches. In this paper, we propose Alecsa as an attentive learning approach for automatic email categorization. Alecsa aims to simulate the dynamic behavior of users while they attempt to categorize a new email. For this purpose, email categorization problem in Alecsa is cast to a decision-making problem, and an attention control framework is employed to dynamically choose a sequence of structural aspects of the email as the distinguishing factors for categorization. We have analytically evaluated the proposed approach on the Enron–Bekkerman datasets. The evaluation results indicate the unprecedented power of Alecsa toward modeling the dynamic essence of the email categorization problem in terms of effectiveness as well as efficiency. [ABSTRACT FROM AUTHOR]

Transfer learning, which aims to exploit the knowledge in the source domains to promote the learning tasks in the target domains, has attracted extensive research interests recently. The general idea of the previous approaches is to model the shared structure in one latent space as the bridge across domains by reducing the distribution divergences. However, there exist some latent factors in the other latent spaces, which can also be utilized to draw the corresponding distributions closer for establishing the bridges. In this paper, we propose a novel transfer learning method, referred to as Multi-Bridge Transfer Learning (MBTL), to learn the distributions in the different latent spaces together. Therefore, more latent factors shared can be utilized to transfer knowledge. Additionally, an iterative algorithm with convergence guarantee based on non-negative matrix tri-factorization techniques is proposed to solve the optimization problem. Comprehensive experiments demonstrate that MBTL can significantly outperform state-of-the-art learning methods on the topic and sentiment classification tasks. [ABSTRACT FROM AUTHOR]

The twin hyper-sphere support vector machine (THSVM) classifies two classes of samples via two hyper-spheres instead of a pair of nonparallel hyper-planes as in the conversional twin support vector machine (TSVM). Moreover THSVM avoids the matrix inverse operation when solving two dual quadratic programming problems (QPPs). However it cannot yield a desirable result when dealing with the imbalanced data classification. To improve the generalization performance, we propose a maximum margin and minimum volume hyper-spheres machine with pinball loss (Pin-M 3 HM) for the imbalanced data classification in this paper. The basic idea is to construct two hyper-spheres with different centers and radiuses in a sequential order. The first one contains as many examples in majority class as possible, and the second one covers minority class of examples as possible. Moreover the margin between two hyper-spheres is as large as possible. Besides, the pinball loss function is introduced into it to avoid the noise disturbance. Experimental results on 24 imbalanced datasets from the repositories of UCI and KEEL, and a real spectral dataset of Chinese grape wines indicate that our proposed Pin-M 3 HM yields a good generalization performance for the imbalanced data classification. [ABSTRACT FROM AUTHOR]

Noise filters are preprocessing techniques designed to improve data quality in classification tasks by detecting and eliminating examples that contain errors or noise. However, filtering can also remove correct examples and examples containing valuable information, which could be useful for learning. This fact usually implies a margin of improvement on the noise detection accuracy for almost any noise filter. This paper proposes a scheme to improve the performance of noise filters in multi-class classification problems, based on decomposing the dataset into multiple binary subproblems. Decomposition strategies have proven to be successful in improving classification performance in multi-class problems by generating simpler binary subproblems. Similarly, we adapt the principles of the One-vs-One decomposition strategy to noise filtering, making the noise identification process simpler. In order to integrate the filtering results achieved in the binary subproblems, our proposal uses a soft voting approach considering a reliability level based on the aggregation of the noise degree prediction calculated for each binary classifier. The experimental results show that the One-vs-One decomposition strategy usually increases the performance of the noise filters studied, which can detect more accurately the noisy examples. [ABSTRACT FROM AUTHOR]

Learning from imbalanced data is a problem which arises in many real-world scenarios, so does the need to build classifiers able to predict more than one class label simultaneously (multilabel classification). Dealing with imbalance by means of resampling methods is an approach that has been deeply studied lately, primarily in the context of traditional (non-multilabel) classification. In this paper the process of synthetic instance generation for multilabel datasets (MLDs) is studied and MLSMOTE (Multilabel Synthetic Minority Over-sampling Technique), a new algorithm aimed to produce synthetic instances for imbalanced MLDs, is proposed. An extensive review on how imbalance in the multilabel context has been tackled in the past is provided, along with a thorough experimental study aimed to verify the benefits of the proposed algorithm. Several multilabel classification algorithms and other multilabel oversampling methods are considered, as well as ensemble-based algorithms for imbalanced multilabel classification. The empirical analysis shows that MLSMOTE is able to improve the classification results produced by existent proposals. [ABSTRACT FROM AUTHOR]

The insufficiency of labeled training data for representing the distribution of entire dataset is a major obstacle in various practical data mining applications. Semi-supervised learning algorithms, which attempt to learn from both labeled and unlabeled data, provide possibilities to solve this problem. Graph-based semi-supervised learning has recently become one of the most active research areas. In this paper, a novel graph-based semi-supervised learning approach entitled Class Dissimilarity based Linear Neighborhood Propagation (CD-LNP) is proposed, which assumes that each data point can be linearly reconstructed from its neighborhood. The neighborhood graph of the input data is constructed according to a certain kind of dissimilarity between data points, which is specially designed to integrate the class information. Our algorithm can propagate the labels from the labeled points to entire data set using these linear neighborhoods with sufficient smoothness. Experiment results demonstrate that our approach outperforms other popular graph-based semi-supervised learning methods. [ABSTRACT FROM AUTHOR]

Transfer learning aims to provide a framework to utilize previously-acquired knowledge to solve new but similar problems much more quickly and effectively. In contrast to classical machine learning methods, transfer learning methods exploit the knowledge accumulated from data in auxiliary domains to facilitate predictive modeling consisting of different data patterns in the current domain. To improve the performance of existing transfer learning methods and handle the knowledge transfer process in real-world systems, computational intelligence has recently been applied in transfer learning. This paper systematically examines computational intelligence-based transfer learning techniques and clusters related technique developments into four main categories: (a) neural network-based transfer learning; (b) Bayes-based transfer learning; (c) fuzzy transfer learning, and (d) applications of computational intelligence-based transfer learning. By providing state-of-the-art knowledge, this survey will directly support researchers and practice-based professionals to understand the developments in computational intelligence-based transfer learning research and applications. [ABSTRACT FROM AUTHOR]

An intelligent tutoring system plays vital role in education and its importance is constantly increasing, meanwhile the key challenge in the teaching learning process is assessing students’ learning efficiently. In this paper, a hybrid assessment based-on ACT-R cognitive learning theory, combining ontology knowledge map with skills is proposed. In order to assess how well students master knowledge structure, an ontology knowledge map is constructed to describe declarative knowledge; and in order to assess how well students master knowledge skills, a problem solving process is constructed to describe procedural knowledge based on ACT-R. Finally, a student’s mastery of knowledge is assessed through both the knowledge map and skills in the problem solving process, as well as auxiliary indicators like time usage, prior knowledge level, self-assessment, etc. This method is implemented in a geometric intelligent assessment system and is evaluated in a junior high school. Experiments show that the assessment results are consistent with students’ actual learning levels. The hybrid cognitive assessment method can not only obtain the score of students’ mastery of knowledge points and the structure through knowledge map, but also assess the learning skills in problem solving process through exercises quantitatively. [ABSTRACT FROM AUTHOR]

The dual coordinate descent (DCD) algorithm solves the dual problem of support vector machine (SVM) by minimizing a series of single-variable sub-problems with a random order at inner iterations. Apparently, this DCD algorithm gives a sightless update for all variables at each iteration, which leads to a slow speed. In this paper, we present a clipping dual coordinate descent (clipDCD) algorithm for solving the dual problem of SVM. In each iteration, this clipDCD algorithm only solves one single-variable sub-problem according to the maximal possibility-decrease strategy on objective value. We can easily implement this clipDCD algorithm since it has a much simpler formulation compared with the DCD algorithm. Our experiment results indicate that, if the clipDCD algorithm is employed, SVM, twin SVM (TWSVM) and its extensions not only obtain the same classification accuracies, but also take much faster learning speeds than those classifiers employing the DCD algorithm. [ABSTRACT FROM AUTHOR]

The traditional Multi-view Learning (MVL) studies how to process patterns with multiple information sources. In practice, the MVL is proven to have a significant advantage over the Single-view Learning (SVL). But in most real-world cases, there are only single-source patterns to be dealt with and the existing MVL is unable to be directly applied. In order to solve this problem, an alternative MVL technique was developed for the single-source patterns through reshaping the original vector representation of the single-source patterns into multiple matrix representations in our previous work. Doing so can effectively bring an improved classification performance. This paper aims to generalize the previous MVL through taking advantage of the Universum examples which do not belong to either class of the classification problem. The newly-proposed generalization can not only inherit the advantage of the previous MVL, but also get a prior domain knowledge of the whole data distribution. To our knowledge, it introduces the Universum technique into the MVL for the first time. In the implementation, our previous MVL named MultiV-MHKS is selected as the learning paradigm and incorporate MultiV-MHKS with the Universum technique, which forms a more flexible MVL with the Universum called UMultiV-MHKS for short. The subsequent experiments validate that the proposed UMultiV-MHKS can effectively improve the classification performance over both the original MultiV-MHKS and some other state-of-the-art algorithms. Finally, it is demonstrated that the UMultiV-MHKS can get a tighter generalization risk bound in terms of the Rademacher complexity. [ABSTRACT FROM AUTHOR]

Instance selection is one of the most important preprocessing steps in many machine learning tasks. Due to the increasing size of the problems, removing useless, erroneous or noisy instances is frequently an initial step that is performed before other data mining algorithms are applied. Instance selection as part of this data reduction task is one of the most relevant problems in current data mining research. Over the past decades, many different instance selection algorithms have been proposed, each with its own strengths and weaknesses. However, as in the case of classification, it is unlikely that a single instance selection algorithm would be able to achieve good results across many different datasets and application fields. In classification, one of the most successful ways of consistently improving the performance of a single learner is the construction of ensembles using boosting methods. In this paper, we propose a novel approach for instance selection based on boosting instance selection algorithms in the same way boosting is applied to classification. The proposed approach opens a new field of research in which to apply the many techniques developed for boosting classifiers, for instance selection and other data reduction techniques such as feature selection and simultaneous instance and feature selection. Using 60 datasets for balanced problems and 45 datasets for class-imbalanced problems, the experiments reported show a clear improvement in several state-of-the-art instance selection algorithms using the proposed methodology. [ABSTRACT FROM AUTHOR]

Knowledge graphs (KGs) are an important source repository for a wide range of applications and rule mining from KGs recently attracts wide research interest in the KG-related research community. Many solutions have been proposed for the rule mining from large-scale KGs, which however are limited in the inefficiency of rule generation and ineffectiveness of rule evaluation. To solve these problems, in this paper we propose a generation-then-evaluation rule mining approach guided by reinforcement learning. Specifically, a two-phased framework is designed. The first phase aims to train a reinforcement learning agent for rule generation from KGs, and the second is to utilize the value function of the agent to guide the step-by-step rule generation. We conduct extensive experiments on several datasets and the results prove that our rule mining solution achieves state-of-the-art performance in terms of efficiency and effectiveness. • We propose a reinforcement learning algorithm for rule generation. • We propose an efficient rule mining algorithm guided by the trained RL agent. • Our method achieves the state-of-the-art performance on serveral benchmark datasets. [ABSTRACT FROM AUTHOR]

Similarity learning plays a fundamental role in the problem of database retrieval and nearest classification problem. Traditional pairwise similarity measure ignores the contextual information, and the Graph Transduction (GT) has been proposed as a contextual similarity learning algorithm to utilize the contextual information, which is embedded in a nearest neighbor graph. On main shortage of this method is that it is difficult to choose the optimal graph since different graphs may focus on different aspects of the objects. Co-Transduction (CT) is lately proposed by fusing two different graphs. In this paper, we generalize this problem by using the ensemble of many candidate graphs with different models and parameters for transduction, by assuming that the optimal graph could be obtained by the weighted linear ensemble of these candidate graphs. The similarities and graph weights are modeled within one unified objective function, and optimized alternately in an iterative algorithm. The new proposed algorithm, named as Ensemble Transduction (ET), is tested on two challenging tasks and the experimental results show that it can outperform both the GT and CT. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we study the problem of multi-label collective classification (MLCC) where instances are related and associated with multiple class labels. Such correlation of class labels among interrelated instances exists in a wide variety of data, e.g., a web page can belong to multiple categories since its semantics can be recognized in different ways, and the linked web pages are more likely to have the same classes than the unlinked pages. We propose an effective and novel Markov chain based learning method for MLCC problems. Our idea is to model the problem as a Markov chain with restart on transition probability graphs, and to propagate the ranking score of labeled instances to unlabeled instances based on the affinity among instances. The affinity among instances is set up by explicitly using the attribute features derived from the content of instances as well as the correlation features constructed from the links of instances. Intuitively, an instance which contains linked neighbors that are highly similar to the other instances with a high rank of a particular class label, has a high chance of this class label. Extensive experiments have been conducted on two DBLP datasets to demonstrate the effectiveness of the proposed algorithm. The performance of the proposed algorithm is shown to be better than those of the binary relevance multi-label algorithm, collective classification algorithms (wvRN, ICA and Gibbs), and the ICML algorithm for the tested MLCC problems. [Copyright &y& Elsevier]

Highlights: [•] This paper proposes a Column Generation Newton (CGN) algorithm for finding solution of the kernel 1-norm SVM. [•] CGN is combining the Column Generation and the Newton Linear Programming SVM method. [•] CGN is fast when solving the kernel 1-norm SVM. [ABSTRACT FROM AUTHOR]

Abstract: One of the most challenging steps in learning algebra is the translation of word problems into symbolic notation. This paper describes an Intelligent Tutoring System (ITS) that focuses on this stage of the problem solving process. On the one hand, a domain specific inference engine and a knowledge representation mechanism are proposed. These are based on a description language based on hypergraphs, and the idea of using conceptual schemes to represent the student’s knowledge. As a result, the system is able to simultaneously: (a) represent all potential algebraic solutions to a given word problem; (b) keep track of the student’s actions; (c) univocally determine the current state of the resolution process; (d) build a student model; and (e) provide adaptive automatic remediation. On the other hand, the Graphical User Interface (GUI) has been designed to force the student to follow the sequence of steps described in the Cartesian method. The ITS has been evaluated in an educational environment. The results show significant gains for the experimental group and hence support the use of the ITS in practice. [Copyright &y& Elsevier]

Abstract: The success of kernel methods is very much dependent on the choice of kernels. Multiple kernel learning (MKL) aims at learning a combination of different kernels in order to better match the underlying problem instead of using a single fixed kernel. In this paper, we propose a simple but effective multiclass MKL method by a two-stage strategy, in which the first stage finds the kernel weights to combine the kernels, and the second stage trains a standard multiclass support vector machine (SVM). Specifically, we first present an evaluation criterion named multiclass kernel polarization (MKP) to assess the quality of a kernel in the multiclass classification scenario, and then develop a heuristic rule to directly assign a weight to each kernel based on the quality of the individual kernel. MKP is a multiclass extension of the kernel polarization, which is a universal kernel evaluation criterion for kernel design and learning. Comprehensive experiments are conducted on several UCI benchmark examples and the results well demonstrate the effectiveness and efficiency of our approach. [Copyright &y& Elsevier]

Abstract: Traditionally, each instance selection proposal applies the same selection criterion to any problem. However, the performance of such criteria depends on the input data and a single one is not sufficient to guarantee success over a wide range of environments. An option to adapt the selection criteria to the input data is the use of meta-learning to build knowledge-based systems capable to choose the most appropriate selection strategy among several available candidates. Nevertheless, there is not in the literature a theoretical framework that guides the design of instance selection techniques based on meta-learning. This paper presents a framework for this purpose as well as a case study in which the framework is instantiated and an experimental study is carried out to show that the meta-learning approach offers a good compromise between efficiency and versatility in instance selection. [Copyright &y& Elsevier]

Abstract: In the family of Learning Classifier Systems, the classifier system XCS is most widely used and investigated. However, the standard XCS has difficulties solving large multi-step problems, where long action chains are needed to get delayed rewards. Up to the present, the reinforcement learning technique in XCS has been based on Q-learning, which optimizes the discounted total reward received by an agent but tends to limit the length of action chains. However, there are some undiscounted reinforcement learning methods available, such as R-learning and average reward reinforcement learning in general, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by XCS, to replace Q-learning. The modification results in a classifier system that is rapid and able to solve large maze problems. In addition, it produces uniformly spaced payoff levels, which can support long action chains and thus effectively prevent the occurrence of overgeneralization. [Copyright &y& Elsevier]

Traditional machine learning algorithms are always trapped by the class-imbalance problem due to they are biased to the majority class. As one of the most efficient techniques to solve the class-imbalance problem, oversampling technique has attracted many researchers' attention. An obvious observation of an imbalanced dataset is that there is a clear density difference between minority class and majority class. In view of this, we propose a new density-adaptive space partition method called Dannoy. It can distinguish minority class from dataset easily. After that, a novel space partitioning based synthetic minority oversampling technique named SP-SMOTE is also proposed in this paper to deal with the class imbalance problem. Experiments on four synthetic and fifteen real-world datasets are performed and the results on all real-world datasets demonstrate that the average performances (Accuracy, F1-measure, G-mean and AUC) of SP-SMOTE is superior to the other existing popular algorithms SMOTE, ADASYN, K-means SMOTE, Borderline-SMOTE (1 & 2), polynom_fit_SMOTE and ProWSyn. [ABSTRACT FROM AUTHOR]

Small data means that the number of samples in data is small. In such case, their features are often hand-crafted so that they may be insufficient and inconsistent. Currently some feature augmentation methods are proposed to solve the problem. They are mainly for the larger data composed of images and videos, instead of small data. In the case of small data, the human cognition such as cognitive relativity can be more beneficial to machine learning, as humans routinely classify objects according to both their individual features and their environments. The relative transformation is an efficient way to formalize the cognitive relativity and has been validated in improving the performance of the machine learning. Although relationships among categorical central vectors of small data can greatly improve the performance, they have not been considered as features in current methods. This paper uses the relative transformation to model these relationships and then proposes a new feature augmentation method. These relationships as features are learned automatically for each sample through the neural network, instead of determined by the predefined rules. As it only uses the information of small data itself, without requiring any additional knowledge, it can be applied to any data with features. Lots of experimental results on small data validate the proposed method. [ABSTRACT FROM AUTHOR]

Most existing object tracking methods have poor adaptability to complex scenes, and cannot achieve a good balance between tracking accuracy and real-time performance. To solve the above problems, this paper proposes a lightweight UAV object real-time tracking algorithm based on strategy gradient and attention. Firstly, a lightweight E-Mobile Net is designed as the backbone network of feature extraction; secondly, a feature enhanced attention assistant module is designed to enhance the adaptability and discrimination ability of the model; with multi-layer feature fusion regional suggestion network, foreground background classification and boundary box regression response map are obtained by cross-correlation, and the tracking results are calculated. The strategy network based on strategy gradient is used to optimize the template update and re detection strategy, which improves the overall tracking accuracy and efficiency of the model. Simulation experiments on an embedded device and multiple standard data sets show that compared with the current mainstream algorithms, the tracking accuracy is significantly improved 20% ∼ 30%, the algorithm robustness also has obvious advantages, and the tracking speed on an embedded device is 56 fps can meet the real-time requirements. [ABSTRACT FROM AUTHOR]

Hashing technology has been widely used in approximate nearest neighbor search algorithms for large-scale cross-modal retrieval due to its significantly reduced storage and high-speed search capabilities. However, most existing supervised cross-modal hashing methods either mainly rely on binary pairwise similarity and fail to exploit the rich semantic information contained in the label matrix, or the use of single linear projections which suffer from limited information completeness. In this paper, we propose a novel method, named N onlinear S upervised D iscrete H ashing (NSDH). Specifically, NSDH consists of two components, (1) a semantic enhancement descriptor consisting of multiple linear projections that is used to extract comprehensive latent representations of heterogeneous multimedia data, which aligns the original heterogeneous features and integrates the rich semantic label matrix; (2) a fast discrete optimization module used to learn discriminative compact hash codes, which preserves the similarity information using an inner product between the real-valued embeddings of the output of the semantic enhancement descriptors. Therefore, NSDH leverages both the label matrix and similarity information in order to enhance the semantic information of the learned hash codes. In this way, the representation learning capability of the output layer of semantic enhancement descriptors can be greatly enhanced and as a result the learned hash codes are more discriminative. In addition, we present a fast discrete optimization algorithm to efficiently learn the binary hash codes. Results from our experiments on two benchmark datasets highlight the superiority of NSDH in comparison to many state-of-the-art cross-modal hashing methods. • Proposed a novel supervised asymmetric cross-modal hashing method. • Designed a semantic enhancement descriptor and a fast discrete optimization module. • Proposed a well-designed alternative optimization to solve the discrete problem. • Evaluated the effectiveness of the proposed method via two famous datasets. [ABSTRACT FROM AUTHOR]