Search

Forecasting natural gas consumption, considering seasonality and trends, is crucial in planning its supply and consumption and optimizing the cost of obtaining it, mainly by industrial entities. However, in times of threats to its supply, it is also a critical element that guarantees the supply of this raw material to meet individual consumers' needs, ensuring society's energy security. This article introduces a novel multistep forecasting of natural gas consumption with change point detection integration for model collection selection with continual learning capabilities using data stream processing. The performance of the forecasting models based on the proposed approach is evaluated in a complex real-world use case of natural gas consumption forecasting. Furthermore, the methodology generability was verified in an electricity load forecasting task. We employed Hoeffding tree predictors as forecasting models and the Pruned Exact Linear Time (PELT) algorithm for the change point detection procedure. The change point detection integration enables the selection of a different model collection for successive time frames. Thus, three model collection selection procedures are defined and evaluated for forecasting scenarios with various densities of detected change points. These models were compared with change point agnostic baseline approaches and deep learning models. Our experiments show that the proposed approach provides superior results to deep learning models for both datasets and that fewer change points result in a lower forecasting error regardless of the model collection selection procedure employed. • Continual learning methodology for multistep forecasting of time series is defined. • Impact of integrating a change point detection and model selection is studied. • Methods are compared with a multiple baseline change point agnostic approaches. • Experiments were conducted over a recent online available real-world data. • Method accuracy in a dynamic and complex data stream environment is demonstrated. [ABSTRACT FROM AUTHOR]

Imbalanced data classification presents a significant challenge when there is a substantial disparity in sample sizes across different classes. This issue severely affects classifier accuracy in predicting minority classes, hampering numerous real-world applications. Traditional methods address data imbalance by using undersampling or oversampling techniques. However, these methods may lead to information loss during sample reduction or introduce noise and model bias through synthetic sample generation. In this paper, we introduce DRIL , an innovative clustering-based incremental learning approach designed to overcome these limitations and improve the classification of minority class samples. Specifically, we employ a "two-step clustering" method to rebalance the dataset, partitioning it into similar and representative sub-dataset. Subsequently, incremental learning is applied to enable the classifier to gradually acquire knowledge about these sub-data, establishing a comprehensive understanding of all features present in the imbalanced dataset. Experimental results on twenty datasets demonstrate that our incremental learning-based algorithm outperforms baseline methods in correctly classifying minority classes while exhibiting improved precision and F1 score performance. [ABSTRACT FROM AUTHOR]

In real-world fault diagnosis tasks, it is never possible to enumerate all fault types beforehand since there are always unseen situations that may arise unexpectedly. Therefore, fault diagnosis is essentially an open-world recognition task. A desirable open-world fault diagnosis model must be able to perform open-set recognition (OSR) and incremental learning (IL). Classifier calibration and prototype replay are popular in two subtasks. However, existing calibration and replay strategies suffer from tampering with raw data and ignoring resource allocation, respectively. In this work, we formulate an open-world fault diagnosis scheme to address both issues. Firstly, we calibrate the uncertainty of the classifier's prediction by assigning soft labels to the samples based on their distance from the class center. Then, Shannon entropy is employed to quantify the uncertainty as an estimate of the probability that the sample belongs to the unknown. Secondly, we adaptively manage the memory of old and new known classes according to the training accuracy. Then, based on this memory budget, a dissimilarity-based sparse subset selection algorithm is utilized to pick more diverse exemplars as prototypes for replay. Furthermore, we extend the open-set F-Measure (OSFM) to the weighted OSFM to make it applicable even when the classes are imbalanced. Experimental results on multiphase flow facility and wastewater treatment plant demonstrate the feasibility and superiority of the proposed method. • Open-set recognition by uncertainty calibration and estimation. • Incremental learning by memory management and prototype selection. • The proposed weighted Open-set F-Measure works even in the case of class imbalance. [ABSTRACT FROM AUTHOR]

Machine learning has demonstrated remarkable success in the field of intelligent fault diagnosis. In current machine learning systems, models tend to be fixed after training, which makes them can only generalize to classes that appear in the training set, and cannot continuously learn newly emerging classes. However, in real industrial scenarios, industrial data is constantly growing, which requires models to be able to acquire new knowledge while retaining existing knowledge. Furthermore, a substantial proportion of the acquired samples in industry are unlabeled and only few samples are labeled. To tackle the above two challenges, this article proposes a novel semi-supervised class incremental broad network (SSCIBN) for incremental intelligent diagnosis of rotating machinery faults under limited labeled samples. Specifically, a semi-supervised graph embedding loss function is designed. On the one hand, this loss function can learn the structural information of a large amount of unlabeled data, which overcomes the limitation that the scarcity of labeled samples leads to poor model performance. On the other hand, local structure information within old and new classes is considered in the continuous learning process to fully learn the unique manifold structure information of different classes, which in turn enhances the discriminative performance between old and new classes. Furthermore, a novel semi-supervised class incremental learning mechanism is proposed, which does not need to utilize the old class data during the incremental learning process, but can effectively retain the old class knowledge. The effectiveness of the proposed method is evaluated through multiple mechanical failure increment cases. Experimental analysis demonstrates that the designed SSCIBN has significant advantages in the class incremental application with the lack of labeled samples. [ABSTRACT FROM AUTHOR]

Compared with conventional close-domain relation classification, incremental few-shot relation classification requires incrementally to learn novel relations through very few samples without forgetting base relations, which is more fitting to the needs of real scenarios. Existing incremental few-shot learning methods commonly leverage memory to freeze knowledge of the learned base classes, and then incrementally learn new classes from very few labeled samples. However, there are still two challenges to these existing approaches: overfitting novel relations and the catastrophic forgetting of base relations. In this paper, we propose a relational concept enhanced prototypical network (RC-EProto) to address these problems. First, we build a concept alignment module to eliminate the difference between relational concepts and samples, and we incorporate relational concepts for sample representation by this module to construct a more appropriate prototype. Second, we develop a contrastive approach on the base relations to address the forgetting of old knowledge. The samples of the base relation are pulled tightly with its prototype in the embedding space, while the samples of the other relations are pulled far from this base prototype. Extensive experiments on FewRel 1.0 and 2.0 datasets demonstrate the superiority of our method compared with the state-of-the-art methods. Notably, RC-EProto maintains an excellent incremental few-shot domain adaptation learning ability. Compared to existing methods, our model is able to achieve near-optimal results for the base relations while improving the classification accuracy of the novel relations by 13.48% and 9.63% for the 1-increment and 5-increment settings on FewRel 2.0, respectively. • We propose a relational concept enhanced prototypical network to address incremental few-shot relation classification. • We propose a base-class augmented contrastive learning method for forgetting problems. • We propose incorporating relational concepts as external knowledge to alleviate the overfitting problem. • Our proposed model achieves better results compared to the baseline models. [ABSTRACT FROM AUTHOR]

Abstract Multi-view learning has attracted increasing attention in recent years, and the existing multi-view learning methods learn a consensus result by collecting all views. These methods have two obvious limitations. First, it is not scalable; with limited computational resources it would be difficult, if not impossible, to collect and process a large collection of views together. Second, in many applications views of data are available over time; it is infeasible to apply the existing multi-view learning methods to such streaming views. To address the two limitations, in this paper we propose a novel incremental multi-view spectral clustering (IMSC) method. In IMSC, instead of ensembling the collection of all views simultaneously, we integrate them one by one in an incremental way. We first learn an initial model from a small number of views; next when a new view is available, we need only use it to update the model and apply the updated model to learn a consensus result. This method is scalable and applicable to streaming views. To further reduce the time and space complexity, we apply low rank approximation by means of the well-known random Fourier features to construct the base kernels and do low rank SVD decompositions accordingly. The theoretical analysis and experimental results on benchmark data sets show that our incremental multi-view spectral clustering method is significantly faster in efficiency than the existing state-of-the-art non-incremental ones and is comparable or even better in clustering quality. [ABSTRACT FROM AUTHOR]

With the massive explosion of machine learning in our day-to-day life, incremental and adaptive learning has become a major topic, crucial to keep up-to-date and improve classification models and their corresponding feature extraction processes. This paper presents a categorized overview of incremental feature extraction based on linear subspace methods which aim at incorporating new information to the already acquired knowledge without accessing previous data. Specifically, this paper focuses on those linear dimensionality reduction methods with orthogonal matrix constraints based on global loss function, due to the extensive use of their batch approaches versus other linear alternatives. Thus, we cover the approaches derived from Principal Components Analysis, Linear Discriminative Analysis and Discriminative Common Vector methods. For each basic method, its incremental approaches are differentiated according to the subspace model and matrix decomposition involved in the updating process. Besides this categorization, several updating strategies are distinguished according to the amount of data used to update and to the fact of considering a static or dynamic number of classes. Moreover, the specific role of the size/dimension ratio in each method is considered. Finally, computational complexity, experimental setup and the accuracy rates according to published results are compiled and analyzed, and an empirical evaluation is done to compare the best approach of each kind. [ABSTRACT FROM AUTHOR]

Short-term electric load forecasting plays an important role in the management of modern power systems. Improving the accuracy and efficiency of electric load forecasting can help power utilities design reasonable operational planning which will lead to the improvement of economic and social benefits of the systems. A hybrid incremental learning approach composed of Discrete Wavelet Transform (DWT), Empirical Mode Decomposition (EMD) and Random Vector Functional Link network (RVFL) is presented in this work. RVFL network is a universal approximator with good efficiency because of the randomly generated weights between input and hidden layers and the close form solution for parameter computation. By introducing incremental learning, along with ensemble approach via DWT and EMD into RVFL network, the forecasting performance can be significantly improved with respect to both efficiency and accuracy. The electric load datasets from Australian Energy Market Operator (AEMO) were used to evaluate the effectiveness of the proposed incremental DWT-EMD based RVFL network. Moreover, the attractiveness of the proposed method can be demonstrated by the comparison with eight benchmark forecasting methods. [ABSTRACT FROM AUTHOR]

Multigranulation rough sets have attracted more and more attentions in recent years. In real-life applications, with the development of information technology, the attribute values often dynamically evolve over time. How to update useful knowledge is of great importance for dynamic information systems. Approximations of a concept are fundamental concepts of multigranulation rough sets, which need to be updated incrementally while refining or coarsening attribute values. Motivated by the requirements of dynamic knowledge acquisition due to refining or coarsening attribute values, in this paper, we present the dynamic mechanisms for updating approximations in multigranulation rough sets while refining or coarsening attribute values. Then, the corresponding dynamic algorithms for updating multigranulation approximations are designed on the basis of the proposed mechanisms. Extensive experiments on six data sets from UCI demonstrate that the proposed dynamic algorithms for updating approximations in multigranulation rough sets are more effective in comparison with the static algorithm. [ABSTRACT FROM AUTHOR]

Set-valued information systems are important type of data tables in many real applications, where the attribute values are described by sets to characterize uncertain and incomplete information. However, in some real situations, set-values may be depicted by probability distributions, which results in that the traditional tolerance relation based on intersection operation could not reasonably describe the indiscernibility relation of objects. To address this issue, we introduce the concept of probabilistic set-valued information systems (PSvIS), and present the extended variable precision rough set model (VPRS) based on the λ -tolerance relation in terms of Bhattacharyya distance. Considering the features of information systems will evolve over time in a dynamic data environment, it will lead to the change of information granulation and approximation structures. A matrix representation of rough approximation is presented based on two matrix operators and two vector functions in PSvIS. Then incremental mechanisms by the utilization of previously learned approximation results and region relation matrices for updating rough approximations are proposed, and the corresponding algorithms are developed and analyzed. Experimental results show that the proposed algorithms outperform the static algorithms and related incremental algorithms while inserting into or removing from attributes in PSvIS. [ABSTRACT FROM AUTHOR]

In a dynamic environment, the data collected from real applications varies not only with the amount of objects but also with the number of features, which will result in continuous change of knowledge over time. The static methods of updating knowledge need to recompute from scratch when new data are added every time. This makes it potentially very time-consuming to update knowledge, especially as the dataset grows dramatically. Calculation of approximations is one of main mining tasks in rough set theory, like frequent pattern mining in association rules. Considering the fuzzy descriptions of decision states in the universe under fuzzy environment, this paper aims to provide an efficient approach for computing rough approximations of fuzzy concepts in dynamic fuzzy decision systems (FDS) with simultaneous variation of objects and features. We firstly present a matrix-based representation of rough fuzzy approximations by a Boolean matrix associated with a matrix operator in FDS. While adding the objects and features concurrently, incremental mechanisms for updating rough fuzzy approximations are introduced, and the corresponding matrix-based dynamic algorithm is developed. Unlike the static method of computing approximations by updating the whole relation matrix, our new approach partitions it into sub-matrices and updates each sub-matrix locally by utilizing the previous matrix information and the interactive information of each sub-matrix to avoid unnecessary calculations. Experimental results on six UCI datasets shown that the proposed dynamic algorithm achieves significantly higher efficiency than the static algorithm and the combination of two reference incremental algorithms. [ABSTRACT FROM AUTHOR]

Rolling bearings are susceptible to failure because of their complex and severe working environments. Deep learning-driven intelligent fault diagnosis methods have been widely introduced and exhibit satisfactory performance. However, bearing fault diagnosis during various working conditions is challenging; catastrophic forgetting occurs when test data are gathered under different conditions. In this paper, we develop an incremental learning-based multi-task shared classifier (IL-MTSC) for bearing fault diagnosis under various conditions. We use a one-dimensional convolutional neural network model as the principal framework. Then, we create a knowledge distillation method that allows the model to retain learned knowledge. Finally, we use a shared classifier that operates under various scenarios to establish a unified structure. The method can continuously learn and preserve knowledge when diagnoses bearing faults under various working conditions. The results show that IL-MTSC optimally diagnoses bearing marks under various conditions. • We develop an incremental learning-based multi-task shared classifier (IL-MTSC) for bearing fault diagnosis. • IL-MTSC can continuously learn new tasks with data under various conditions. • We compare performance of our method with several salient models on a benchmark dataset. • IL-MTSC exhibits superior effectiveness and accuracy. [ABSTRACT FROM AUTHOR]

With the continuous addition of new classes of data, incremental learning is widely followed and studied, especially for few-shot scenarios. Few-shot class incremental learning (FSCIL) aims to continually learn new knowledge with limited data, without forgetting old knowledge. Owing to limited new class samples and special incremental processes, FSCIL faces the problems of old knowledge catastrophic forgetting and new class adaption. We focus on a multi-feature space similarity supplement (MFS3) to alleviate the two dilemmas. We first train different feature spaces for different sessions, and use an inter-feature space similarity supplement (IFS3) to focus on boundary-sensitive sample points to improve the expression ability of every single session. To handle old knowledge catastrophic forgetting, we use the base feature space as the major and take multiple new feature spaces as supplements. For better adaption of new classes, we further design an outer-feature space similarity supplement (OFS3). OFS3 can utilize the supplement of base feature space with new feature space to rejudge the sample points, whose similarity results of base feature space are changed due to the addition of new classes. In general, IFS3 facilitates a single session, while OFS3 contributes a multi-feature space similarity supplement between different sessions in FSCIL. Sufficient experiments on three benchmark datasets, including CIFAR100, miniImageNet, and CUB200 demonstrate the advantage of our method. With the multi-feature space similarity supplement strategy, our method outperforms the state-of-the-art approaches by a large margin. [ABSTRACT FROM AUTHOR]

Probabilistic rough set model, which is established by incorporating the theory of probability into rough set theory, aims to model imprecise data with the tolerance of decision errors in terms of conditional probability and probabilistic parameters. The volume of data is frequently varied dynamically. It is very time consuming to analyze the updates of data incessantly from computation perspective. Incremental learning technique is desired to improve computational efficiency, which poses an incremental capability for adaptive knowledge maintenance to accommodate varied data. In this paper, we focus on efficient updating of probabilistic approximations with incremental objects in a dynamic information table. The dynamic characteristics of conditional partition and decision classification on the universe are analyzed when the insertion or deletion of objects occurs. Different updating patterns of conditional probability are presented for different combinatorial varieties of the conditional and decision classes. Meanwhile, incremental algorithms for updating probabilistic approximations are proposed, which are proficient to efficiently classify the incremental objects into decision regions by avoiding re-computation efforts. Experiments on benchmark data sets indicate that the proposed algorithms outperform the static algorithm in the presence of dynamic variation of the universe. [ABSTRACT FROM AUTHOR]

Rough set provides a theoretical framework for classification learning in data mining and knowledge discovery. As an important application of rough set, attribute reduction, also called feature selection, aims to reduce the redundant attributes in a given decision system while preserving a particular classification property, e.g., information entropy and knowledge granularity. In view of the dynamic changes of the object set in a decision system, in this paper, we focus on knowledge granularity-based attribute reduction approach when some objects vary dynamically. We first introduce incremental mechanisms to compute new knowledge granularity. Then, the corresponding incremental algorithms for attribute reduction are developed when some objects are added into and deleted from the decision system. Experiments conducted on different data sets from UCI show that the proposed incremental algorithm can achieve better performance than the non-incremental counterpart and incremental algorithm based on entropy. [ABSTRACT FROM AUTHOR]

Classification trees are a powerful tool for mining non-stationary data streams. In these situations, massive data are constantly generated at high speed and the underlying target function can change over time. The iadem family of algorithms is based on Hoeffding’s and Chernoff’s bounds and induces online decision trees from data streams, but is not able to handle concept drift. This study extends this family to deal with time-changing data streams. The new online algorithm, named iadem-3 , performs two main actions in response to a concept drift. Firstly, it resets the variables affected by the change and maintains unbroken the structure of the tree, which allows for changes in which consecutive target functions are very similar. Secondly, it creates alternative models that replace parts of the main tree when they significantly improve the accuracy of the model, thereby rebuilding the main tree if needed. An online change detector and a non-parametric statistical test based on Hoeffding’s bounds are used to guarantee this significance. A new pruning method is also incorporated in iadem-3 , making sure that all split tests previously installed in decision nodes are useful. The learning model is also viewed as an ensemble of classifiers, and predictions of the main and alternative models are combined to classify unlabeled examples. iadem-3 is empirically compared with various well-known decision tree induction algorithms for concept drift detection. We empirically show that our new algorithm often reaches higher levels of accuracy with smaller decision tree models, maintaining the processing time bounded, irrespective of the number of instances processed. [ABSTRACT FROM AUTHOR]

Decision-theoretic rough sets is a generalized probabilistic model for the expression of uncertainties and the representation of knowledge from data. It provides a semantic explanation and systematically computation of probabilistic thresholds to define probabilistic rough set approximations, which offers a ternary classification framework based on Bayesian decision theory. In practice, data for decision making process resides in a dynamic database whose data is typically evolving through the periodical or occasional updating, e.g. , new data are appended and obsolete data are removed. It is impractical to have a maturity decision model, stalled until the preparation of all helpful training data. To address this issue, incremental learning appeared to be a feasible solution for continuous knowledge modeling from evolving data with the incorporation of unlearned knowledge embedded in the updating data. In this paper, we exploit matrix approaches to study incremental decision-theoretic rough set approach for evolving data. Starting from the representation of object subset and indiscernibility relation in matrix form, we obtain a matrix characterization of probabilistic rough set approximations in decision-theoretic rough sets by using matrix properties associated with the multiplication operator. We also develop incremental algorithms for updating probabilistic rough set approximations with respect to the addition/deletion of objects, which enables decision theoretic rough sets to deal gracefully with evolving data. A detailed experimental study is conducted to examine the performance of the proposed incremental algorithms on UCI data sets. [ABSTRACT FROM AUTHOR]

Utilizing machine learning methods to detect intrusion into computer networks is a trending topic in information security research. The limitation of labeled samples is one of the challenges in this area. This challenge makes it difficult to build accurate learning models for intrusion detection. Transfer learning is one of the methods to counter such a challenge in machine learning topics. On the other hand, the emergence of new technologies and applications might bring new vulnerabilities to computer networks. Therefore, the learning process cannot occur all at once. Incremental learning is a practical standpoint to confront this challenge. This research presents a new framework for intrusion detection systems called ITL-IDS that can potentially start learning in a network without prior knowledge. It begins with an incremental clustering algorithm to detect clusters' numbers and shape without prior assumptions about the attacks. The outcomes are candidates to transfer knowledge between other instances of ITL-IDS. In each iteration, transfer learning provides target environments with incremental knowledge. Our evaluation shows that this method can combine incremental and transfer learning to identify new attacks. [ABSTRACT FROM AUTHOR]

As news events on the same subject occur, our knowledge about the subject will accumulate and become more comprehensive. In this paper, we formally define the problem of incremental knowledge learning from similar news events on the same subject, where each event consists of a set of news articles reporting about it. The knowledge is represented by a topic hierarchy presenting topics at different levels of granularity. Though topic (hierarchy) mining from text has been researched a lot, incremental learning from similar events remains under developed. In this paper, we propose a scalable two-phase framework to incrementally learn a topic hierarchy for a subject from events on the subject as the events occur. First, we recursively construct a topic hierarchy for each event based on a novel topic model considering the named entities and entity types in news articles. Second, we incrementally merge the topic hierarchies through top-down hierarchical topic alignment. Extensive experimental results on real datasets demonstrate the effectiveness and efficiency of the proposed framework in terms of both qualitative and quantitative measures. [ABSTRACT FROM AUTHOR]

Few-shot incremental intent recognition aims at continually identifying users' intents from utterances with limited labeled novel data. To mitigate the effect of catastrophic forgetting inherent in incremental learning, existing methods generally resort to storing informative base exemplars as the model memory that are further replied to when learning novel classes. However, merely preserving these base exemplars and ignoring the relationship between them can easily trigger some concerns in the incremental training process. First, in each iteration, the participation of novel samples further updates the embedding space to make the novel labels well separated, which instead poisons the learned labels, reducing their separability. Second, the overfitting risk is further amplified in incremental learning, causing models to be overconfident on seen base classes but hardly generalized to unseen novel classes. In view of these problems, a geometry-aware learning (GAL) model is proposed to address few-shot incremental intent recognition and bridge the two research gaps mentioned above. Specifically, in our proposal, for the embedding shifting issue, GAL constructs a geometric structure of selected exemplars based on their spatial distribution in the embedding space, which is treated as a strong constraint in the following training. For the overfitting issue, GAL introduces an episodic-based pretraining strategy as well as a multisource contrastive-based loss to enhance the lack of supervised signals in the classification of data-scarce novel labels. Experimental results on a public dataset OOS (CLINC-150) verify the effectiveness of our proposal by beating the state-of-the-art baselines. Specifically, our model outperforms the best baseline by 1.34% and 0.39% on the 5-way 1-shot meta task for the base and novel classes, respectively. It also outperforms the best baseline by 1.89% and 0.80% on the 5-way 5-shot meta task for the base and novel classes, respectively. Furthermore, cross-domain experiments between two datasets (CLINC-150 and ATIS) reflect that GAL has better generalization ability across different domains. The method is superior to traditional intent recognition models in the application scenario where the model is required to accurately distinguish both new and trained categories under the low-data dilemma. Analogously, this common application scenario can also provide a broad developing direction for the dialog recommender system, which helps it recommend continuously and accurately with only a few labeled samples. [ABSTRACT FROM AUTHOR]

Abstract: Incremental learning is an efficient technique for knowledge discovery in a dynamic database, which enables acquiring additional knowledge from new data without forgetting prior knowledge. Rough set theory has been successfully used in information systems for classification analysis. Set-valued information systems are generalized models of single-valued information systems, which can be classified into two categories: disjunctive and conjunctive. Approximations are fundamental concepts of rough set theory, which need to be updated incrementally while the object set varies over time in the set-valued information systems. In this paper, we analyze the updating mechanisms for computing approximations with the variation of the object set. Two incremental algorithms for updating the approximations in disjunctive/conjunctive set-valued information systems are proposed, respectively. Furthermore, extensive experiments are carried out on several data sets to verify the performance of the proposed algorithms. The results indicate the incremental approaches significantly outperform non-incremental approaches with a dramatic reduction in the computational speed. [Copyright &y& Elsevier]

Abstract: The complete linear discriminant analysis (CLDA) algorithm has been successfully employed for face recognition. The CLDA method can make full use of the discriminant information of the training samples. However, CLDA suffers from the scalability problem. In this paper, we propose an incremental CLDA (ICLDA) to overcome this limitation. We first propose a new implementation of CLDA in which two steps of QR decomposition, rather than singular value decomposition (SVD), are used to get the orthonormal bases of the range and null spaces of the within-class scatter matrix. Then, by using efficient QR-updating technique, we propose the ICLDA method which can accurately incrementally update the discriminant vectors of CLDA instead of recomputing the CLDA again. Experiments on PIE and FERET face databases show the efficiency of our proposed CLDA algorithms over the original implementation of CLDA. [Copyright &y& Elsevier]

Abstract: The Matrix-Factorization (MF) based models have become popular when building Collaborative Filtering (CF) recommenders, due to the high accuracy and scalability. However, most of the current MF based models are batch models that are incapable of being incrementally updated; while in real world applications users always enjoy receiving quick responses from the system once they have made feedbacks. In this work, we aim to design an incremental CF recommender based on the Regularized Matrix Factorization (RMF). To achieve this objective, we first simplify the training rule of RMF to propose the SI-RMF, which provides a simple mathematic form for further investigation; whereby we design two Incremental RMF models, respectively are the Incremental RMF (IRMF) and the Incremental RMF with linear biases (IRMF-B). The experiments on two large, real datasets suggest positive results, which prove the efficiency of our strategy. [Copyright &y& Elsevier]

Abstract: In a typical application of association rule learning from market basket data, a set of transactions for a fixed period of time is used as input to rule learning algorithms. For example, the well-known Apriori algorithm can be applied to learn a set of association rules from such a transaction set. However, learning association rules from a set of transactions is not a one time only process. For example, a market manager may perform the association rule learning process once every month over the set of transactions collected through the last month. For this reason, we will consider the problem where transaction sets are input to the system as a stream of packages. The sets of transactions may come in varying sizes and in varying periods. Once a set of transactions arrive, the association rule learning algorithm is executed on the last set of transactions, resulting in new association rules. Therefore, the set of association rules learned will accumulate and increase in number over time, making the mining of interesting ones out of this enlarging set of association rules impractical for human experts. We refer to this sequence of rules as “association rule set stream” or “streaming association rules” and the main motivation behind this research is to develop a technique to overcome the interesting rule selection problem. A successful association rule mining system should select and present only the interesting rules to the domain experts. However, definition of interestingness of association rules on a given domain usually differs from one expert to another and also over time for a given expert. This paper proposes a post-processing method to learn a subjective model for the interestingness concept description of the streaming association rules. The uniqueness of the proposed method is its ability to formulate the interestingness issue of association rules as a benefit-maximizing classification problem and obtain a different interestingness model for each user. In this new classification scheme, the determining features are the selective objective interestingness factors related to the interestingness of the association rules, and the target feature is the interestingness label of those rules. The proposed method works incrementally and employs user interactivity at a certain level. It is evaluated on a real market dataset. The results show that the model can successfully select the interesting ones. [Copyright &y& Elsevier]

Abstract: Machine-learning research is to study and apply the computer modeling of learning processes in their multiple manifestations, which facilitate the development of intelligent system. In this paper, we have introduced a clustering based machine-learning algorithm called clustering algorithm system (CAS). The CAS algorithm is tested to evaluate its performance and find fruitful results. We have been presented some heuristics to facilitate machine-learning authors to boost up their research works. The InfoBase of the Ministry of Civil Services is used to analyze the CAS algorithm. The CAS algorithm is compared with other machine-learning algorithms like UNIMEM, COBWEB, and CLASSIT, and was found to have some strong points over them. The proposed algorithm combined advantages of two different approaches to machine learning. The first approach is learning from Examples, CAS supports Single and Multiple Inheritance and Exceptions. CAS also avoids probability assumptions which are well understood in concept formation. The second approach is learning by Observation. CAS applies a set of operators that have proven to be effective in conceptual clustering. We have shown how CAS builds and searches through a clusters hierarchy to incorporate or characterize an object. [Copyright &y& Elsevier]

In this paper, a hybrid learning approach named hybrid decision tree (HDT) is proposed. HDT simulates human reasoning by using symbolic learning to do qualitative analysis and using neural learning to do subsequent quantitative analysis. It generates the trunk of a binary HDT according to the binary information gain ratio criterion in an instance space defined by only original unordered attributes. If unordered attributes cannot further distinguish training examples falling into a leaf node whose diversity is beyond the diversity-threshold, then the node is marked as a dummy node. After all those dummy nodes are marked, a specific feedforward neural network named Fannc that is trained in an instance space defined by only original ordered attributes is exploited to accomplish the learning task. Moreover, this paper distinguishes three kinds of incremental learning tasks. Two incremental learning procedures designed for example-incremental learning with different storage requirements are provided, which enables HDT to deal gracefully with data sets where new data are frequently appended. Also a hypothesis-driven constructive induction mechanism is provided, which enables HDT to generate compact concept descriptions. [Copyright &y& Elsevier]

Incremental learning strategy based feature selection approaches can improve the efficiency of reduction algorithm used for datasets with dynamic characteristic, which has attracted increasing research attention. Nevertheless, there is currently no work on incremental feature selection approaches for dynamic interval-valued ordered data. Interval-valued ordered data is a generalized form of single-valued ordered data, which is more widely used in practice. However, the endpoints of the interval numbers are easily polluted by noise, thereby the knowledge granules are very sensitive. Motivated by these two issues, we study incremental feature selection approaches based on a fuzzy dominance neighborhood rough set (FDNRS) for dynamic interval-valued ordered data in this work. First, we propose the FDNRS model for an interval-valued ordered decision system (IvODS) and investigate its related properties. Second, a conditional entropy with robustness is proposed based on the proposed model. This conditional entropy can measure the degree of monotonic consistency of the IvODS, so it is used as a metric and combined with a heuristic feature selection algorithm. Finally, two incremental feature selection algorithms are proposed on the basis of the above researches. Experiments are performed on nine public datasets to evaluate the robustness of the proposed metric and the performance of the incremental algorithms. Experimental results verify that the proposed metric is robust and our incremental algorithms are effective and efficient for updating reducts in dynamic IvODS. [ABSTRACT FROM AUTHOR]

With the vigorous development of the information technology industry, the information data available to mankind has shown an explosive growth trend. Dynamic concept learning is an approach that can effectively process the acquired massive data and extract valuable information from them. Concept cognitive learning (CCL) is a very active research direction in the field of dynamic concept learning, while partial order formal structure analysis (POFSA) is a concrete and practical model of CCL. However, the existing CCL algorithms in POFSA face some challenges when processing constantly changing data. Therefore, this paper is devoted to explore an incremental CCL algorithm based on three-way object partial order structure diagram (OPOSD) in POFSA with the incorporation of the thoughts of incremental learning. The features of five object categories are considered, and their incremental influences on three-way OPOSD are analyzed and their incremental CCL algorithms in three-way OPOSD are established. Based on some real famous formal contexts, this paper conducts numerical experiments, and the results show that the incremental CCL algorithm based on three-way OPOSD is consistent with human cognitive principles, and can improve the CCL performance of POFSA as well. [ABSTRACT FROM AUTHOR]

Cross-domain object detection uses knowledge from source domain tasks to enhance the object detection in target domain. It can reduce the workload of data annotations in the new domain and significantly improve the adaptation ability of the network. We consider a more realistic transfer scenario, that is, our target domain samples cannot be obtained at the same time so that learners cannot forget the old domain knowledge while learning the new domain knowledge, which is consistent with the basic assumption of incremental learning. However, the existing methods do not consider the sequential learning process of the multiple target domains which may cause performance degradation. To address this issue, we propose a novel multi-domain adaptation method for object detection based on incremental learning. Specifically, the incremental learning network saves the knowledge of multiple domains and makes the model to fuse the knowledge of different domains during the training effectively. To make it work better, the progressive training strategy is proposed to make the model gradually adapt to multiple domains. Moreover, we use a multi-level feature alignment module to ensure that domain alignment is realized on features at various levels. We perform experiments on two sets of 6 datasets, which demonstrate that our model can effectively solve the problem of domain knowledge forgetting in multi-target domain adaptation and significantly improve the detection accuracy in each domain. [ABSTRACT FROM AUTHOR]

In Deep Learning, training a model properly with a high quantity and quality of data is crucial in order to achieve a good performance. In some tasks, however, the necessary data is not available at a particular moment and only becomes available over time. In which case, incremental learning is used to train the model correctly. An open problem remains, however, in the form of the stability–plasticity dilemma: how to incrementally train a model that is able to respond well to new data (plasticity) while also retaining previous knowledge (stability). In this paper, an incremental learning model inspired in Rehearsal (recall of past memories based on a subset of data) named CRIF is proposed, and two instances for the framework are employed — one using a random-based selection of representative samples (Naive Incremental Learning, NIL), the other using Crowding Distance and Best vs. Second Best metrics in conjunction for this task (RILBC). The experiments were performed on five datasets — MNIST, Fashion-MNIST, CIFAR-10, Caltech 101, and Tiny ImageNet, in two different incremental scenarios: a strictly class-incremental scenario, and a pseudo class-incremental scenario with unbalanced data. In Caltech 101, Transfer Learning was used, and in this scenario as well as in the other three datasets, the proposed method, NIL, achieved better results in most of the quality metrics than comparison algorithms such as RMSProp Inc (base line) and iCaRL (state-of-the-art proposal) and outperformed the other proposed method, RILBC. NIL also requires less time to achieve these results. • An incremental learning model inspired in Rehearsal (recall of past memories based on a subset of data) is proposed. • Experiments were performed over MNIST, Fashion-MNIST, CIFAR-10 and Caltech 101 in two different scenarios. • Several metrics were used to compare learning quality results when each new megabatch of data is used. • Friedman's non-parametric statistical test and Holm post-hoc test were used for supporting the analysis of the results. • Random-based selection of representative samples obtains the best results. [ABSTRACT FROM AUTHOR]

Double-quantitative decision-theoretic rough sets (Dq-DTRS) provide more comprehensive description methods for rough approximations of concepts, which lay foundations for the development of attribute reduction and rule extraction of rough sets. Existing researches on concept approximations of Dq-DTRS pay more attention to the equivalence class of each object in approximating a concept, and calculate concept approximations from the whole data set in a batch. This makes the calculation of approximations time consuming in dynamic data sets. In this paper, we first analyze the variations of equivalence classes, decision classes, conditional probability, internal grade and external grade in dynamic data sets while objects vary sequentially or simultaneously over time. Then we propose the updating mechanisms for the concept approximations of two types of Dq-DTRS models from incremental perspective in dynamic decision information systems with the sequential and batch variations of objects. Meanwhile, we design incremental sequential insertion, sequential deletion, batch insertion, batch deletion algorithms for two Dq-DTRS models. Finally, we present experimental comparisons showing the feasibility and efficiency of the proposed incremental approaches in calculating approximations and the stability of the incremental updating algorithms from the perspective of the runtime under different inserting and deleting ratios and parameter values. • We study dynamic maintenance approaches for the approximations of Dq-DTRS. • We propose incremental updating mechanisms of Dq-DTRS with the variation of objects. • We design incremental sequential and batch updating algorithms for Dq-DTRS models. • The validity, stability and efficiency of our methods are verified by comparisons. [ABSTRACT FROM AUTHOR]