Search

A learner model with fast learning and compact architecture is expected for industrial data modeling. To achieve these goals during stochastic configuration networks (SCNs) construction, we propose an improved version of SCNs in this paper. Unlike the original SCNs, the improved one employs a new inequality constraint in the construction process. In addition, to speed up the construction efficiency of SCNs, a node selection method is proposed to adaptively select nodes from a candidate pool. Moreover, to reduce the redundant nodes of the built SCNs model, we further compress the model based on the singular value decomposition algorithm. The improved SCNs are compared with other methods over four datasets and then applied to the ammonia–nitrogen concentration prediction task in the wastewater treatment process. Experimental results indicate that the proposed method has good potential for industrial data analytics. [ABSTRACT FROM AUTHOR]

The optimal balance and dispatch of power plants in a smart grid require an accurate short-term forecast of photovoltaic (PV) power generation. The climatic condition may have an impact on the PV output, but it is difficult to be used in forecasting due to untimely sampling of meteorological data. To this end, this paper presents an incremental learning using privileged information (LUPI) paradigm for PV power forecasting by using stochastic configuration network. This novel algorithm can employ the meteorological data as privileged information for building PV power forecasting model in the training stage. Additionally, the model performance has been fully discussed in this paper. Finally, experimental results indicate that the proposed model indeed performs favorably in PV power forecasting. [ABSTRACT FROM AUTHOR]

Deep neural networks have achieved breakthrough improvement in various application fields. Nevertheless, they usually suffer from a time-consuming training process because of the complicated structures of neural networks with a huge number of parameters. As an alternative, a fast and efficient discriminative broad learning system (BLS) is proposed, which takes the advantages of flat structure and incremental learning. The BLS has achieved outstanding performance in classification and regression problems. However, the previous studies ignored the reason why the BLS can generalize well. In this article, we focus on the interpretation from the viewpoint of the frequency domain. We discover the existence of the frequency principle in BLS, i.e., the BLS preferentially captures low-frequency components quickly and then fits the high frequencies during the incremental process of adding feature nodes and enhancement nodes. The frequency principle may be of great inspiration for expanding the application of BLS. [ABSTRACT FROM AUTHOR]

There are two main categories of face sketch synthesis: data- and model-driven. The data-driven method synthesizes sketches from training photograph–sketch patches at the cost of detail loss. The model-driven method can preserve more details, but the mapping from photographs to sketches is a time-consuming training process, especially when the deep structures require to be refined. We propose a face sketch synthesis method via regularized broad learning system (RBLS). The broad learning-based system directly transforms photographs into sketches with rich details preserved. Also, the incremental learning scheme of broad learning system (BLS) ensures that our method easily increases feature mappings and remodels the network without retraining when the extracted feature mapping nodes are not sufficient. Besides, a Bayesian estimation-based regularization is introduced with the BLS to aid further feature selection and improve the generalization ability and robustness. Various experiments on the CUHK student data set and Aleix Robert (AR) data set demonstrated the effectiveness and efficiency of our RBLS method. Unlike existing methods, our method synthesizes high-quality face sketches much efficiently and greatly reduces computational complexity both in the training and test processes. [ABSTRACT FROM AUTHOR]

• We introduce the mechanism of a binary tree and propose a parallel incremental acceleration strategy based on the attribute tree. • The branch threshold coefficient is added into the calculation process to guide the algorithm to jump out of the loop, avoid redundant calculations, and reduce the number of attribute evaluations. • When multiple incremental objects are added to the decision system, the incremental mechanism can be used to update the reduction. • We combine IARAT and Spark parallel technology to parallelize data processing to accelerate the calculation process. Attribute reduction is an important application of rough sets. Efficiently reducing massive dynamic data sets quickly has always been a major goal of researchers. Traditional incremental methods focus on reduction by updated approximations. However, these methods must evaluate all attributes and repeatedly calculate their importance. When these algorithms are applied to large datasets with high time complexity, reducing large decision systems becomes inefficient. We propose an incremental acceleration strategy based on attribute trees to solve this problem. The key step is to cluster all attributes into multiple trees for incremental attribute evaluation. Specifically, we first select the appropriate attribute tree for attribute evaluation according to the attribute tree correlation measure to reduce the time complexity. Next, the branch coefficient is added to the stop criterion, increasing with the branch depth and guiding a jump out of the loop after reaching the maximum threshold. This avoids redundant calculation and improves efficiency. Furthermore, we propose an algorithm for incremental attribute reduction based on attribute trees using these improvements. Finally, a Spark parallel mechanism is added to parallelize data processing to implement the parallel incremental efficient attribute reduction based on the attribute tree. Experimental results on the Shuttle dataset show that the time consumption of our algorithm is more than 40% lower than that of the classical IARC algorithm while maintaining its good classification performance. In addition, the time is shortened by more than 87% from the benchmark after adding the Spark parallelizing mechanism. [ABSTRACT FROM AUTHOR]

• The CSWBLS is proposed for imbalanced heartbeat classification. • The CSWBLS constructs the least squares error term with specific weight by class. • Minority classes get higher weights to increase their contribution to the model. • New incremental learning algorithms of CSWBLS for additional nodes are developed. • The proposed CSWBLS has similar learning efficiency but better performance than BLS. Accurate classification of heartbeats is essential for the treatment of cardiac arrhythmia. In real-world applications, many existing methods suffer from the imbalance between heartbeat classes since the number of normal heartbeats heavily outnumbers that of abnormal heartbeats. To address the class imbalance problem effectively, this study proposes a novel class-specific weighted broad learning system (CSWBLS) based on the broad learning system (BLS). The proposed CSWBLS constructs the least squares error term by class and uses weights to constrain the contribution of each class to the model. The weights are computed by employing class distribution and fine-tuning with preset scale factors. Minority classes are allocated higher weights to increase their contributions. Furthermore, the proposed CSWBLS can be quickly remodeled by our newly developed incremental learning algorithms when additional nodes are added. The proposed CSWBLS improves the generalization performance for imbalanced classification tasks while inheriting the fast learning efficiency compared to traditional BLS. To validate the proposed method, we conducted experiments on five imbalanced heartbeat datasets obtained from the MIT-BIH Arrhythmia Database. The results show that the overall accuracy and G-mean of our method exceed 99% and 97%, respectively, outperforming traditional BLS and some state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A new incremental clustering and density-based outlier detection method is proposed that simultaneously performs both clustering and outlier detection. • To the best of our knowledge, this is the first study to combine the concepts of incremental DBSCAN (iDBSCAN) and iLOF to detect outliers from streaming data. • To minimize the negative effects of the selection of parameters, iLDCBOF automatically adjusts its own hyperparameters for different, real-time applications. • To detect outliers from data streams and prevent their clustering, a newly-developed, core kNN (CkNN) concept is introduced. • The incremental Mahalanobis metric is used in all distance computations to reduce the impact of the data dimensions in both iLOF and iDBSCAN. In this paper, a novel, parameter-free, incremental local density and cluster-based outlier factor (iLDCBOF) method is presented that unifies incremental versions of local outlier factor (LOF) and density-based spatial clustering of applications with noise (DBSCAN) to detect outliers efficiently in data streams. The iLDCBOF has many advanced advantages compared to previously reported iLOF-based studies: (1) it is based on a newly-developed core k-nearest neighbor (CkNN) concept to reliably and scalably detect outliers from data streams and prevent the clustering of outliers; 2) it uses a newly-developed algorithm that automatically adjusts the value of the k (number of neighbors) parameter for different real-time applications; and 3) it uses the Mahalanobis distance metric, so its performance is not affected even for large amounts of data. The iLDCBOF method is well suited for different data stream applications because it requires no distribution assumptions, it is parameterless (determined automatically), and it is easy to implement. ROC-AUC and statistical test analysis results from extensive experiments performed on 16 different real-world datasets showed that the iLDCBOF method significantly outperformed benchmark methods. [ABSTRACT FROM AUTHOR]

In the open-world environment, the incremental updating approaches to attribute reduction based on rough sets are efficient and effective to evaluate and search an optimal subset of attributes from two-dimensionally time-evolving data, which can be interpreted as the complex changes of dynamic data, i.e., four types of combinations induced by the insertion/deletion of objects and the addition/remove of attributes. To avoid the time-consuming and repetitive computation from scratch in such dynamic data, this paper mainly focuses on constructing a unified incremental framework to attribute reduction by the matrix-based accelerated updating strategies. We systematically discuss and present a series of incremental updating mechanisms and algorithms of approximation quality in the neighborhood-based probabilistic rough sets. Besides, a unified framework of dynamic attribute reduction in four situations of changes is proposed to develop the performance of updating reduct. Finally, we report the comparative experiments between the non-incremental and incremental algorithms of reduct to demonstrate the feasibility and efficiency of proposed approaches. [ABSTRACT FROM AUTHOR]

Class incremental learning has made great progress in solving the problem of catastrophic forgetting through knowledge distillation method and sample playback method. However, the existing class incremental learning methods still face the problems of limited feature representation and lack of normalized feature space, which makes them perform poorly in long-term incremental tasks. To address the above problems in class incremental learning, we propose a non-exemplar based method named Hybrid Rotation with Feature Space Normalization (HRFSN). Firstly, a novel self-supervised method called Hybrid Rotation Self-supervision (HRS) is designed to overcome the problem of limited features. HRS uses random positive samples to perform rotation prediction tasks, and makes the feature extractor learn more rich feature expression ability through complex rotation prediction tasks. Secondly, to make the learned features more generalized, Feature Space Normalization (FSN) is introduced to constrain the feature value to a normal distribution, which is well matched with HRS. Experimental results on benchmark datasets such as CIFAR-100 and Tiny-Imagenet show that our approach significantly outperforms mainstream incremental learning methods and achieves comparable performance compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Multimodal multi-objective optimization problems aim to identify multiple equivalent Pareto-optimal solution sets in the decision space, each associated with the same Pareto front in the objective space. It remains challenging to combine multimodal multi-objective optimization algorithms with appropriate machine learning techniques to balance diversity and convergence and acquire complete and evenly distributed Pareto-optimal solution sets and fronts. Therefore, we propose a particle swarm optimizer named TCAPSO, which is based on topological clustering via adaptive resonance theory. This algorithm leverages incremental learning to construct a topology, which adaptively learns from new solutions while retaining previously valuable information. The knowledge accumulated within the topology is fully applied during the algorithmic iterations, wherein a learnable solution generator and discriminator are developed. The generator advances solutions toward promising regions by employing a neighborhood construction strategy that utilizes topological node adjacency. The discriminator selects solutions by integrating historical crowding information from topological nodes in the decision space with crowding distance in the objective space, ensuring that the solutions are uniformly distributed across both spaces. In comparative experiments across 34 multimodal multi-objective benchmark problems, TCAPSO outperformed 10 advanced multimodal multi-objective algorithms, surpassing the closest competitors by 21.64% and 10.95% respectively in the decision and objective spaces. [ABSTRACT FROM AUTHOR]

Artificial neural networks, celebrated for their human-like cognitive learning abilities, often encounter the well-known catastrophic forgetting (CF) problem, where the neural networks lose the proficiency in previously acquired knowledge. Despite numerous efforts to mitigate CF, it remains the significant challenge particularly in complex changing environments. This challenge is even more pronounced in cross-domain adaptation following the continual learning (CL) setting, which is a more challenging and realistic scenario that is under-explored. To this end, this article proposes a cross-domain CL approach making possible to deploy a single model in such environments without additional labelling costs. Our approach, namely continual learning approach for many processes (CLAMP), integrates a class-aware adversarial domain adaptation strategy to align a source domain and a target domain. An assessor-guided learning process is put forward to navigate the learning process of a base model assigning a set of weights to every sample controlling the influence of every sample and the interactions of each loss function in such a way to balance the stability and plasticity dilemma thus preventing the CF problem. The first assessor focuses on the negative transfer problem rejecting irrelevant samples of the source domain while the second assessor prevents noisy pseudo labels of the target domain. Both assessors are trained in the meta-learning approach using random transformation techniques and similar samples of the source domain. Theoretical analysis and extensive numerical validations demonstrate that CLAMP significantly outperforms established baseline algorithms across all experiments by at least 10% margin. • This paper presents a scarcely addressed problem, cross-domain continual learning. • This paper proposes a new algorithm, continual learning approach for many processes (CLAMP), for cross-domain continual learning. • Extensive numerical validations and theoretical studies are performed to guarantee the advantage of CLAMP. [ABSTRACT FROM AUTHOR]

In data-driven modeling, the generalization performance of a neural network based on the minimum mean square error criterion may be affected when the samples contain noise. To handle uncertain data modeling problems, this study contributes a novel robust data modeling method called information entropy-based fuzzy stochastic configuration networks (IEFSCNs), which incorporates the maximum correntropy criterion (MCC) to reflect the squared error of the training sample set and uses the quantized minimum error entropy (QMEE) criterion to mine the distribution structure of the modeling data and suppress outlier data. Fuzzy systems are designed to generate nodes in the fuzzy transformation layer. A nonlinear enhancement layer is established as a hidden layer to enhance the representational capacity of the target system. The fuzzy transformation and nonlinear enhancement layers are fully connected to determine the outputs of the IEFSCNs. In addition, an incremental parameter learning algorithm is developed based on a supervision mechanism for optimizing the cost function, and its convergence property is demonstrated. Owing to the superiorities of the cost function by synthesizing MCC and QMEE, IEFSCNs can improve the modeling accuracy and reduce noise suppression. The performances are evaluated using various regression and classification tasks. Compared with existing methods, including SCNs, RVFL, F-SCNs, RSC-KDE, RSC-MCC, and SM-RSC, the proposed method demonstrates an effective robust modeling performance by reducing network structure redundancy while enhancing its ability to suppress noisy data. [ABSTRACT FROM AUTHOR]

• To handle the uncertainty decision problems in incomplete hybrid data, a generalized three-way neighborhood decision model is proposed by distributing the interval-valued loss function to each object and averaging the interval-valued loss functions of all objects in the data-driven neighborhood class. • A matrix-based approach for representing three-way regions in the generalized three-way neighborhood decision model is presented by introducing the matrix forms of related concepts and the matrix operators. • An efficient framework for dynamically updating the three-way regions is provided when objects and attributes increase simultaneously. • An incremental algorithm based on matrix is designed for maintaining the three-way regions. • Experimental results demonstrate that the proposed incremental algorithm has an advantage in improving the computational performance. The theory of three-way decisions, as a powerful methodology of granular computing, has been widely used in making decision under uncertainty environments. Decision tasks in incomplete hybrid data including heterogeneous and missing features are of abundance in realistic situations. To deal with these tasks, some work based on three-way decisions has been investigated. However, the losses used for evaluating objects are precise real numbers, which makes these decision models have some limitations in applications when there exist missing values in incomplete hybrid data. Thus, this paper constructs a generalized three-way neighborhood decision model by assigning the interval-valued loss function to each object and further adopting an average strategy to integrate the interval-valued loss functions of objects in each data-driven neighborhood class. Moreover, considering that the objects and attributes of incomplete hybrid data will simultaneously change over time, this paper also provides an efficient framework to dynamically maintain three-way regions of the proposed model. An approach based on matrix to compute the three-way regions is first presented by introducing the matrix operations and the matrix forms of related concepts. Then, with the simultaneous variation of objects and attributes, the matrix-based incremental mechanism and algorithm are proposed for updating the three-way regions, respectively. Experimental results on nine datasets indicate that the proposed incremental algorithm can effectively improve the computational performance for evolving data in comparison with the static algorithm. [ABSTRACT FROM AUTHOR]

In dynamic and open data environment, how to improve the performance of reduction is of great importance from incremental evaluation of attributes and quick search of attributes. In this paper, by considering both two perspectives, we first combine the incremental technology and the accelerated strategy in attribute reduction. On the one hand, we utilize the stable attribute group generated by DBSCAN to accelerate the process of searching reduction. On the other hand, we propose the matrix-based incremental mechanisms to dynamic attribute reduction when the objects are evolved over time. Moreover, these two methods are fused together in a unified algorithm of reduction. Finally, a series of comparative experiments is conducted to verify the effectiveness of proposed approach from stability, computational cost, and classification accuracy. [ABSTRACT FROM AUTHOR]

• We consider an incremental learning mechanism based on progressive fuzzy three-way concept. • We design a novel classification mechanism and develop its corresponding algorithm for individual recognition. • For the changing individual, a dynamic update mechanism is studied for progressive concepts. • Effectiveness of the proposed mechanism is verified by comparing with classification mechanism. Fuzzy concept has been an important methodology for data analysis, especially in the classification research. Particularly, fuzzy concept could directly process the continuous data through contrasting the numerical data into the membership degree of object to attribute. However, the classical fuzzy concept only focuses on the positive information, that is, the information about membership degree, while ignoring non-membership degree. Meanwhile, since the limitations of individual cognition and cognitive environment, the concept learning is progressive. Inspired by these thoughts, we design an incremental learning mechanism based on progressive fuzzy three-way concept for object classification in dynamic environment. In this paper, the object and attribute learning operators are first defined to obtain fuzzy three-way concept. Then, a progressive fuzzy three-way concept and its corresponding concept space are learned considering the progressive process of concept learning. Moreover, the object classify mechanism and dynamic update mechanism based on the progressive concept space are proposed, and their effectiveness is verified by numerical experiments. Finally, an incremental learning mechanism is further designed for dynamic increased data and compared with other fuzzy classify methods. All the experimental results carried on ten datasets from UCI and KEEL illustrate the proposed learning mechanism is an excellent object classify algorithm. [ABSTRACT FROM AUTHOR]

The data collected from the real world are diverse and include categorical data, numerical data, incomplete data and noisy data. In addition, many real data sets may dynamically vary, and dynamic data display characteristics with multi-dimensional variations. However, for mixed incomplete data systems, most of the existing incremental methods only work well with single-dimensional dynamic data sets and are not suitable for processing specific multi-dimensional variations of objects and attributes. In this paper, we focus on researching dynamic approaches to efficiently update three-way regions based on the simultaneous variations of the object set and the attribute set in an incomplete neighborhood decision system (INDS). First, considering the complexity of data, we utilize matrix approaches to calculate three-way regions of the INDS based on a proposed neighborhood tolerance relation. Then, under the simultaneous addition of the object set and the attribute set in the INDS, we research incremental mechanisms based on the matrix to obtain three-way regions from previous knowledge. Subsequently, an incremental algorithm for updating three-way regions is proposed when the object set and the attribute set are simultaneously added to the INDS. Finally, the results of a series of experiments and comparisons based on UCI data sets show that the performance of the proposed incremental algorithm is much better than that of the traditional static algorithm, the integrated single-dimensional incremental algorithm and the single-level combined incremental algorithm. [ABSTRACT FROM AUTHOR]

In order to quickly discover the low-dimensional representation of high-dimensional noisy data in online environments, we transform the linear dimensionality reduction problem into the problem of learning the bases of linear feature subspaces. Based on that, we propose a fast and robust dimensionality reduction framework for incremental subspace learning named evolutionary orthogonal component analysis (EOCA). By setting adaptive thresholds to automatically determine the target dimensionality, the proposed method extracts the orthogonal subspace bases of data incrementally to realize dimensionality reduction and avoids complex computations. Besides, EOCA can merge two learned subspaces that are represented by their orthonormal bases to a new one to eliminate the outlier effects, and the new subspace is proved to be unique. Extensive experiments and analysis demonstrate that EOCA is fast and achieves competitive results, especially for noisy data. [ABSTRACT FROM AUTHOR]

Fuzzy rough sets (FRS) provides effective ways for selecting the compact/relevant feature subset for hybrid decision systems. However, the underlying complexity of the existing FRS methods through batch processing is often costly or intractable on large data and also suffer from continuous model adaptation on dynamic data. This paper proposes a FRS based incremental feature subset selection (IvFMFRS) framework using fuzzy min-max neural network (FMNN) as a preprocessor step in aiding to deal with data dynamically without sacrificing classification performance. FMNN is a single epoch learning algorithm employed to construct fuzzy hyperboxes (information granules) of pattern spaces very fast. Fuzzy hyperboxes facilitate the formation of interval-valued decision system (IDS) from the numerical decision system of much smaller size. In IvFMFRS, on each sample subset arrival, an incremental mechanism for updating fuzzy discernibility matrix (FDM) based on constructed IDS is first formulated and then update feature subset by adding and deleting features based on updated FDM. A comparative analysis has been conducted comprehensively to assess the performance of the proposed algorithm with the existing FRS methods on numerical datasets. And, the results show that the IvFMFRS obtained the relevant feature subsets with similar classification accuracy with significantly less computational time than existing FRS methods. [ABSTRACT FROM AUTHOR]

Sensor-based human activity recognition (HAR), i.e., the ability to discover human daily activity patterns from wearable or embedded sensors, is a key enabler for many real-world applications in smart homes, personal healthcare, and urban planning. However, with an increasing number of applications being deployed, an important question arises: how can a HAR system autonomously learn new activities over a long period of time without being re-engineered from scratch? This problem is known as continual learning and has been particularly popular in the domain of computer vision, where several techniques to attack it have been developed. This paper aims to assess to what extent such continual learning techniques can be applied to the HAR domain. To this end, we propose a general framework to evaluate the performance of such techniques on various types of commonly used HAR datasets. Then, we present a comprehensive empirical analysis of their computational cost and of their effectiveness of tackling HAR specific challenges (i.e., sensor noise and labels' scarcity). The presented results uncover useful insights on their applicability and suggest future research directions for HAR systems. [ABSTRACT FROM AUTHOR]

Continual learning models allow them to learn and adapt to new changes and tasks over time. However, in continual and sequential learning scenarios, in which the models are trained using different data with various distributions, neural networks (NNs) tend to forget the previously learned knowledge. This phenomenon is often referred to as catastrophic forgetting. The catastrophic forgetting is an inevitable problem in continual learning models for dynamic environments. To address this issue, we propose a method, called continual Bayesian learning networks (CBLNs), which enables the networks to allocate additional resources to adapt to new tasks without forgetting the previously learned tasks. Using a Bayesian NN, CBLN maintains a mixture of Gaussian posterior distributions that are associated with different tasks. The proposed method tries to optimize the number of resources that are needed to learn each task and avoids an exponential increase in the number of resources that are involved in learning multiple tasks. The proposed method does not need to access the past training data and can choose suitable weights to classify the data points during the test time automatically based on an uncertainty criterion. We have evaluated the method on the MNIST and UCR time-series data sets. The evaluation results show that the method can address the catastrophic forgetting problem at a promising rate compared to the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Knowledge obtained from the properties of a Pareto-optimal set can guide an evolutionary search. Learning models for multiobjective estimation of distributions have led to improved search efficiency, but they incur a high computational cost owing to their use of a repetitive learning or iterative strategy. To overcome this drawback, we propose an algorithm for incremental-learning model-based multiobjective estimation of distributions. A learning mechanism based on an incremental Gaussian mixture model is embedded within the search procedure. In the proposed algorithm, all new solutions generated during the evolution are passed to a data stream, which is fed incrementally into the learning model to adaptively discover the structure of the Pareto-optimal set. The parameters of the model are updated continually as each newly generated datum is collected. Each datum is learned only once for the model, regardless of whether it has been preserved or deleted. Moreover, a sampling strategy based on the learned model is designed to balance the exploration/exploitation dilemma in the evolutionary search. The proposed algorithm is compared with six state-of-the-art algorithms for several benchmarks. The experimental results show that there is a significant improvement over the representative algorithms. [ABSTRACT FROM AUTHOR]

Cross-version defect prediction (CVDP) refers to training an excellent model on tagged defect data in a previously released project version, and then performing defect prediction on an unlabeled instance module in the current version. Nevertheless, the complicated internal intrinsic construction hidden behind the code defects makes it difficult for the previous cross-version defect models to capture more discriminative software features, and seriously restrains the CVDP performance. In this study, we propose an intelligent data-driven CVDP model named WSBCV based on multi-objective optimization and incremental representation learning. We firstly leverage an advanced deep generation adversarial network – WGAN-GP (Wasserstein GAN with Gradient Penalty) to perform data augmentation, including balancing defect classes and synthesizing abundant training instances. Secondly, a multi-objective SPEA/R (Strength Pareto-based Evolutionary Algorithm / Reference) feature selection optimization method is built to effectively search the fewest representative feature subsets while achieving the minimum error. Finally, a powerful defect predictor for CVDP based on the BLS (Broad Learning System) with incremental learning is built to learn excellent feature representations and achieve incremental online model update quickly. Experimental results across 32 cross-version pairs from 45 version demonstrate that the proposed SPEA/R, BLS and WSBCV all have statistically significant difference advantages compared to ten multi-objective feature selection approaches, six defect predictors and two CVDP models, respectively. [ABSTRACT FROM AUTHOR]

Zero-shot learning (ZSL) is a successful paradigm for categorizing objects from the previously unseen classes. However, it suffers from severe performance degradation in the generalized ZSL (GZSL) setting, i.e., to recognize the test images that are from both seen and unseen classes. In this article, we present a simple but effective mechanism for GZSL and more open scenarios based on a transfer-increment strategy. On the one hand, a dual-knowledge-source-based generative model is constructed to tackle the missing data problem. Specifically, the local relational knowledge extracted from the label-embedding space and the global relational knowledge, which is the estimated data center in the feature-embedding space, are concurrently considered to synthesize the virtual exemplars. On the other hand, we further explore the training issue for the generative models under the GZSL setting. Two incremental training modes are designed to learn directly the unseen classes from the synthesized exemplars instead of the training classifiers with the seen and synthesized unseen exemplars together. It not only presents an effective unseen class learning but also requires less computing and storage resources in practical application. Comprehensive experiments are conducted based on five benchmark data sets. In comparison with the state-of-the-art methods, both the generating and training processes are considered for virtual exemplars by the proposed transfer-increment strategy, which results in a significant improvement in the conventional and GZSL tasks. [ABSTRACT FROM AUTHOR]

• Tackle dynamic incomplete hybrid data based on three-way decision theory. • A Three-Way Neighborhood Decision Model (TWNDM) is proposed. • Matrix-based approaches for computing three-way regions in TWNDM are presented. • Incremental methods for updating three-way regions are proposed when objects vary. • Experiments are conducted to verify the effectiveness of the proposed approaches. In practical applications, there generally exist incomplete hybrid data with heterogeneous and missing features. The complex structures and the fast update of incomplete hybrid data bring a series of challenges for decision making in dynamic data environments. Three-way decisions, as an important cognitive method for analyzing uncertain problems, have been extensively applied into various fields. However, the existing studies rarely focus on exploring three-way decisions with incomplete hybrid information. To tackle this issue, we propose a Three-Way Neighborhood Decision Model (TWNDM) based on the data-driven neighborhood relation in terms of two pseudo-distance functions only satisfying the reflexivity. Considering that the addition and deletion of objects will result in the variation of information granules and decision structures, this paper presents a matrix-based dynamic framework for updating three-way regions (positive, boundary and negative regions) in TWNDM. A novel relation matrix is first constructed by using a pair of values to replace single value in the classical relation matrix. Then, the matrix-based approach for computing the three-way regions is established in the light of the new relation matrix, the decision matrix and the related induced matrices. Moreover, the matrix-based incremental mechanisms and algorithms for the maintenance of the three-way regions are presented when adding and removing objects, respectively. The results of comparative experiments demonstrate that the proposed incremental algorithms can improve the computational performance for maintaining three-way regions in TWNDM compared with the static algorithm. [ABSTRACT FROM AUTHOR]

In practical decision making applications, it is computationally time-consuming to maintain multigranulation approximations from scratch in dynamic ordered decision information systems (ODISs) with incremental granular structures consisting of the changing of granular structures by adding granular structures, or by adding an attribute set into each granular structure. The time consumed in the process of maintaining approximations from scratch makes it natural to take into account incremental strategies in order to reduce computational complexity in dynamic multigranulation contexts. To address this challenge, we propose two matrix-based incremental strategies that can dynamically update the lower and upper approximations of each decision class with incremental granular structures in dominance-based multigranulation rough sets (DMGRSs). Moreover, the corresponding incremental algorithms are designed for handling dynamic multi-source ordered data. Ultimately, empirical experiments conducted on UCI data sets depict that the proposed algorithms exhibit a better computational performance compared with the matrix-based static algorithm. [ABSTRACT FROM AUTHOR]

Incremental feature selection can improve learning of accumulated data. We focus on incremental feature selection based on rough sets, which along with their generalizations (e.g., fuzzy rough sets), reduce dimensionality without requiring domain knowledge, such as data distributions. By analyzing the basic concepts of fuzzy rough sets on incremental datasets, we propose incremental mechanisms of information measure. Moreover, we introduce a key instance set containing representative instances to select supplementary features when new instances arrive. As the key instance set is much smaller than the whole dataset, the proposed incremental feature selection mostly suppresses redundant computations. We experimentally compare the proposed method with various non-incremental and two state-of-the-art incremental methods on a variety of datasets. The comparison results demonstrate that the proposed method achieves compact results with reduced computation time, especially on high-dimensional datasets. [ABSTRACT FROM AUTHOR]

In order to extract useful information from data streams, incremental learning has been introduced in more and more data mining algorithms. For instance, a self-organizing incremental neural network (SOINN) has been proposed to extract a topological structure that consists of one or more neural networks to closely reflect the data distribution of data streams. However, SOINN has the tradeoff between deleting previously learned nodes and inserting new nodes, i.e., the stability–plasticity dilemma. Therefore, it is not guaranteed that the topological structure obtained by the SOINN will closely represent data distribution. For solving the stability–plasticity dilemma, we propose a Gaussian membership-based SOINN (Gm-SOINN). Unlike other SOINN-based methods that allow only one node to be identified as a “winner” (the nearest node), the Gm-SOINN uses a Gaussian membership to indicate to which degree the node is a winner. Hence, the Gm-SOINN avoids the topological structure that cannot represent the data distribution because previously learned nodes overly deleted or noisy nodes inserted. In addition, an evolving Gaussian mixture model is integrated into the Gm-SOINN to estimate the density distribution of nodes, thereby avoiding the wrong connection between two nodes. Experiments involving both artificial and real-world data sets indicate that our proposed Gm-SOINN achieves better performance than other topology learning methods. [ABSTRACT FROM AUTHOR]

In many real-life applications, the data collected from different information sources are always located in diverse sites and characterized by different types of attributes. In this paper, we call this type of data as multi-source hybrid data. Existed rough set models only work well with single source data or multi-source data with one type of attributes, which are not suitable for processing multi-source hybrid data. Furthermore, in dynamic data environment, multi-source hybrid data are typically evolving over time, including the addition of objects and attributes and the revision of missing values in each information source. It will result in the update of information granulation and rough approximation structures. To address these issues, we present an extended rough set model, named as multi-source composite rough sets (MCRS), by integrating different types of attributes and fusing multiple composite relations derived from different information sources. Two novel matrix operators are proposed to construct the matrix-based representation of rough approximations. Then, under the variations of objectives, attributes and attribute values in the multi-source hybrid information system, we present incremental mechanisms based on matrices for the maintenance of approximations by utilizing the previously accumulated matrices information and transmitting location information of part elements in each composite relation matrix. In addition, we develop the incremental algorithms for updating composite rough approximations. A series of comparative experiments show that the proposed algorithm outperforms the static algorithm and other extended incremental algorithms in terms of the efficiency of computation and transmission. [ABSTRACT FROM AUTHOR]

• We propose a novel incremental and decremental variant of the TWSVM called Fuzzy Bounded Twin Support Vector Machine (FBTWSVM) for solving incremental learning problems. • The proposed approach can continuously integrate new information into already-built models and it is adherent to the structural risk minimization principle, and it uses the dual coordinate descent (DCD) algorithm with active shrinking to create the off-line classifier. • The incremental and decremental strategies are based on the DCD with shrinking, exploiting the relevance of each support vector. • We propose the use of our linear formulation with a kernel approximation to speed up training and classification while maintaining the non-linearity. In this paper, we present an incremental variant of the Twin Support Vector Machine (TWSVM) called Fuzzy Bounded Twin Support Vector Machine (FBTWSVM) to deal with large datasets and to learn from data streams. We combine the TWSVM with a fuzzy membership function, so that each input has a different contribution to each hyperplane in a binary classifier. To solve the pair of quadratic programming problems (QPPs), we use a dual coordinate descent algorithm with a shrinking strategy, and to obtain a robust classification with a fast training we propose the use of a Fourier Gaussian approximation function with our linear FBTWSVM. Inspired by the shrinking technique, the incremental algorithm re-utilizes part of the training method with some heuristics, while the decremental procedure is based on a scoring window. The FBTWSVM is also extended for multi-class problems by combining binary classifiers using a Directed Acyclic Graph (DAG) approach. Moreover, we analyzed the theoretical foundation's properties of the proposed approach and its extension, and the experimental results on benchmark datasets indicate that the FBTWSVM has a fast training and retraining process while maintaining a robust classification performance. [ABSTRACT FROM AUTHOR]

• Matrix-based incremental mechanisms for updating the positive, boundary, and negative regions of a target concept in neighborhood multigranulation rough sets are explored when adding or deleting multiple objects. • On the basis of the updating mechanisms, the corresponding matrix-based incremental updating algorithms are developed for maintaining the neighborhood multigranulation positive, boundary, and negative regions of a target concept. • The experimental results on publicly available UCI datasets demonstrate the effectiveness and efficiency of the proposed dynamic algorithms. Rough approximations play a significant role in rule extraction and decision making. However, re-scanning the entire data set to update the approximations is time-consuming due to the dynamic characteristics of objects in a neighborhood multigranulation space. In order to reduce the computational time, the neighborhood multigranulation approximations need to be updated in an incremental manner based on previously saved knowledge. Therefore, in this study, we establish a dynamic framework for maintaining the positive, boundary, and negative regions in neighborhood multigranulation spaces when adding or deleting objects from the matrix perspective. First, we explore the incremental mechanisms for updating relevant matrices when adding or deleting multiple objects. Based on the proposed mechanisms, we design the corresponding dynamic algorithms to incrementally update the positive, boundary, and negative regions. Finally, we conduct empirical experiments on benchmark UCI data sets to assess the feasibility and efficiency of our updating algorithms, which demonstrate their promising performance. [ABSTRACT FROM AUTHOR]

Neighborhood decision systems (NDSs) are commonly-used systems in real-world applications such as credit scoring. Probabilistic neighborhood rough sets (PNRSs) model, which is a generalized rough set model by combining neighborhood rough sets (NRSs) with probabilistic theory, can efficiently handle numerical data with noise values and allow tolerance of errors. Approximations of a target concept are fundamental and vital notions of the PNRSs, which can induce the certain and uncertain decision rules. Nevertheless, data evolves over time due to dynamic characteristics, especially, the addition of new attributes and the deletion of redundant or irrelevant attributes. Therefore, the potential meaningful knowledge may alter over time accordingly. To improve the efficiency of acquiring decision rules, the three-way regions (i.e., the positive, negative, and boundary regions) of the PNRSs need to be updated in an incremental fashion. To address this issue, two incremental algorithms for updating the three-way regions of each decision class of the PNRSs are investigated from the matrix perspective with time-evolving attributes (namely, adding attributes and deleting attributes). First, a matrix-based characterization of the three-way regions of each decision class of the PNRSs is constructed. Subsequently, considering the variation of the attributes, matrix-based updating mechanisms for relevant matrices are developed by means of the previously calculated information. Furthermore, the incremental algorithms are developed by taking advantage of the matrix-based updating mechanisms. Comprehensive experiments are performed on benchmark UCI data sets, and comparative results demonstrate our proposed algorithms consistently outperform the non-incremental algorithm in terms of computational efficiency. [ABSTRACT FROM AUTHOR]

In rough set philosophy, each set of data can be seen as a fuzzy decision table. Since a decision table dynamically increases with time and space, these decision tables are integrated into a new one called fused decision table. In this paper, we focus on designing an incremental feature selection method on fused decision table by optimizing the space constraint of storing discernibility matrix. Here discernibility matrix is a known way of discernibility information measure in rough set theory. This paper applies the quasi/pseudo value of discernibility matrix rather than the true value of discernibility matrix to design an incremental mechanism. Unlike those discernibility matrix based non-incremental algorithms, the improved algorithm needs not save the whole discernibility matrix in main memory, which is desirable for the large data sets. More importantly, with the increment of decision tables, the discernibility matrix-based feature selection algorithm could constrain the computational cost by applying efficient information updating techniques—quasi/pseudo approximation operators. Finally, our experiments reveal that the proposed algorithm needs less computational cost, especially less occupied space, on the condition that the accuracy is limitedly lost. [ABSTRACT FROM AUTHOR]

We present a novel method for concept drift detection, based on: 1) the development and continuous updating of online sequential extreme learning machines (OS-ELMs) and 2) the quantification of how much the updated models are modified by the newly collected data. The proposed method is verified on two synthetic case studies regarding different types of concept drift and is applied to two public real-world data sets and a real problem of predicting energy production from a wind plant. The results show the superiority of the proposed method with respect to alternative state-of-the-art concept drift detection methods. Furthermore, updating the prediction model when the concept drift has been detected is shown to allow improving the overall accuracy of the energy prediction model and, at the same time, minimizing the number of model updatings. [ABSTRACT FROM AUTHOR]

Interval type-2 fuzzy neural networks (IT2FNNs) have gained sustainable attention and wide applications because of their power of adaptive fuzzy modeling. Although the existing models accurately simulate the relationships between inputs and outputs by various adaptive learning algorithms, the parameters and fuzzy rules characterizing the model behaviors no longer have explicit interpretations. To achieve good interpretability and generalization performance, this paper proposes a novel data-knowledge-driven IT2FNN (DKIT2FNN). Unlike accuracy-oriented model fitting, DKIT2FNN pioneers the use of famous association rule mining techniques to automatically explore the interpretable rule knowledge, which intuitively presents conditional relation patterns among features. Using the obtained knowledge, the DKIT2FNN model is accomplished in a single-hidden layer network framework, in which the antecedent structure of each fuzzy rule is determined by the relation patterns provided by rule knowledge. In addition, an adversarial incremental learning algorithm is proposed for the self-organizing modeling of DKIT2FNN. Besides automatically assembling a compact rule base, it integrates the label adversarial training to reinforce the model's capability against label noise. We validate the DKIT2FNN on classification benchmark datasets with label noise and compare it with state-of-the-art non-fuzzy and neuro-fuzzy methods. The results demonstrate that the proposed DKIT2FNN achieves robust classification performance and good interpretability. • Propose a data-knowledge-driven interval type-2 fuzzy neural network (DKIT2FNN). • Realize an automated knowledge mining to ensure reliable fuzzy rule interpretation. • Develop an incremental learning algorithm integrating label adversarial training. • DKIT2FNN achieves a good and robust generalization performance facing label noise. • DKIT2FNN can self-organize a compact model structure with good interpretability. [ABSTRACT FROM AUTHOR]

RNA modifications are important for deciphering the function of cells and their regulatory mechanisms. In recent years, researchers have developed many deep learning methods to identify specific modifications. However, these methods require model retraining for each new RNA modification and cannot progressively identify the newly identified RNA modifications. To address this challenge, we propose an innovative incremental learning framework that incorporates multiple incremental learning methods. Our experimental results confirm the efficacy of incremental learning strategies in addressing the RNA modification challenge. By uniquely targeting 10 RNA modification types in a class incremental setting, our framework exhibits superior performance. Notably, it can be extended to new category methylation predictions without the need for retraining with previous data, improving computational efficiency. Through the accumulation of knowledge, the model is able to evolve and continuously learn the differences across methylation, mitigating the problem of catastrophic forgetting during deep learning model training. Overall, our framework provides various alternatives to enhance the prediction of novel RNA modifications and illuminates the potential of incremental learning in tacking numerous genome data. [ABSTRACT FROM AUTHOR]

Concept-cognitive learning (CCL) concerns the manner in which to incorporate new information into itself by mimicking human cognitive processes. Numerous CCL systems have been proposed to meet different requirements based on formal concept analysis. However, most of these can only operate effectively on small-scale datasets and lack classification ability. To overcome these two challenges, based on a regular formal decision context and multi-thread technique, a new concurrent CCL model is proposed in this study as an extension of classical CCL. More precisely, to enhance computational efficiency, a new concurrent learning framework is designed and its corresponding learning algorithms at the initial concept construction and CCL stages are developed. We propose a concurrent incremental learning technique by continuously accommodating newly added data to meet the requirements of classification tasks. Finally, experiments on various datasets, including real-world and synthetic data, demonstrate that the proposed concurrent method can achieve comparable classification effectiveness and significantly improve the concept learning performance simultaneously. [ABSTRACT FROM AUTHOR]

Abstract Major assumptions in computational intelligence and machine learning consist of the availability of a historical dataset for model development, and that the resulting model will, to some extent, handle similar instances during its online operation. However, in many real-world applications, these assumptions may not hold as the amount of previously available data may be insufficient to represent the underlying system, and the environment and the system may change over time. As the amount of data increases, it is no longer feasible to process data efficiently using iterative algorithms, which typically require multiple passes over the same portions of data. Evolving modeling from data streams has emerged as a framework to address these issues properly by self-adaptation, single-pass learning steps and evolution as well as contraction of model components on demand and on the fly. This survey focuses on evolving fuzzy rule-based models and neuro-fuzzy networks for clustering, classification and regression and system identification in online, real-time environments where learning and model development should be performed incrementally. [ABSTRACT FROM AUTHOR]

Abstract Three-way decisions theory has been developed and applied to decision making in uncertain environments. Decision tasks under ambiguity involving both missing values and hierarchical concept are abundant in real-world applications. In this paper, we study the update problem of three-way decisions with dynamic variation of scales in incomplete multi-scale information systems. The updating mechanisms of decision granules induced by the similarity relation are exploited with the cut refinement and coarsening through the attribute value taxonomies, and the dynamic tendencies of conditional probability are hence estimated with evolving granularity structure. Reasonable updating behaviors of probabilistic parameters are characterized in the framework of decision-theoretic rough sets, in order to relate optimism and pessimism decision making on the basis of the variation in attitudes to losses or costs of decisions during the periodic transformation between coarser granular-scale and refiner granular-scale of data. Then the dynamic tendencies of conditional probability with the updating behaviors of probabilistic parameters together are incorporated into the incremental process of updating three-way decisions. Theoretical justifications and semantic interpretations are provided to guarantee the satisfaction of the proposed method for incremental three-way decisions in multi-scale incomplete information systems. Experimental results on several UCI datasets show that the proposed incremental algorithms efficiently update three-way decisions for handling transformation of data scales caused by the cut refinement and coarsening through the attribute value taxonomies. [ABSTRACT FROM AUTHOR]

Broad learning system (BLS) has attracted widespread attention owing to its concise structure and efficient incremental learning based on ridge regression approximation of pseudo-inverse. However, BLS struggles with expensive computational costs and high memory usage when processing complex problems and training large networks due to the inverse operation of a large matrix. There has been some attempts to address this issue. In this work, we attempt to ameliorate the computational efficiency motivated by the state-of-the-art randomized iterative least squares solver. First, we improve the iteration manner of randomized extended Kaczmarz (REK) and then propose the iterated projection learning for training BLS without pseudo-inverse. The iterated projection learning is suitable for both offline and online learning scenarios and can adjust the model structure by incremental learning. Thanks to its linear convergence of least norm in expectation, the obtained model is expected to have better generalization ability without additional regularization procedure. Our model is found to maintain stable performance during incremental learning. Moreover, instead of grid search, we present an evolutionary bilevel programming (EBP) method with dispersion operator to further optimize the hyperparameters of the network structure. Numerical experiments indicate that the proposed methods can improve the efficiency, robustness, and generalization ability compared with pseudo-inverse and grid search based methods. [ABSTRACT FROM AUTHOR]

Collecting sufficient labeled data is time-consuming and even infeasible in streaming data applications. Unsupervised domain adaptation is a reasonable solution that has recently gained attention. However, most efforts have focused on stationary environments and have ignored the uncertainties that arise from mismatched distributions. Therefore, the proposed method addresses the classification of samples in streaming data in the presence of concept drift, employing a heterogeneous unsupervised domain adaptation method. Accordingly, a fuzzy rough set-based sample weighting approach is introduced to modulate the impact of uncertainties on feature alignment in non-stationary environments. The domain adaptation is carried out through a fuzzy approach and is optimized by a heuristic optimization technique that reduces the sensitivity to tunable parameters. Moreover, an incremental classifier is designed to enable rapid adaptation to changes. The advantages of the proposed method encompass an effective avoidance of excessive alignment, training cost optimization, and the gradual reduction in dependency on the source domain for domain adaptation. Regarding different types of concept drift, experiments on several tasks taken from four benchmark datasets demonstrate the superiority of the proposed method in terms of accuracy and computational time. [ABSTRACT FROM AUTHOR]

As an important preprocessing step for data mining, attribute reduction has become a hot research topic in rough set theory. In practice, many real data may vary dynamically with time, therefore, reduct will change dynamically under the variation of objects and attributes in decision systems. The classical attribute reduction methods need to recompute from scratch, which are ineffective to deal with dynamic decision systems. How to implement updating reducts by utilizing previous results is vital for improving the efficiency of attribute reduction approaches. In the paper, we firstly introduce incremental mechanisms of computing reduct when objects and attributes of the decision system change dynamically. Then, incremental methods are developed to update reduct when attributes and objects increase simultaneously. Finally, a series of experiments are conducted to validate the proposed incremental attribute reduction methods. Experimental results show that they are effective to update reduct with change of attributes and objects in the decision systems. [ABSTRACT FROM AUTHOR]

Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel adaptive piecewise linear model for dynamic system identification. It has four unique features. First, the model designs a new kind of basis function for function approximation. It maintains the uniform shape for each basis function, so as to achieve a satisfactory tradeoff between generalization ability and model complexity. Second, the model takes the structure of basis functions as decision variables to optimize the formulated identification problems instead of taking expansion coefficients as decision variables as proposed by many existing approaches. Third, we establish an incremental design strategy to solve the system identification problems. In each step of the identification, the selection of optimal basis function is a Lipschitz continuous optimization problem that is likely to be easily handled with some mature toolboxes. This incremental design strategy greatly reduces the estimation cost. Fourth, we introduce a smoothing mechanism to avoid overfitting, when the output of dynamic systems is disturbed by noise. Tests on several benchmark dynamic systems demonstrate the potential of the proposed model. [ABSTRACT FROM AUTHOR]

Multicriteria classification refers to classify objects evaluated by a set of criteria to preference-ordered decision classes. Dominance-based rough set approach has been successfully introduced to express and reason inconsistencies with a dominance principle in multicriteria classification problems. Hierarchical attribute values exist extensively within many real-world applications, which provide a hierarchical form to organize, view and analyze data from different perspectives for accommodating the preference variability. In this study, we consider an extension of dominance-based rough set approach by applying an incremental learning technique for hierarchical multicriteria classification while attribute values dynamically vary across different levels of granulations. We formalize the dynamic characteristics of knowledge granules with the cut refinement and coarsening through attribute value taxonomies in the hierarchical multicriteria decision systems. In consequence, incremental algorithms for computing dominance-based rough approximations of preference-ordered decision classes are developed by applying the resulted prior-knowledge as the input, and only recomputing those outputs which depend on the changed attribute values. This paper presents the theoretical foundation of the proposed approach. Example analysis and experimental evaluation are also provided for illustration of the feasibility and efficiency. [ABSTRACT FROM AUTHOR]

The amount of data being generated in industrial and scientific applications is constantly increasing. These are often generated as a chronologically ordered unlabeled data flow which exceeds usual storage and processing capacities. Association stream mining is an appealing field which models complex environments online by finding relationships among the attributes without presupposing any a priori structure. The discovered relationships are continuously adapted to the dynamics of the problem in a pure online way, being able to deal with both categorical and continuous attributes. This paper presents a new advanced version, Fuzzy-CSar-AFP, of an online genetic fuzzy system designed to obtain interesting fuzzy association rules from data streams. It is capable of managing partitions of different granularity for the variables, which allows the algorithm to adapt to the precision requirements of each variable in the rule. It can also work with data streams without needing to know the domains of the attributes as it includes a mechanism which updates them in real-time. Fuzzy-CSar-AFP performance is validated in an original real-world Psychophysiology problem where associations between different electroencephalogram signals in subjects which are put through different stimuli are analyzed. [ABSTRACT FROM AUTHOR]

Broad Learning System (BLS) that aims to offer an alternative way of learning in deep structure is proposed in this paper. Deep structure and learning suffer from a time-consuming training process because of a large number of connecting parameters in filters and layers. Moreover, it encounters a complete retraining process if the structure is not sufficient to model the system. The BLS is established in the form of a flat network, where the original inputs are transferred and placed as “mapped features” in feature nodes and the structure is expanded in wide sense in the “enhancement nodes.” The incremental learning algorithms are developed for fast remodeling in broad expansion without a retraining process if the network deems to be expanded. Two incremental learning algorithms are given for both the increment of the feature nodes (or filters in deep structure) and the increment of the enhancement nodes. The designed model and algorithms are very versatile for selecting a model rapidly. In addition, another incremental learning is developed for a system that has been modeled encounters a new incoming input. Specifically, the system can be remodeled in an incremental way without the entire retraining from the beginning. Satisfactory result for model reduction using singular value decomposition is conducted to simplify the final structure. Compared with existing deep neural networks, experimental results on the Modified National Institute of Standards and Technology database and NYU NORB object recognition dataset benchmark data demonstrate the effectiveness of the proposed BLS. [ABSTRACT FROM AUTHOR]

Modern multi-function radars (MFRs) can flexibly generate multiple fine-grained working modes for different missions through programmable parameters. Precise recognition of these working modes by analyzing the parameter combinations lays foundation for situational awareness. This recognition becomes more challenging to electromagnetic reconnaissance system when prior information of radiation source is unavailable and effective labeled signal data is not sufficient in adversarial scenarios. The presence of incremental mode involved in radar data flow requires that the recognition model enables to dynamic adjust and online perceive current data. This issue incorporates online learning to few-shot learning to accomplish efficiently recognition to a novel mode with the support of small amount of data, and still retain model's ability to recognize existing modes. This paper designed a backtracking contextual prototypical memory (BCPM) network for online MFR mode recognition. The proposed method learns temporal and spatial information from data stream as a reference predicting the signal segments as existing working modes or a novel mode. The BCPM network also designs a backtracking module and an alignment regularization term to utilize the data without annotation adequately and obtain a more reliable category representation. The experimental results verified the proposed method's good performance and high robustness to non-ideal factors and distractors. [ABSTRACT FROM AUTHOR]

This paper proposes an assessor-guided learning strategy for continual learning where an assessor guides the learning process of a base learner by controlling the direction and pace of the learning process thus allowing an efficient learning of new environments while protecting against the catastrophic interference problem. The assessor is trained in a meta-learning manner with a meta-objective to boost the learning process of the base learner. It performs a soft-weighting mechanism of every sample accepting positive samples while rejecting negative samples. The training objective of a base learner is to minimize a meta-weighted combination of the cross entropy loss function, the dark experience replay (DER) loss function and the knowledge distillation loss function whose interactions are controlled in such a way to attain an improved performance. A compensated over-sampling (COS) strategy is developed to overcome the class imbalanced problem of the episodic memory due to limited memory budgets. Our approach, Assessor-Guided Learning Approach (AGLA), has been evaluated in the class-incremental and task-incremental learning problems. AGLA achieves improved performances compared to its competitors while the theoretical analysis of the COS strategy is offered. Source codes of AGLA, baseline algorithms and experimental logs are shared publicly in https://github.com/anwarmaxsum/AGLA for further study. [ABSTRACT FROM AUTHOR]

As a commonly used framework for uncertainty reasoning, tolerance rough set has achieved remarkable success in handling incomplete information systems with missing values. Three-way regions generated from tolerance rough set model play an increasingly crucial role in decision making and intelligent data analysis. Nevertheless, the dynamic change of attributes often exists in incomplete information systems. With this dynamic characteristic, three-way regions must be effectively updated for potential decision-making processes. Therefore, we develop incremental algorithms for maintenance of three-way regions in incomplete information systems when adding or deleting attributes, accelerating the calculation by making use of prior information. First, we put forward an effective matrix-based approach to calculate three-way regions in incomplete data. With the dynamic change of attributes, we further investigate the updating strategies of related matrices for constructing three-way regions. Accordingly, matrix-based algorithms for incrementally updating three-way regions are developed and discussed while the attributes vary over time. In addition, the complexity comparisons of non-incremental and incremental algorithms are illustrated. Finally, empirical experiments are performed to reveal the efficiency of the incremental algorithms compared with matrix-based non-incremental and related incremental algorithms. • We develop efficient matrix-based incremental approaches for updating three-way regions when adding or deleting attributes. • We evaluate and verify the performance of the proposed incremental algorithms from the perspectives of different sizes of objects and incremental ratios of attributes. • The proposed incremental algorithms can provide a solid foundation for dynamic extraction of decision rules and matrix-based attribute reduction of tolerance rough sets. [ABSTRACT FROM AUTHOR]

Stochastic Configure Networks (SCNs) are an incremental variant of randomly weighted neural networks of which one of the key highlights lies in the constraints while adding hidden layer nodes. Recent studies show two weaknesses of SCNs. One is the redundancy of added hidden layer nodes while the other is the lack of interpretability to the designed constraints. Motivated by overcoming these weaknesses, this paper proposes a new model named Evolving SCN to increase the interpretability of the design for constraints from the viewpoint of evolution by a sampling mechanism and to promote the model compactness by optimizing random weights within the space of constraint parameters. Surprisingly, although an evolving method is used in our model, the effectiveness and efficiency are significantly improved with respect to running time and prediction accuracy in comparison with the existing versions of SCNs. This work makes a first attempt to enhance the interpretability of incrementally adding nodes and simultaneously reduce the redundancy of hidden nodes in SCNs, which brings some new insights to understand the model compactness from the view of Occam's razor and further to the nature of incremental learning. • An accessible interpretation of the designed constraints in SCN is summarized, extending the fundamental theory. • A new evolution mechanism is proposed to optimize random weights, addressing hidden node redundancy in SCN, i.e., E-SCN. • Real-world dataset experiments reveal E-SCN's improved compactness, training efficiency, and generalization ability. [ABSTRACT FROM AUTHOR]