Search

Incremental randomized neural networks have been widely applied in industrial data modeling. However, incremental randomized neural networks may generate redundant hidden nodes. These nodes may lead to weak performance in real world data modeling tasks. An important factor is that the neural nodes constructed in the early stages may influence the quality of subsequently generated nodes. To resolve this drawback, an incremental regularization kernel randomized neural network (IRKRNN) is proposed in this work. IRKRNN adopts a kernel learning method to enhance the feature expression during the parameter-learning. Meanwhile, the uncertainty of random mapping is reduced. Each randomly generated node is projected into kernel space. When new nodes are generated, the kernel space is constantly updated. Moreover, the number of nodes and the expected tolerance are used as criteria for terminating the kernel-based incremental learning process to achieve a compact network structure. Finally, IRKRNN is compared with state-of-the-art randomized neural networks on six real world regression datasets and the electrical energy output prediction task of the combined cycle power plant. Experimental results indicate that the proposed IRKRNN achieves satisfactory generalization performance, and it has good potential for industrial data modeling tasks.

With the widespread application of wireless body area networks in medical monitoring and human health management, improving its network performance and reliability has become a key issue. However, the inherent complex spatiotemporal characteristics of WBAN channels pose great challenges to the improvement of network performance. This paper proposes a human body channel prediction model based on deep learning and incremental learning to effectively capture the dynamic changes of human body channels. This model can adaptively predict future channel quality by combining historical channel data and real-time update capabilities. At the same time, this paper also designs an optimal adaptive relay selection algorithm, which achieves intelligent selection of relay nodes by evaluating channel quality and node energy consumption, further enhancing the robustness and transmission efficiency of the network. Simulation results show that the proposed method has significant advantages in channel prediction accuracy, relay selection effect and network performance improvement, providing new technical support for the practical application of WBAN.

We address an important and practical problem facing deep-learning-based image classification: class incremental learning with a large domain shift. Most previous efforts on class incremental learning focus on one aspect of the problem, i.e., learning to classify additional new classes (with a little shift). However, in the real world, when new classes are added, the domain changes simultaneously (with a large domain shift). To obtain a model that is robust to these situations, we need to consider incrementally learning not only new labels but also domain-shifted labels. We target a continual and simultaneous shift of class and domain distribution and propose a new incremental learning method named Momentum Contrastive learning enhancing Orthogonality of Negative pairs (MoCo-ON). We employ a momentum encoder framework augmented with rehearsal memory to mitigate the risk of forgetting while leveraging contrastive learning to extract versatile features capable of adapting to the progressively shifting domain. Specifically, when training with a knowledge distillation loss, we introduce a novel supervised contrastive loss designed to closely embed positive pairs of the same class, even in the presence of a substantial domain gap. Additionally, we leverage feature embedding from momentum encoders for exemplar selection, aiming to mitigate the risk of forgetting previously acquired information from earlier tasks. We conduct comprehensive experiments involving inter-domain shifted class incremental learning scenarios using widely adopted datasets commonly employed for studying domain generalization in image classification. Our proposed method consistently outperforms other methods by a significant margin.

Phishing is the process of deceiving or stealing private or confidential information through illicit means. This could lead to financial loss, loss of reputation, and identity theft. Hence, identifying and preventing the use of such phishing sites becomes crucial. In data science, the term outlier, also termed an anomaly refers to points or series of data that opt out of the normal behaviour of the system under study. Anomaly detection touches down on the concepts related to studying the authentic outlier in a data set. The paper aims to present the optimised techniques and multiple modes of executing the process for detecting phishing websites. The most relevant features are chosen for execution by applying feature extraction. The Mendley phishing websites dataset is used to detect phishing websites, along with the SPAM-HAM publicly available dataset, which is used for detecting SPAM/HAM classification for SMS data in this research study. The experiments are also carried out on a custom dataset to avoid any bias present in a publicly available dataset. The study is carried out in three modes, namely offline, batch, and incremental, using machine learning models. The performance evaluation metrics such as accuracy, f1 score, precision, recall, and time complexity of the machine learning models and accuracy and loss metrics of the deep learning models are compared between the different modes. The study is then summarised by detailing the pros and cons of each of the modes and models used for the study. The incremental mode of execution suits better for real-time processing, with an accuracy of 97.1% on the custom dataset using the adaptive random forest (ARF) classifier available in the Python’s River Framework. But if we make use of the deep learning approach with Keras sequential model, the accuracy obtained was 99.28%.

Real-world multiple or streaming tabular datasets, such as electronic health records from various sources and internet-of-things data generated from different devices, typically exhibit varied feature spaces depending on the datasets. Batch-mode learning with these types of datasets is often inefficient or impractical due to time constraints or privacy regulations. Therefore, an incremental-learning model capable of handling dynamically varying feature spaces, without relying on previous data, is required. To address this need, we propose a new incremental-learning method called Robust and Adaptive Incremental Learning (RAIL). RAIL comprises two core components: an incremental classifier based on naïve Bayes, and a novel adaptive feature-weighting component that utilizes feature-to-feature and feature-to-class relations. RAIL robustly handles missing and new features and adaptively assigns feature weights to improve representation capability while maintaining robustness. Based on public tabular datasets from diverse categories, we demonstrate that RAIL exhibits effective incremental-learning performance for various scenarios where the feature space regularly or arbitrarily varies. Furthermore, we validate that the proposed adaptive feature-weighting method significantly improves prediction accuracy. Additionally, we show that RAIL is more robust in preserving acquired knowledge than the existing state-of-the-art methods. Thus, our approach provides a viable incremental-learning solution for dynamic environments involving varying features.

Internet of Things (IoT) devices are constantly producing vast amounts of data, necessitating efficient storage and processing to extract useful information. However, the models used to extract relevant information from IoT data are often hindered by the lack of useful data and the ever-changing distribution of this data. This paper introduces an incremental data clustering technique on a continuous stream of data through a Dirichlet process-based approach that is adept at handling the formation of clusters in streaming data. The complete approach is twofold; firstly, it starts with an estimated distribution of data and allocates an incoming data point to the estimated data distribution. Secondly, it refines the estimated data distribution after the allocation of the current point and over the subsequent arrival of data points. The influx of data leads to greater challenges in determining clusters for incoming points and preserving the current clusters for improved decision-making. In this context, our proposed approach deals with the increasing amount of data using a selective elimination technique on both existing and incoming data. To assess the performance of the proposed approach, benchmark experiments have been performed using benchmark datasets. The results of the experiments demonstrate that the proposed model has a gain ranging from 2% to 4% as compared to the existing state-of-the-art and recent adaptive clustering approaches in terms of clustering accuracy with incremental data addition and variable clustering parameters. The proposed method shows a high gain in terms of running time ranging from 2% to 20% as compared to the existing approaches depending on the data reduction parameter. Furthermore, research findings through this work indicate that it is possible to set a trade-off between accuracy and running time by adjusting the elimination parameter depending on the requirements of the considered application.

With the rapid growth of digitalization and the increasing volume of data, the cybersecurity threat landscape is expanding at an alarming rate. Intrusion Detection Systems (IDS) have been widely employed in conjunction with firewalls to safeguard networks. However, traditional IDS systems operate in a static manner, rendering them vulnerable to obsolescence and necessitating costly retraining efforts. As a result, the demand for dynamic models capable of handling continuous streams of network traffic has surged as they can learn from the incoming traffic without the need to old data and costly retraining. In response to this challenge, we have implemented an enhanced approach: an incremental majority voting IDS system, which utilizes existing tools and techniques to improve the robustness and adaptability of intrusion detection By leveraging the collective decision-making power of multiple machine learning models such as: KNN classifier, Softmax Regressor and Adaptive Random Forest classifier, our system aims to improve the accuracy, especially reducing false alarm rates, and effectiveness of intrusion detection in real-time scenarios. Through this research, we have managed to obtain a model which scores 96.43% of accuracy as well as giving 100% precision for majority type of attacks. By successfully handling the imbalanced nature of streaming data, our adopted model shows promising potential as a high-performing solution for IDS and can be considered one of the robust IDS models capable of dealing with real-world imbalanced datasets.

Modern manufacturing industry relies on complex machinery that requires skills, attention, and precise safety certifications. Protecting operators in the machine’s surroundings while at the same time reducing the impact on the normal workflow is a major challenge. In particular, safety systems based on proximity sensing of humans or obstacles require that the detection is accurate, low-latency, and robust against variations in environmental conditions. This work proposes a functional safety solution for collision avoidance relying on Ultrasounds (US) and a Temporal Convolutional Network (TCN) suitable for deployment directly at the edge on a low-power Microcontroller Unit (MCU). The setup allowed to acquire a sensor-fusion dataset with 9 US sensors mounted on a real industrial woodworking machine. Applying incremental training, the proposed TCN achieved sensitivity 90.5%, specificity 95.2%, and AUROC 0.972 on data affected by the typical acoustic noise of an industrial facility, an accuracy comparable with the State-of-the-Art (SoA). Deployment on an STM32H7 MCU yielded a memory footprint of 560 B (3× less than SoA), with an extremely low latency of 5.0 ms and an energy consumption of 8.2 mJ per inference (both >2.3× less than SoA). The proposed solution increases its robustness against acoustic noise by leveraging new data, and it fits the resource budget of real-time operation execution on resource-constrained embedded devices. It is thus promising for generalization to different industrial settings and for scale-up to wider monitored spaces.

This paper is motivated by the challenge of providing accurate and contextually relevant answers to natural language questions about visual scenes, particularly in support of individuals with visual impairments. We present a system that is capable of incrementally learning both visual concepts and symbolic facts to answer natural language questions about visual scenes via rich concepts. Deep neural networks are used to learn a feature space from which visual classes are learned as independent probability distributions, allowing new classes to be added arbitrarily with small sample sizes and without the risk of catastrophic forgetting associated with traditional neural networks. Visual classes are not limited to object labels, but also include visual attributes. A knowledge graph is used to represent facts about objects, such as their actions, locations and the relationships between different objects. This allows facts to be stored explicitly and added incrementally. A large language model is used to translate between natural language questions and knowledge graph traversal queries, providing a natural visual question answering process.

Edge-based intelligent data analytics supports the Industrial Internet of Things (IIoT) to enable efficient manufacturing. Incremental learning in the edge-based data analytics has the potential to analyze continuously collected real-time data. However, additional efforts are needed to address performance, latency, resource utilization and storage of historical data challenges. This paper introduces an incremental learning approach based on Long-Short Term Memory (LSTM) autoencoders, by sparsening the weight matrix and taking samples from previously trained sub-datasets. The aim is to minimize the resources utilized while training redundant knowledge for edge devices of IIoT. The degree of sparsity can be determined by the redundancy of patterns, and the inverse of the coefficient of variation has been utilized to recognize it. A higher value of the inverse of the coefficient of variation shows that the values of the weight matrix are close to each other, which indicates the redundancy of knowledge, and vice versa. In addition, the coefficient of variation has been applied for limiting the size of samples from the previously trained sub-datasets. The experiment conducted using the IIoT testbed dataset demonstrates substantial enhancements in resource optimization without compromising performance.

In order to improve the performance of online learning in the real-time distribution of streaming data, a streaming data classification algorithm based on hierarchical concept drift and online ensemble(SCHCDOE) is proposed in this paper. The concept drift index is calculated based on the newly arrived data instance, and the streaming data is divided into three states: stable state, concept drift warning state, and concept drift occurrence state. When the streaming data is in a stable state, the classifier is not updated. When the streaming data is in a concept drift warning state, online ensemble learning is achieved through random subspaces method to perform feature selection and efficiently update the classifier. When the streaming data is in a concept drift occurrence state, anomaly detection mechanism is used to eliminate abnormal data, and online ensemble learning method and incremental learning method are combined for learning. Local information and global distribution information of the streaming data are fully utilized to train the model, so that the learning model can respond quickly after concept drift occurs. Experiments are conducted on both synthetic and real datasets, and the experimental results show that the proposed algorithm performs well. Compared with other classic algorithms, classification accuracy and concept drift adaptability of the proposed algorithm are improved.

Deep neural networks have shown excellent performance in the field of pattern classification and are widely used. However, real-world data are often cannot be obtained at once, and the knowledge of old classes will be heavily forgotten when training new classes of data on the network, which is called catastrophic forgetting. Therefore, the incremental learning method to solve this problem came into being. In this paper, we propose a class-incremental learning method based on a big data pre-trained model, which makes full use of the large amount of public knowledge in the pre-trained model’s front network to reduce the forgetting problem of the network in subsequent classification tasks. On the basis of our previous incremental learning method based on PEDCC, we discuss the effects of different pre-trained models, training strategy, training hyperparameters, etc. PEDCC-Loss is used to constrain the cosine distance between the latent feature and the pre-defined class center, and finally the joint prediction is determined by multiple network prediction results. The algorithm in this paper is verified on the CIFAR100, Tiny ImageNet, and FaceScrub datasets with and without partial retention of old samples, and achieves the best results compared to the previous typical class-incremental learning methods. The performance in coarse-grained datasets even exceeds the accuracy of non-incremental learning without pre-trained model. Code is available in https://github.com/byBinWen/Class-Incremental-Learning-Based-on-Big-Dataset-Pre-trained-Models.

Incremental learning aims to alleviate the catastrophic forgetting problem of deep neural networks during learning sequential data stream. This problem is even more challenging when old data is unavailable, since learning system can only be trained under the supervision of current data. To address this problem, we proposed a prototype sampling mechanism based on K-means clustering method. On the one hand, we proposed to use K-means clustering to pick out class-representative prototypes for each old class. During incremental stages, prototypes and deep features from current data are trained together to maintain the distinction and balance between old and new classes. On the other hand, we proposed to attach a mask to the loss function based on the cosine similarity between the prototypes and the current data. Which further enhances the discrimination between old and new classes compared to naive knowledge distillation schemes. Extensive experiments conducted on three benchmark datasets including CIFAR100, Tiny-ImageNet and vggface2 verified the effectiveness and advantages of our proposed method. Specifically, we improved class incremental performance by 1.6%, 1.2% and 1.7% on three datasets respectively.

An accurate prediction for the stored-grain situation is a necessary means to ensure grain security. It is difficult for traditional machine learning methods to process large-scale stored-grain monitoring data, while deep learning methods have the problems of difficult model training and high resource consumption. To solve these problems, a risk prediction model for stored-grain situations was proposed based on a broad learning network. First, the concept of stored-grain risks was proposed and defined in three categories: low, medium, and high. Then, based on the existing broad learning network, two improved broad learning algorithms were proposed: an enhanced node incremental algorithm and an input data incremental algorithm. Based on the multi-modal features of stored-grain situation data, canonical correlation analysis is introduced to the feature extraction and fusion methods. Finally, an accurate prediction model of stored-grain risk based on improved broad learning and correlation analysis is constructed. The experimental results show that, compared to the traditional broad learning model, the two improved broad learning models improve the accuracy of stored-grain risk prediction by more than 2.3%. Meanwhile, compared with the existing deep learning models, the improved broad learning model reduces training time by more than 20 times without reducing the prediction accuracy and has better robustness. In short, the incremental learning training method can make the prediction accuracy of the broad learning model close to or reach the level of the deep learning model with the increase of training data, which proved that the proposed methods may be an effective alternative to deep learning models.

The interconnection of instruments (i.e., actuators and sensors) networked together for industrial applications brings about the Industrial Internet of Things (IIoT). This connectivity enables the collection, sharing, and analysis of data to enhance the efficiency and productivity of manufacturing. Machine learning models are popular methods for analyzing massive time-series data collected by industrial control systems. Classical Long Short-Term Memory (LSTM), which is a widely used time-series model, learns patterns by feeding the entire dataset at once, and the model remains fixed. However, real-world industrial control system nodes generate new data. This paper proposes Resource Aware Long Short-Term Memory Model (RALSTMM) based incremental learning for edge devices in the IIoT. In RALSTMM edge devices can collect and analyze data for various predictive applications. The proposed RALSTMM can be deployed on these tiny edge devices and can be updated to enhance existing knowledge using newly generated data. The RALSTMM gradually learns from newly collected data by leveraging crucial information from previously analyzed data, thereby minimizing the resources needed for training. Hence, previous data that has been processed earlier would not undergo further analysis as the model has already extracted the necessary knowledge from it. The performance of RALSTMM has been evaluated with Mean Squared Error (MSE), accuracy, recall, precision, information criteria and processing time using three IoT testbed datasets. A comparative experimental demonstration of the RALSTMM with the existing LSTM proves the effectiveness of the RALSTMM by reducing processing time and maintaining its performance.

Incremental learning-based fault diagnosis systems (IFD) are widely used because of their ability to handle constantly updated fault data and types. However, the catastrophic forgetting problem remains the most crucial contemporary challenge facing IFD. This paper proposes an incremental fault diagnosis method based on metric feature distillation (MFD) and improved sample memory to solve this problem. First, the metric feature distillation is designed with metric learning methods and feature distillation. It uses distillation and triplet loss to constrain the network parameters of old and new tasks in the same feasible region, effectively alleviating catastrophic forgetting. Then, for a small amount of data that can be stored scenario, an improved sample memory strategy is introduced to reduce catastrophic forgetting further, called the center and hard sample memory (CAHM). It can better preserve the global information of the data, reducing the forgetting of old data information that needs to be preserved during the training process. Experimental results on CWRU and MFPT datasets verify the proposed method’s effectiveness.

When deep-learning classifiers try to learn new classes through supervised learning, they exhibit catastrophic forgetting issues. In this paper we propose the Gaussian Mixture Model - Incremental Learner (GMM-IL), a novel two-stage architecture that couples unsupervised visual feature learning with supervised probabilistic models to represent each class. The key novelty of GMM-IL is that each class is learnt independently of the other classes. New classes can be incrementally learnt using a small set of annotated images with no requirement to relearn data from existing classes. This enables the incremental addition of classes to a model, that can be indexed by visual features and reasoned over based on perception. Using Gaussian Mixture Models to represent the independent classes, we outperform a benchmark of an equivalent network with a Softmax head, obtaining increased accuracy for sample sizes smaller than 12 and increased weighted F1 score for 3 imbalanced class profiles in that sample range. This novel method enables new classes to be added to a system with only access to a few annotated images of the new class.

In practical applications, the raw input to a Knowledge Based Question Answering (KBQA) system may vary in forms, expressions, sources, etc. As a result, the actual input to the system may contain various errors caused by various noise in raw data and processes of transmission, transformation, translation, etc. As a result, it is significant to evaluate and enhance the robustness of a KBQA model to various noisy questions. In this paper, we generate 29 datasets of various noisy questions based on the original SimpleQuestions dataset to evaluate and enhance the robustness of a KBQA model, and propose a model which is more robust to various noisy questions. Compared with traditional methods, the main contribution in this paper is that we propose a method of generating datasets of different noisy questions to evaluate the robustness of a KBQA model, and propose a KBQA model which contains incremental learning and Mask Language Model (MLM) in the question answering process, so that our model is less affected by different kinds of noise in questions and achieves higher accuracies on datasets of different noisy questions, which shows its robustness. Experimental results show that our model achieves an average accuracy of 78.1% on these datasets and outperforms the baseline BERT-based model by an average margin of 5.0% with the similar training cost. In addition, further experiments show that our model is compatible with other pre-trained models such as ALBERT and ELECTRA.

Traditional deep convolutional neural networks have achieved excellent performance on various machine learning tasks. Still, they perform poorly in continuous data stream environments, where models trained on new datasets often suffer from a significant drop in performance on old datasets, a phenomenon known as “catastrophic forgetting.” Incremental learning can help solve the “catastrophic forgetting” problem in deep learning by learning new knowledge while retaining what has already been learned. In practice, incremental learning algorithms usually need to be deployed on edge devices with limited memory and restricted access to training data, facing the problems of high model complexity and imbalance between old and new categories of data. We propose an Adaptive Threshold Hierarchical Incremental Learning (ATHIL) method to address the above problems. Our proposed method does not require additional data and model storage space during the training process, combines local weight discrete coefficient thresholding and the mean nearest neighbor principle, uses a sparse matrix hierarchical masking network, and flexibly adjusts the network structure according to different tasks to achieve learning multiple image classification tasks in a single network. The experimental results show that the performance of the proposed method significantly outperforms existing methods on fine-grained classification datasets under three evaluation metrics.

In this study, a novel framework using multiple novelty detection filters is developed to learn a model of the normality of a robot’s visual perception, which is called multichannel novelty detection. Subsequently, the acquired model was used to highlight dissimilar perceptions when the robot explored an environment. The main purpose of fusing multiple novelty filters is that each novelty filter performs well in detecting specific types of novelties; therefore, a new framework is proposed that demonstrates a new way to combine multiple different purposed novelty detection filters together in order to yield an overall more robust novelty status on the visual features. To develop a multichannel novelty detection system, expectation- and appearance-based novelty detection models were used in this study. To become experts in detecting different types of novelty using these models, different features from the input image were extracted as inputs for the models. The expectation-based novelty detection model uses the MobileNetV2 deep network to extract the deep features of the input image, which is subsequently used to learn a sequential and temporal model of normality to detect novelty. By contrast, the appearance-based novelty detection model uses speeded up robust features (SURF), which provide more region-focused features within the input image, to identify whether a specific region of the image is novel. The proposed multichannel novelty detection system is a completely online and real-time approach that is very important for mobile robotics applications. The proposed framework was tested in three novel environments, and it was reported that the proposed multichannel novelty detection system performs better than expectation-based and appearance-based novelty filters separately. Statistically, in the three novel environments, the Matthews correlation coefficients are reported to be 0.94, 0.97, and 0.93, and $F_{1}$ scores are reported to be 0.95, 0.97, and 0.93, respectively, which proves that and can be concluded as almost perfect statistically.

Incremental learning involves processing continuous streams of information in real time without much incorporation of previous knowledge. It is crucial to humans and machine learning models. In this work, to alleviate the catastrophic forgetting characteristic of incremental learning in low-complexity models, we aim to combine the low complexity of single-task learning (STL) with the stability of multitask learning (MTL) through a two-stage similarity harmonizing mechanism (SHM). In the initialization stage, we construct an explainer by a multitask learning Shapley additive explanation (MTLSHAP) on a pretrained MTL model and return the model feature importance. In the incremental stage, the sample feature importance of incoming data computed by the explainer is compared with the model feature importance to establish disharmonies among samples. Hence, SHM impels the performance of an STL model to approach the designed MTL model. Numerical experiments show that the STL model improved by SHM is more stable while maintaining low complexity due to the acceptable additional cost of the initialization stage.

We introduce a neural network architecture that logarithmically reduces the number of self-rehearsal steps in the generative rehearsal of continually learned models. In continual learning (CL), training samples come in subsequent tasks, and the trained model can access only a current task. Contemporary CL methods employ generative models to replay previous samples and train them recursively with a combination of current and regenerated past data. This recurrence leads to superfluous computations as the same past samples are regenerated after each task, and the reconstruction quality successively degrades. In this work, we address these limitations and propose a new generative rehearsal architecture that requires, at most, a logarithmic number of retraining sessions for each sample. Our approach leverages the allocation of past data in a set of generative models such that most of them do not require retraining after a task. The experimental evaluation of our logarithmic continual learning approach shows the superiority of our method with respect to the state-of-the-art generative rehearsal methods.

Financial fraud cases are on the rise even with the current technological advancements. Due to the lack of inter-organization synergy and because of privacy concerns, authentic financial transaction data is rarely available. On the other hand, data-driven technologies like machine learning need authentic data to perform precisely in real-world systems. This study proposes a blockchain and smart contract-based approach to achieve robust Machine Learning (ML) algorithm for e-commerce fraud detection by facilitating inter-organizational collaboration. The proposed method uses blockchain to secure the privacy of the data. Smart contract deployed inside the network fully automates the system. An ML model is incrementally upgraded from collaborative data provided by the organizations connected to the blockchain. To incentivize the organizations, we have introduced an incentive mechanism that is adaptive to the difficulty level in updating a model. The organizations receive incentives based on the difficulty faced in updating the ML model. A mining criterion has been proposed to mine the block efficiently. And finally, the blockchain network is tested under different difficulty levels and under different volumes of data to test its efficiency. The model achieved 98.93% testing accuracy and 98.22% Fbeta score (recall-biased f measure) over eight incremental updates. Our experiment shows that both data volume and difficulty level of blockchain impacts the mining time. For difficulty level less than five, mining time and difficulty level has a positive correlation. For difficulty level two and three, less than a second is required to mine a block in our system. Difficulty level five poses much more difficulties to mine the blocks.

In this paper, a combined framework is proposed that includes Hyperdimensional (HD) computing, neural networks, and k-means clustering to fulfill a computationally simple incremental learning framework in a facial recognition system. The main advantages of HD computing algorithms are the simple computations needed, the high resistance to noise,and the ability to store excessive amounts of information into a single HD vector. The problem of incremental learning revolves around the ability to regularly update the knowledge within the framework to include new subjects in an online manner. Using an HD computing classifier proved efficient and highly accurate to implement an incremental learning framework as no re-training was required after each online update to the framework wbich is HD computing biggest advantage. Another advantage is that HD computing classifiers can achieve a high degree of generalization. The framework was tested on a total of 11 open source benchmark data sets. A number of experimental tests were preformed to ensure consistent performance of the framework under different conditions against different data sets.

Traditional models for various machine learning problems such as image classification perform well only under the assumption of a closed set. This implies that inputs must belong to the classes for which the models were trained. Data collected in the real world may not belong to any finite set of classes, and training a model with an infinite number of classes would obviously be impossible. Rather than incorrectly classifying outlier inputs that belong to unknown classes as members of one of the classes on which the model was trained, learning models must recognize and reject such data samples, or request human assistance in labeling them. For example, when a self-driving vehicle detects unfamiliar scenes or objects, it must notify the driver and hand over control. Various models have been proposed to address the open-set problem. However, the existing models are generally complex or use complicated techniques such as generative adversarial networks and auto-encoders, and their efficiency is not proportional to their complexity. In this study, we propose Triple-Sigmoid as a simple activation function comprised of three Sigmoid functions. Using Triple-Sigmoid in the last activation layer of any deep neural network model enables the model to recognize outliers. Although Triple-Sigmoid can be applied to a variety of machine learning problems, including semi-supervised learning, active learning, and incremental learning, we only investigated the open-set recognition problem in this work. The results of numerous experiments are presented to validate that substituting Triple-Sigmoid for conventional activations such as Softmax and Sigmoid not only retained performance in closed-set settings, but also significantly improved performance in open-set configurations. Furthermore, these results demonstrate that Triple-Sigmoid significantly outperformed existing state-of-the-art methods. Source code for the proposed model is available at https://github.com/dinhtuantran/triple-sigmoid.

This paper addresses an incremental learning problem, in which tasks are learned sequentially without access to the previously trained dataset. Catastrophic forgetting is a significant bottleneck to incremental learning as the network performs poorly on previous tasks when it is trained on a new task. We propose an adaptive model search method that uses a different part of the backbone network depending on an input image to mitigate catastrophic forgetting. Our model search method prevents forgetting by minimizing the update of critical parameters for the previous tasks while learning a new task. This model search involves a trainable network that selects the model structure for an input image. We also propose a method for approximating the loss function of previous tasks without the previous dataset. The critical parameters for the previous tasks can be found, considering the relationship between the approximated loss function and the parameters. The proposed framework is the first method of model search that can consider the performance of both current and previous tasks in the incremental learning problem. Empirical studies and theoretical analysis show that the proposed method outperforms other competitors for old and new tasks while requiring less computation.

This paper presents a novel Flow-based reinforcement learning strategy to model agent systems that can adapt to complex and dynamic problem environments by incrementally mastering their skills. It is inspired by the psychological notion of Flow that describes the optimal mental state experienced by an individual when they are fully immersed in a task and find it intrinsically rewarding to engage with. The proposed model presents an algorithm to describe the Flow experience such that agents can be trained through finer distinctions to the challenges across training time to maintain them in the Flow zone. In contrast to the traditional and incremental learning approaches that suffer from limitations associated with overfitting, the Flow-based model drives agent behaviours not simply through external goals but also through intrinsic curiosity to improve their skills and thus the performance levels. Experimental evaluations are conducted across two simulation environments on a maze navigation task and a reward collection task with comparisons against a generic reinforcement learning model and an incremental reinforcement learning model. The results reveal that these two models are prone to overfit under different design decisions and loose the ability to perform in dynamic variations of the tasks in varying degrees. Conversely, the proposed Flow-based model is capable of achieving near optimal solutions with random environmental factors, appropriately utilising the previously learned knowledge to identify robust solutions to complex problems.

Within the machine learning field, the main purpose of lifelong learning, also known as continuous learning, is to enable neural networks to learn continuously, as humans do. Lifelong learning accumulates the knowledge learned from previous tasks and transfers it to support the neural network in future tasks. This technique not only avoids the catastrophic forgetting problem with previous tasks when training new tasks, but also makes the model more robust with the temporal evolution. Motivated by the recent intervention of the lifelong learning technique, this paper presents a novel feature-based knowledge distillation method that differs from the existing methods of knowledge distillation in lifelong learning. Specifically, our proposed method utilizes the features from intermediate layers and compresses them in a unique way that involves global average pooling and fully connected layers. The authors then use the output of this branch network to deliver information from previous tasks to the model in the future. Extensive experiments show that our proposed model consistency outperforms the state-of-the-art baselines with the accuracy metric by at least two percent improvement under different experimental settings.

Existing techniques for incremental learning are computationally expensive and produce duplicate features leading to higher false positive and true negative rates. We propose a novel privacy-preserving intrusion detection pipeline for distributed incremental learning. Our pre-processing technique eliminates redundancies and selects unique features by following innovative extraction techniques. We use autoencoders with non-negativity constraints, which help us extract less redundant features. More importantly, the distributed intrusion detection model reduces the burden on the edge classifier and distributes the load among IoT and edge devices. Theoretical analysis and numerical experiments have shown lower space and time costs than state of the art techniques, with comparable classification accuracy. Extensive experiments with standard data sets and real-time streaming IoT traffic give encouraging results.

With the exponential rise of the number of IoT devices, the amount of data being produced is massive. Thus, it is unfeasible to send all the raw data directly to the cloud for processing, especially for data that is high dimensional. Training deep learning models incrementally evolves the model over time and eliminates the need to statically training the models with all the data. However, the integration of class incremental learning and the Internet of Things (IoT) is a new concept and is not yet mature. In the context of IoT and deep learning, the transmission cost of data in the edge-cloud architecture is a challenge. We demonstrate a novel sample selection method that discards certain training images on the IoT edge device that reduces transmission cost and still maintains class incremental learning performance. It can be unfeasible to transmit all parameters of a trained model back to the IoT edge device. Therefore, we propose an algorithm to find only the useful parameters of a trained model in an efficient way to reduce the transmission cost from the cloud to the edge devices. Results show that our proposed methods can effectively perform class-incremental learning in an edge-cloud setting.

The broad learning system (BLS) framework gives an efficient solution for training flat-structured feedforward networks and flat structured deep neural networks. However, the classical BLS model and other variants focus on the faultless situation only, where enhancement nodes, feature mapped nodes, and output weights of a BLS network are assumed to be realized in a perfect condition. When a trained BLS network suffers from coexistence of weight/node failures, the trained network has a greatly degradation in its performance if a countermeasure is not taken. In order to reduce the effect of weight/node failures on the BLS network’s performance, this paper proposes an objective function for enhancing the fault aware performance of BLS networks. The objective function contains a fault aware regularizer term which handles the weight/node failures. A learning algorithm is then derived based on the objective function. The simulation results show that the performance of the proposed fault aware BLS (FABLS) algorithm is superior to the classical BLS and two state-of-the-arts BLS algorithms, namely correntropy criterion BLS (CBLS) and weighted BLS (WBLS).

With the rapid development of research on machine learning models, especially deep learning, more and more endeavors have been made on designing new learning models with properties such as fast training with good convergence, and incremental learning to overcome catastrophic forgetting. In this paper, we propose a scalable wide neural network (SWNN), composed of multiple multi-channel wide RBF neural networks (MWRBF). The MWRBF neural network focuses on different regions of data and nonlinear transformations can be performed with Gaussian kernels. The number of MWRBFs for proposed SWNN is decided by the scale and difficulty of learning tasks. The splitting and iterative least squares (SILS) training method is proposed to make the training process easy with large and high dimensional data. Because the least squares method can find pretty good weights during the first iteration, only a few succeeding iterations are needed to fine tune the SWNN. Experiments were performed on different datasets including gray and colored MNIST data, hyperspectral remote sensing data (KSC, Pavia Center, Pavia University, and Salinas), and compared with main stream learning models. The results show that the proposed SWNN is highly competitive with the other models.

Human activity recognition (HAR) based on wearable sensors has attracted significant research attention in recent years due to its advantages in availability, accuracy, and privacy-friendliness. HAR baseline model is essentially a general-purpose classifier trained to recognized multiple activity patterns of most user types. It provides the input for subsequent steps of model personalization. Training a good baseline model is of fundamental importance because it has significant impacts on the ultimate HAR accuracy. In practice, baseline model training in HAR is a non-trivial problem that faces two challenges: insufficient training data and biased training data. This paper proposes a novel baseline model training scheme to tackle the two challenges using Deep InfoMax (DIM)-based unsupervised feature extraction and Broad Learning System (BLS)-based incremental learning, respectively. Experimental results demonstrate that the proposed scheme outperform conventional methods in terms of overall accuracy, computational efficiency, and the ability to adapt to dynamic scenarios with changing data characteristics.

Incremental learning requires a learning model to learn new tasks without forgetting the learned tasks continuously. However, when a deep learning model learns new tasks, it will catastrophically forget tasks it has learned before. Researchers have proposed methods to alleviate catastrophic forgetting; these methods only consider extracting features related to tasks learned before, suppression to extract features for unlearned tasks. As a result, when a deep learning model learns new tasks incrementally, the model needs to learn to extract the relevant features of the newly learned task quickly; this requires a significant change in the model’s behavior of extracting features, which increases the learning difficulty. Therefore, the model is caught in the dilemma of reducing the learning rate to retain existing knowledge or increasing the learning rate to learn new knowledge quickly. We present a study aiming to alleviate this problem by introducing self-supervised learning into incremental learning methods. We believe that the task-independent self-supervised learning signal helps the learning model extract features not only effective for the current learned task but also suitable for other tasks that have not been learned. We give a detailed algorithm combining self-supervised learning signals and incremental learning methods. Extensive experiments on several different datasets show that self-supervised signal significantly improves the accuracy of most incremental learning methods without the need for additional labeled data. We found that the self-supervised learning signal works best for the replay-based incremental learning method.

Machine learning has recently become a popular algorithm in building reliable intrusion detection systems (IDSs). However, most of the models are static and trained using datasets containing all targeted intrusions. If new intrusions emerge, these trained models must be retrained using old and new datasets to classify all intrusions accurately. In real-world situations, new threats continuously appear. Therefore, machine learning algorithms used for IDSs should have the ability to learn incrementally when these new intrusions emerge. To solve this issue, we propose T-DFNN. T-DFNN is an algorithm capable of learning new intrusions incrementally as they emerge. A T-DFNN model is composed of multiple deep feedforward neural network (DFNN) models connected in a tree-like structure. We examined our proposed algorithm using CICIDS2017, an open and widely used network intrusion dataset covering benign traffic and the most common network intrusions. The experimental results showed that the T-DFNN algorithm can incrementally learn new intrusions and reduce the catastrophic forgetting effect. The macro average of the F1-score of the T-DFNN model was over 0.85 for every retraining process. In addition, our proposed T-DFNN model has some advantages in several aspects compared to other models. Compared to the DFNN and Hoeffding tree models trained with a dataset containing only the latest targeted intrusions, our proposed T-DFNN model has higher F1-scores. Moreover, our proposed T-DFNN model has significantly shorter training times than a DFNN model trained using a dataset containing all targeted intrusions. Even though several factors can affect the duration of the training process, the T-DFNN algorithm shows promising results in solving the problem of ever-evolving network intrusion variants.

In this paper, we propose an efficient algorithm to accelerate the existing Broad Learning System (BLS) algorithm for new added nodes. The existing BLS algorithm computes the output weights from the pseudoinverse with the ridge regression approximation, and updates the pseudoinverse iteratively. As a comparison, the proposed BLS algorithm computes the output weights from the inverse Cholesky factor of the Hermitian matrix in the calculation of the pseudoinverse, and updates the inverse Cholesky factor efficiently. Since the Hermitian matrix in the definition of the pseudoinverse is smaller than the pseudoinverse, the proposed BLS algorithm can reduce the computational complexity, and usually requires less than $\frac {2}{3}$ of complexities with respect to the existing BLS algorithm. Our experiments on the Modified National Institute of Standards and Technology (MNIST) dataset show that the speedups in accumulative training time and each additional training time of the proposed BLS over the existing BLS are 24.81%~ 37.99% and 36.45%~ 58.96%, respectively, and the speedup in total training time is 37.99%. In our experiments, the proposed BLS and the existing BLS both achieve the same testing accuracy when the tiny differences (≤ 0.05%) caused by the numerical errors are neglected, and the above-mentioned tiny differences and numerical errors become zeroes and ignorable, respectively, when the ridge parameter is not too small.

Detecting outliers in real time from multivariate streaming data is a vital and challenging research topic in many areas. Recently introduced the incremental Local Outlier Factor (iLOF) approach and its variants have received considerable attention as they achieve high detection performance in data streams with varying distributions. However, these iLOF-based approaches still have some major limitations: i) Poor detection in high-dimensional data; ii) The difficulty of determining the proper nearest neighbor number $k$ ; iii) Instead of labeling the outlier, assigning a score to each sample that indicates the probability to be an outlier; iv) Inability to detect a long sequence (small cluster) of outliers. This article proposes a new robust outlier detection method (RiLOF) based on iLOF that can effectively overcome these limitations. In the RiLOF method, a novel metric called Median of Nearest Neighborhood Absolute Deviation (MoNNAD) has been developed that uses the median of the local absolute deviation of the samples LOF values. Unlike the previously reported LOF-based approaches, RiLOF is capable of achieving outlier detection in different data stream applications using the same hyperparameters. Extensive experiments performed on 15 different real-world data sets demonstrate that RiLOF remarkably outperforms 12 different state-of-the-art competitors.

Deep convolutional neural network (DCNN) based supervised learning is a widely practiced approach for large-scale image classification. However, retraining these large networks to accommodate new, previously unseen data demands high computational time and energy requirements. Also, previously seen training samples may not be available at the time of retraining. We propose an efficient training methodology and incrementally growing DCNN to learn new tasks while sharing part of the base network. Our proposed methodology is inspired by transfer learning techniques, although it does not forget previously learned tasks. An updated network, for learning new set of classes, is formed using previously learned convolutional layers (shared from initial part of base network) with addition of few newly added convolutional kernels included in the later layers of the network. We employed a `clone-and-branch' technique with calibration, which allows the network to learn new tasks (containing classes with similar features as old tasks) one after another without any performance loss in old tasks. We evaluated the proposed scheme on several recognition applications. The classification accuracy achieved by our approach is comparable to the regular incremental learning approach (where networks are updated with new training samples only, without any network sharing), while achieving energy efficiency, reduction in storage requirements, memory access and training time.

At present, cloud robots tend to be intelligent and cooperative. Based on this, we proposed a teaching method based on Imitation and a learning method that incorporates Incremental Learning and Meta Learning. We use Imitation Learning to teach robots, and more concretely, we propose a natural teaching method based on visual sense by using a depth camera, the robot can learn from the trajectory caught by the camera. Meta Learning helps robots understand the task and split it into some subtasks which enhances the level of generalization. Besides, once the circumstances change the robot can update the cloud database using Incremental Learning. Using proposed method, we make robots capable of learning and cooperating with other robots. It is no longer necessary for robots to learn based on a great number of data which is a shortcoming of traditional robots. The greatest advantage of this method is that we improve the learning efficiency of robots and enhance the level of generalization of the model. Our method was experimentally verified in a laboratory and the results indicated that the method improved the learning efficiency of robots.

At present, regions of the same class determined by Support Vector Machines (SVM) classifier, Support Vector Domain Description (SVDD) classifier and Deep Learning (DL) classifier may occupy regions of other classes or unknown classes in feature space. There exists a risk that samples of other classes or unknown classes are wrongly classified as a known class. In this paper, the Support Vector Domain Tightly Wrapping Description Design (SVDTWDD) method with appropriate rejection regions and the corresponding incremental learning algorithm are proposed to overcome the above problem. The main work includes: (1) We develop a construction algorithm of the tightly wrapping set for the homogeneous feature set; (2) Based on the homogeneous feature set and tightly wrapping set, a novel algorithm is presented for obtaining the tightly wrapping surface of the homogeneous feature region; (3) The method for constructing all the public regions outside of the tightly wrapping surface and the intersections of wrapping regions in two different tightly wrapping surfaces, as the rejection region of the classifier; (4) An incremental algorithm is also presented based on the SVD-TWDD method. The experimental results with UCI data sets show that the correct recognition rate of our proposed method is nearly100% even if with a low rejection rate.

In multi-agent networked systems, parameter estimation problems arising in many practical applications are often required to solve Non-Linear Least Squares (NLLS) problems with the usual objective function (i.e., sum of squared residuals). The aim is to estimate a global parameter of interest across the network, such that the discrepancy between the estimation model and the real output of the system is minimized. There are challenges to face when applying the conventional Gauss-Newton method, such as non-cooperation and prosaic learning behavior. In this paper, we propose two Gauss-Newton type fusion estimation algorithms for solving overdetermined NLLS optimization problems arising frequently in multi-agent networked environment. One is the cycle-based Gauss-Newton (CGN) algorithm that is more attractive in performance due to its distributed nature than its peer: the known centralized Gauss-Newton algorithm. On the basis of CGN, we put emphasis on developing a simple but effective learning scheme leveraging an incremental technique, which is distributed on each computing agent over network. Such scheme results in the Incremental Gauss-Newton (IGN) algorithm that achieves a clear increase on convergence rate at the expense of higher computation cost than the CGN algorithm as well as the centralized one by deeper learning over the networking cycle. Both algorithms utilize Gauss-Newton iteration update in a cyclic cooperative manner, which offers the flexibility in exploiting the network topology. We provide the detailed analysis and the sufficient conditions for convergence of proposed IGN algorithm. By applying to target localization in wireless sensor networks, the numerical results confirm our convergence analysis and show that the proposed incremental scheme outperforms the centralized one in term of convergence performance.

One major challenge for modern artificial neural networks (ANNs) is that they typically does not handle incremental learning well. In other words, while learning the new features, the performances of existing features usually deteriorate. This phenomenon is called catastrophic forgetting, which causes great problems for continuous, incremental, and intelligent learning. In this work, we propose a dynamic correction vector based algorithm to address both the bias problem from knowledge distillation and the overfitting problem. Specifically, we have made the following contributions: 1) we have designed a novel dynamic correction vector based algorithm; 2) we have proposed new loss functions accordingly. Experimental results on MNIST and CIFAR-100 datasets demonstrate that our technique can outperform state-of-the-art incremental learning methods by 4% on large datasets.

Class incremental learning is an online learning paradigm wherein the classes to be recognized are gradually increased with limited memory, storing only a partial set of examples of past tasks. At a task transition, we observe an unintentional imbalance of confidence or likelihood between the classes of the past and the new task. We argue that the imbalance aggravates a catastrophic forgetting for class incremental learning. We propose a simple yet effective learning objective to balance the confidence of classes of old tasks and new task in the class incremental learning setup. In addition, we compare various sample memory configuring strategies and propose a novel sample memory management policy to alleviate the forgetting further. The proposed method outperforms the state of the arts in many evaluation metrics including accuracy and forgetting F by a large margin (up to 5.71% in A10 and 17.1% in F10) in extensive empirical validations on multiple visual recognition datasets such as CIFAR100, TinyImageNet and a subset of the ImageNet.

In recent years, the quantity of digital data being generated has increased considerably and is overwhelming the storage capacity. To overcome this problem, acquiring more and larger data storage is the simplest solution. But this solution is rather costly and may produce poisonous electronic garbage. A new fast and memory-efficient algorithm for learning and classifying these data without increasing the space and time complexities more than those of current learning and classifying algorithms is desirable. Although many one-pass online or incremental learning algorithms based on hyperellipsoidal functions for streaming data without retaining any learned data in fixed storage have been successfully developed for training streaming data, achieving high accuracy of any testing dataset is unstable and uncontrollable, depending on the experimental datasets. This paper proposes an improvement to these one-pass and fixed-storage learning algorithms so that the high accuracy of testing data can be significantly improved and stabilized, regardless of the experimental datasets. The concept is based on animal recoil behavior, which occurs when an animal moves away suddenly from something it dislikes. The behavior is mathematically modeled in forms of shrinking and shifting the hyperellipsoidal function during the training period to improve testing accuracy. The experimental results on 15 datasets improved the accuracy up to 8.16 % and also provided the highest or near-highest accuracy results in 10 datasets when compared to other algorithms.

Catastrophic forgetting is a key challenge for class-incremental learning with deep neural networks, where the performance decreases considerably while dealing with long sequences of new classes. To tackle this issue, in this paper, we propose a new exemplar-supported representation for incremental learning (ESRIL) approach that consists of three components. First, we use memory aware synapses (MAS) pre-trained on the ImageNet to retain the ability of robust representation learning and classification for old classes from the perspective of the model. Second, exemplar-based subspace clustering (ESC) is utilized to construct the exemplar set, which can keep the performance from various views of the data. Third, the nearest class multiple centroids (NCMC) is used as the classifier to save the training cost of the fully connected layer of MAS when the criterion is met. Intensive experiments and analyses are presented to show the influence of various backbone structures and the effectiveness of different components in our model. Experiments on several general-purpose and fine-grained image recognition datasets have fully demonstrated the efficacy of the proposed methodology.

Learning streaming data with limited size of memory storage becomes an interesting problem. Although there have been several learning methods recently proposed, based on the interesting concept of discard-after-learn, the performance of these issues: the learning speed, number of redundant neurons, and classification accuracy of these methods can be further improved in terms of faster speed, less number of neurons, and higher accuracy. The following new concepts and approaches were proposed in this paper: (1) a more generic structure of hyper-ellipsoidal function called Scalable Hyper-Ellipsoidal Function (SHEF) capable of handling the problem of a curse of dimensionality by introducing a regularization parameter into the covariance matrix of SHEF; (2) a new recursive function to update the covariance matrix of SHEF based on only the incoming data chunk; (3) a fast and easy conditions to test the states of being overlapped, inside, and touching of two SHEFs; (4) a new distance measure for determining the class of a queried datum based on the projected distance on only one discriminant vector, namely the Projection Ratio. The experimental results show the significant improvement when compared with the results from VLLDA, ILDA, LOL, VEBF, and CIL in terms of classification accuracy, the number of generated neurons, and computational time.

Caches are widely applied to improve data delivery performance in distributed systems like edge networks and content delivery networks (CDNs). We consider caching mechanism in those networks that deliver contents to end users. The challenge comes from the dynamic content distribution problem. The distribution of data popularity is highly skewed and changing over time. Besides, the access pattern of the user requests also varies over time. Some learning algorithms for edge caching problems need to rebuild a new model periodically to adapt to system dynamics, where the knowledge learned from the past is discarded. Besides, each model updating needs a large amount of data, leading to outdated models for consecutive user requests. Inspired by the success of incremental learning approaches in processing massive data in real time, we propose an incremental learning based framework at an edge caching server. The incremental learning algorithm is used to preserve valuable knowledge and to adapt to dynamic workloads faster. We implement our incremental learning based cache system prototype and evaluate its performance under various real-world workloads. The experimental results show that our algorithm can boost cache hit ratio for dynamic workloads compared with the state-of-the-art caching algorithms.

Event cameras are bio-inspired vision sensors measuring brightness changes (referred to as an `event') for each pixel independently, instead of capturing brightness images at a fixed rate using conventional cameras. Asynchronous event data mixed with noise information is challenging for event-based vision tasks. In this paper, we propose a broad learning network for object detection using the event data. The broad learning network consists of two distinct layers, a feature-node layer and an enhancement-node layer. Different to convolutional neural networks, the broad learning network can be extended by adding nodes into layers during training. We design a gradient descent algorithm to train network parameters, which creates an event-based broad learning network in an end-to-end manner. Our model outperforms state-of-the-art models, specifically, because of the small scale and increased speed displayed by our model during training. This demonstrates the superiority of event cameras towards online training and inference.

Classification over data streams is a crucial task of explosive social stream mining and computing. Efficient learning techniques provide high-quality services in the aspect of content distribution and event browsing. Due to the concept drift and concept evolution in data streams, the classification performance degrades drastically over time. Many existing methods utilize supervised and unsupervised learning strategies. However, supervised strategies require labeled emerging records to update the classifiers, which is unfeasible to work in the practical social stream applications. Although unsupervised strategies are popularly applied to detect concept evolution, it takes tremendous run-time computation cost to run online clustering. To this end, in this paper, we address these major challenges of social stream classification by proposing an efficient incremental semi-supervised classification method named CODES (Classification Over Drifting and Evolving Stream). The proposed CODES method consists of an efficient incremental semi-supervised learning module and a dynamic novelty threshold update module. Thus, in the drifting and evolving social streams, CODES is able to provide: 1) semi-supervised learning ability to eliminate dependency on the labels of emerging records; 2) fast incremental learning with real-time update ability to tackle concept drift; 3) efficient novel class detection ability to tackle concept evolution. Extensive experiments are conducted on several real-world datasets. The results indicate a higher performance than several state-of-the-art methods. CODES achieves efficient learning performance over drifting and evolving social streams, which improves practical significance in the real-world social stream applications.

Incremental learning or online learning as a branch of machine learning has attracted more attention recently. For large-scale problems and dynamic data problem, incremental learning overwhelms batch learning, because of its efficient treatment for new data. However, class imbalance problem, which always appears in online classification brings a considerable challenge for incremental learning. The serious class imbalance problem may directly lead to a useless learning system. Cost-sensitive learning is an important learning paradigm for class imbalance problems and widely used in many applications. In this article, we propose an incremental cost-sensitive learning method to tackle the class imbalance problems in the online situation. This proposed algorithm is based on a novel cost-sensitive support vector machine, which uses the Linear-exponential (LINEX) loss to implement high cost for minority class and low cost for majority class. Using the half-quadratic optimization, we first put forward the algorithm for the cost-sensitive support vector machine, called CSLINEX-SVM*. Then we propose the incremental cost-sensitive algorithm, ICSL-SVM. The results of numeric experiments demonstrate that the proposed incremental algorithm outperforms some conventional batch algorithms except the proposed CSLINEX-SVM*.