1,579 results on '"incremental learning"'
Search Results
2. Incremental Learning Using a Grow-and-Prune Paradigm With Efficient Neural Networks
- Author
-
Niraj K. Jha, Hongxu Yin, and Xiaoliang Dai
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,Computer science ,Inference ,02 engineering and technology ,Neural network synthesis ,Machine learning ,computer.software_genre ,Machine Learning (cs.LG) ,0202 electrical engineering, electronic engineering, information engineering ,Computer Science (miscellaneous) ,Redundancy (engineering) ,Neural and Evolutionary Computing (cs.NE) ,computer.programming_language ,Artificial neural network ,business.industry ,Computer Science - Neural and Evolutionary Computing ,020207 software engineering ,Computer Science Applications ,Human-Computer Interaction ,Scratch ,Incremental learning ,Deep neural networks ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer ,MNIST database ,Information Systems - Abstract
Deep neural networks (DNNs) have become a widely deployed model for numerous machine learning applications. However, their fixed architecture, substantial training cost, and significant model redundancy make it difficult to efficiently update them to accommodate previously unseen data. To solve these problems, we propose an incremental learning framework based on a grow-and-prune neural network synthesis paradigm. When new data arrive, the neural network first grows new connections based on the gradients to increase the network capacity to accommodate new data. Then, the framework iteratively prunes away connections based on the magnitude of weights to enhance network compactness, and hence recover efficiency. Finally, the model rests at a lightweight DNN that is both ready for inference and suitable for future grow-and-prune updates. The proposed framework improves accuracy, shrinks network size, and significantly reduces the additional training cost for incoming data compared to conventional approaches, such as training from scratch and network fine-tuning. For the LeNet-300-100 (LeNet-5) neural network architectures derived for the MNIST dataset, the framework reduces training cost by up to 64% (67%), 63% (63%), and 69% (73%) compared to training from scratch, network fine-tuning, and grow-and-prune from scratch, respectively. For the ResNet-18 architecture derived for the ImageNet dataset (DeepSpeech2 for the AN4 dataset), the corresponding training cost reductions against training from scratch, network fine-tunning, and grow-and-prune from scratch are 64% (67%), 60% (62%), and 72% (71%), respectively. Our derived models contain fewer network parameters but achieve higher accuracy relative to conventional baselines.
- Published
- 2022
- Full Text
- View/download PDF
3. Fast and Progressive Misbehavior Detection in Internet of Vehicles Based on Broad Learning and Incremental Learning Systems
- Author
-
Yushan Zhu, Haixia Gu, Han Shuangshuang, Xiao Wang, Fei-Yue Wang, and Linyao Yang
- Subjects
Computer Networks and Communications ,Computer science ,business.industry ,Deep learning ,Key features ,Machine learning ,computer.software_genre ,Computer Science Applications ,Nonlinear system ,Hardware and Architecture ,Signal Processing ,Incremental learning ,Scalability ,In vehicle ,The Internet ,Artificial intelligence ,Raw data ,business ,computer ,Information Systems - Abstract
In recent years, deep learning has been widely used in vehicle misbehavior detection and has attracted great attention due to its powerful nonlinear mapping ability. However, because of the large number of network parameters, the training processes of these methods are time-consuming. Besides, the existing detection methods lack scalability, thus they are not suitable for Internet of Vehicles (IoV) where new data is constantly generated. In this paper, the concept of Broad Learning System (BLS) is innovatively introduced into vehicle misbehavior detection. In order to make better use of vehicle information, key features are firstly extracted from the collected raw data. Then, a BLS is established, which is able to calculate the connection weight of the network efficiently and effectively by ridge regression approximation. Finally, the system can be updated and refined by incremental learning algorithm based on the newly generated data in IoV. Experimental results show that the proposed method performs much better than deep learning or traditional classifiers, and could update and optimize the old model fastly and progressively while improving the system’s misbehavior detection accuracy.
- Published
- 2022
- Full Text
- View/download PDF
4. HarMI: Human Activity Recognition Via Multi-Modality Incremental Learning
- Author
-
Yujun Li, Fuzhen Zhuang, Jingjing Gu, Dongxiao Yu, Zhaochun Ren, Yang Yang, Xiao Zhang, and Hongzheng Yu
- Subjects
business.industry ,Computer science ,Machine learning ,computer.software_genre ,Multi modality ,Computer Science Applications ,Machine Learning ,Activity recognition ,Wearable Electronic Devices ,Text mining ,Health Information Management ,Incremental learning ,Humans ,Human Activities ,Neural Networks, Computer ,Smartphone ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,computer ,Biotechnology - Abstract
Nowadays, with the development of various kinds of sensors in smartphones or wearable devices, human activity recognition (HAR) has been widely researched and has numerous applications in healthcare, smart city, etc. Many techniques based on hand-crafted feature engineering or deep neural network have been proposed for sensor based HAR. However, these existing methods usually recognize activities offline, which means the whole data should be collected before training, occupying large-capacity storage space. Moreover, once the offline model training finished, the trained model can't recognize new activities unless retraining from the start, thus with a high cost of time and space. In this paper, we propose a multi-modality incremental learning model, called HarMI, with continuous learning ability. The proposed HarMI model can start training quickly with little storage space and easily learn new activities without storing previous training data. In detail, we first adopt attention mechanism to align heterogeneous sensor data with different frequencies. In addition, to overcome catastrophic forgetting in incremental learning, HarMI utilizes the elastic weight consolidation and canonical correlation analysis from a multi-modality perspective. Extensive experiments based on two public datasets demonstrate that HarMI can achieve a superior performance compared with several state-of-the-arts.
- Published
- 2022
- Full Text
- View/download PDF
5. Polarity Classification of Social Media Feeds Using Incremental Learning — A Deep Learning Approach
- Author
-
Sathya Madhusudhanan and Suresh Jaganathan
- Subjects
business.industry ,Polarity (physics) ,Computer science ,Applied Mathematics ,Deep learning ,computer.software_genre ,Computer Graphics and Computer-Aided Design ,Signal Processing ,Incremental learning ,Social media ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,computer ,Natural language processing - Published
- 2022
- Full Text
- View/download PDF
6. Secure and efficient parameters aggregation protocol for federated incremental learning and its applications
- Author
-
Xiaoying Wang, Qintai Yang, Arthur Sandor Voundi Koe, Zhiwei Liang, Haitao Li, Qingwu Wu, and Xiaodong Zhang
- Subjects
Human-Computer Interaction ,Artificial Intelligence ,business.industry ,Computer science ,Incremental learning ,business ,Protocol (object-oriented programming) ,Software ,Theoretical Computer Science ,Computer network - Published
- 2021
- Full Text
- View/download PDF
7. Online Semisupervised Broad Learning System for Industrial Fault Diagnosis
- Author
-
Xiaokun Pu and Chunguang Li
- Subjects
Training set ,Manifold regularization ,Generalization ,business.industry ,Computer science ,Deep learning ,Supervised learning ,Process (computing) ,Construct (python library) ,Machine learning ,computer.software_genre ,Fault (power engineering) ,Computer Science Applications ,Data modeling ,Control and Systems Engineering ,Incremental learning ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,computer ,Information Systems - Abstract
Recently, broad learning system (BLS) has been introduced to solve industrial fault diagnosis problems and has achieved impressive performance. As a flat network, BLS enjoys a simple linear structure, which enables BLS to train and update the model efficiently in an incremental manner, and it potentially has better generalization capacity than deep learning methods when training data are limited. The basic BLS is a supervised learning method that requires all the training data to be labeled. However, in many practical industrial scenarios, data labels are usually difficult to obtain. Existing semisupervised variant uses manifold regularization framework to capture the information of unlabeled data, however, such a method will sacrifice the incremental learning capacity of BLS. Considering that in many practical applications, training data are sequentially generated, in this article, an online semisupervised broad learning system (OSSBLS) is proposed for fault diagnosis in these cases. The proposed method not only can efficiently construct and incrementally update the model, but also can take advantage of unlabeled data to improve the model's diagnostic performance. Experimental results on the Tennessee Eastman process and a real-world air compressor working process demonstrate the superiority of OSSBLS in terms of both diagnostic performance and time consumption.
- Published
- 2021
- Full Text
- View/download PDF
8. Fast and robust supervised machine learning approach for classification and prediction of Parkinson’s disease onset
- Author
-
Kadambari K and Lavanya Madhuri Bollipo
- Subjects
Parkinson's disease ,business.industry ,Computer science ,Biomedical Engineering ,Computational Mechanics ,Disease ,medicine.disease ,Machine learning ,computer.software_genre ,Computer Science Applications ,Support vector machine ,Frank–Wolfe algorithm ,Motor system ,Incremental learning ,medicine ,Radiology, Nuclear Medicine and imaging ,Artificial intelligence ,business ,computer - Abstract
Parkinson’s disease (PD) is an incurable long-term neurodegenerative disorder that mainly influence the motor system and eventually results in significant morbidity. The use of computational tools ...
- Published
- 2021
- Full Text
- View/download PDF
9. Broad Learning System Based on Maximum Correntropy Criterion
- Author
-
Badong Chen, Yunfei Zheng, Weiqun Wang, and Shiyuan Wang
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,Computer Networks and Communications ,Computer science ,Gaussian ,Machine Learning (stat.ML) ,02 engineering and technology ,Machine Learning (cs.LG) ,symbols.namesake ,Statistics - Machine Learning ,Artificial Intelligence ,Robustness (computer science) ,0202 electrical engineering, electronic engineering, information engineering ,Moore–Penrose pseudoinverse ,Minimum mean square error ,ComputingMilieux_THECOMPUTINGPROFESSION ,business.industry ,Pattern recognition ,Computer Science Applications ,Incremental learning ,Outlier ,symbols ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Software ,Discriminative learning - Abstract
As an effective and efficient discriminative learning method, Broad Learning System (BLS) has received increasing attention due to its outstanding performance in various regression and classification problems. However, the standard BLS is derived under the minimum mean square error (MMSE) criterion, which is, of course, not always a good choice due to its sensitivity to outliers. To enhance the robustness of BLS, we propose in this work to adopt the maximum correntropy criterion (MCC) to train the output weights, obtaining a correntropy based broad learning system (C-BLS). Thanks to the inherent superiorities of MCC, the proposed C-BLS is expected to achieve excellent robustness to outliers while maintaining the original performance of the standard BLS in Gaussian or noise-free environment. In addition, three alternative incremental learning algorithms, derived from a weighted regularized least-squares solution rather than pseudoinverse formula, for C-BLS are developed.With the incremental learning algorithms, the system can be updated quickly without the entire retraining process from the beginning, when some new samples arrive or the network deems to be expanded. Experiments on various regression and classification datasets are reported to demonstrate the desirable performance of the new methods.
- Published
- 2021
- Full Text
- View/download PDF
10. Multimodal continual learning with sonographer eye-tracking in fetal ultrasound
- Author
-
Yifan Cai, Aris T. Papageorghiou, Harshita Sharma, Lior Drukker, Arijit Patra, Pierre Chatelain, and J. Alison Noble
- Subjects
Forgetting ,business.industry ,Computer science ,Continual learning ,Machine learning ,computer.software_genre ,Article ,Image (mathematics) ,Reduction (complexity) ,Incremental learning ,Sonographer ,Eye tracking ,Artificial intelligence ,Extended time ,business ,computer - Abstract
Deep networks have been shown to achieve impressive accuracy for some medical image analysis tasks where large datasets and annotations are available. However, tasks involving learning over new sets of classes arriving over extended time is a different and difficult challenge due to the tendency of reduction in performance over old classes while adapting to new ones. Controlling such a 'forgetting' is vital for deployed algorithms to evolve with new arrivals of data incrementally. Usually, incremental learning approaches rely on expert knowledge in the form of manual annotations or active feedback. In this paper, we explore the role that other forms of expert knowledge might play in making deep networks in medical image analysis immune to forgetting over extended time. We introduce a novel framework for mitigation of this forgetting effect in deep networks considering the case of combining ultrasound video with point-of-gaze tracked for expert sonographers during model training. This is used along with a novel weighted distillation strategy to reduce the propagation of effects due to class imbalance.
- Published
- 2022
- Full Text
- View/download PDF
11. SpaceNet: Make Free Space for Continual Learning
- Author
-
Ghada Sokar, Mykola Pechenizkiy, Decebal Constantin Mocanu, Data Mining, Process Science, EAISI Health, EAISI Foundational, Digital Society Institute, and Datamanagement & Biometrics
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,0209 industrial biotechnology ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Cognitive Neuroscience ,cs.LG ,Lifelong learning ,Computer Science - Computer Vision and Pattern Recognition ,Inference ,Machine Learning (stat.ML) ,02 engineering and technology ,Machine learning ,computer.software_genre ,Regularization (mathematics) ,Machine Learning (cs.LG) ,020901 industrial engineering & automation ,Statistics - Machine Learning ,Artificial Intelligence ,Robustness (computer science) ,Deep neural networks ,0202 electrical engineering, electronic engineering, information engineering ,cs.CV ,Sparse training ,Forgetting ,Artificial neural network ,business.industry ,stat.ML ,Computer Science Applications ,Class incremental learning ,Incremental learning ,020201 artificial intelligence & image processing ,Continual learning ,Artificial intelligence ,business ,computer ,MNIST database - Abstract
The continual learning (CL) paradigm aims to enable neural networks to learn tasks continually in a sequential fashion. The fundamental challenge in this learning paradigm is catastrophic forgetting previously learned tasks when the model is optimized for a new task, especially when their data is not accessible. Current architectural-based methods aim at alleviating the catastrophic forgetting problem but at the expense of expanding the capacity of the model. Regularization-based methods maintain a fixed model capacity; however, previous studies showed the huge performance degradation of these methods when the task identity is not available during inference (e.g. class incremental learning scenario). In this work, we propose a novel architectural-based method referred as SpaceNet for class incremental learning scenario where we utilize the available fixed capacity of the model intelligently. SpaceNet trains sparse deep neural networks from scratch in an adaptive way that compresses the sparse connections of each task in a compact number of neurons. The adaptive training of the sparse connections results in sparse representations that reduce the interference between the tasks. Experimental results show the robustness of our proposed method against catastrophic forgetting old tasks and the efficiency of SpaceNet in utilizing the available capacity of the model, leaving space for more tasks to be learned. In particular, when SpaceNet is tested on the well-known benchmarks for CL: split MNIST, split Fashion-MNIST, and CIFAR-10/100, it outperforms regularization-based methods by a big performance gap. Moreover, it achieves better performance than architectural-based methods without model expansion and achieved comparable results with rehearsal-based methods, while offering a huge memory reduction., Published in Neurocomputing Journal
- Published
- 2021
- Full Text
- View/download PDF
12. Online Tensor-Based Learning Model for Structural Damage Detection
- Author
-
Ali Anaissi, Seid Miad Zandavi, and Basem Suleiman
- Subjects
Damage detection ,General Computer Science ,business.industry ,Computer science ,Online learning ,020206 networking & telecommunications ,02 engineering and technology ,Machine learning ,computer.software_genre ,Online analysis ,Tensor (intrinsic definition) ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Anomaly detection ,Artificial intelligence ,Structural health monitoring ,business ,computer - Abstract
The online analysis of multi-way data stored in a tensor has become an essential tool for capturing the underlying structures and extracting the sensitive features that can be used to learn a predictive model. However, data distributions often evolve with time and a current predictive model may not be sufficiently representative in the future. Therefore, incrementally updating the tensor-based features and model coefficients are required in such situations. A new efficient tensor-based feature extraction, named Nesterov Stochastic Gradient Descent (NeSGD), is proposed for online (CP) decomposition. According to the new features obtained from the resultant matrices of NeSGD, a new criterion is triggered for the updated process of the online predictive model. Experimental evaluation in the field of structural health monitoring using laboratory-based and real-life structural datasets shows that our methods provide more accurate results compared with existing online tensor analysis and model learning. The results showed that the proposed methods significantly improved the classification error rates, were able to assimilate the changes in the positive data distribution over time, and maintained a high predictive accuracy in all case studies.
- Published
- 2021
- Full Text
- View/download PDF
13. Deep Neural Networks Techniques using for Learning Automata Based Incremental Learning Method
- Author
-
C. Swetha Reddy et.al
- Subjects
Learning automata ,business.industry ,Computer science ,General Mathematics ,Machine learning ,computer.software_genre ,Training (civil) ,Education ,Visual recognition ,Computational Mathematics ,Visual language ,Computational Theory and Mathematics ,Incremental learning ,Deep neural networks ,Artificial intelligence ,business ,Set (psychology) ,computer ,MNIST database - Abstract
Surprisingly comprehensive learning methods are implemented in many large learning machine data, such as visual recognition and visual language processing. Much of the success of advanced training in recent years is due to leadership training, which requires a set of information for specific tasks, before such training. However, in reality, selected tasks related to personal study are gradually accumulated over time as it is difficult to collect and submit training data manually. It provides a way to continue learning some information columns and examples of steps that are specific to the new class and called additional learning. In this post, we recommend the best machine training method for further training for deep neural networks. The basic idea is to learn a deep system with strong connections that can be "activated" or "turned off" at different stages. The approach you suggest allows you to reduce the distribution of old services as you learn new for example new training, which increases the effectiveness of training in the additional training phase. Experiments with MNIST and CIFAR-100 show that our approach can be implemented in other long-term phases in deep neuron models and achieve better results from zero-base training.
- Published
- 2021
- Full Text
- View/download PDF
14. On the Challenges of Open World Recognition Under Shifting Visual Domains
- Author
-
Barbara Caputo, Massimiliano Mancini, Fabio Cermelli, and Dario Fontanel
- Subjects
FOS: Computer and information sciences ,Control and Optimization ,Computer science ,Generalization ,Computer Vision and Pattern Recognition (cs.CV) ,media_common.quotation_subject ,Computer Science - Computer Vision and Pattern Recognition ,Biomedical Engineering ,visual learning ,02 engineering and technology ,Machine learning ,computer.software_genre ,Computer Science - Robotics ,Artificial Intelligence ,020204 information systems ,Deep Learning ,Computer Vision ,Incremental Learning ,Open World Recognition ,Domain Shift ,0202 electrical engineering, electronic engineering, information engineering ,Set (psychology) ,Function (engineering) ,media_common ,Point (typography) ,business.industry ,Mechanical Engineering ,Cognitive neuroscience of visual object recognition ,Deep learning for visual perception ,Computer Science Applications ,Variety (cybernetics) ,Human-Computer Interaction ,recognition ,Control and Systems Engineering ,Benchmark (computing) ,Robot ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,Robotics (cs.RO) ,computer - Abstract
Robotic visual systems operating in the wild must act in unconstrained scenarios, under different environmental conditions while facing a variety of semantic concepts, including unknown ones. To this end, recent works tried to empower visual object recognition methods with the capability to i) detect unseen concepts and ii) extended their knowledge over time, as images of new semantic classes arrive. This setting, called Open World Recognition (OWR), has the goal to produce systems capable of breaking the semantic limits present in the initial training set. However, this training set imposes to the system not only its own semantic limits, but also environmental ones, due to its bias toward certain acquisition conditions that do not necessarily reflect the high variability of the real-world. This discrepancy between training and test distribution is called domain-shift. This work investigates whether OWR algorithms are effective under domain-shift, presenting the first benchmark setup for assessing fairly the performances of OWR algorithms, with and without domain-shift. We then use this benchmark to conduct analyses in various scenarios, showing how existing OWR algorithms indeed suffer a severe performance degradation when train and test distributions differ. Our analysis shows that this degradation is only slightly mitigated by coupling OWR with domain generalization techniques, indicating that the mere plug-and-play of existing algorithms is not enough to recognize new and unknown categories in unseen domains. Our results clearly point toward open issues and future research directions, that need to be investigated for building robot visual systems able to function reliably under these challenging yet very real conditions. Code available at https://github.com/DarioFontanel/OWR-VisualDomains, RAL/ICRA 2021
- Published
- 2021
- Full Text
- View/download PDF
15. An Incremental Learning Based Convolutional Neural Network Model for Large-Scale and Short-Term Traffic Flow
- Author
-
Yanli Shao, Bin Chen, Feng Yu, and Jinglong Fang
- Subjects
Information Systems and Management ,Scale (ratio) ,business.industry ,Computer science ,Traffic flow ,Machine learning ,computer.software_genre ,Convolutional neural network ,Computer Science Applications ,Term (time) ,Artificial Intelligence ,Incremental learning ,Artificial intelligence ,business ,computer - Abstract
Traffic flow prediction is very important for smooth road conditions in cities and convenient travel for residents. With the explosive growth of traffic flow data size, traditional machine learning algorithms cannot fit large-scale training data effectively and the deep learning algorithms do not work well because of the huge training and update costs, and the prediction accuracy may need to be further improved when an emergency affecting traffic occurs. In this study, an incremental learning based convolutional neural network model, TF-net, is proposed to achieve the efficient and accurate prediction of large-scale and short-term traffic flow. The key idea is to introduce the uncertainty features into the model without increasing the training cost to improve the prediction accuracy. Meanwhile, based on the idea of combining incremental learning with active learning, a certain percentage of typical samples in historical traffic flow data are sampled to fine-tune the prediction model, so as to further improve the prediction accuracy for special situations and ensure the real-time requirement. The experimental results show that the proposed traffic flow prediction model has better performance than the existing methods.
- Published
- 2021
- Full Text
- View/download PDF
16. An object recognition system based on convolutional neural networks and angular resolutions
- Author
-
Achmad Lukman and Chuan-Kai Yang
- Subjects
Network architecture ,Computer Networks and Communications ,business.industry ,Computer science ,Deep learning ,Process (computing) ,Cognitive neuroscience of visual object recognition ,020207 software engineering ,Pattern recognition ,02 engineering and technology ,Object (computer science) ,Convolutional neural network ,Hardware and Architecture ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,Media Technology ,Artificial intelligence ,business ,Software - Abstract
The development of 3D object recognition often requires a huge amount of data in the training process, especially when deep learning methods are involved so that the training can be convergent. The problem is that the availability of free 3D object datasets is usually quite limited, so some researchers have proposed several techniques to overcome this problem. In this work, we propose a novel algorithm, making use of angular resolutions and convolutional neural networks for 3D object recognition, and it collects image shapes or contours from real objects by placing them on a rotating display to record the appearances from multiple angular views. The chosen angular resolution is in the range of 0-180 degrees, and the selection of viewing angle is done by a binary search. We have conducted a comparative experiment on the accuracy of 6 well-known network architectures, including GoogleNet, CaffeNet, SqueezeNet, ResNet18, ResNet32, and ResNet50, to see how far these architecture networks can adapt to the angular resolution techniques that we propose for the classification of objects outside the lab environment. We also propose another way with the use of incremental learning, where we integrate our proposed method that uses GoogleNet model with two existing weights pre-trained models, i.e., AlexNet and VGG16. In other words, our proposed method helps address the limitations of other models with the weights of existing pre-trained methods to recognize new classes that were not recognized.
- Published
- 2021
- Full Text
- View/download PDF
17. Concept-Cognitive Learning Model for Incremental Concept Learning
- Author
-
Yong Shi, Yunlong Mi, Wenqi Liu, and Jinhai Li
- Subjects
Context model ,Computer science ,business.industry ,02 engineering and technology ,Machine learning ,computer.software_genre ,Computer Science Applications ,Data modeling ,Human-Computer Interaction ,Control and Systems Engineering ,020204 information systems ,Concept learning ,Incremental learning ,Still face ,Cognitive learning ,0202 electrical engineering, electronic engineering, information engineering ,Task analysis ,020201 artificial intelligence & image processing ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,computer ,Classifier (UML) ,Software - Abstract
Concept-cognitive learning (CCL) is an emerging field of concerning incremental concept learning and dynamic knowledge processing in the context of dynamic environments. Although CCL has been widely researched in theory, the existing studies of CCL have one problem: the concepts obtained by CCL systems do not have generalization ability. In the meantime, the existing incremental algorithms still face some challenges that: 1) classifiers have to adapt gradually and 2) the previously acquired knowledge should be efficiently utilized. To address these problems, based on the advantage that CCL can naturally integrate new data into itself for enhancing flexibility of concept learning, we first propose a new CCL model (CCLM) to extend the classical methods of CCL, which is not only a new classifier but also good at incremental learning. Unlike the existing CCL systems, the theory of CCLM is mainly based on a formal decision context rather than a formal context. In learning concepts from dynamic environments, we show that CCLM can naturally incorporate new data into itself with a sufficient theoretical guarantee for incremental learning. For classification task and knowledge storage, our results on various data sets demonstrate that CCLM can simultaneously: 1) achieve the state-of-the-art static and dynamic classification task and 2) directly accomplish preservation of previously acquired knowledge (or concepts) under dynamic environments.
- Published
- 2021
- Full Text
- View/download PDF
18. Incremental learning framework for real‐world fraud detection environment
- Author
-
Farzana Anowar and Samira Sadaoui
- Subjects
Computational Mathematics ,Artificial Intelligence ,Computer science ,business.industry ,Incremental learning ,Artificial intelligence ,Machine learning ,computer.software_genre ,business ,computer ,Imbalanced data - Published
- 2021
- Full Text
- View/download PDF
19. Bringing AI to the edge: a formal M&S specification to deploy effective IoT architectures
- Author
-
José Luis Risco Martín, Román Cárdenas, and Patricia Arroba
- Subjects
Networking and Internet Architecture (cs.NI) ,FOS: Computer and information sciences ,050210 logistics & transportation ,021103 operations research ,Computer Science - Artificial Intelligence ,business.industry ,Computer science ,Distributed computing ,05 social sciences ,0211 other engineering and technologies ,Model-based systems engineering ,02 engineering and technology ,Computer Science - Networking and Internet Architecture ,Artificial Intelligence (cs.AI) ,Modeling and Simulation ,0502 economics and business ,Incremental learning ,Computation offloading ,Enhanced Data Rates for GSM Evolution ,Ubiquitous network ,Internet of Things ,business ,Software ,Edge computing - Abstract
The Internet of Things is transforming our society, providing new services that improve the quality of life and resource management. These applications are based on ubiquitous networks of multiple distributed devices, with limited computing resources and power, capable of collecting and storing data from heterogeneous sources in real-time. To avoid network saturation and high delays, new architectures such as fog computing are emerging to bring computing infrastructure closer to data sources. Additionally, new data centers are needed to provide real-time Big Data and data analytics capabilities at the edge of the network, where energy efficiency needs to be considered to ensure a sustainable and effective deployment in areas of human activity. In this research, we present an IoT model based on the principles of Model-Based Systems Engineering defined using the Discrete Event System Specification formalism. The provided mathematical formalism covers the description of the entire architecture, from IoT devices to the processing units in edge data centers. Our work includes the location-awareness of user equipment, network, and computing infrastructures to optimize federated resource management in terms of delay and power consumption. We present an effective framework to assist the dimensioning and the dynamic operation of IoT data stream analytics applications, demonstrating our contributions through a driving assistance use case based on real traces and data.
- Published
- 2021
- Full Text
- View/download PDF
20. Context-aware incremental learning-based method for personalized human activity recognition
- Author
-
Pekka Siirtola and Juha Röning
- Subjects
human activity recognition ,General Computer Science ,Computer science ,Decision tree ,Word error rate ,Computational intelligence ,Context (language use) ,02 engineering and technology ,Machine learning ,computer.software_genre ,Personalization ,Activity recognition ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,incremental learning ,adaptive models ,Ensemble forecasting ,business.industry ,context-awareness ,Quadratic classifier ,Linear discriminant analysis ,Weighting ,Incremental learning ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer - Abstract
This study introduces an ensemble-based personalized human activity recognition method relying on incremental learning, which is a method for continuous learning, that can not only learn from streaming data but also adapt to different contexts and changes in context. This adaptation is based on a novel weighting approach which gives bigger weight to those base models of the ensemble which are the most suitable to the current context. In this article, contexts are different body positions for inertial sensors. The experiments are performed in two scenarios: (S1) adapting model to a known context, and (S2) adapting model to a previously unknown context. In both scenarios, the models had to also adapt to the data of previously unknown person, as the initial user-independent dataset did not include any data from the studied user. In the experiments, the proposed ensemble-based approach is compared to non-weighted personalization method relying on ensemble-based classifier and to static user-independent model. Both ensemble models are experimented using three different base classifiers (linear discriminant analysis, quadratic discriminant analysis, and classification and regression tree). The results show that the proposed ensemble method performs much better than non-weighted ensemble model for personalization in both scenarios no matter which base classifier is used. Moreover, the proposed method outperforms user-independent models. In scenario 1, the error rate of balanced accuracy using user-independent model was 13.3%, using non-weighted personalization method 13.8%, and using the proposed method 6.4%. The difference is even bigger in scenario 2, where the error rate using user-independent model is 36.6%, using non-weighted personalization method 36.9%, and using the proposed method 14.1%. In addition, F1 scores also show that the proposed method performs much better in both scenarios that the rival methods. Moreover, as a side result, it was noted that the presented method can also be used to recognize body position of the sensor.
- Published
- 2021
- Full Text
- View/download PDF
21. T-DFNN: An Incremental Learning Algorithm for Intrusion Detection Systems
- Author
-
Masayoshi Aritsugi and Mahendra Data
- Subjects
Structure (mathematical logic) ,incremental learning ,Forgetting ,General Computer Science ,Computer science ,business.industry ,Process (engineering) ,catastrophic forgetting ,General Engineering ,deep learning ,Intrusion detection system ,Machine learning ,computer.software_genre ,TK1-9971 ,Tree (data structure) ,Feedforward neural network ,General Materials Science ,Incremental learning algorithm ,Artificial intelligence ,Electrical engineering. Electronics. Nuclear engineering ,Macro ,business ,classification algorithm ,computer ,Network intrusion detection - Abstract
Machine learning has recently become a popular algorithm in building reliable intrusion detection systems (IDSs). However, most of the models are static and trained using datasets containing all targeted intrusions. If new intrusions emerge, these trained models must be retrained using old and new datasets to classify all intrusions accurately. In real-world situations, new threats continuously appear. Therefore, machine learning algorithms used for IDSs should have the ability to learn incrementally when these new intrusions emerge. To solve this issue, we propose T-DFNN. T-DFNN is an algorithm capable of learning new intrusions incrementally as they emerge. A T-DFNN model is composed of multiple deep feedforward neural network (DFNN) models connected in a tree-like structure. We examined our proposed algorithm using CICIDS2017, an open and widely used network intrusion dataset covering benign traffic and the most common network intrusions. The experimental results showed that the T-DFNN algorithm can incrementally learn new intrusions and reduce the catastrophic forgetting effect. The macro average of the F1-score of the T-DFNN model was over 0.85 for every retraining process. In addition, our proposed T-DFNN model has some advantages in several aspects compared to other models. Compared to the DFNN and Hoeffding tree models trained with a dataset containing only the latest targeted intrusions, our proposed T-DFNN model has higher F1-scores. Moreover, our proposed T-DFNN model has significantly shorter training times than a DFNN model trained using a dataset containing all targeted intrusions. Even though several factors can affect the duration of the training process, the T-DFNN algorithm shows promising results in solving the problem of ever-evolving network intrusion variants.
- Published
- 2021
22. Autonomous cognition development with lifelong learning: A self-organizing and reflecting cognitive network
- Author
-
Yibin Li, Rui Song, Xin Ma, Ke Huang, and Xuewen Rong
- Subjects
0209 industrial biotechnology ,Reflection (computer programming) ,Artificial neural network ,business.industry ,Computer science ,Cognitive Neuroscience ,Lifelong learning ,Cognition ,02 engineering and technology ,Object (computer science) ,Cognitive network ,Computer Science Applications ,020901 industrial engineering & automation ,Artificial Intelligence ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,Cognitive development ,020201 artificial intelligence & image processing ,Artificial intelligence ,Cluster analysis ,business - Abstract
Lifelong learning is still a great challenge for cognitive robots since the continuous streaming data they encounter is usually enormous and no-stationary. Traditional cognitive methods suffer from large storage and computation consumption in this situation. Therefore, we propose a self-organizing and reflecting cognitive network (SORCN) to realize robotic lifelong cognitive development through incremental learning and regular reflecting. The network integrates a self-organizing incremental neural network (SOINN) with a modified CFS clustering algorithm. SOINN develops concise object concepts to alleviate storage consumption. Moreover, we modify SOINN by an efficient competitive method based on reflection results to reduce the learning computation. The modified CFS clustering algorithm is designed for reflecting knowledge learned by SOINN periodically. It improves the traditional CFS as a three-step clustering method including clustering, merging and splitting. Specifically, an autonomous center selection strategy is employed for CFS to cater to online learning. Moreover, a series of cluster merging and splitting strategies are proposed to enable CFS to cluster data incrementally and improve its clustering effect. Additionally, the reflection results are utilized to adjust the topological structure of SOINN and guide the future learning. Experimental results demonstrate that SORCN can achieve better learning effectiveness and efficiency over several state-of-art algorithms.
- Published
- 2021
- Full Text
- View/download PDF
23. A Novel Approach of IoT Stream Sampling and Model Update on the IoT Edge Device for Class Incremental Learning in an Edge-Cloud System
- Author
-
Wong Yee Wan, Hermawan Nugroho, and Swaraj Dube
- Subjects
General Computer Science ,Edge device ,Computer science ,IoT edge device ,Distributed computing ,convolutional neural network ,Context (language use) ,Cloud computing ,02 engineering and technology ,Data modeling ,0202 electrical engineering, electronic engineering, information engineering ,cloud ,General Materials Science ,Incremental learning ,Class (computer programming) ,business.industry ,Deep learning ,General Engineering ,020206 networking & telecommunications ,Transmission (telecommunications) ,data sampling ,020201 artificial intelligence & image processing ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Artificial intelligence ,Enhanced Data Rates for GSM Evolution ,business ,lcsh:TK1-9971 - Abstract
With the exponential rise of the number of IoT devices, the amount of data being produced is massive. Thus, it is unfeasible to send all the raw data directly to the cloud for processing, especially for data that is high dimensional. Training deep learning models incrementally evolves the model over time and eliminates the need to statically training the models with all the data. However, the integration of class incremental learning and the Internet of Things (IoT) is a new concept and is not yet mature. In the context of IoT and deep learning, the transmission cost of data in the edge-cloud architecture is a challenge. We demonstrate a novel sample selection method that discards certain training images on the IoT edge device that reduces transmission cost and still maintains class incremental learning performance. It can be unfeasible to transmit all parameters of a trained model back to the IoT edge device. Therefore, we propose an algorithm to find only the useful parameters of a trained model in an efficient way to reduce the transmission cost from the cloud to the edge devices. Results show that our proposed methods can effectively perform class-incremental learning in an edge-cloud setting.
- Published
- 2021
- Full Text
- View/download PDF
24. Baseline Model Training in Sensor-Based Human Activity Recognition: An Incremental Learning Approach
- Author
-
Linlin Chen, Jianyu Xiao, Haipeng Chen, and Xuemin Hong
- Subjects
General Computer Science ,Computer science ,Feature extraction ,Wearable computer ,02 engineering and technology ,Machine learning ,computer.software_genre ,baseline model ,Data modeling ,Personalization ,Activity recognition ,Classifier (linguistics) ,0202 electrical engineering, electronic engineering, information engineering ,General Materials Science ,Infomax ,DIM ,incremental learning ,business.industry ,General Engineering ,020206 networking & telecommunications ,TK1-9971 ,broad learning system ,Task analysis ,020201 artificial intelligence & image processing ,Artificial intelligence ,Human activity recognition ,Electrical engineering. Electronics. Nuclear engineering ,business ,computer - Abstract
Human activity recognition (HAR) based on wearable sensors has attracted significant research attention in recent years due to its advantages in availability, accuracy, and privacy-friendliness. HAR baseline model is essentially a general-purpose classifier trained to recognized multiple activity patterns of most user types. It provides the input for subsequent steps of model personalization. Training a good baseline model is of fundamental importance because it has significant impacts on the ultimate HAR accuracy. In practice, baseline model training in HAR is a non-trivial problem that faces two challenges: insufficient training data and biased training data. This paper proposes a novel baseline model training scheme to tackle the two challenges using Deep InfoMax (DIM)-based unsupervised feature extraction and Broad Learning System (BLS)-based incremental learning, respectively. Experimental results demonstrate that the proposed scheme outperform conventional methods in terms of overall accuracy, computational efficiency, and the ability to adapt to dynamic scenarios with changing data characteristics.
- Published
- 2021
25. A Fault Aware Broad Learning System for Concurrent Network Failure Situations
- Author
-
Muideen Adegoke, Hiu Tung Wong, and Chi Sing Leung
- Subjects
0209 industrial biotechnology ,General Computer Science ,Linear programming ,Computer science ,02 engineering and technology ,Fault (power engineering) ,Multiplicative noise ,020901 industrial engineering & automation ,0202 electrical engineering, electronic engineering, information engineering ,Feature (machine learning) ,General Materials Science ,incremental learning ,multiplicative noise ,ComputingMilieux_THECOMPUTINGPROFESSION ,business.industry ,Node (networking) ,General Engineering ,Feed forward ,Fault tolerance ,Term (time) ,broad learning system ,regression ,020201 artificial intelligence & image processing ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Artificial intelligence ,open fault ,business ,lcsh:TK1-9971 - Abstract
The broad learning system (BLS) framework gives an efficient solution for training flat-structured feedforward networks and flat structured deep neural networks. However, the classical BLS model and other variants focus on the faultless situation only, where enhancement nodes, feature mapped nodes, and output weights of a BLS network are assumed to be realized in a perfect condition. When a trained BLS network suffers from coexistence of weight/node failures, the trained network has a greatly degradation in its performance if a countermeasure is not taken. In order to reduce the effect of weight/node failures on the BLS network’s performance, this paper proposes an objective function for enhancing the fault aware performance of BLS networks. The objective function contains a fault aware regularizer term which handles the weight/node failures. A learning algorithm is then derived based on the objective function. The simulation results show that the performance of the proposed fault aware BLS (FABLS) algorithm is superior to the classical BLS and two state-of-the-arts BLS algorithms, namely correntropy criterion BLS (CBLS) and weighted BLS (WBLS).
- Published
- 2021
- Full Text
- View/download PDF
26. Parallel Multistage Wide Neural Network
- Author
-
Jiangbo Xi, Jianwu Fang, Xin Wei, Okan K. Ersoy, Tianjun Wu, and Chaoying Zhao
- Subjects
Artificial neural network ,Computer Networks and Communications ,Computer science ,business.industry ,Deep learning ,Decision tree ,Machine learning ,computer.software_genre ,Multistage wide learning ,Ensemble learning ,Computer Science Applications ,Support vector machine ,Tree (data structure) ,Artificial Intelligence ,Multilayer perceptron ,Parallel testing ,Artificial intelligence ,business ,computer ,Software ,MNIST database ,Incremental learning - Abstract
Deep learning networks have achieved great success in many areas, such as in large-scale image processing. They usually need large computing resources and time and process easy and hard samples inefficiently in the same way. Another undesirable problem is that the network generally needs to be retrained to learn new incoming data. Efforts have been made to reduce the computing resources and realize incremental learning by adjusting architectures, such as scalable effort classifiers, multi-grained cascade forest (gcForest), conditional deep learning (CDL), tree CNN, decision tree structure with knowledge transfer (ERDK), forest of decision trees with radial basis function (RBF) networks, and knowledge transfer (FDRK). In this article, a parallel multistage wide neural network (PMWNN) is presented. It is composed of multiple stages to classify different parts of data. First, a wide radial basis function (WRBF) network is designed to learn features efficiently in the wide direction. It can work on both vector and image instances and can be trained in one epoch using subsampling and least squares (LS). Second, successive stages of WRBF networks are combined to make up the PMWNN. Each stage focuses on the misclassified samples of the previous stage. It can stop growing at an early stage, and a stage can be added incrementally when new training data are acquired. Finally, the stages of the PMWNN can be tested in parallel, thus speeding up the testing process. To sum up, the proposed PMWNN network has the advantages of: 1) optimized computing resources; 2) incremental learning; and 3) parallel testing with stages. The experimental results with the MNIST data, a number of large hyperspectral remote sensing data, and different types of data in different application areas, including many image and nonimage datasets, show that the WRBF and PMWNN can work well on both image and nonimage data and have very competitive accuracy compared to learning models, such as stacked autoencoders, deep belief nets, support vector machine (SVM), multilayer perceptron (MLP), LeNet-5, RBF network, recently proposed CDL, broad learning, gcForest, ERDK, and FDRK.
- Published
- 2021
- Full Text
- View/download PDF
27. Reduce the Difficulty of Incremental Learning With Self-Supervised Learning
- Author
-
Linting Guan and Yan Wu
- Subjects
Forgetting ,General Computer Science ,Artificial neural network ,Computer science ,business.industry ,Deep learning ,Feature extraction ,General Engineering ,deep learning ,Machine learning ,computer.software_genre ,Data modeling ,Task (project management) ,TK1-9971 ,Learning disability ,self-supervised learning ,medicine ,Task analysis ,General Materials Science ,Artificial intelligence ,Electrical engineering. Electronics. Nuclear engineering ,medicine.symptom ,business ,computer ,Incremental learning - Abstract
Incremental learning requires a learning model to learn new tasks without forgetting the learned tasks continuously. However, when a deep learning model learns new tasks, it will catastrophically forget tasks it has learned before. Researchers have proposed methods to alleviate catastrophic forgetting; these methods only consider extracting features related to tasks learned before, suppression to extract features for unlearned tasks. As a result, when a deep learning model learns new tasks incrementally, the model needs to learn to extract the relevant features of the newly learned task quickly; this requires a significant change in the model’s behavior of extracting features, which increases the learning difficulty. Therefore, the model is caught in the dilemma of reducing the learning rate to retain existing knowledge or increasing the learning rate to learn new knowledge quickly. We present a study aiming to alleviate this problem by introducing self-supervised learning into incremental learning methods. We believe that the task-independent self-supervised learning signal helps the learning model extract features not only effective for the current learned task but also suitable for other tasks that have not been learned. We give a detailed algorithm combining self-supervised learning signals and incremental learning methods. Extensive experiments on several different datasets show that self-supervised signal significantly improves the accuracy of most incremental learning methods without the need for additional labeled data. We found that the self-supervised learning signal works best for the replay-based incremental learning method.
- Published
- 2021
28. Sentiment analysis for customer relationship management: an incremental learning approach
- Author
-
Pierluigi Ritrovato, Mario Vento, Nicola Capuano, and Luca Greco
- Subjects
business.industry ,Computer science ,Customer relationship management ,Hierarchical attention networks ,Machine learning ,Natural language processing ,Sentiment analysis ,02 engineering and technology ,computer.software_genre ,Loyalty business model ,Artificial Intelligence ,020204 information systems ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Customer satisfaction ,Artificial intelligence ,business ,computer ,Classifier (UML) ,Corporate management - Abstract
In recent years there has been a significant rethinking of corporate management, which is increasingly based on customer orientation principles. As a matter of fact, customer relationship management processes and systems are ever more popular and crucial to facing today’s business challenges. However, the large number of available customer communication stimuli coming from different (direct and indirect) channels, require automatic language processing techniques to help filter and qualify such stimuli, determine priorities, facilitate the routing of requests and reduce the response times. In this scenario, sentiment analysis plays an important role in measuring customer satisfaction, tracking consumer opinion, interacting with consumers and building customer loyalty. The research described in this paper proposes an approach based on Hierarchical Attention Networks for detecting the sentiment polarity of customer communications. Unlike other existing approaches, after initial training, the defined model can improve over time during system operation using the feedback provided by CRM operators thanks to an integrated incremental learning mechanism. The paper also describes the developed prototype as well as the dataset used for training the model which includes over 30.000 annotated items. The results of two experiments aimed at measuring classifier performance and validating the retraining mechanism are also presented and discussed. In particular, the classifier accuracy turned out to be better than that of other algorithms for the supported languages (macro-averaged f1-score of 0.89 and 0.79 for Italian and English respectively) and the retraining mechanism was able to improve the classification accuracy on new samples without degrading the overall system performance.
- Published
- 2020
- Full Text
- View/download PDF
29. Active and incremental learning for semantic ALS point cloud segmentation
- Author
-
Yaping Lin, George Vosselman, Yanpeng Cao, Michael Ying Yang, Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation, and UT-I-ITC-ACQUAL
- Subjects
Active learning ,010504 meteorology & atmospheric sciences ,Computer science ,UT-Hybrid-D ,0211 other engineering and technologies ,Point cloud ,02 engineering and technology ,Machine learning ,computer.software_genre ,01 natural sciences ,ITC-HYBRID ,Entropy (information theory) ,Segmentation ,Computers in Earth Sciences ,Engineering (miscellaneous) ,Incremental learning ,021101 geological & geomatics engineering ,0105 earth and related environmental sciences ,Artificial neural network ,business.industry ,Deep learning ,Mutual information ,Semantic segmentation ,Atomic and Molecular Physics, and Optics ,Computer Science Applications ,Lidar ,Photogrammetry ,ITC-ISI-JOURNAL-ARTICLE ,Artificial intelligence ,Point clouds ,business ,computer - Abstract
Supervised training of a deep neural network for semantic segmentation of point clouds requires a large amount of labelled data. Nowadays, it is easy to acquire a huge number of points with high density in large-scale areas using current LiDAR and photogrammetric techniques. However it is extremely time-consuming to manually label point clouds for model training. In this paper, we propose an active and incremental learning strategy to iteratively query informative point cloud data for manual annotation and the model is continuously trained to adapt to the newly labelled samples in each iteration. We evaluate the data informativeness step by step and effectively and incrementally enrich the model knowledge. The data informativeness is estimated by two data dependent uncertainty metrics (point entropy and segment entropy) and one model dependent metric (mutual information). The proposed methods are tested on two datasets. The results indicate the proposed uncertainty metrics can enrich current model knowledge by selecting informative samples, such as considering points with difficult class labels and choosing target objects with various geometries in the labelled training pool. Compared to random selection, our metrics provide valuable information to significantly reduce the labelled training samples. In contrast with training from scratch, the incremental fine-tuning strategy significantly save the training time.
- Published
- 2020
- Full Text
- View/download PDF
30. An integrated classification model for incremental learning
- Author
-
Hu Ji, Zhiyuan Li, Xin Liu, Chengwei Ren, Yi Yang, Chenggang Yan, Dongliang Peng, and Jiyong Zhang
- Subjects
Computer Networks and Communications ,Process (engineering) ,Computer science ,Image classification ,Feature vector ,Masked-face dataset ,02 engineering and technology ,Machine learning ,computer.software_genre ,Field (computer science) ,Article ,0202 electrical engineering, electronic engineering, information engineering ,Media Technology ,Artificial Intelligence & Image Processing ,Confidence weight ,Incremental learning ,Contextual image classification ,business.industry ,Software Engineering ,020207 software engineering ,Transfer learning ,Statistical classification ,Hardware and Architecture ,Face (geometry) ,0801 Artificial Intelligence and Image Processing, 0803 Computer Software, 0805 Distributed Computing, 0806 Information Systems ,Noise (video) ,Artificial intelligence ,Transfer of learning ,business ,computer ,Software - Abstract
Incremental Learning is a particular form of machine learning that enables a model to be modified incrementally, when new data becomes available. In this way, the model can adapt to the new data without the lengthy and time-consuming process required for complete model re-training. However, existing incremental learning methods face two significant problems: 1) noise in the classification sample data, 2) poor accuracy of modern classification algorithms when applied to modern classification problems. In order to deal with these issues, this paper proposes an integrated classification model, known as a Pre-trained Truncated Gradient Confidence-weighted (Pt-TGCW) model. Since the pre-trained model can extract and transform image information into a feature vector, the integrated model also shows its advantages in the field of image classification. Experimental results on ten datasets demonstrate that the proposed method outperform the original counterparts.
- Published
- 2020
31. BNGBS: An efficient network boosting system with triple incremental learning capabilities for more nodes, samples, and classes
- Author
-
Honglin Qiao, Min Zhou, Chunhui Zhao, Chuan Fu, Yuanlong Li, C. L. Philip Chen, and Liangjun Feng
- Subjects
0209 industrial biotechnology ,Boosting (machine learning) ,Computer science ,business.industry ,Cognitive Neuroscience ,02 engineering and technology ,Machine learning ,computer.software_genre ,Computer Science Applications ,020901 industrial engineering & automation ,Artificial Intelligence ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Artificial intelligence ,Gradient boosting ,business ,Additive model ,computer - Abstract
As an ensemble algorithm, network boosting enjoys a powerful classification ability but suffers from the tedious and time-consuming training process. To tackle the problem, in this paper, a broad network gradient boosting system (BNGBS) is developed by integrating gradient boosting machine with broad networks, in which the classification loss caused by a base broad network is learned and eliminated by followed networks in a cascade manner. The proposed system is constructed as an additive model and can be easily optimized by a greedy strategy instead of the tedious back-propagation algorithm, resulting in a more efficient learning process. Meanwhile, triple incremental learning capabilities including the increment of feature nodes, increment of input samples, and increment of target classes are designed. The proposed system can be efficiently updated and expanded based on the current status instead of being entirely retrained when the demands for more feature nodes, input samples, and target classes are proposed. The node-increment ability allows to add more feature nodes into the built system if the current structures are not effective for learning. The sample-increment ability is developed to allow the model to keep learning from the coming batch data. The class-increment ability is used to tackle the issue that the coming batch data may contain unseen categories. In comparison with existing popular machine learning methods, comprehensive results based on eight benchmark datasets illustrate the effectiveness of the proposed broad network gradient boosting system for the classification task.
- Published
- 2020
- Full Text
- View/download PDF
32. Beyond Cross-Validation—Accuracy Estimation for Incremental and Active Learning Models
- Author
-
Helge Ritter, Christian Limberg, and Heiko Wersing
- Subjects
lcsh:Computer engineering. Computer hardware ,Computer science ,online learning ,lcsh:TK7885-7895 ,02 engineering and technology ,Machine learning ,computer.software_genre ,Cross-validation ,accuracy estimation ,020204 information systems ,active learning ,0202 electrical engineering, electronic engineering, information engineering ,benchmarking ,incremental learning ,business.industry ,error prediction ,Cognitive neuroscience of visual object recognition ,Regression analysis ,Benchmarking ,Standard methods ,ComputingMethodologies_PATTERNRECOGNITION ,classifier evaluation ,Robot ,020201 artificial intelligence & image processing ,Artificial intelligence ,Benchmark data ,business ,Classifier (UML) ,computer - Abstract
For incremental machine-learning applications it is often important to robustly estimate the system accuracy during training, especially if humans perform the supervised teaching. Cross-validation and interleaved test/train error are here the standard supervised approaches. We propose a novel semi-supervised accuracy estimation approach that clearly outperforms these two methods. We introduce the Configram Estimation (CGEM) approach to predict the accuracy of any classifier that delivers confidences. By calculating classification confidences for unseen samples, it is possible to train an offline regression model, capable of predicting the classifier&rsquo, s accuracy on novel data in a semi-supervised fashion. We evaluate our method with several diverse classifiers and on analytical and real-world benchmark data sets for both incremental and active learning. The results show that our novel method improves accuracy estimation over standard methods and requires less supervised training data after deployment of the model. We demonstrate the application of our approach to a challenging robot object recognition task, where the human teacher can use our method to judge sufficient training.
- Published
- 2020
33. Broad Reinforcement Learning for Supporting Fast Autonomous IoT
- Author
-
Jialin Zhao, Xin Wei, Liang Zhou, and Yi Qian
- Subjects
Computer Networks and Communications ,Computer science ,business.industry ,Control (management) ,Big data ,020206 networking & telecommunications ,02 engineering and technology ,Machine learning ,computer.software_genre ,Computer Science Applications ,Dilemma ,Action (philosophy) ,Hardware and Architecture ,Signal Processing ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,Reinforcement learning ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Internet of Things ,computer ,Information Systems - Abstract
The emergence of a massive Internet-of-Things (IoT) ecosystem is changing the human lifestyle. In several practical scenarios, IoT still faces significant challenges with reliance on human assistance and unacceptable response time for the treatment of big data. Therefore, it is very urgent to establish a new framework and algorithm to solve problems specific to this kind of fast autonomous IoT. Traditional reinforcement learning and deep reinforcement learning (DRL) approaches have abilities of autonomous decision making, but time-consuming modeling and training procedures limit their applications. To get over this dilemma, this article proposes the broad reinforcement learning (BRL) approach that fits fast autonomous IoT as it combines the broad learning system (BLS) with a reinforcement learning paradigm to improve the agent’s efficiency and accuracy of modeling and decision making. Specifically, a BRL framework is first constructed. Then, the associated learning algorithm, containing training pool introduction, training sample preparation, and incremental learning for BLS, is carefully designed. Finally, as a case study of fast autonomous IoT, the proposed BRL approach is applied to traffic light control, aiming to alleviate traffic congestion in the intersections of smart cities. The experimental results show that the proposed BRL approach can learn better action policy at a shorter execution time when compared with competing approaches.
- Published
- 2020
- Full Text
- View/download PDF
34. Class Boundary Exemplar Selection Based Incremental Learning for Automatic Target Recognition
- Author
-
Zongjie Cao, Nengyuan Liu, Zongyong Cui, Sihang Dang, and Yiming Pi
- Subjects
Training set ,Computer science ,business.industry ,0211 other engineering and technologies ,Boundary (topology) ,02 engineering and technology ,Machine learning ,computer.software_genre ,Class (biology) ,Data modeling ,Support vector machine ,Set (abstract data type) ,Automatic target recognition ,Incremental learning ,Task analysis ,General Earth and Planetary Sciences ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,computer ,Selection (genetic algorithm) ,021101 geological & geomatics engineering - Abstract
When adding new tasks/classes in an incremental learning scenario, the previous recognition capabilities trained on the previous training data can be lost. In the real-life application of automatic target recognition (ATR), part of the previous samples may be able to be used. Most incremental learning methods have not considered how to save the previous key samples. In this article, the class boundary exemplar selection-based incremental learning (CBesIL) is proposed to save the previous recognition capabilities in the form of the class boundary exemplars. For exemplar selection, the class boundary selection method based on local geometrical and statistical information is proposed. And when adding new classes continually, a class-boundary-based data reconstruction method is introduced to update the exemplar set. Thus, when adding new classes, the previous class boundaries could be kept complete. Experimental results demonstrate that the proposed CBesIL outperforms the other state of the art on the accuracy of multiclass recognition and class-incremental recognition.
- Published
- 2020
- Full Text
- View/download PDF
35. Distributed Incremental Clustering Algorithms: A Bibliometric and Word-Cloud Review Analysis
- Author
-
Rahul Joshi, Archana Chaudhari, and Preeti Mulay
- Subjects
business.industry ,Computer science ,Incremental learning ,Artificial intelligence ,Library and Information Sciences ,Tag cloud ,Machine learning ,computer.software_genre ,Review analysis ,Cluster analysis ,business ,computer - Abstract
“Incremental Learning (IL)” is the niche area of “Machine Learning.” It is of utmost essential to keep learning incremental for ever-increasing data from all domains for effectual decisions, predic...
- Published
- 2020
- Full Text
- View/download PDF
36. DEVDAN: Deep evolving denoising autoencoder
- Author
-
Andri Ashfahani, Edwin Lughofer, Mahardhika Pratama, Yew-Soon Ong, Ashfahani, Andri, Pratama, Mahardhika, Lughofer, Edwin, Ong, Yew-Soon, and School of Computer Science and Engineering
- Subjects
FOS: Computer and information sciences ,Data stream ,Computer Science - Machine Learning ,0209 industrial biotechnology ,Computer science ,Cognitive Neuroscience ,data streams ,Machine Learning (stat.ML) ,02 engineering and technology ,Machine Learning (cs.LG) ,020901 industrial engineering & automation ,Discriminative model ,Statistics - Machine Learning ,Artificial Intelligence ,denoising autoencoder ,0202 electrical engineering, electronic engineering, information engineering ,Protocol (object-oriented programming) ,incremental learning ,Flexibility (engineering) ,Denoising autoencoder ,business.industry ,Pattern recognition ,Computer Science Applications ,Computer science and engineering [Engineering] ,020201 artificial intelligence & image processing ,Artificial intelligence ,business - Abstract
The Denoising Autoencoder (DAE) enhances the flexibility of the data stream method in exploiting unlabeled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves an in-depth study because it characterizes a fixed network capacity that cannot adapt to rapidly changing environments. Deep evolving denoising autoencoder (DEVDAN), is proposed in this paper. It features an open structure in the generative phase and the discriminative phase where the hidden units can be automatically added and discarded on the fly. The generative phase refines the predictive performance of the discriminative model exploiting unlabeled data. Furthermore, DEVDAN is free of the problem-specific threshold and works fully in the single-pass learning fashion. We show that DEVDAN can find competitive network architecture compared with state-of-the-art methods on the classification task using ten prominent datasets simulated under the prequential test-then-train protocol., This paper has been accepted for publication in Neurocomputing 2019. arXiv admin note: substantial text overlap with arXiv:1809.09081
- Published
- 2020
- Full Text
- View/download PDF
37. The unordered time series fuzzy clustering algorithm based on the adaptive incremental learning
- Author
-
Jinfeng Fan, Huanchun Xu, Liang Zhou, Hongxuan Yue, Liusheng Wang, Jiayue Liu, and Rui Hou
- Subjects
Statistics and Probability ,Fuzzy clustering ,Series (mathematics) ,Artificial Intelligence ,business.industry ,Computer science ,Incremental learning ,General Engineering ,Artificial intelligence ,business - Published
- 2020
- Full Text
- View/download PDF
38. Fuzzy clustering algorithm for time series based on adaptive incremental learning
- Author
-
Mingye Wang, Xiaohui Hu, and Wei Wang
- Subjects
Statistics and Probability ,Fuzzy clustering ,Series (mathematics) ,Artificial Intelligence ,business.industry ,Computer science ,Incremental learning ,General Engineering ,Artificial intelligence ,business - Published
- 2020
- Full Text
- View/download PDF
39. Downsizing and enhancing broad learning systems by feature augmentation and residuals boosting
- Author
-
Runshan Xie and Shitong Wang
- Subjects
Boosting (machine learning) ,ComputingMilieux_THECOMPUTINGPROFESSION ,Artificial neural network ,Computer science ,business.industry ,Computational intelligence ,General Medicine ,Overfitting ,Machine learning ,computer.software_genre ,Incremental learning ,Systems architecture ,Artificial intelligence ,Architecture ,business ,computer - Abstract
Recently, a broad learning system (BLS) has been theoretically and experimentally confirmed to be an efficient incremental learning system. To get rid of deep architecture, BLS shares the same architecture and learning mechanism of the well-known functional link neural networks (FLNN), but works in broad learning way on both the randomly mapped features of original features of data and their randomly generated enhancement nodes. As such, BLS often requires a huge heap of hidden nodes to achieve the prescribed or satisfactory performance, which may inevitably cause both overwhelming storage requirement and overfitting phenomenon. In this study, a stacked architecture of broad learning systems called D&BLS is proposed to achieve enhanced performance and simultaneously downsize the system architecture. By boosting the residuals between previous and current layers and simultaneously augmenting the original input space with the outputs of the previous layer as the inputs of current layer, D&BLS stacks several lightweight BLS sub-systems to guarantee stronger feature representation capability and better classification/regression performance. Three fast incremental learning algorithms of D&BLS are also developed, without the need for the whole re-training. Experimental results on some popular datasets demonstrate the effectiveness of D&BLS in the sense of both enhanced performance and reduced system architecture.
- Published
- 2020
- Full Text
- View/download PDF
40. A comparative study of general fuzzy min-max neural networks for pattern classification problems
- Author
-
Bogdan Gabrys and Thanh Tung Khuat
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,0209 industrial biotechnology ,Computer science ,Cognitive Neuroscience ,68T30, 68T20, 68T37, 68W27 ,Fuzzy set ,Machine Learning (stat.ML) ,02 engineering and technology ,Machine learning ,computer.software_genre ,Fuzzy logic ,Machine Learning (cs.LG) ,020901 industrial engineering & automation ,Empirical research ,Statistics - Machine Learning ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,Artificial Intelligence & Image Processing ,Cluster analysis ,I.5.0 ,I.5.1 ,I.2.1 ,I.2.6 ,I.2.m ,I.5.2 ,I.5.3 ,I.5.4 ,Artificial neural network ,business.industry ,Computer Science Applications ,Hierarchical clustering ,ComputingMethodologies_PATTERNRECOGNITION ,Incremental learning ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Classifier (UML) ,computer - Abstract
General fuzzy min-max (GFMM) neural network is a generalization of fuzzy neural networks formed by hyperbox fuzzy sets for classification and clustering problems. Two principle algorithms are deployed to train this type of neural network, i.e., incremental learning and agglomerative learning. This paper presents a comprehensive empirical study of performance influencing factors, advantages, and drawbacks of the general fuzzy min-max neural network on pattern classification problems. The subjects of this study include (1) the impact of maximum hyperbox size, (2) the influence of the similarity threshold and measures on the agglomerative learning algorithm, (3) the effect of data presentation order, (4) comparative performance evaluation of the GFMM with other types of fuzzy min-max neural networks and prevalent machine learning algorithms. The experimental results on benchmark datasets widely used in machine learning showed overall strong and weak points of the GFMM classifier. These outcomes also informed potential research directions for this class of machine learning algorithms in the future., Comment: 18 pages, 7 figures, 12 tables
- Published
- 2020
- Full Text
- View/download PDF
41. Prediction of blood glucose concentration for type 1 diabetes based on echo state networks embedded with incremental learning
- Author
-
Jianyong Tuo, Ning Li, Menghui Wang, and Youqing Wang
- Subjects
0209 industrial biotechnology ,Type 1 diabetes ,Computer science ,business.industry ,Cognitive Neuroscience ,Echo (computing) ,02 engineering and technology ,Hypoglycemia ,medicine.disease ,Machine learning ,computer.software_genre ,Artificial pancreas ,Computer Science Applications ,020901 industrial engineering & automation ,Artificial Intelligence ,Diabetes mellitus ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,020201 artificial intelligence & image processing ,State (computer science) ,Artificial intelligence ,business ,computer - Abstract
Valid prediction of blood glucose concentration can help people to manage diabetes mellitus, alert hypoglycemia/hyperglycemia, exploit artificial pancreas, and plan a treatment program. Along the development of continuous glucose monitoring system (CGMS), the massive historical data require a new modeling framework based on a data-driven perspective. Studies indicate that the glucose time series (i.e., CGMS readings) involve chaotic properties; therefore, echo state networks (ESN) and its improved variants are proposed to establish subject-specific prediction models owing to their superiority in processing chaotic systems. This study mainly has two innovations: (1) a novel combination of incremental learning and ESN is developed to obtain a suitable network structure through partial optimization of parameters; (2) a feedback ESN is proposed to excavate the relationship of different predictions. These methods are assessed on ten patients with diabetes mellitus. Experimental results substantiate that the proposed methods achieve superior prediction performance in terms of four evaluation metrics compared with three conventional methods.
- Published
- 2020
- Full Text
- View/download PDF
42. GrowingNet: An end-to-end growing network for semi-supervised learning
- Author
-
Xiaomo Yu and Qifei Zhang
- Subjects
Computer Networks and Communications ,Computer science ,business.industry ,Sample (material) ,Word error rate ,020206 networking & telecommunications ,Pattern recognition ,02 engineering and technology ,Semi-supervised learning ,Overfitting ,End-to-end principle ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,Benchmark (computing) ,Labeled data ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Network model - Abstract
Semi-supervised learning (SSL) typically involves a small quantity of labeled data and a large quantity of unlabeled data. As such, the successful application of semi-supervised learning (SSL) depends on distinguishing easy and hard samples which contributes substantial recognition, as well as obtaining a more accurate pseudo target for a hard sample. However, existing SSL network models with deeper layers will suffer from overfitting or optimization difficulties. To address these problems, we propose a growing network (GrowingNet) where the convolution depth of the model can expand and contract. We also propose an incremental learning method by which the amount of pseudo labeled data can be increased uniformly. During training, the goal is to increase the convolutional layers of our model and the number of pseudo labeled data synchronously. We divide training epochs, the convolutional layers of GrowingNet, and pseudo labeled data into u equal parts. During each part of training epochs, we increase one part of convolutional layers, select one division of pseudo labeled data into the training process. The accuracy of the model will improve as training progresses, which distinguishes easy and hard samples and also provides more reliable pseudo labels during subsequent part of training epochs. This provides significant improvements over state-of-the-art networks in most of cases on SSL benchmark tasks (CIFAR-10, CIFAR-100, and SVHN). Specifically, without data augmentation, our model produces error rates of 20.86%, 18.22%, and 12.02% on CIFAR-10 with 1000, 2000, and 4000 labeled data, as well as error rates of 5.03% and 3.46% on SVHN with 500 and 1000 labeled data, respectively. With data augmentation, the error rate reaches 12.16% on CIFAR-10 with 2000 labeled data and 31.06% on CIFAR-100 with 10,000 labeled data.
- Published
- 2020
- Full Text
- View/download PDF
43. Active and Incremental Learning with Weak Supervision
- Author
-
Clemens-Alexander Brust, Christoph Käding, and Joachim Denzler
- Subjects
FOS: Computer and information sciences ,Training set ,Computer science ,business.industry ,Computer Vision and Pattern Recognition (cs.CV) ,Computer Science - Computer Vision and Pattern Recognition ,02 engineering and technology ,Pascal (programming language) ,010501 environmental sciences ,Machine learning ,computer.software_genre ,01 natural sciences ,Object detection ,Artificial Intelligence ,Active learning ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer ,0105 earth and related environmental sciences ,computer.programming_language - Abstract
Large amounts of labeled training data are one of the main contributors to the great success that deep models have achieved in the past. Label acquisition for tasks other than benchmarks can pose a challenge due to requirements of both funding and expertise. By selecting unlabeled examples that are promising in terms of model improvement and only asking for respective labels, active learning can increase the efficiency of the labeling process in terms of time and cost. In this work, we describe combinations of an incremental learning scheme and methods of active learning. These allow for continuous exploration of newly observed unlabeled data. We describe selection criteria based on model uncertainty as well as expected model output change (EMOC). An object detection task is evaluated in a continuous exploration context on the PASCAL VOC dataset. We also validate a weakly supervised system based on active and incremental learning in a real-world biodiversity application where images from camera traps are analyzed. Labeling only 32 images by accepting or rejecting proposals generated by our method yields an increase in accuracy from 25.4% to 42.6%., Comment: Accepted for publication in KI - K\"unstliche Intelligenz
- Published
- 2020
- Full Text
- View/download PDF
44. MineCap: super incremental learning for detecting and blocking cryptocurrency mining on software-defined networking
- Author
-
Natalia Castro Fernandes, Martin Andreoni Lopez, Diogo M. F. Mattos, and Helio N. Cunha Neto
- Subjects
Cryptocurrency ,business.industry ,Computer science ,Online processing ,Machine learning ,computer.software_genre ,Flow network ,Network interface controller ,Covert ,Incremental learning ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Software-defined networking ,computer ,Classifier (UML) - Abstract
Covert mining of cryptocurrency implies the use of valuable computing resources and high energy consumption. In this paper, we propose MineCap, a dynamic online mechanism for detecting and blocking covert cryptocurrency mining flows, using machine learning on software-defined networking. The proposed mechanism relies on Spark Streaming for online processing of network flows, and, when identifying a mining flow, it requests the flow blocking to the network controller. We also propose a learning technique called super incremental learning, a variant of the super learner applied to online learning, which takes the classification probabilities of an ensemble of classifiers as features for an incremental learning classifier. Hence, we design an accurate mechanism to classify mining flows that learn with incoming data with an average of 98% accuracy, 99% precision, 97% sensitivity, and 99.9% specificity and avoid concept drift–related issues.
- Published
- 2020
- Full Text
- View/download PDF
45. Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing
- Author
-
Syed Shakib Sarwar, Aayush Ankit, and Kaushik Roy
- Subjects
FOS: Computer and information sciences ,Scheme (programming language) ,General Computer Science ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,lifelong learning ,Computer Science - Computer Vision and Pattern Recognition ,02 engineering and technology ,Machine learning ,computer.software_genre ,Convolutional neural network ,energy-efficient learning ,Reduction (complexity) ,Set (abstract data type) ,0202 electrical engineering, electronic engineering, information engineering ,General Materials Science ,Incremental learning ,computer.programming_language ,Contextual image classification ,business.industry ,catastrophic forgetting ,020208 electrical & electronic engineering ,Supervised learning ,General Engineering ,020201 artificial intelligence & image processing ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Artificial intelligence ,network sharing ,Transfer of learning ,business ,lcsh:TK1-9971 ,computer ,Efficient energy use - Abstract
Deep convolutional neural network (DCNN) based supervised learning is a widely practiced approach for large-scale image classification. However, retraining these large networks to accommodate new, previously unseen data demands high computational time and energy requirements. Also, previously seen training samples may not be available at the time of retraining. We propose an efficient training methodology and incrementally growing DCNN to learn new tasks while sharing part of the base network. Our proposed methodology is inspired by transfer learning techniques, although it does not forget previously learned tasks. An updated network for learning new set of classes is formed using previously learned convolutional layers (shared from initial part of base network) with addition of few newly added convolutional kernels included in the later layers of the network. We employed a `clone-and-branch' technique which allows the network to learn new tasks one after another without any performance loss in old tasks. We evaluated the proposed scheme on several recognition applications. The classification accuracy achieved by our approach is comparable to the regular incremental learning approach (where networks are updated with new training samples only, without any network sharing), while achieving energy efficiency, reduction in storage requirements, memory access and training time., Comment: 18 pages, 13 figures. IEEE Access 2019
- Published
- 2020
- Full Text
- View/download PDF
46. Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations
- Author
-
Selvarajah Thuseethan, John Yearwood, and Sutharshan Rajasegarar
- Subjects
sparse data ,Active learning ,General Computer Science ,Computer science ,media_common.quotation_subject ,Emotion classification ,02 engineering and technology ,Anger ,Machine learning ,computer.software_genre ,020204 information systems ,Perception ,emotion recognition ,0202 electrical engineering, electronic engineering, information engineering ,complex emotions ,Profiling (information science) ,General Materials Science ,GeneralLiterature_REFERENCE(e.g.,dictionaries,encyclopedias,glossaries) ,media_common ,incremental learning ,business.industry ,General Engineering ,Disgust ,Sadness ,Surprise ,Incremental learning ,020201 artificial intelligence & image processing ,Artificial intelligence ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,business ,computer ,lcsh:TK1-9971 - Abstract
Generally, in-the-wild emotions are complex in nature. They often occur in combinations of multiple basic emotions, such as fear, happy, disgust, anger, sadness and surprise. Unlike the basic emotions, annotation of complex emotions, such as pain, is a time-consuming and expensive exercise. Moreover, there is an increasing demand for profiling such complex emotions as they are useful in many real-world application domains, such as medical, psychology, security and computer science. The traditional emotion recognition systems require a significant amount of annotated training samples to understand the complex emotions. This limits the direct applicability of those methods for complex emotion detection from images and videos. Therefore, it is important to learn the profile of the in-the-wild complex emotions accurately using limited annotated samples. In this paper, we propose a deep framework to incrementally and actively profile in-the-wild complex emotions, from sparse data. Our approach consists of three major components, namely a pre-processing unit, an optimization unit and an active learning unit. The pre-processing unit removes the variations present in the complex emotion images extracted from an uncontrolled environment. Our novel incremental active learning algorithm along with an optimization unit effectively predicts the complex emotions present in-the-wild. Evaluation using multiple complex emotions benchmark datasets reveals that our proposed approach performs close to the human perception capability in effectively profiling complex emotions. Further, our proposed approach shows a significant performance enhancement, in comparison with the state-of-the-art deep networks and other benchmark complex emotion profiling approaches.
- Published
- 2020
47. Confidence Calibration for Incremental Learning
- Author
-
Yeongwoo Nam, Yeonsik Jo, Dongmin Kang, and Jonghyun Choi
- Subjects
General Computer Science ,Computer science ,Calibration (statistics) ,Sample (statistics) ,02 engineering and technology ,010501 environmental sciences ,Machine learning ,computer.software_genre ,01 natural sciences ,Task (project management) ,Margin (machine learning) ,0202 electrical engineering, electronic engineering, information engineering ,General Materials Science ,Set (psychology) ,continual learning ,Incremental learning ,0105 earth and related environmental sciences ,Class (computer programming) ,Forgetting ,business.industry ,General Engineering ,confidence calibration ,Memory management ,020201 artificial intelligence & image processing ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Artificial intelligence ,business ,lcsh:TK1-9971 ,computer - Abstract
Class incremental learning is an online learning paradigm wherein the classes to be recognized are gradually increased with limited memory, storing only a partial set of examples of past tasks. At a task transition, we observe an unintentional imbalance of confidence or likelihood between the classes of the past and the new task. We argue that the imbalance aggravates a catastrophic forgetting for class incremental learning. We propose a simple yet effective learning objective to balance the confidence of classes of old tasks and new task in the class incremental learning setup. In addition, we compare various sample memory configuring strategies and propose a novel sample memory management policy to alleviate the forgetting further. The proposed method outperforms the state of the arts in many evaluation metrics including accuracy and forgetting $F$ by a large margin (up to 5.71% in $A_{10}$ and 17.1% in $F_{10}$ ) in extensive empirical validations on multiple visual recognition datasets such as CIFAR100, TinyImageNet and a subset of the ImageNet.
- Published
- 2020
- Full Text
- View/download PDF
48. Traffic classification for connectionless services with incremental learning
- Author
-
C. Mala and V. Punitha
- Subjects
Voice over IP ,Computer Networks and Communications ,business.industry ,Computer science ,ComputerSystemsOrganization_COMPUTER-COMMUNICATIONNETWORKS ,Botnet ,020206 networking & telecommunications ,02 engineering and technology ,Connectionless communication ,Traffic classification ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,Resource allocation ,File transfer ,020201 artificial intelligence & image processing ,business ,Classifier (UML) ,Computer network - Abstract
The technological advancement in VoIP technology and P2P streaming led to the development of novel applications. Most of these applications use UDP traffic. The availability of UDP services for applications such as streaming, trivial file transfer, are denied to legitimate users due to malicious traffic, intentionally created by abnormal requesting behaviour of the botnets. Categorizing the traffic is required to discriminate the malicious traffic that occur due to attacks from normal traffic for better real time resource allocation. For this purpose, this paper proposes a two level hybrid classification model based on incremental learning to detect high and low rate attacks that deny the legitimate access to connectionless services. The simulation results show that the proposed incremental learning strategy improves the classification accuracy of the proposed hybrid classifier compared to existing traditional learning methods.
- Published
- 2020
- Full Text
- View/download PDF
49. Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning
- Author
-
Deboleena Roy, Kaushik Roy, and Priyadarshini Panda
- Subjects
FOS: Computer and information sciences ,0209 industrial biotechnology ,Computer Science - Artificial Intelligence ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Cognitive Neuroscience ,Computer Science - Computer Vision and Pattern Recognition ,Machine Learning (stat.ML) ,02 engineering and technology ,Convolutional neural network ,Pattern Recognition, Automated ,Reduction (complexity) ,Deep Learning ,020901 industrial engineering & automation ,Statistics - Machine Learning ,Artificial Intelligence ,FOS: Electrical engineering, electronic engineering, information engineering ,0202 electrical engineering, electronic engineering, information engineering ,Animals ,Humans ,Sensitivity (control systems) ,Forgetting ,business.industry ,Deep learning ,Image and Video Processing (eess.IV) ,Electrical Engineering and Systems Science - Image and Video Processing ,Tree (data structure) ,Artificial Intelligence (cs.AI) ,Incremental learning ,020201 artificial intelligence & image processing ,Neural Networks, Computer ,Artificial intelligence ,business ,Transfer of learning ,Photic Stimulation - Abstract
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100., Comment: 8 pages, 6 figures, 7 tables Accepted in Neural Networks, 2019
- Published
- 2020
- Full Text
- View/download PDF
50. Class-Incremental Learning With Deep Generative Feature Replay for DNA Methylation-Based Cancer Classification
- Author
-
Erdenebileg Batbaatar, Van Huy Pham, Lkhagvadorj Munkhdalai, Kwang Ho Park, Tsatsral Amarbayasgalan, Khishigsuren Davagdorj, and Keun Ho Ryu
- Subjects
0301 basic medicine ,Information privacy ,General Computer Science ,Computer science ,Lifelong learning ,Feature extraction ,Feature selection ,010501 environmental sciences ,Machine learning ,computer.software_genre ,01 natural sciences ,Data modeling ,Computational biology ,03 medical and health sciences ,class-incremental learning ,Feature (machine learning) ,variational autoencoder ,General Materials Science ,continual learning ,0105 earth and related environmental sciences ,business.industry ,Deep learning ,General Engineering ,deep learning ,deep generative model ,Autoencoder ,Generative model ,030104 developmental biology ,Incremental learning ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Artificial intelligence ,business ,lcsh:TK1-9971 ,computer - Abstract
Developing lifelong learning algorithms are mandatory for computational systems biology. Recently, many studies have shown how to extract biologically relevant information from high-dimensional data to understand the complexity of cancer by taking the benefit of deep learning (DL). Unfortunately, new cancer growing up into the hundred types that make systems difficult to classify them efficiently. In contrast, the current state-of-the-art continual learning (CL) methods are not designed for the dynamic characteristics of high-dimensional data. And data security and privacy are some of the main issues in the biomedical field. This article addresses three practical challenges for class-incremental learning (Class-IL) such as data privacy, high-dimensionality, and incremental learning problems. To solve this, we propose a novel continual learning approach, called Deep Generative Feature Replay (DGFR), for cancer classification tasks. DGFR consists of an incremental feature selection (IFS) and a scholar network (SN). IFS is used for selecting the most significant CpG sites from high-dimensional data. We investigate different dimensions to find an optimal number of selected CpG sites. SN employs a deep generative model for generating pseudo data without accessing past samples and a neural network classifier for predicting cancer types. We use a variational autoencoder (VAE), which has been successfully applied to this research field in previous works. All networks are sequentially trained on multiple tasks in the Class-IL setting. We evaluated the proposed method on the publicly available DNA methylation data. The experimental results show that the proposed DGFR achieves a significantly superior quality of cancer classification tasks with various state-of-the-art methods in terms of accuracy.
- Published
- 2020
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.