Search

Due to the ever-growing nature of dataset sizes, the need for scalable and accurate machine learning algorithms has become evident. Stochastic gradient descent methods are popular tools used to optimize large-scale learning problems because of their generalization performance, simplicity, and scalability. This paper proposes a novel stochastic, also known as online, learning algorithm for solving the L1 support vector machine (SVM) problem, named OnLine Learning Algorithm using Worst-Violators (OLLAWV). Unlike other stochastic methods, OLLAWV eliminates the need for specifying the maximum number of iterations and the use of a regularization term. OLLAWV uses early stopping for controlling the size of the margin instead of the regularization term. The experimental study, performed under very strict nested cross-validation (a.k.a., double resampling), evaluates and compares the performance of this proposal with state-of-the-art SVM kernel methods that have been shown to outperform traditional and widely used approaches for solving L1-SVMs such as Sequential Minimal Optimization . OLLAWV is also compared to 5 other traditional non-SVM algorithms. The results over 23 datasets show OLLAWV's superior performance in terms of accuracy, scalability, and model sparseness, making it suitable for large-scale learning. [ABSTRACT FROM AUTHOR]

Takagi–Sugeno–Kang (TSK) fuzzy systems have been widely applied for solving function approximation and regression-centric problems. Existing dynamic TSK models proposed in the literature can be broadly classified into two classes. Class I TSK models are essentially fuzzy systems that are limited to time-invariant environments. Class II TSK models are generally evolving systems that can learn in time-variant environments. This paper attempts to address the issues of achieving compact, up-to-date fuzzy rule bases and interpretable knowledge bases in TSK models. It proposes a novel rule pruning method which is simple, computationally efficient and biologically plausible. This rule pruning algorithm applies a gradual forgetting approach and adopts the Hebbian learning mechanism behind the long-term potentiation phenomenon in the brain. It also proposes a merging approach which is used to improve the interpretability of the knowledge bases. This approach can prevent derived fuzzy sets from expanding too many times to protect their semantic meanings. These two approaches are incorporated into a generic self-evolving Takagi–Sugeno–Kang fuzzy framework (GSETSK) which adopts an online data-driven incremental-learning-based approach. Extensive experiments were conducted to evaluate the performance of the proposed GSETSK against other established evolving TSK systems. GSETSK has also been tested on real world dataset using the high-way traffic flow density and Dow Jones index time series. The results are encouraging. GSETSK demonstrates its fast learning ability in time-variant environments. In addition, GSETSK derives an up-to-date and better interpretable fuzzy rule base while maintaining a high level of modeling accuracy at the same time. [ABSTRACT FROM AUTHOR]

Imbalanced learning is important and challenging since the problem of the classification of imbalanced datasets is prevalent in machine learning and data mining fields. Sampling approaches are proposed to address this issue, and cluster-based oversampling methods have shown great potential as they aim to simultaneously tackle between-class and within-class imbalance issues. However, all existing clustering methods are based on a one-time approach. Due to the lack of a priori knowledge, improper setting of the number of clusters often exists, which leads to poor clustering performance. Besides, the existing methods are likely to generate noisy instances. To solve these problems, this paper proposes a deep instance envelope network-based imbalanced learning algorithm with the multilayer fuzzy c-means (MlFCM) and a minimum interlayer discrepancy mechanism based on the maximum mean discrepancy (MIDMD). This algorithm can guarantee high quality balanced instances using a deep instance envelope network in the absence of prior knowledge. First, the MlFCM is designed for the original minority class instances to obtain deep instances and increase the diversity of instances. Then, the MIDMD is proposed to avoid the generation of noisy instances and maintain the consistency of the interlayers of instances. Next, the multilayer FCM and minimum interlayer discrepancy mechanism are combined to construct a deep instance envelope network – the MlFC&IDMD. Finally, an imbalance learning algorithm is proposed based on the MlFC&IDMD. In the experimental section, thirty-three popular public datasets are used for verification, and over ten representative algorithms are used for comparison. The experimental results show that the proposed approach significantly outperforms other popular methods. • For the first time, a multilayer FCM (MlFCM) algorithm for classification is proposed to mine the instances for more information. • A minimum interlayer discrepancy mechanism (MIDMD) is proposed to make the distribution of instances before and after clustering consistent. • A deep instance envelope network (MlFC&IDMD) is constructed by combining the MlFCM and MIDMD. • An efficient strategy is proposed to determine the number of layers L of the deep instance envelope network. • A new imbalance learning algorithm is proposed based on the MlFC&IDMD. [ABSTRACT FROM AUTHOR]

Nowadays, many real applications comprise data-sets where the distribution of the classes is significantly different. These data-sets are commonly known as imbalanced data-sets. Traditional classifiers are not able to deal with these kinds of data-sets because they tend to classify only majority classes, obtaining poor results for minority classes. The approaches that have been proposed to address this problem can be categorized into three types: resampling methods, algorithmic adaptations and cost sensitive techniques. Radial Basis Function Networks (RBFNs), artificial neural networks composed of local models or RBFs, have demonstrated their efficiency in different machine learning areas. Centers, widths and output weights for the RBFs must be determined when designing RBFNs. Taking into account the locally tuned response of RBFs, the objective of this paper is to study the influence of global and local paradigms on the weights training phase, within the RBFNs design methodology, for imbalanced data-sets. Least Mean Square and the Singular Value Decomposition have been chosen as representatives of local and global weights training paradigms respectively. These learning algorithms are inserted into classical RBFN design methods that are run on imbalanced data-sets and also on these data-sets preprocessed with re-balance techniques. After applying statistical tests to the results obtained, some guidelines about the RBFN design methodology for imbalanced data-sets are provided. [ABSTRACT FROM AUTHOR]

Multilayer perceptron (MLP) and support vector machine (SVM), two popular learning machines, are increasingly being used as alternatives to classical statistical models for ground-level ozone prediction. However, employing learning machines without sufficient awareness about their limitations can lead to unsatisfactory results in modeling the ozone evolving mechanism, especially during ozone formation episodes. With the spirit of literature review and justification, this paper discusses, with respect to the concerning of ozone prediction, the recently developed algorithms/technologies for treating the most prominent model-performance-degradation limitations. MLP has the “black-box” property, i.e., it hardly provides physical explanation for the trained model, overfitting and local minima problems, and SVM has parameters identification and class imbalance problems. This commentary article aims to stress that the underlying philosophy of using learning machines is by no means as trivial as simply fitting models to the data because it causes difficulties, controversies or unresolved problems. This article also aims to serve as a reference point for further technical readings for experts in relevant fields. [ABSTRACT FROM AUTHOR]