Search

Quite often a machine learning problem lends itself to be split in several well-defined subproblems, or tasks. The goal of Multi-Task Learning (MTL) is to leverage the joint learning of the problem from two different perspectives: on the one hand, a single, overall model, and on the other hand task-specific models. In this way, the found solution by MTL may be better than those of either the common or the task-specific models. Starting with the work of Evgeniou et al., support vector machines (SVMs) have lent themselves naturally to this approach. This paper proposes a convex formulation of MTL for the L1-, L2- and LS-SVM models that results in dual problems quite similar to the single-task ones, but with multi-task kernels; in turn, this makes possible to train the convex MTL models using standard solvers. As an alternative approach, the direct optimal combination of the already trained common and task-specific models can also be considered. In this paper, a procedure to compute the optimal combining parameter with respect to four different error functions is derived. As shown experimentally, the proposed convex MTL approach performs generally better than the alternative optimal convex combination, and both of them are better than the straight use of either common or task-specific models. [ABSTRACT FROM AUTHOR]

• We propose a class of kernel interpolation-based methods, called kernel reconstruction learning, for solving machine learning problems. • We prove a reconstruction representer theorem, which indicates that conventional kernel methods can be viewed as special cases of kernel reconstruction learning. • We propose the kernel reconstruction vector machine, kernel reconstruction logistic regression, and kernel reconstruction density estimation methods, and show that they outperform popular kernel methods. This paper proposes a class of kernel interpolation-based methods, called kernel reconstruction learning, for solving machine learning problems. Kernel reconstruction learning uses kernel interpolators to reconstruct the unknown functions, which are needed to estimate in the problem, with estimated function values at selected knots. It can be applied to any learning problem that involves function estimation. We prove a reconstruction representer theorem, which indicates that conventional kernel methods, including kernel ridge regression, kernel support vector machine, and kernel logistic regression, can be viewed as special cases of kernel reconstruction learning. Furthermore, kernel reconstruction learning provides new algorithms for large datasets. The kernel reconstruction vector machine, kernel reconstruction logistic regression, and kernel reconstruction density estimation are discussed in detail. With appropriate implementations, they are shown to have higher prediction/estimation accuracy and/or less computational cost than popular kernel methods. [ABSTRACT FROM AUTHOR]

The classification of imbalanced datasets is one of the main challenges in machine learning techniques. Support vector machine (SVM), which generates a model biased within the majority class, usually has bad performance on the minority class because this class may be considered incorrectly as noises. Moreover, datasets often include noises and outliers, and SVM cannot effectively deal with those datasets. In this paper, to defeat the aforementioned challenges, we propose intuitionistic fuzzy twin support vector machines for imbalanced data (IFTSVM-ID). The proposed method can easily handle imbalanced datasets in the presence of noises and outliers. A reasonable weighting strategy is offered to deal with imbalanced classes, and a margin-based technique is assigned to reduce the impact of noise and outliers. We formulate the linear and non-linear kernel functions to find two non-parallel hyperplanes. One real-world and thirty-two imbalanced datasets are selected to validate the performance of IFTSVM-ID. The Friedman test and the bootstrap technique with 95% confidence interval are applied to quantify the results statistically. The experimental results show that our proposed method has much better performance in comparison with other similar techniques. [ABSTRACT FROM AUTHOR]

The expensive optimization problem, widespread in real-world applications, poses great challenges to traditional evolutionary algorithms (EAs) due to its costly evaluations. A lot of investigations have proved that it is a practical way to incorporate surrogate models into EAs to reduce the number of time-consuming evaluations. Although various types of surrogate models from the community of machine learning, such as regression models that approximate the fitness values of candidate solutions or classification models that predict candidate solutions' category, have been widely applied separately, none of them consistently outperforms the other. Considering the intrinsic consistency of alleviating computational burden to improve the optimization efficiency, this paper introduces a two-layer architecture to integrate regression and classification models and synergize global and local evolution for high-dimensional expensive many-objective optimization problem (EMaOP). To implement this idea, the Kriging model based on incremental learning is utilized to explore the potential promising space. Further, multi local support vector machine (SVM) classifiers assisted improved differential evolution is run by turn with the global search to accelerate convergence. Accordingly, a novel surrogate-assisted EA, named Classification- and Regression-assisted evolutionary algorithm (CR-SAEA), is proposed on this basis. Experimental studies on several benchmark functions with up to 11 objectives and 200 dimensions confirm the effectiveness of the proposed algorithm and its competitiveness against state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In the fields of computer vision and machine learning, domain adaptation has been extensively studied and the main challenge in the case is how to transform the existing classifier(s) into an effective adaptive classifier to exploit the latent information in the new data source which typically has a different distribution compared with the original data source. Currently, the Adaptive Support Vector Machines (A-SVM) has been proposed to deal with the domain adaptation problem, which is an effective strategy. However, the resulting optimization task by minimizing a convex quadratic function in A-SVM can not effectively minimize the distance between a source and a target domain as much as possible and typically has high computational complexity. In order to handle these problems, in this paper, we extend the A-SVM by determining a pair of nonparallel up- and down-bound functions solved by two smaller sized quadratic programming problems (QPPs) to achieve a faster learning speed. Notably, our method yields two nonparallel separating hyperplanes to exploit the latent discriminant information based on SVM classification mechanism, which can naturally enhance the classification performance. This method is named as Adaptive Twin Support Vector Machine Learning (A-TSVM). Moreover, we consider a high-level learning paradigm with privilege information (LUPI) to learn a induced model that further constrains the solution in the target space. The learned model is named as domain Adaptive Twin Support Vector Machine Learning Using Privileged Information (A-TSVM+). Finally, a series of comparative experiments with many other methods are performed on three datasets. The experimental results effectively indicate that the proposed method can not only greatly improve the accuracy of classification, but also save computing time. [ABSTRACT FROM AUTHOR]

It is well known that the Leave-One-Out Cross-Validation (LOO-CV) is a highly reliable procedure in terms of model selection. Unfortunately, it is an extremely tedious method and has rarely been deployed in practical applications. In this paper, a highly efficient Leave-One-Out Cross-Validation (LOO-CV) formula has been developed and integrated with the popular Regularized Extreme Learning Machine (RELM). The main contribution of this paper is the proposed algorithm, termed as Efficient LOO-CV-based RELM (ELOO-RELM), that can effectively and efficiently update the LOO-CV error with every regularization parameter and automatically select the optimal model with limited user intervention. Rigorous analysis of computational complexity shows that the ELOO-RELM, including the tuning process, can achieve similar efficiency as the original RELM with pre-defined parameter, in which both scale linearly with the size of the training data. An early termination criterion is also introduced to further speed up the learning process. Experimentation studies on benchmark datasets show that the ELOO-RELM can achieve comparable generalization performance as the Support Vector Machines (SVM) with significantly higher learning efficiency. More importantly, comparing to the trial and error tuning procedure employed by the original RELM, the ELOO-RELM can provide more reliable results by the virtue of incorporating the LOO-CV procedure. [ABSTRACT FROM AUTHOR]

In recent years, an enormous amount of research has been carried out on support vector machines (SVMs) and their application in several fields of science. SVMs are one of the most powerful and robust classification and regression algorithms in multiple fields of application. The SVM has been playing a significant role in pattern recognition which is an extensively popular and active research area among the researchers. Research in some fields where SVMs do not perform well has spurred development of other applications such as SVM for large data sets, SVM for multi classification and SVM for unbalanced data sets. Further, SVM has been integrated with other advanced methods such as evolve algorithms, to enhance the ability of classification and optimize parameters. SVM algorithms have gained recognition in research and applications in several scientific and engineering areas. This paper provides a brief introduction of SVMs, describes many applications and summarizes challenges and trends. Furthermore, limitations of SVMs will be identified. The future of SVMs will be discussed in conjunction with further applications. The applications of SVMs will be reviewed as well, especially in the some fields. [ABSTRACT FROM AUTHOR]

This paper presents a recurrent neural network to support vector machine (SVM) learning in pattern classification arising widespread applications in a variety of setting. The SVM learning problem in classification is first converted into an equivalent quadratic programming (QP) formulation, and then a recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification. It is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the QP problem. Several illustrative examples are provided to show the feasibility and the efficiency of the proposed method in this paper. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks have been found to be very efficient universal approximators. Single Layer Feedforward Networks (SLFN) are the most popular and easy to train. The neurons in these networks can use both sigmoidal functions and radial basis functions (RBF) as activation functions. Both functions have been shown to work very efficiently. Sigmoidal networks are already very well described in the literature. This paper will focus on the construction of a SLFN architecture using RBF neurons. There are many algorithms that are used to construct or train networks to solve function approximation problems. In this paper, an algorithm which is a modification of the Incremental Extreme Learning Machine (I-ELM) family of algorithms is proposed. The proposed algorithm eliminates randomness in the learning process with respect to center positions and widths of the RBF neurons. To do this, the input with the highest error magnitude is saved during error calculation and then used as the center for the next incrementally added neuron. Then the radius of the new neuron is iteratively chosen using Nelder–Mead׳s Simplex method. This allows the universal approximation properties of I-ELM to be preserved while greatly reducing the sizes of the trained RBF networks. [ABSTRACT FROM AUTHOR]

The recently proposed Minimal Complexity Machine (MCM) learns a hyperplane classifier by minimizing a bound on the Vapnik-Chervonenkis (VC) dimension. Both the linear and kernel versions of the MCM solve a linear programming problem, in order to minimize a bound on the VC dimension. This paper proposes a new quadratic programming formulation, termed as the Quadratic MCM (QMCM), that minimizes a tighter bound on the VC dimension. We present two variants of the QMCM, that differ in the norm of the error vector being minimized. We also explore a scalable variant of the QMCM for large datasets using Stochastic Gradient Descent (SGD), and present the use of the QMCM as a viable featureselection method, in view of the the sparse nature of the models it learns. We compare the performance of the QMCM variants with LIBLinear, a linear Support Vector Machine (SVM) library; as well as against Pegasos and the linear MCM for large datasets, along with sequential feature selection methods and ReliefF. Our results validate the superiority of the QMCM in terms of statistically significant improvements on benchmark datasets from the UCI Machine Learning repository. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is a popular machine learning method with a solid theoretical foundation, and has shown promising performance on different classification problems. However, it suffers from an expensive training cost, which makes it not be very suitable for the application with a large-scale training set. To this end, as a data pre-processing technique, training set selection (TSS) for SVM has received much attention recently, since it can reduce the size of SVM training set without degrading the performance. In this paper, a subregion division based multi-objective evolutionary algorithm termed SDMOEA-TSS is proposed for SVM training set selection, where objective space is divided into several subregions for effectively searching good solutions. Specifically, in SDMOEA-TSS, a divided based initialization strategy is firstly suggested to initialize the population to locate in different regions of objective space. Then a subregion based evolutionary (including crossover, mutation and update) strategy is developed, which not only makes full use of individuals in each subregion for local search but also maintains the whole population's global search ability. Empirical studies on 21 public data sets demonstrate the superiority of the proposed algorithm over the state-of-the-arts in terms of both quality and diversity of the selected SVM training subset. [ABSTRACT FROM AUTHOR]

Semi-Supervised Support Vector Machines (S3VMs) provide a powerful framework for Semi-Supervised Learning (SSL) tasks which leverage widely available unlabeled data to improve performance. However, there exist three issues in S3VMs: (i) S3VMs require concurrently training c one-against-all (OAA) classifiers (c is the number of classes) for multiclass classification, which is prohibitive for large c ; (ii) S3VMs require huge computational time and large storage (because of the large kernel matrix) in large-scale training and testing; (iii) S3VMs require the balance constraint in the unlabeled data, which not only needs prior knowledge from the unlabeled data (the prior knowledge is unavailable in some applications), but also makes their nonconvex optimization problem more intractable. To address these issues, a novel method called Extreme Semi-Supervised Learning (ESSL) is proposed in this paper. First, the framework of Extreme Learning Machine (ELM) is adopted to handle both binary and multiclass classification problems in a unified model. Second, the hidden layer is encoded by an extremely small approximate empirical kernel map (AEKM) to greatly reduce the computational cost and the memory usage for training and testing. Third, the balance constraint (prior knowledge) in the unlabeled data is removed through the elaborative design of weighting function (which emphasizes the importance of labeled data and the minority pattern in the labeled data).By these three ways, ESSL can be solved effectively and efficiently based on alternating optimization (AO). More specifically, ESSL can be analytically and simply solved by generalized pseudoinverse and oneHotMap function (without any optimization solver and the OAA strategy) in the AO procedure, and consequently, better performance and much faster training speed are always achieved in ESSL. Our empirical study shows that ESSL significantly outperforms existing efficient SSL methods (e.g., meanS3VM and SS-ELM) in terms of accuracy, efficiency and memory, especially for large-scale multiclass problems. As an example, on the 20Newsgroups dataset, ESSL respectively runs 45 and 120 times faster than meanS3VM for training and testing with the improvement in accuracy of 3%, while the memory usage is reduced to 1/14. It is noteworthy that even though all the model parameters are with default values, ESSL already produces very excellent performance without fine-tuning parameters. [ABSTRACT FROM AUTHOR]

Slow Feature Analysis (SFA) is an unsupervised learning algorithm which extracts slowly varying features from a temporal vectorial signal. In SFA, feature slowness is measured by the average value of its squared time-derivative. In this paper, we introduce Frequency-Based Slow Feature Analysis (FSFA) and prove that it is a generalization of SFA in the frequency domain. In FSFA, the low pass filtered versions of the extracted slow features have maximum energy, making slowness a filter dependent measurement. Experimental results show that the extracted features depend on the selected filter kernel and are different than the signals extracted using SFA. However, it is proven that there is one filter kernel that makes FSFA equivalent to SFA. Furthermore, experiments on UCF-101 video action recognition dataset, showcase that the features extracted by FSFA, with proper filter kernels, result in improved classification performance when compared to the features extracted by standard SFA. Finally, an experiment on UCF-101, with an indicative, simple and shallow neural network, being composed of FSFA and SFA nodes, demonstrates that the previously mentioned network, can transform the features extracted by a known Convolutional Neural Network to a new feature space, where classification performance through Support Vector Machine can be improved. [ABSTRACT FROM AUTHOR]

Many machine learning applications involve imbalance class prior probabilities, multi-class classification with many classes (often addressed by one-versus-rest strategy), or "cost-sensitive" classification. In such domains, each class (or in some cases, each sample) requires special treatment. In this paper, we use a constructive procedure to extend SVM's standard loss function to optimize the classifier with respect to class imbalance or class costs. By drawing connections between risk minimization and probability elicitation, we show that the resulting classifier guarantees Bayes consistency. We further analyze the primal and the dual objective functions and derive the objective function in a regularized risk minimization framework. Finally, we extend the classifier to the with cost-sensitive learning with example dependent costs. We perform experimental analysis on class imbalance, cost-sensitive learning with given class and example costs and show that proposed algorithm provides superior generalization performance, compared to conventional methods. [ABSTRACT FROM AUTHOR]

Support Vector Machines (SVMs) are a state-of-the-art and powerful learning algorithm that can effectively solve many real world problems. SVMs are the transposition of the Vapnik–Chervonenkis (VC) theory into a learning algorithm. In this paper, we present the Local Rademacher Complexity Machine (LRCM), a transposition of the Local Rademacher Complexity (LRC) theory, the state-of-the-art evolution of the VC theory, into a learning algorithm. Analogously to what has been done for the SVMs, we will present first the theoretical ideas behind the LRC theory, we will show how these ideas can be translated into a learning algorithm, the LRCM, and then how the LRCM can be made efficient and kernelizable. By exploiting a series of real world datasets, we will show the effectiveness of the LRCM against the SVMs. [ABSTRACT FROM AUTHOR]

Machine learning started as an academic-oriented domain, but nowadays it is becoming more and more widespread across diverse domains, such as retail, healthcare, finance, and many more. This non-academic face of machine learning creates a new set of challenges. The usage of such complex methods by non-expert users has increased the need for interpretable models. To this end, in this paper we propose an approach for extracting explanation rules from support vector machines. The core idea is based on using kernels with feature spaces composed by logical propositions. On top of that, a searching algorithm tries to retrieve the most relevant features/rules that can be used to explain the trained model. Experiments on both categorical and real-valued datasets show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract In recent years, least squares support vector machines (LSSVMs) with various kernel functions have been widely used in the field of machine learning. However, the selection of kernel functions is often ignored in practice. In this paper, an improved LSSVM method based on self-organizing multiple kernel learning is proposed for black-box problems. To strengthen the generalization ability of the LSSVM, some appropriate kernel functions are selected and the corresponding model parameters are optimized using a differential evolution algorithm based on an improved mutation strategy. Due to the large computation cost, a sparse selection strategy is developed to extract useful data and remove redundant data without loss of accuracy. To demonstrate the effectiveness of the proposed method, some benchmark problems from the UCI machine learning repository are tested. The results show that the proposed method performs better than other state-of-the-art methods. In addition, to verify the practicability of the proposed method, it is applied to a real-world converter steelmaking process. The results illustrate that the proposed model can precisely predict the molten steel quality and satisfy the actual production demand. [ABSTRACT FROM AUTHOR]

Abstract A general approach for incorporating imprecise prior knowledge and for robustifying the machine learning SVM-based models is proposed in the paper. The main idea underlying the approach is to use a double duality representation in the framework of the minimax strategy of decision making. This idea allows us to get simple extensions of SVMs including additional constraints for optimization variables (the Lagrange multipliers) formalizing the incorporated imprecise information. The approach is applied to regression, binary classification and one-class classification SVM-based problems. Moreover, it is adopted to set-valued or interval-valued training data. For every problem, numerical examples are provided which illustrate how imprecise information may improve the machine learning algorithm performance. [ABSTRACT FROM AUTHOR]

Abstract Hallucinations are an elusive phenomena that has been associated with psychotic behavior, but which has a high prevalence in healthy population. Some models of Auditory Hallucinations (AH) have been proposed in the literature, but so far empirical evidence is scarce. The current study aims to obtain measurement of effective connectivity on rs-fMRI showing discrimination of subjects with a history of AH to give empirical validation of AH models. In this paper, we consider the Dynamic Causal Modeling (DCM) for the estimation of the effective connection between brain regions on resting state functional magnetic resonance imaging (rs-fMRI). Applying DCM to rs-fMRI data is a very recent approach, that allows studies of effective connectivity on rs-fMRI data. Brain regions are selected on the basis of previous results on functional connectivity in agreement with the models proposed in the literature. The selection of the most significant connection structure is carried as a wrapper feature selection process using Support Vector Machine (SVM) classifiers. Results on a dataset of Schizophrenia patients with and without a history of AH are provided which are in good agreement with the models in the literature. [ABSTRACT FROM AUTHOR]

Abstract N6-methyladenosine (m6A), as one of the most well-studied RNA modifications, has been found to be involved with a wide range of biological processes. Recently, diverse computational methods have been developed for automated identification of m6A sites within RNAs. To identify m6A sites accurately, one of the major challenges is to extract informative features to describe characteristics of m6A sites. However, existing feature representation methods are usually hand-crafted based, and cannot capture discriminative information of m6A sites. In this paper, we develop a m6A site predictor, named DeepM6APred. In this predictor, we propose to use a deep learning based feature descriptor with deep belief network (DBN) to extract high-level latent features. By integrating the deep features with traditional handcrafted features, we train a classification model based on support vector machine and successfully improve the predictive ability of m6A sites. Experimental results on a benchmark dataset show that our proposed method outperforms the state-of-the-art predictors, at least 2% higher in terms of Matthew's correlation coefficient (MCC). Moreover, a webserver that implements the DeepM6APred is established, which is currently available at the website: http://server.malab.cn/DeepM6APred. It is expected to be a useful tool to assist biologists to reveal the functional mechanisms of m6A sites. [ABSTRACT FROM AUTHOR]

Abstract The massive amount of available data potentially used to discover patterns in machine learning is a challenge for kernel based algorithms with respect to runtime and storage capacities. Local approaches might help to relieve these issues. From a statistical point of view local approaches allow additionally to deal with different structures in the data in different ways. This paper analyses properties of localized kernel based, non-parametric statistical machine learning methods, in particular of support vector machines (SVMs) and methods close to them. We will show there that locally learnt kernel methods are universally consistent. Furthermore, we give an upper bound for the maxbias in order to show statistical robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure between two arbitrary variables, and it has been applied in a variety of learning algorithms to improve noise insensitivity. In this paper, based on the correntropy, a non-convex and bounded loss function is obtained which contains second and higher order moments of the classification margin. And the novel loss function is robust to noises and close to the 0–1 loss function. Then we introduce it into extreme learning machine (ELM), and propose a correntropy-based robust ELM framework for classification, trained by half quadratic optimization to cope with non-convexity of the algorithm. To evaluate robustness, feature noise and label noise are simulated to provide noisy environments. Experimental results on benchmark datasets demonstrate that the proposed algorithm is better than original algorithms and robust algorithms. Moreover, the superiority of proposed algorithm in noisy environment is more evident, which further proves its robustness to noises. [ABSTRACT FROM AUTHOR]

Twin support vector machines (TWSVM) is a new machine learning method based on the theory of Support Vector Machine (SVM). Unlike SVM, TWSVM would generate two non-parallel planes, such that each plane is closer to one of the two classes and is as far as possible from the other. In TWSVM, a pair of smaller sized quadratic programming problems (QPPs) is solved, instead of solving a single large one in SVM, making the computational speed of TWSVM approximately 4 times faster than the standard SVM. At present, TWSVM has become one of the popular methods because of its excellent learning performance. In this paper, the research progress of TWSVM is reviewed. Firstly, it analyzes the basic theory of TWSVM, then tracking describes the research progress of TWSVM including the learning model and specific applications in recent years, finally points out the research and development prospects. This helps researchers to effectively use TWSVM as an emerging research approach, encouraging them to work further on performance improvement. [ABSTRACT FROM AUTHOR]

In many small-data-learning problems, owing to the incomplete data structure, explicit information for decision makers is limited. Although machine learning algorithms are extensively applied to extract knowledge, most of them are developed without considering whether the training sets can fully represent the population properties. Focusing on small data which contains nominal inputs and continuous outputs, this paper develops an effective sample generating procedure based on fuzzy theories to tackle the learning issue by data preprocessing. According to the derived fuzzy relations between categories and continuous outputs, the possibilities of the combinations of categories (virtual samples) can be aggregated when continuous outputs are given. Proper virtual samples are further selected by using fuzzy alpha-cut on the possibility distributions, and these are added to the training sets to form new ones. In the experiment, sixteen datasets taken from the UC Irvine Machine Learning Repository are examined with back-propagation neural networks and support vector regressions. The results reveal that the forecasting accuracies of the two models are significantly improved when they are built with the proposed new training sets. Moreover, the results also indicate the proposed method outperforms bootstrap aggregating and the synthetic minority over-sampling technique-Nominal-Continuous with the greatest amount of statistical support. [ABSTRACT FROM AUTHOR]

Approaches based on scene parts are deemed to be one of the main streams for scene categorization. In previous methods, before one can utilize scene parts, models need to be constructed for them first. The quality of part models has a great influence on the final results. However, building high-quality scene part models is still an open question. To perform scene categorization based on parts effectively, in this paper we propose to explore scene part information without constructing explicit models for them. For this purpose, a cascading framework is used, at each of whose stages we aim to process image patches potentially corresponding to scene parts from different perspectives. Specifically, the first stage of the framework uses the selective search algorithm to extract possible object patches from images and represents obtained patches based on convolutional neural networks. Then, spectral clustering and linear support vector machines are adopted to select representative visual patterns for images in the second stage. In the third stage, random forest and multi-class support vector machines are combined to mine and classify image features for determining the categories of the images. Through using the cascading framework, we can explore scene part information step by step without needing to construct explicit models for them. Finally, extensive experiments are conducted to evaluate the proposed method on three well-known benchmark scene datasets, i.e. MIT Indoor 67, SUN397 and Places. Experiment results demonstrated the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Olive fruit fly trap measurements are used as one of the indicators for olive grove infestation, and therefore, as a consultation tool on spraying parameters. In this paper, machine learning techniques are used to predict the next olive fruit fly trap measurement, given input knowledge of previous trap measurements as well as an attribute that acts as a correlation model between the temperature and the development of a pest’s population, known as the Degree Day model. This is the first time the Degree Day model is utilized as input in classification algorithms for the prediction of olive fruit fly trap measurements. Various classification algorithms are employed and applied to different environmental settings, in extensive comparative experiments, in order to detect the impact of the latter on olive fruit fly population prediction. [ABSTRACT FROM AUTHOR]

The design of robust data-based fault diagnostic systems can be formulated in terms of classification tasks. A diagnostic classifier designed to effectively minimize the false and missing alarm rates resulting from noise, uncertainty, and unknown disturbances while maintaining a relatively high performance can be defined as robust. This paper presents a novel criterion to compare off-line the robustness of classifiers. The proposed index allows to complement the estimated misclassification rate and to quantify the quality of any data-based diagnostic system more rigorously. In order to evaluate the effectiveness of the proposed index, both Artificial Neural Networks and Support Vector Machines are used as diagnostic classifiers for the Continuous Stirred-Tank Reactor benchmark. [ABSTRACT FROM AUTHOR]

As having the closed form solution, the least squares support vector machine (LSSVM) has been widely used for classification and regression problems owing to its competitive performance compared with other types of SVMs. However, the LSSVM has two drawbacks: it is sensitive to outliers and its solution lacks sparseness. The robust LSSVM (R-LSSVM) partially overcomes the first drawback via its nonconvex truncated loss function, but it is unable to address the second drawback because its current algorithms produce dense solutions that are inefficient for training large-scale problems. In this paper, we interpret the robustness of the R-LSSVM from a re-weighted viewpoint and develop a primal R-LSSVM using the representer theorem. The new model may have a sparse solution. Then, we design a convergent sparse R-LSSVM (SR-LSSVM) algorithm to achieve a sparse solution of the primal R-LSSVM after obtaining a low-rank approximation of the kernel matrix. The new algorithm not only overcomes the two drawbacks of LSSVM simultaneously, it also has lower complexity than the existing algorithms. Therefore, it is very efficient at training large-scale problems. Numerous experimental results demonstrate that the SR-LSSVM can achieve better or comparable performance to other related algorithms in less training time, especially when used to train large-scale problems. [ABSTRACT FROM AUTHOR]

Multiple kernel learning (MKL) serves as an attractive research direction in current kernel machine learning field. It can flexibly process diverse characteristics of patterns such as heterogeneous information or irregular data, non-flat distribution of high-dimensional samples etc. The existing MKL models are usually built on SVM. However, there is still potential to improve the performance of MKL instead of learning based on SVM. Nonparallel support vector machine (NPSVM), as a novel classifier, pursues two nonparallel proximal hyperplanes with several incomparable advantages over the state-of-the-art classifiers. In this paper, we propose a new model termed as MKNPSVM for classification. By integrating NPSVM into the MKL framework, MKNPSVM inherits the advantages of them and opens a new perspective to extend NPSVM to the MKL field. To solve MKNPSVM efficiently, we provide an alternating optimization algorithm (Alter-MKNPSVM for short) as the solution. We theoretically analyze the performance of MKNPSVM from three viewpoints: the generalization capability analysis, the convergence analysis and the comparisons with NPSVM and MKL. Experimental results on eighteen publicly available UCI data sets confirm the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Stream Processing has recently become one of the current commercial trends to face huge amounts of data. However, normally these techniques need specific infrastructures and high resources in terms of memory and computing nodes. This paper shows how mini-batch techniques and topology extraction methods can help making gigabytes of data to be manageable for just one server using computationally costly Machine Learning techniques as Support Vector Machines. The algorithm iGNGSVM is proposed to improve the performance of Support Vector Machines in datasets where the data is continuously arriving. It is benchmarked against a mini-batch version of LibSVM, achieving good accuracy rates and performing faster than this. [ABSTRACT FROM AUTHOR]

In this paper, we propose a simple yet effective method for video based action recognition referred to as dual-layer kernel extreme learning machine (DKELM). Our approach takes advantages of both early and late fusion techniques into a unified framework. In particular, the first layer in DKELM adopts linear kernel extreme learning machine (KELM) on handcrafted feature kernel, deep-learned feature kernel, and the fused kernel to provide various perspectives about the video. The second layer trains a radial basis function based KELM classifier on different fusion scores obtained from the first layer to predict the final action class label. Finally, we empirically show the superior performance of DKELM, both in terms of accuracy and computational time, over some state-of-the-art human action recognition methods on two large-scale datasets. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is an efficient and effective learning algorithm for pattern classification. For binary classification problem, traditional ELM learns only one hyperplane to separate different classes in the feature space. In this paper, we propose a novel twin extreme learning machine (TELM) to simultaneously train two ELMs with two nonparallel classification hyperplanes. Specifically, TELM first utilizes the random feature mapping mechanism to construct the feature space, and then two nonparallel separating hyperplanes are learned for the final classification. For each hyperplane, TELM jointly minimizes its distance to one class and requires it to be far away from the other class. TELM incorporates the idea of twin support vector machine (TSVM) into the basic framework of ELM, thus TELM could have the advantages of the both algorithms. Moreover, compared to TSVM, TELM has fewer optimization constraint variables but with better classification performance. We also introduce a successive over-relaxation technique to speed up the training of our algorithm. Comprehensive experimental results on a large number of datasets verify the effectiveness and efficiency of TELM. [ABSTRACT FROM AUTHOR]

Assessing liver fibrosis with chronic hepatitis B (CHB) in patients is quite important. Some non-invasive approaches for evaluating liver fibrosis include blood tests and ultrasound elastography. How to effectively combine multiple methods to improve the diagnostic performance remains a challenging problem. The main goal of this paper is to assess and stage liver fibrosis in CHB using feature selection and machine learning methods based on multimodal data. Popular machine learning approaches (e.g., support vector machine (SVM)) and feature selection (FS) were explored to stage the CHB. 16 volunteers and 92 patients with CHB were investigated for liver fibrosis staging based on transient elastography (TE) and acoustical radiation force impulse imaging (ARFI) data. The accuracy of the staging result using FS and a SVM classifier was an accuracy of 90.68% for significant fibrosis (≥F2) and an accuracy of 93.52% for cirrhosis (F4), respectively. The proposed method also increased the sensitivity, specificity, and area under curve (AUC) values for both significant fibrosis and cirrhosis diagnosis, which is very promising for staging liver fibrosis from multimodal information. It outperforms any single method and their linear combination and also achieves a state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Over the past decades, features generated by different models have been designed to describe various aspects of object. To connect the complementary information and represent the data properly, effective heterogeneous feature fusion methods are required. Multiple kernel learning (MKL) methods are widely adopted to learn the feature weights and to fuse features on score-level. In this paper, we exploit score distribution to address the feature fusion problem and propose a novel method named score-distribution MKL (SD-MKL) for image classification. Different from existing MKL methods, SD-MKL uses weights which are learned from score curves as a constraint on the weights of kernels. It contains two stages in off-line part: (1) independent data is used to construct reference curves according to classes and feature type; (2) samples and corresponding score-distribution weights are put into multi-kernel support vector machine (MKSVM) to learn feature weights. Our experimental results demonstrate the effect of exploiting score-distribution information on two datasets, which significantly benefits the performance of image classification. [ABSTRACT FROM AUTHOR]

Predictions regarding the solar greenhouse temperature and humidity are important because they play a critical role in greenhouse cultivation. On account of this, it is important to set up a predictive model of temperature and humidity that would precisely predict the temperature and humidity, reducing potential financial losses. This paper presents a novel temperature and humidity prediction model based on convex bidirectional extreme learning machine (CB-ELM). Simulation results show that the convergence rate of the bidirectional extreme learning machine (B-ELM) can further be improved while retaining the same simplicity, by simply recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added. The performance of the CB-ELM model is compared with other modeling approaches by applying it to predict solar greenhouse temperature and humidity. The experiment results show that the CB-ELM model predictions are more accurate than those of the B-ELM, Back Propagation Neural Network (BPNN), Support Vector Machine (SVM), and Radial Basis Function (RBF). Therefore, it can be considered as a suitable and effective method for predicting the solar greenhouse temperature and humidity. [ABSTRACT FROM AUTHOR]

one of the most important tasks of Mars rover, a robot which explores the Mars surface, is the process of automatic segmentation of images taken by front-line Panoramic Camera (Pancam). This procedure is highly significant since the transformation cost of images from Mars to earth is extremely high. Also, image analysis may help Mars rover for its navigation and localization. In this paper, a new feature vector including wavelet and color features for Mars images is proposed. Then, this feature vector is presented for extreme learning machine (ELM) classifier which leads to a high accuracy pixel classifier. It is shown that this system statistically outperforms support vector machine (SVM) and k-nearest neighbours (KNNs) classifiers with respect to both accuracy and run time. After that, dimension reduction in feature space is done by two proposed feature section algorithms based on ant colony optimization (ACO) to decrease the time complexity which is very important in Mars on-board applications. In the first proposed feature selection algorithm, the same feature subset is selected among the feature vector for all pixel classes, while in the second proposed algorithm, the most significant features are selected for each pixel class, separately. Proposed pixel classifier with complete feature set outperforms prior methods by 6.44% and 5.84% with respect to average Fmeasure and accuracy, respectively. Finally, proposed feature selection methods decrease the feature vector size up to 76% and achieves Fmeasure and accuracy of 91.72% and 91.05%, respectively, which outperforms prior methods with 87.22% and 86.64%. [ABSTRACT FROM AUTHOR]

Twin support vector machine is a machine learning algorithm developing from standard support vector machine. The performance of twin support vector machine is always better than support vector machine on datasets that have cross regions. Recently proposed wavelet twin support vector machine introduces the wavelet kernel function into twin support vector machine to make the combination of wavelet analysis techniques and twin support vector machine come true. Wavelet twin support vector machine not only expands the range of the kernel function selection, but also greatly improves the generalization ability of twin support vector machine. However, similar with twin support vector machine, wavelet twin support vector machine cannot deal with the parameter selection problem well. Unsuitable parameters reduce the classification capability of the algorithm. In order to solve the parameter selection problem in wavelet twin support vector machine, in this paper, we use glowworm swarm optimization method to optimize the parameters of wavelet twin support vector machine and propose wavelet twin support vector machine based on glowworm swarm optimization. Wavelet twin support vector machine based on glowworm swarm optimization takes the parameters of wavelet twin support vector machine as the position information of glowworms, regards the function to calculate the wavelet twin support vector machine classification accuracy as objective function and starts glowworm swarm optimization algorithm to update the glowworms. The optimal parameters are the position information of glowworms that we get when the glowworm swarm optimal algorithm stops. Wavelet twin support vector machine based on glowworm swarm optimization determines the parameters in wavelet twin support vector machine automatically before the training process to avoid difficulty of parameter selection. Reasonable parameters promote the performance of wavelet twin support vector machine and improve the accuracy. The experimental results on benchmark datasets indicate that the proposed approach is efficient and has high classification accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we propose and solve a new machine learning problem called the extensive semi-quantitative regression, where the information about some target values is incomplete; we only know their lower bounds and/or upper bounds instead of their exact values. To employ the information efficiently in extensive semi-quantitative regression, we introduce a local graph to capture the geometric structure for the samples with the exact target values and the target bounds, and construct a graph-based support vector regressor, called ESQ-SVR. The efficiency of our ESQ-SVR is supported by the results of preliminary experiments conducted on both the artificial and real world datasets. [ABSTRACT FROM AUTHOR]

In this paper, based on the representative instances and feature mapping, we propose two Multi-Instance Learning (MIL) algorithms, i.e. Representative Instance and Feature Mapping for instances (RIFM-I) and Representative Instance and Feature Mapping for bags (RIFM-B). These two algorithms firstly select representative positive and negative instances from positive and negative bags, respectively, and then map selected instances and bags to the feature space, in which MIL problem is converted into conventional single-instance learning problem. Finally, Support Vector Data Description (SVDD) method is introduced to solve the converted problem. The experiment on the MUSK dataset shows that RIFM-I performs better than RIFM-B and provides highest classification accuracies compared with the best results obtained among all the methods, and RIFM-B achieves a competitive average accuracy performance. Furthermore, RIFM-I is applied on COREL image repository for the content-based image retrieval. The experimental results show that RIFM-I outperforms the other image retrieval methods, such as MILES and MissSVM, and is able to distinguish two easily confused categories, Beach and Mountains, quite well. In addition, The results in ten data sets commonly used in MIL also show that RIFM-I can achieve better results in most cases. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of hierarchical image classification with multiple semantic views of object categories. A novel method is proposed for computing an image-semantic measure by determining the weights for the semantic similarity among the concepts of each view. After obtaining the new image-semantic measure, we construct a semantic hierarchy with the existing method called TRUST-ME. For the hierarchical classification, we translate the classification task with a learned taxonomy into a structured support vector machine (SVM) learning framework. We demonstrate our method on VOC2010 and a subset of the Animals with Attributes dataset, and show that the structured SVM using the weighted semantic hierarchy provides better accuracy. [ABSTRACT FROM AUTHOR]

The use of learning algorithms for text classification assumes the availability of a large amount of documents which have been organized and labeled correctly by human experts for use in the training phase. Unless the text documents in question have been in existence for some time, using an expert system is inevitable because manual organizing and labeling of thousands of groups of text documents can be a very labor intensive and intellectually challenging activity. Also, in some new domains, the knowledge to organize and label different classes might not be unavailable. Therefore unsupervised learning schemes for automatically clustering data in the training phase are needed. Furthermore, even when knowledge exists, variation is high when the subject under classification depends on personal opinions and is open to different interpretations. This paper describes a methodology which uses Self Organizing Maps (SOM) and alternatively does the automatic clustering by using the Correlation Coefficient (CorrCoef). Consequently the clusters are used as the labels to train the Support Vector Machine (SVM). Experiments and results are presented based on applying the methodology to some standard text datasets in order to verify the accuracy of the proposed scheme. We will also present results which are used to evaluate the effect that dimensionality reduction and changes in the clustering schemes have on the accuracy of the SVM. Results show that the proposed combination has better accuracy compared to training the learning machine using the expert knowledge. [ABSTRACT FROM AUTHOR]

Shape classification is an intriguing and challenging problem that lies at the crossroads of computer vision, geometry processing and machine learning. In this paper, we introduce a graph-theoretic approach for 3D shape classification using graph regularized sparse coding in conjunction with the biharmonic distance map. Our unified framework exploits both sparsity and dependence among the features of shape descriptors in a bid to design robust shape signatures that are effective in discriminating between shapes from different classes. In an effort to coherently capture the similarity between feature descriptors, we use multiclass support vector machines for 3D shape classification on mid-level features that are learned via graph regularized sparse coding. Our experiments on two standard 3D shape benchmarks show that the proposed framework not only outperforms the state-of-the-art methods in classification accuracy, but also provides attractive scalability in terms of computational efficiency. [ABSTRACT FROM AUTHOR]

Current service selection is unable to perform proactively. When a service provider overloads, the services list is ever-lengthening, which leads to backlog and failure of service composition. To compensate for this deficiency, this paper improves the proactive service selection. In this strategy, the service provider analyses a time series of services received to forecast the backlog and consign services to the others through a negotiation process. The least squares support vector learning is used to predict a random list of services, and an acquaintance model (AM) makes a consigner allocate backlog services to other providers with high degree of relationship. The backlog of services by forecasting is entrusted to the provider who can implement the same service, and negotiation between the providers with the same role would allow generation of a new service selection solution before a fault occurs. Experiments showed that the least squares support vector machine (LS-SVM) algorithm was more accurate in predicting a services list and a negotiation mechanism using AM decreased communication time effectively, which improved the success rate of service selection and reduced the execution time of service composition. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) has been a promising method for classification and regression areas due to its solid statistical foundations, such as margin maximization and kernel methods. However, SVM is not typically used for large-scale data mining problems because its training complexity is highly dependent on the dataset size. This paper presents an improved granular support vector machine learning model based on hierarchical and dynamical granulation, namely, HD_GSVM, to solve the low learning efficiency and generalization performance problem of traditional granular support vector machines (GSVM). For HD_GSVM, the original data will be mapped into a high-dimensional space by a Mercer kernel. Then, the data are divided into several granules, and those granules near the approximate hyperplane are extracted and re-granulated on a subtle level by their density and radius degree. Finally, the decision hyperplane will be obtained through all of the granules at different hierarchical and dynamical granulation levels effectively. During the granulation process, the granulation level of all granules can be dynamically changed continuously. With this method, different classification information can be obtained from different levels of granules; to meet a variety of needs for various practical problems from different perspectives. The experimental results on the UCI benchmark datasets demonstrate that the proposed HD_GSVM model can improve the generalization performance greatly with high efficiency synchronously. [ABSTRACT FROM AUTHOR]

Support Vector Machine (SVM) classifier is widely used in analogue circuit diagnosis. However, the penalty parameter C and the kernel parameter γ of SVM classifier with the radial basis function (RBF) affect the classification performance seriously. A double-chains-quantum-genetic-algorithm (DCQGA) based method is proposed to optimize C and γ . In DCQGA, each chromosome carries two gene chains, and each of gene chains represents an optimization solution, which can accelerate the search process and help to find the global solution. Thereafter, the optimal parameters C and γ are obtained by optimizing the parameter searching process with DCQGA. Two common datasets named Iris and Wine from UCI Machine Learning Repository are used to test the performance of the presented SVM classifier. The simulation results illustrate that the population's best fitness and the classifying accuracy of the proposed DCQGA-SVM are higher than that of the Particle-Swarm-Optimization based SVM (PSO-SVM), the Quantum Genetic Algorithm based SVM (QGA-SVM) and the classifier based on grid search method (GS-SVM). Finally, the proposed DCQGA-SVM is applied to analogue circuit diagnosis, a Sallen–Key bandpass filter circuit and a four-opamp biquad high-pass filter are chosen as circuits under test (CUT). Wavelet packet analysis is performed to extract the fault features before classifying. The experimental results show that the SVM parameters selected by DCQGA-SVM contribute to higher diagnosis accuracy than other methods referred in this paper. [ABSTRACT FROM AUTHOR]

This paper proposes a novel probabilistic detection framework of weighted combining semi-supervised k -means clustering and Posterior Probability SVM (PPSVM) for unbalanced data based on robot vision. Within the framework, an algorithm for learning synchronously the k in k -means and features is introduced based on hybrid wrapper and filter criterion. Then the optimal hierarchical probabilistic model by combining k -means and PPSVM is used to anomaly detection so as to alleviate the problems of imbalanced data with small samples, improve the detection accuracy, and deal with the difficult problem of defining the anomaly classes. The other contributions of our approach include the following three aspects: (1) it classifies anomaly candidates by using their class probability distributions rather than the direct extracted features; (2) the relevant classes are automatically built by learning the samples׳ multimodal Gaussian distribution; and (3) the cost-sensitive idea and filter criterion are integrated in learning k and features via cost function of Tabu search. Experimental results on real-world data sets show the proposed approach obtains a satisfactory detection performance within limited time in inspecting the condition of Heating, and Ventilation and Air-Conditioning (HVAC) ductwork. [ABSTRACT FROM AUTHOR]

Many organizations recognize the necessities of utilizing sophisticated tools and systems to protect their computer networks and reduce the risk of compromising their information. Although many machine-learning-based data classification algorithm has been proposed in network intrusion detection problem, each of them has its own strengths and weaknesses. In this paper, we propose an effective intrusion detection framework by using a new adaptive, robust, precise optimization method, namely, time-varying chaos particle swarm optimization (TVCPSO) to simultaneously do parameter setting and feature selection for multiple criteria linear programming (MCLP) and support vector machine (SVM). In the proposed methods, a weighted objective function is provided, which takes into account trade-off between the maximizing the detection rate and minimizing the false alarm rate, along with considering the number of features. Furthermore, to make the particle swarm optimization algorithm faster in searching the optimum and avoid the search being trapped in local optimum, chaotic concept is adopted in PSO and time varying inertia weight and time varying acceleration coefficient is introduced. The performance of proposed methods has been evaluated by conducting experiments with the NSL-KDD dataset, which is derived and modified from well-known KDD cup 99 data sets. The empirical results show that the proposed method performs better in terms of having a high detection rate and a low false alarm rate when compared with the obtained results using all features. [ABSTRACT FROM AUTHOR]

Nowadays by employing many machine learning and pattern classification methods in object classification, the view-based 3D object classification, an emerging research topic, becomes a major research focus. However, most existing researches focus on only a single modality of image features for the object classification, although recent studies have shown that different kinds of features may provide complementary information for 3D object classification. In this paper, we propose the multimodal support vector machine to combine three modalities of image features, i.e., Sift descriptor, Outline Fourier transform descriptor, and Zernike Moments descriptor to discriminate the multiple classes of object, where each kernel corresponds to each modality. In this way, not only the independence of each modality but also the interrelation between them are both taken into considered. And we further employ multi-task feature selection via the l 2 -norm regularization after feature extraction to improve the performance of final classification. The experiments conducted in ETH-80 image set demonstrate the effectiveness and superiority of our method. [ABSTRACT FROM AUTHOR]

Multiple kernel learning method has more advantages over the single one on the model’s interpretability and generalization performance. The existing multiple kernel learning methods usually solve SVM in the dual which is equivalent to the primal optimization. Research shows solving in the primal achieves faster convergence rate than solving in the dual. This paper provides a novel L P -norm( P >1) constraint non-spare multiple kernel learning method which optimizes the objective function in the primal. Subgradient and Quasi-Newton approach are used to solve standard SVM which possesses superlinear convergence property and acquires inverse Hessian without computing a second derivative, leading to a preferable convergence speed. Alternating optimization method is used to solve SVM and to learn the base kernel weights. Experiments show that the proposed algorithm converges rapidly and that its efficiency compares favorably to other multiple kernel learning algorithms. [ABSTRACT FROM AUTHOR]

The availability of multi-core processors has motivated an increasing interest in research lines about parallelization of machine learning algorithms. Kernel methods such as Support Vector Machines (SVMs) or Gaussian Processes (GPs), in spite of their efficacy solving problems of classification and regression, have a very high computational cost and usually produce very large models. In this paper we present parallel algorithmic implementations of Semiparametric SVM (Parallel Semiparametric SVM, PS-SVM) and Gaussian Processes (Parallel full GP, P-GP and Parallel Semiparametric GP, PS-GP). We have implemented the proposed methods using OpenMP and benchmarked them against other state of the art methods, showing their good performance and advantages in both computation time and final model size. [ABSTRACT FROM AUTHOR]