Search

The paper addresses the problem of forecasting in manufacturing systems. The main aim of the research is to propose new hybrid forecasting models combining artificial intelligencebased methods with traditional techniques based on time series – namely: Hybrid econometric model, Hybrid artificial neural network model, Hybrid support vector machine model and Hybrid extreme learning machine model. The study focuses on solving the problem of limited access to independent variables. Empirical verification of the proposed models is built upon real data from the three manufacturing system areas – production planning, maintenance and quality control. Moreover, in the paper, an algorithm for the forecasting accuracy assessment and optimal method selection for industrial companies is introduced. It can serve not only as an efficient and costless tool for advanced manufacturing companies willing to select the right forecasting method for their particular needs but also as an approach supporting the initial steps of transformation towards smart factory and Industry 4.0 implementation. [ABSTRACT FROM AUTHOR]

Celebrity Profiling is a type of text classification problem which is used for predicting the profiling features like birthyear, gender, fame and occupation of celebrity authors by analysing their writing styles. PAN competition introduced this task in 2019 competition. They provided a corpus for celebrity profiling task and the corpus contains four characteristics like gender, birth-year, fame and occupation of celebrity authors. In order to differentiate the authors writing style the researchers extracted a different types of features such as style based, content based, lexical, character based, syntactic and structural features in the approaches of celebrity profiling. The researchers found that content based features play a crucial role when contrasted with other features in the identification of the author. In this work, the content based features are used in the experiment of celebrity profiling. The frequencies of terms in the total corpus are considered to recognize the important features for the experiment. The most frequent terms are used as features for representing the document vectors. The term value in the vector representation plays a vital role to enhance the performance of celebrity profiling. The Term Weight Measures (TWMs) are used for this purpose to compute the importance of a term in a document. In existing literature, various TWMs are proposed by the researchers in various research domains. In this paper, a term weight measures based approach is proposed for celebrity profiling. In this approach, a new TWM is proposed and compared the performance of proposed term weight measure with existing term weight measures. We observed from the results of celebrity profiling the proposed term weight measure attained best accuracies for profiles prediction than other term weight measures. Three Machine Learning (ML) algorithms such as Naïve Bayes (NB), Support Vector Machine (SVM) and Random Forest (RF) are used for evaluating the performance of proposed approach. The PAN 2019 competition celebrity profiling corpus is used in this work. We considered gender, fame and occupation profiles prediction. [ABSTRACT FROM AUTHOR]

For decades, machine learning techniques have been used to process Amharic texts. The potential application of deep learning on Amharic document classification has not been exploited due to a lack of language resources. In this paper, we present a deep learning model for Amharic news document classification. The proposed model uses fastText to generate text vectors to represent semantic meaning of texts and solve the problem of traditional methods. The text vectors matrix is then fed into the embedding layer of a convolutional neural network (CNN), which automatically extracts features. We conduct experiments on a data set with six news categories, and our approach produced a classification accuracy of 93.79%. We compared our method to well-known machine learning algorithms such as support vector machine (SVM), multilayer perceptron (MLP), decision tree (DT), XGBoost (XGB), and random forest (RF) and achieved good results. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is one of the most efficient machine learning tools, and it is fast, simple to use, reliable, and provides accurate classification results. Despite its generalization capability, SVM is usually posed as a quadratic programming (QP) problem to find a separation hyperplane in nonlinear cases. This needs huge quantities of computational time and memory for large datasets, even for moderately sized ones. SVM could be used for classification tasks whose number of samples is limited but does not scale well to large datasets. The idea is to solve this problem by a smoothing technique to get a new smaller dataset representing the original one. This paper proposes a fast and less time and memory-consuming algorithm to solve the problems represented by a nonlinear support vector machine tool, based on generating a Gaussian pyramid to minimize the size of the dataset. The reduce operation between dataset points and the Gaussian pyramid is reformulated to get a smoothed copy of the original dataset. The new dataset points after passing the Gaussian pyramid will be closed to each other, and this will minimize the degree of nonlinearity in the dataset, and it will be 1/4 of the size of the original large dataset. The experiments demonstrate that our proposed techniques can reduce the classical SVM tool complexity, more accurately, and are applicable in real time. [ABSTRACT FROM AUTHOR]

The presence of machine learning, data mining and related disciplines is increasingly evident in everyday environments. The support for the applications of learning techniques in topics related to economic risk assessment, among other financial topics of interest, is relevant for us as human beings. The content of this paper consists of a proposal of a new supervised learning algorithm and its application in real world datasets related to finance, called D1-NN (Dynamic 1-Nearest Neighbor). The D1-NN performance is competitive against the main state of the art algorithms in solving finance-related problems. The effectiveness of the new D1-NN classifier was compared against five supervised classifiers of the most important approaches (Bayes, nearest neighbors, support vector machines, classifier ensembles, and neural networks), with superior results overall. [ABSTRACT FROM AUTHOR]

Classification problem is an important research direction in machine learning. υ-nonparallel support vector machine (υ-NPSVM) is an important classifier used to solve classification problems. It is widely used because of its structural risk minimization principle, kernel trick, and sparsity. However, when solving classification problems, υ-NPSVM will encounter the problem of sample noises and heteroscedastic noise structure, which will affect its performance. In this paper, two improvements are made on the υ-NPSVM model, and a υ-nonparallel parametric margin fuzzy support vector machine (par-υ-FNPSVM) is established. On the one hand, for the noises that may exist in the data set, the neighbor information is used to add fuzzy membership to the samples, so that the contribution of each sample to the classification is treated differently. On the other hand, in order to reduce the effect of heteroscedastic structure, an insensitive loss function is introduced. The advantages of the new model are verified through UCI machine learning standard data set experiments. Finally, Friedman test and Bonferroni-Dunn test are used to verify the statistical significance of it. [ABSTRACT FROM AUTHOR]

Learning from categorical data is a critical yet challenging task. Current research focuses on either leveraging the complex interaction between and within categorical values to generate a numerical representation, or designing a model that can tackle this types of data directly. However, both of these paradigms overlook the relation between the data characteristics and learning model hypothesis. In this paper, we propose a model-aware categorical data embedding framework that jointly reveals the intrinsic categorical data characteristics and optimizes the fitness of the representation for the follow-up learning model. An ELM-aware and a SVM-aware representation methods have been instantiated under this framework. Extensive experiments of classification with the embedded representation on 17 data sets demonstrate that the proposed framework can significantly improve the categorical data representation performance compared with state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of tag completion. Given an image and a set of tags, only a few of the tags are known to be associated with this image or not, and the problem is to predict whether the other tags are associated with the image. To solve this problem, we propose to learn a tag scoring vector for each image and use it to predict the associated tags of the image. To learn the tag scoring vector, we use the method of local linear learning. A local linear function is used in the neighborhood of each image to predict the tag scoring vectors of its neighboring images. We construct a unified objective function for the learning of both tag scoring vectors and local linear function parameters. In this objective, we impose the learned tag scoring vectors to be consistent with the known associations to the tags of each image and also minimize the prediction error of each local linear function, while reducing the complexity of each local function. The objective function is optimized by an alternate optimization strategy and gradient descent methods in an iterative algorithm. We compare the proposed algorithm against different state-of-the-art tag completion methods, and the results show its advantages. [ABSTRACT FROM AUTHOR]

In multiple instance learning problems, patterns are often given as bags and each bag consists of some instances. Most of existing research in the area focuses on multiple instance classification and multiple instance regression, while very limited work has been conducted for multiple instance clustering (MIC). This paper formulates a novel framework, maximum margin multiple instance clustering (M^3IC), for MIC. However, it is impractical to directly solve the optimization problem of M^3IC. Therefore, M^3IC is relaxed in this paper to enable an efficient optimization solution with a combination of the constrained concave-convex procedure and the cutting plane method. Furthermore, this paper presents some important properties of the proposed method and discusses the relationship between the proposed method and some other related ones. An extensive set of empirical results are shown to demonstrate the advantages of the proposed method against existing research for both effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Relevance ranking has been a popular and interesting topic over the years, which has a large variety of applications. A number of machine learning techniques were successfully applied as the learning algorithms for relevance ranking, including neural network, regularized least square, support vector machine and so on. From machine learning point of view, extreme learning machine actually provides a unified framework where the aforementioned algorithms can be considered as special cases. In this paper, pointwise ELM and pairwise ELM are proposed to learn relevance ranking problems for the first time. In particular, ELM type of linear random node is newly proposed together with kernel version of ELM to be linear as well. The famous publicly available dataset collection LETOR is tested to compare ELM-based ranking algorithms with state-of-art linear ranking algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a new incremental learning approach to endow a Takagi–Sugeno-type fuzzy classification model with high generalization ability. The proposed fuzzy model is learned through incremental support vector machine (SVM) and margin-selected gradient descent learning and is called \FM^3 . In this learning approach, training samples are fed incrementally one-by-one instead of all in one batch. The \FM^3 evolves from an empty rule set. A one-pass clustering algorithm is used to determine the number of rules and initial fuzzy sets in the rule antecedent part. An online incremental linear SVM is proposed to tune the rule consequent parameters to endow the \FM^3 with high generalization ability. The use of incremental instead of batch SVM enables the \FM^3 to handle online training problems with only one new sample available at a time. For antecedent parameter learning, a margin-selected gradient descent algorithm is proposed to prevent overtraining. Simulation results and comparisons with SVMs and fuzzy classifiers with different learning algorithms demonstrate the advantage of the \FM^3 . [ABSTRACT FROM PUBLISHER]

Traditional projection twin support vector machines (SVMs) ignore the differences between the categories when establishing the objective functions, which would lessen their generalization performance. To solve the issue, an improved projection twin SVM (abbreviated as IPTSVM) is proposed in this paper, which aims to find two projected directions via a single quadratic programming problem. In their respective subspace, the projected sample points belonging to each category are far from those of the other class. Meanwhile, to enhance the performance, the recursive arithmetic seeks for more than one projection directions for each class. Besides, an effective clipping dual coordinate descent model is adopted to solve the dual problem to accelerate the training process. The linear IPTSVM model could be changed into the nonlinear model by using the kernel metric. Furthermore, the multi-label version of IPTSVM model is developed to deal with the multi-label learning problems. Experiments on a set of public datasets show that the IPTSVM model has significant advantages over the other models in terms of generalization performance. [ABSTRACT FROM AUTHOR]

Featured Application: Deep learning, decision tree, linear discriminant analysis (LDA), support vector machines (SVMs), k-nearest neighbors algorithm (K-NN), and ensemble learning are evaluated for detecting hairy scalp problems. To the best of our knowledge, we are the first case study to apply modern machine learning to the diagnosis and analysis of hairy scalp issues. Deep learning has become the most popular research subject in the fields of artificial intelligence (AI) and machine learning. In October 2013, MIT Technology Review commented that deep learning was a breakthrough technology. Deep learning has made progress in voice and image recognition, image classification, and natural language processing. Prior to deep learning, decision tree, linear discriminant analysis (LDA), support vector machines (SVM), k-nearest neighbors algorithm (K-NN), and ensemble learning were popular in solving classification problems. In this paper, we applied the previously mentioned and deep learning techniques to hairy scalp images. Hairy scalp problems are usually diagnosed by non-professionals in hair salons, and people with such problems may be advised by these non-professionals. Additionally, several common scalp problems are similar; therefore, non-experts may provide incorrect diagnoses. Hence, scalp problems have worsened. In this work, we implemented and compared the deep-learning method, the ImageNet-VGG-f model Bag of Words (BOW), with machine-learning classifiers, and histogram of oriented gradients (HOG)/pyramid histogram of oriented gradients (PHOG) with machine-learning classifiers. The tools from the classification learner apps were used for hairy scalp image classification. The results indicated that deep learning can achieve an accuracy of 89.77% when the learning rate is 1 × 10−4, and this accuracy is far higher than those achieved by BOW with SVM (80.50%) and PHOG with SVM (53.0%). [ABSTRACT FROM AUTHOR]

Large-scale problems have been a very active topic in machine learning area. In the time of big data, it is a challenge and meaningful work to solve such problems. Standard SVM can make linear classification on large-scale problems effectively, with acceptable training time and excellent prediction accuracy. However, nonparallel SVM (NPSVM) and ramp loss nonparallel SVM (RNPSVM) are proposed with better performance than SVM on benchmark datasets. It is motivated to introduce NPSVMs into the area of large-scale issues. In this paper, we propose large-scale linear NPSVMs, solved by the alternating direction method of multipliers (ADMM), to handle large-scale classification problems. ADMM breaks large problems into smaller pieces, avoiding solving intractable problems and leading to higher training speed. The primal problems of NPSVM are convex and differentiable, and they can be managed directly by ADMM. But the objective functions of RNPSVM, composed of convex ones and concave ones, should first be processed by CCCP algorithm and transformed as a series of convex programs. Then, we apply ADMM to solve these programs in every iteration. Experiments of NPSVMs on large-scale problems verify that the algorithms can classify large-scale tasks effectively. [ABSTRACT FROM AUTHOR]

Data imbalance occurs on most real-world classification problems and decreases the performance of classifiers. An oversampling method addresses the imbalance problem by generating synthetic samples to balance the data distribution. However, many of the existing oversampling methods have potential problems in generating wrong and unnecessary synthetic samples, which makes the learning tasks difficult. This paper proposes a new segmented oversampling method for imbalanced data classification. The input space is first partitioned into several linearly separable local partitions along the potential separation boundary by introducing a bottom-up, minimal-spanning-tree-based clustering method; an oversampling method is then applied within each local linear partition to prevent the generation of wrong and unnecessary synthetic samples; a quasi-linear support vector machine is finally used to realize the classification by taking advantages of the local linear partitions. Simulation results on different real-world datasets show that the proposed segmented oversampling method is effective for imbalanced data classifications. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. [ABSTRACT FROM AUTHOR]

The graduation projects (GP) are important because it reflects the academic profile and achievement of the students. For many years' graduation projects are done by the information technology department students. Most of these projects have great value, and some were published in scientific journals and international conferences. However, these projects are stored in an archive room haphazardly and there is a very small part of it is a set of electronic PDF files stored on hard disk, which wastes time and effort and cannot benefit from it. However, there is no system to classify and store these projects in a good way that can benefit from them. In this paper, we reviewed some of the best machine learning algorithms to classify text "graduation projects", support vector machine (SVM) algorithm, logistic regression (LR) algorithm, random forest (RF) algorithm, which can deal with an extremely small amount of dataset after comparing these algorithms based on accuracy. We choose the SVM algorithm to classify the projects. Besides, we will mention how to deal with a super small dataset and solve this problem. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) are among the most robust and accurate methods in all well-known machine learning algorithms, especially for classification. The SVMs train a classification model by solving an optimization problem to decide which instances in the training datasets are the support vectors (SVs). However, SVs are intact instances taken from the training datasets and directly releasing the classification model of the SVMs will carry significant risk to the privacy of individuals, when the training datasets contain sensitive information. In this paper, we study the problem of how to release the classification model of kernel SVMs while preventing privacy leakage of the SVs and satisfying the requirement of privacy protection. We propose a new differentially private algorithm for the kernel SVMs based on the exponential and Laplace hybrid mechanism named DPKSVMEL. The DPKSVMEL algorithm has two major advantages compared with existing private SVM algorithms. One is that it protects the privacy of the SVs by postprocessing and the training process of the non-private kernel SVMs does not change. Another is that the scoring function values are directly derived from the symmetric kernel matrix generated during the training process and does not require additional storage space and complex sensitivity analysis. In the DPKSVMEL algorithm, we define a similarity parameter to denote the correlation or distance between the non-SVs and every SV. And then, every non-SV is divided into a group with one of the SVs according to the maximal value of the similarity. Under some certain similarity parameter value, we replace every SV with a mean value of the top-k randomly selected most similar non-SVs within the group by the exponential mechanism if the number of non-SVs is greater than k. Otherwise, we add random noise to the SVs by the Laplace mechanism. We theoretically prove that the DPKSVMEL algorithm satisfies differential privacy. The extensive experiments show the effectiveness of the DPKSVMEL algorithm for kernel SVMs on real datasets; meanwhile, it achieves higher classification accuracy than existing private SVM algorithms. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs), with their roots in Statistical Learning Theory (SLT) and optimization methods, have become powerful tools for problem solution in machine learning. SVMs reduce most machine learning problems to optimization problems and optimization lies at the heart of SVMs. Lots of SVM algorithms involve solving not only convex problems, such as linear programming, quadratic programming, second order cone programming, semi-definite programming, but also non-convex and more general optimization problems, such as integer programming, semi-infinite programming, bi-level programming and so on. The purpose of this paper is to understand SVM from the optimization point of view, review several representative optimization models in SVMs, their applications in economics, in order to promote the research interests in both optimization-based SVMs theory and economics applications. This paper starts with summarizing and explaining the nature of SVMs. It then proceeds to discuss optimization models for SVM following three major themes. First, least squares SVM, twin SVM, AUC Maximizing SVM, and fuzzy SVM are discussed for standard problems. Second, support vector ordinal machine, semisupervised SVM, Universum SVM, robust SVM, knowledge based SVM and multi-instance SVM are then presented for nonstandard problems. Third, we explore other important issues such as lp-norm SVM for feature selection, LOOSVM based on minimizing LOO error bound, probabilistic outputs for SVM, and rule extraction from SVM. At last, several applications of SVMs to financial forecasting, bankruptcy prediction, credit risk analysis are introduced. [ABSTRACT FROM AUTHOR]

Aiming at the problems of difficult feature extraction and low recognition accuracy of coal and gangue, this paper proposes a recognition method based on dielectric and geometric features, which uses variational mode decomposition (VMD) to denoise the response signal to obtain its dielectric features; uses the marked watershed method to segment the image boundary to obtain its geometric features. The dielectric and geometric characteristic parameters of coal and gangue are extracted to construct the classification model of support vector machines (SVM). It takes only 0.001483 s for the secondary SVM classification model to process 60 groups of test samples, and the recognition accuracy is 98.33%, which is superior to the constructed SVM classification model, k-nearest neighbor (KNN) classification model, and back propagation neural network (BP) classification model. This method opens up a new way to solve the problem of coal and gangue identification. [Display omitted] • A recognition model of coal and gangue based on dielectric characteristics and geometric constraints is proposed. • The effective thickness and position of coal and gangue in capacitor should be strictly controlled. • The geometric features of coal and gangue are extracted by industrial camera. • A VMD- Kalman denoising method is proposed. • Quadratic support vector machine algorithm has excellent performance in processing eigenvectors. [ABSTRACT FROM AUTHOR]

High prediction accuracies are not the only objective to consider when solving problems using machine learning. Instead, particular scientific applications require some explanation of the learned prediction function. For computational biology, positional oligomer importance matrices (POIMs) have been successfully applied to explain the decision of support vector machines (SVMs) using weighted-degree (WD) kernels. To extract relevant biological motifs from POIMs, the motifPOIM method has been devised and showed promising results on real-world data. Our contribution in this paper is twofold: as an extension to POIMs, we propose gPOIM, a general measure of feature importance for arbitrary learning machines and feature sets (including, but not limited to, SVMs and CNNs) and devise a sampling strategy for efficient computation. As a second contribution, we derive a convex formulation of motifPOIMs that leads to more reliable motif extraction from gPOIMs. Empirical evaluations confirm the usefulness of our approach on artificially generated data as well as on real-world datasets. [ABSTRACT FROM AUTHOR]

Quantum computing has rapidly gained prominence for its unprecedented computational efficiency in solving specific problems when compared to classical computing counterparts. This surge in attention is particularly pronounced in the realm of quantum machine learning (QML) following a classical trend. Here we start with a comprehensive overview of the current state-of-the-art in Quantum Support Vector Machines (QSVMs). Subsequently, we analyze the limitations inherent in both annealing and gate-based techniques. To address these identified weaknesses, we propose a novel hybrid methodology that integrates aspects of both techniques, thereby mitigating several individual drawbacks while keeping the advantages. We provide a detailed presentation of the two components of our hybrid models, accompanied by the presentation of experimental results that corroborate the efficacy of the proposed architecture. These results pave the way for a more integrated paradigm in quantum machine learning and quantum computing at large, transcending traditional compartmentalization. [ABSTRACT FROM AUTHOR]

The traditional vector-based classifiers, such as support vector machine (SVM) and twin support vector machine (TSVM), cannot handle tensor data directly and may not utilize the data informations effectively. In this paper, we propose a novel classifier based on tensor data, called twin bounded support tensor machine (TBSTM) which is an extension of twin bounded support vector machine (TBSVM). Similar to TBSVM, TBSTM gets two hyperplanes and obtains the solution by solving two quadratic programming problems (QPPs). The computational complexity of each QPPs is smaller than that of support tensor machine (STM). TBSTM not only retains the advantage of TBSVM, but also has its unique superior characteristics: (1) it makes full use of the structure information of data; (2) it has acceptable or better classification accuracy compared to STM, TBSVM and SVM; (3) the computational cost is basically less than STM; (4) it can deal with large data that TBSVM is not easy to achieve, especially for small-sample-size (S3) problems; (5) it adopts alternating successive over relaxation iteration (ASOR) method to solve optimization problems which accelerates the pace of training. Finally, we demonstrate the effectiveness and superiority by the experiments based on vector and tensor data. [ABSTRACT FROM AUTHOR]

This paper presents a recurrent neural network to support vector machine (SVM) learning in pattern classification arising widespread applications in a variety of setting. The SVM learning problem in classification is first converted into an equivalent quadratic programming (QP) formulation, and then a recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification. It is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the QP problem. Several illustrative examples are provided to show the feasibility and the efficiency of the proposed method in this paper. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) has been introduced as a simple and efficient learning approach for regression and classification applications. From the optimization point of view, optimized ELM is equivalent to SVM, but with less constraints in the optimization formulation and random ELM kernel. This paper introduces an active set based optimized ELM approach to solve bound constrained optimization problem in a straightforward way, which operates on a small working set of variables at each iteration. Thus, the constrained problem can be eventually solved by an unconstrained algorithm, and this enables us to establish a global convergence theory. The approach requires less time for quadratic programming solving and provides better generalization performance. In addition, the proposed approach with much smaller number of non-bound support values is significantly faster than SVM with active set strategy for large training data set. [ABSTRACT FROM AUTHOR]

The paper presents a formalized statement of the problem of selecting parameters and construction of a genomic classifier for medical test systems with mathematical methods of machine learning without the use of biological and medical knowledge. A method is proposed to solve this problem. The results of testing the method using microarray datasets containing information on genome-wide transcriptome of the samples of estrogen positive breast tumors are discussed. Testing showed that the quality of classification provided by the constructed test system and implemented on the basis of assessments of expression of 12 genes is not inferior to the quality of classification carried out by such test systems as OncotypeDX and MammaPrint. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of learning from weakly labeled data, where labels of the training examples are incomplete. This includes, for example, (i) semi-supervised learning where labels are partially known; (ii) multi-instance learning where labels are implicitly known; and (iii) clustering where labels are completely unknown. Unlike supervised learning, learning with weak labels involves a difficult Mixed-Integer Programming (MIP) problem. Therefore, it can suffer from poor scalability and may also get stuck in local minimum. In this paper, we focus on SVMs and propose the WELLSVM via a novel label generation strategy. This leads to a convex relaxation of the original MIP, which is at least as tight as existing convex Semi-Definite Programming (SDP) relaxations. Moreover, the WELLSVM can be solved via a sequence of SVM subproblems that are much more scalable than previous convex SDP relaxations. Experiments on three weakly labeled learning tasks, namely, (i) semi-supervised learning; (ii)multi-instance learning for locating regions of interest in content-based information retrieval; and (iii) clustering, clearly demonstrate improved performance, and WELLSVM is also readily applicable on large data sets. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a classification method named explicit Markov model is applied for text classification. Currently some machine learning technologies, such as support vector machine (SVM), have been discussed widely in text classification. However, these methods consider that any two features are independent and ignore the language structure information. Hidden Markov model is a powerful tool for sequence tagging problems. This paper presents a new method called explicit Markov model (EMM) which is based on HMM for text classification. EMM make better use of the context information between the observation symbols. Our experiments are conducted on three datasets: Reuter's 21578 R8 dataset, WebKB and Fudan University Chinese text classification corpus. Experimental results show that the performance of EMM is comparable to SVM for text classification. [Copyright &y& Elsevier]

Abstract A general approach for incorporating imprecise prior knowledge and for robustifying the machine learning SVM-based models is proposed in the paper. The main idea underlying the approach is to use a double duality representation in the framework of the minimax strategy of decision making. This idea allows us to get simple extensions of SVMs including additional constraints for optimization variables (the Lagrange multipliers) formalizing the incorporated imprecise information. The approach is applied to regression, binary classification and one-class classification SVM-based problems. Moreover, it is adopted to set-valued or interval-valued training data. For every problem, numerical examples are provided which illustrate how imprecise information may improve the machine learning algorithm performance. [ABSTRACT FROM AUTHOR]

This paper concerns the generation of support vector machine classifiers for solving the pattern recognition problem in machine learning. A method is proposed based on interior-point methods for convex quadratic programming. This interior-point method uses a linear preconditioned conjugate gradient method with a novel preconditioner to compute each iteration from the previous. An implementation is developed by adapting the object-oriented package OOQP to the problem structure. Numerical results are provided, and computational experience is discussed. [ABSTRACT FROM AUTHOR]

Learning useful representations from original features is a key issue in classification tasks. Stacked autoencoders (SAEs) are easy to understand and realize, and they are powerful tools that learn deep features from original features, so they are popular for classification problems. The deep features can further combine the original features to construct more representative features for classification. However, existing SAEs do not consider the original features within the network structure and during training, so the deep features have low complementarity with the original features. To solve the problem, this paper proposes an embedded stacked group sparse autoencoder (ESGSAE) for more effective feature learning. Different from traditional stacked autoencoders, the ESGSAE model considers the complementarity between the original feature and the hidden outputs by embedding the original features into hidden layers. To alleviate the impact of the small sample problem on the generalization of the proposed ESGSAE model, an L 1 regularization-based feature selection strategy is designed to further improve the feature quality. After that, an ensemble model with support vector machine (SVM) and weighted local discriminant preservation projection (w_LPPD) is designed to further enhance the feature quality. Based on the designs above, an embedded stacked group sparse autoencoder ensemble with L 1 regularization and manifold reduction is proposed to obtain deep features with high complementarity in the context of the small sample problem. At the end of this paper, several representative public datasets are used for verification of the proposed algorithm. The results demonstrate that the ESGSAE ensemble model with L 1 regularization and manifold reduction yields superior performance compared to other existing and state-of-the-art feature learning algorithms, including some representative deep stacked autoencoder methods. Specifically, compared with the original features, the representative feature extraction algorithms and the improved autoencoders, the algorithm proposed in this paper can improve the classification accuracy by up to 13.33%, 7.33%, and 9.55%, respectively. The data and codes can be found in: https://share.weiyun.com/Jt7qeORm • A hybrid feature is embedded into the training process to construct a novel deep model. • A group sparsity constraint is introduced to obtain the sparse representations. • The ESGSAE ensemble model is constructed to obtain high complementary features. • A three-step feature learning mechanism is realized. [ABSTRACT FROM AUTHOR]

Due to the ever-growing nature of dataset sizes, the need for scalable and accurate machine learning algorithms has become evident. Stochastic gradient descent methods are popular tools used to optimize large-scale learning problems because of their generalization performance, simplicity, and scalability. This paper proposes a novel stochastic, also known as online, learning algorithm for solving the L1 support vector machine (SVM) problem, named OnLine Learning Algorithm using Worst-Violators (OLLAWV). Unlike other stochastic methods, OLLAWV eliminates the need for specifying the maximum number of iterations and the use of a regularization term. OLLAWV uses early stopping for controlling the size of the margin instead of the regularization term. The experimental study, performed under very strict nested cross-validation (a.k.a., double resampling), evaluates and compares the performance of this proposal with state-of-the-art SVM kernel methods that have been shown to outperform traditional and widely used approaches for solving L1-SVMs such as Sequential Minimal Optimization . OLLAWV is also compared to 5 other traditional non-SVM algorithms. The results over 23 datasets show OLLAWV's superior performance in terms of accuracy, scalability, and model sparseness, making it suitable for large-scale learning. [ABSTRACT FROM AUTHOR]

Machine learning algorithms often suffer from good generalization in testing domains especially when the training (source) and test (target) domains do not have similar distributions. To address this problem, several domain adaptation techniques have been proposed to improve the performance of the learning algorithms when they face accuracy degradation caused by the domain shift problem. In this paper, we focus on the non-homogeneous distributed target domains and propose a new latent subdomain discovery model to divide the target domain into subdomains while adapting them. It is expected that applying adaptation on subdomains increase the rate of detection in comparing with the situation that the target domain is seen as one single domain. The proposed division method considers each subdomain as a cluster which has the definite ratio of positive to negative samples, linear discriminability and conditional distribution similarity to the source domain. This method divides the target domain into subdomains while adapting the trained target classifier for each subdomain using Adapt-SVM adaptation method. It also has a simple solution for selecting the appropriate number of subdomains. We call our proposed method Cluster-based Adaptive SVM or CA-SVM in short. We test CA-SVM on two different computer vision problems, pedestrian detection and image classification. The experimental results show the advantage in accuracy rate for our approach in comparison to several baselines. [ABSTRACT FROM AUTHOR]

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects. [ABSTRACT FROM AUTHOR]

Cutting plane algorithm (CPA) is a generalization of iterative first-order gradient method, in which the objective function is approximated successively by supporting hyperplanes. CPA has been tailored to solve regularized loss minimization in machine learning by exploiting the regularization structure. In particular, for linear Support Vector Machine (SVM) embedding a line search procedure effectively remedies the fluctuations of function value and speeds up the convergence in practical issue. However, the existing line search strategy based on sorting algorithm takes O ( m log  m ) time. In this paper, we propose a more effective line search solver which spends only linear time. It can be extended to multiclass SVM in which an optimized explicit piecewise linear function finding algorithm is prearranged. The total SVM training time is proved to reduce theoretically and experiments consistently confirm the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose and solve a new machine learning problem called the extensive semi-quantitative regression, where the information about some target values is incomplete; we only know their lower bounds and/or upper bounds instead of their exact values. To employ the information efficiently in extensive semi-quantitative regression, we introduce a local graph to capture the geometric structure for the samples with the exact target values and the target bounds, and construct a graph-based support vector regressor, called ESQ-SVR. The efficiency of our ESQ-SVR is supported by the results of preliminary experiments conducted on both the artificial and real world datasets. [ABSTRACT FROM AUTHOR]

In this paper, we propose two novel binary classifiers termed as “Improvements on ν -Twin Support Vector Machine: I ν -TWSVM and I ν -TWSVM (Fast)” that are motivated by ν -Twin Support Vector Machine ( ν -TWSVM). Similar to ν -TWSVM, I ν -TWSVM determines two nonparallel hyperplanes such that they are closer to their respective classes and are at least ρ distance away from the other class. The significant advantage of I ν -TWSVM over ν -TWSVM is that I ν -TWSVM solves one smaller-sized Quadratic Programming Problem (QPP) and one Unconstrained Minimization Problem (UMP); as compared to solving two related QPPs in ν -TWSVM. Further, I ν -TWSVM (Fast) avoids solving a smaller sized QPP and transforms it as a unimodal function, which can be solved using line search methods and similar to I ν -TWSVM, the other problem is solved as a UMP. Due to their novel formulation, the proposed classifiers are faster than ν -TWSVM and have comparable generalization ability. I ν -TWSVM also implements structural risk minimization (SRM) principle by introducing a regularization term, along with minimizing the empirical risk. The other properties of I ν -TWSVM, related to support vectors (SVs), are similar to that of ν -TWSVM. To test the efficacy of the proposed method, experiments have been conducted on a wide range of UCI and a skewed variation of NDC datasets. We have also given the application of I ν -TWSVM as a binary classifier for pixel classification of color images. [ABSTRACT FROM AUTHOR]

The Nearest Neighbor classifier is a popular nonparametric classification method that has been successfully applied to many pattern recognition problems. Its usefulness has been limited, however, because of its computational complexity and sensitivity to outliers in the training set. Computational complexity is becoming less of an issue thanks to the availability of inexpensive memory and high processing speeds. To overcome the second limitation, sensitivity to outliers in the training set, researchers have developed editing and condensing techniques that are aimed at selecting a proper set of prototypes from the training set. In this work, we propose a new editing technique based on the idea of rewarding those patterns that make a contribution to a correct classification while punishing those patterns that provide an incorrect classification. This criterion is coupled with an approach for selecting the edited patterns based on the minimization of a criteria index related to the distances in the training patterns and on the calculation of edge and border patterns for determining the number of edited patterns. Extensive experiments were conducted across several classification problems that evaluated both the efficacy of the proposed technique with respect to other editing approaches and the advantages of using our proposed editing technique either in combination with condensing techniques or as a method utilized in the preprocessing stage. Moreover, the proposed editing approach is shown to be particularly effective when combined with a support vector machine (SVM) classifier. The MATLAB code used in the proposed paper will be available at https://www.dei.unipd.it/node/2357 . [ABSTRACT FROM AUTHOR]

Optical Music Recognition (OMR) has received increasing attention in recent years. In this paper, we propose a classifier based on a new method named Directed Acyclic Graph-Large margin Distribution Machine (DAG-LDM). The DAG-LDM is an improvement of the Large margin Distribution Machine (LDM), which is a binary classifier that optimizes the margin distribution by maximizing the margin mean and minimizing the margin variance simultaneously. We modify the LDM to the DAG-LDM to solve the multi-class music symbol classification problem. Tests are conducted on more than 10000 music symbol images, obtained from handwritten and printed images of music scores. The proposed method provides superior classification capability and achieves much higher classification accuracy than the state-of-the-art algorithms such as Support Vector Machines (SVMs) and Neural Networks (NNs). [ABSTRACT FROM AUTHOR]

Source localization and target tracking are among the most challenging problems in wireless sensor networks (WSN). Most of the state-of-the-art solutions are complicated and do not meet the processing and memory limitations of the existing low-cost sensor nodes. In this paper, we propose computationally-cheap solutions based on the support vector machine (SVM) and twin SVM (TWSVM) learning algorithms in which network nodes firstly detect the desired signal. Then, the network is trained to specify the nodes in the vicinity of the source (or target); hence, the region of event is detected. Finally, the centroid of the event region is considered as an estimation of the source location. The efficiency of the proposed methods is shown by simulations. [ABSTRACT FROM AUTHOR]

The ensemble learning paradigm has proved to be relevant to solving most challenging industrial problems. Despite its successful application especially in the Bioinformatics, the petroleum industry has not benefited enough from the promises of this machine learning technology. The petroleum industry, with its persistent quest for high-performance predictive models, is in great need of this new learning methodology. A marginal improvement in the prediction indices of petroleum reservoir properties could have huge positive impact on the success of exploration, drilling and the overall reservoir management portfolio. Support vector machines (SVM) is one of the promising machine learning tools that have performed excellently well in most prediction problems. However, its performance is a function of the prudent choice of its tuning parameters most especially the regularization parameter, C . Reports have shown that this parameter has significant impact on the performance of SVM. Understandably, no specific value has been recommended for it. This paper proposes a stacked generalization ensemble model of SVM that incorporates different expert opinions on the optimal values of this parameter in the prediction of porosity and permeability of petroleum reservoirs using datasets from diverse geological formations. The performance of the proposed SVM ensemble was compared to that of conventional SVM technique, another SVM implemented with the bagging method, and Random Forest technique. The results showed that the proposed ensemble model, in most cases, outperformed the others with the highest correlation coefficient, and the lowest mean and absolute errors. The study indicated that there is a great potential for ensemble learning in petroleum reservoir characterization to improve the accuracy of reservoir properties predictions for more successful explorations and increased production of petroleum resources. The results also confirmed that ensemble models perform better than the conventional SVM implementation. [ABSTRACT FROM AUTHOR]

The ranking problem has received increasing attention in both the statistical and machine learning literature. This paper considers support vector machines for ranking. Under some mild conditions, a learning rate is established. [ABSTRACT FROM AUTHOR]

The dual coordinate descent (DCD) algorithm solves the dual problem of support vector machine (SVM) by minimizing a series of single-variable sub-problems with a random order at inner iterations. Apparently, this DCD algorithm gives a sightless update for all variables at each iteration, which leads to a slow speed. In this paper, we present a clipping dual coordinate descent (clipDCD) algorithm for solving the dual problem of SVM. In each iteration, this clipDCD algorithm only solves one single-variable sub-problem according to the maximal possibility-decrease strategy on objective value. We can easily implement this clipDCD algorithm since it has a much simpler formulation compared with the DCD algorithm. Our experiment results indicate that, if the clipDCD algorithm is employed, SVM, twin SVM (TWSVM) and its extensions not only obtain the same classification accuracies, but also take much faster learning speeds than those classifiers employing the DCD algorithm. [ABSTRACT FROM AUTHOR]

Abstract: Twin support vector machines (TWSVMs), as the representative nonparallel hyperplane classifiers, have shown the effectiveness over standard SVMs from some aspects. However, they still have some serious defects restricting their further study and real applications: (1) They have to compute and store the inverse matrices before training, it is intractable for many applications where data appear with a huge number of instances as well as features; (2) TWSVMs lost the sparseness by using a quadratic loss function making the proximal hyperplane close enough to the class itself. This paper proposes a Sparse Linear Nonparallel Support Vector Machine, termed as -NPSVM, to deal with large-scale data based on an efficient solver—dual coordinate descent (DCD) method. Both theoretical analysis and experiments indicate that our method is not only suitable for large scale problems, but also performs as good as TWSVMs and SVMs. [Copyright &y& Elsevier]

Abstract: In this paper, facial age estimation is discussed in a novel viewpoint – how to jointly exploit the supervised training data and human annotations to improve the age estimation precision. This is motivated by the lacking of data problem in age estimation and the current web booming. To do so, fuzzy age label is firstly defined, and it is then merged into the Support Vector Regression (SVR) framework together with the traditional data labels. The new learning problem is finally formulated into a similar dual form with the standard SVR, which can be easily solved using existing solvers. In experiments, we have compared with the state of the art regression based methods, and the results are very competitive. [Copyright &y& Elsevier]

Highlights: [•] This paper proposes a Column Generation Newton (CGN) algorithm for finding solution of the kernel 1-norm SVM. [•] CGN is combining the Column Generation and the Newton Linear Programming SVM method. [•] CGN is fast when solving the kernel 1-norm SVM. [ABSTRACT FROM AUTHOR]