Search

The kernel method is a tool that converts data to a kernel space where operation can be performed. When converted to a high-dimensional feature space by using kernel functions, the data samples are more likely to be linearly separable. Traditional machine learning methods can be extended to the kernel space, such as the radial basis function (RBF) network. As a kernel-based method, support vector machine (SVM) is one of the most popular nonparametric classification methods, and is optimal in terms of computational learning theory. Based on statistical learning theory and the maximum margin principle, SVM attempts to determine an optimal hyperplane by addressing a quadratic programming (QP) problem. Using Vapnik–Chervonenkis dimension theory, SVM maximizes generalization performance by finding the widest classification margin within the feature space. In this paper, kernel machines and SVMs are systematically introduced. We first describe how to turn classical methods into kernel machines, and then give a literature review of existing kernel machines. We then introduce the SVM model, its principles, and various SVM training methods for classification, clustering, and regression. Related topics, including optimizing model architecture, are also discussed. We conclude by outlining future directions for kernel machines and SVMs. This article functions both as a state-of-the-art survey and a tutorial. [ABSTRACT FROM AUTHOR]

In the world of big data, extracting meaningful insights from large and continually growing distributed datasets is a major challenge. Classical clustering algorithms are effective at identifying clusters with convex structures. However, they fall short in identifying arbitrary-shaped clusters (more irregular and complex patterns), which are often encountered in real-world applications. The process of identifying non-convex cluster representations from very large and growing datasets is a challenge. It is further compounded by the distributed nature of the data, necessitating complex computations across multiple devices. Support Vector Clustering (SVC) is a much-celebrated algorithm capable of finding arbitrarily shaped clusters. However, the major limitation of this algorithm is that it will not scale to large volumes of data as the time and space complexity is high. The second limitation of the SVC algorithm is the requirement for large computation time in finding cluster structures. The adoption of a coreset based methodology is required for finding the true representation of the underlying large datasets. The implementation of hierarchical clustering on these distributed coresets, unlocks the potential to uncover a structured hierarchy of abstractions across the disseminated data. Moreover, a distance-based clustering approach guarantees the identification of clusters with diverse and arbitrary shapes, providing a robust framework for detecting complex structures. This research utilizes the Core Vector Machine (CVM) approach using an approximate Minimum Enclosing Ball (MEB) algorithm to efficiently address the complexities inherent in traditional SVC. Additionally, an enhanced medoid algorithm is employed for cluster head identification across the data sources. Hierarchical clustering is performed in the Reproducing Kernel Hilbert Space (RKHS) using cosine similarity distance matrices. This is used to identify compact non-convex clusters within distributed datasets. Performance assessment involves benchmarking our approach against state-of-the-art improved SVC algorithms using large datasets. The outcomes validate the superior performance of our approach compared to existing methods. [ABSTRACT FROM AUTHOR]

This paper addresses the liquefied natural gas (LNG) sales planning problem over a pipeline network with a focus on uncertain demands. Generically, the total profit is maximized by seeking optimal transportation and inventory decisions, and robust optimization (RO) has been a viable decision-making strategy to this end, which is however known to suffer from over-conservatism. To circumvent this, a new goal-oriented data-driven RO approach is proposed. First, we adopt data-driven polytopic uncertainty sets based on kernel learning, which yields a compact high-density region from data and assures tractability of RO problems. Based on this, a new goal-oriented RO formulation is put forward to satisfy to the greatest extent the target profit while tolerating slight constraint violations. In contrast to traditional min–max RO scheme, the proposed scheme not only ensures a flexible trade-off but also yields parameters with clear interpretation. The resulting optimization problem turns out to be equivalent to a mixed-integer linear program that can be effectively handled using off-the-shelf solvers. We illustrate the merit of the proposed method in satisfying a prescribed goal with optimized robustness by means of a case study.

In order to solve the problems of missing traffic flow data caused by various reasons such as weather effect, detector faults and artificial error etc., this paper proposed a method based on the fuzzy rough set theory to impute missing traffic flow data. We combined the support vector clustering and fuzzy rough set to classify traffic flow data, and then combined the fuzzy neural network and genetic algorithm to impute missing data. The method optimized the support vector clustering parameters, cluster size and weighting factor, and estimated the missing values. The results of the study showed that the proposed novel hybrid method produced sufficient and reasonable data imputation performance results. Compared with the results of fuzzy neural network and other estimation models, the data imputation effect of this model was better than other comparison models. [ABSTRACT FROM AUTHOR]

• Efficient deterministic continuous-time MILP model for scheduling the transportation of oil derivatives via pipeline. • Smaller-sized model and faster computational time compared to previous models. • Scheduling under uncertainty using data-driven robust optimization. • Application of support vector clustering in forming the uncertainty set. • Easy and meaningful trade-off between robustness and optimality. Frequently, parameters in optimization models are subject to a high level of uncertainty coming from several sources and, as such, assuming them to be deterministic can lead to solutions that are infeasible in practice. Robust optimization is a computationally efficient approach that generates solutions that are feasible for realizations of uncertain parameters near the nominal value. This paper develops a data-driven robust optimization approach for the scheduling of a straight pipeline connecting a single refinery with multiple distribution centers, considering uncertainty in the injection rate. For that, we apply support vector clustering to learn an uncertainty set for the robust version of the deterministic model. We compare the performance of our proposed robust model against one utilizing a standard robust optimization approach and conclude that data-driven robust solutions are less conservative. [ABSTRACT FROM AUTHOR]

Rough-Fuzzy Support Vector Clustering (RFSVC) is a novel soft computing derivative of the classical Support Vector Clustering (SVC) algorithm used successfully in many real-world applications. The strengths of RFSVC are its ability to handle arbitrary cluster shapes, identify the number of clusters, and effectively detect outliers by using the membership degrees. However, its current version uses only the closest support vector of each cluster to calculate outliers’ membership degrees, neglecting important information that remaining support vectors can contribute. We present a novel approach based on the ordered weighted average (OWA) operator that aggregates information from all cluster representatives when computing final membership degrees and, at the same time, allows a better interpretation of the cluster structures found. Particularly, we propose the OWA using weights computed by the linguistic and exponential quantifiers. The computational experiments show that our approach obtains comparable results with the current version of RFSVC. However, the former weights all clusters’ support vectors in the computation of membership degrees while maintaining their interpretability level for detecting outliers. [ABSTRACT FROM AUTHOR]

The result of radar signal sorting directly affects the performance of electronic reconnaissance equipment. Sorting method based on intra-pulse features has become a research focus in recent years. However, as the number of extracted features increases, the dimension of the feature vector becomes higher and higher. And too many dimensional feature vectors would make the complexity of the sorting algorithm increase geometrically. In this way, feature selection becomes more and more necessary. Combining the latest research on fuzzy rough sets, this paper proposes two feature selection methods, namely two-steps attribute reduction based on fuzzy dependency (TARFD) algorithm and fuzzy rough artificial bee colony (FRABC) algorithm. The TARFD method uses the candidate attribute set as starting point, according to the definition of the redundant attribute set. Then the less important attributes are successively eliminated. The FRABC method starts from the dependence degree of fuzzy rough set, and constructs a fitness function that reflects the importance of the attribute subset and the reduction rate. Based on this function, the artificial bee colony algorithm is used to reduce the attributes of the dataset. Using the TARFD and FRABC algorithms, the extracted feature sets, including entropy feature set, Zernike moment feature set, pseudo Zernike feature set, gray level co-occurrence matrix (GLCM) feature set, and Hu-invariant moment feature set are processed, then an optimal feature subset was obtained and a sorting test was performed. The results show the effectiveness of the extracted intra-pulse features and the efficiency of the feature selection algorithm.

Identification of coherent generators (CGs) is necessary for the area-based monitoring and protection system of a wide area power system. Synchrophasor has enabled smarter monitoring and control measures to be devised; hence, measurement-based methodologies can be implemented in online applications to identify the CGs. This paper presents a new framework for coherency identification that is based on the dynamic coupling of generators. A distance matrix that contains the dissimilarity indices between any pair of generators is constructed from the pairwise dynamic coupling of generators after the post-disturbance data are obtained by phasor measurement units (PMUs). The dataset is embedded in Euclidean space to produce a new dataset with a metric distance between the points, and then the support vector clustering (SVC) technique is applied to the embedded dataset to identify the final clusters of generators. Unlike other clustering methods that need a priori knowledge about the number of clusters or the parameters of clustering, this information is set in an automatic search procedure that results in the optimal number of clusters. The algorithm is verified by time-domain simulations of defined scenarios in 39 bus and 118 bus test systems. Finally, the clustering result of 39 bus systems is validated by cluster validity measures, and a comparative study investigates the efficacy of the proposed algorithm to cluster the generators with an optimal number of clusters and also its computational efficiency compared with other clustering methods.

Due to the remarkable capability in handling arbitrary cluster shapes, support vector clustering (SVC) benefits data analysis in terms of data description. However, large-scale data such as network traffic frequently makes it suffer from highly intensive pricey computation and storage for solving the dual problem and storing the kernel matrix, respectively. Fortunately, support vectors which describe the clusters, in a sense, are expected in the boundaries. To tackle this issue, we propose an efficient training SVC with appropriate boundary information (ETSVC), which features excellent flexibility and scalability. In ETSVC, we first give a shrinkable boundary selection (SBS) method which collects appropriate boundaries while reducing redundancy and noise. Based on the boundary information, a redefined dual problem is then designed without scarifying the principle of SVC. Finally, we design a reformative solver (RSolver) to reformulate the training phase, which estimates the support vector function by solving the dual problem. Since only a subset of boundaries is employed for model training, theoretical analysis suggests that ETSVC reaches efficiency improvement and consumes much less memory if sacrificing efficiency to reduce storage consumption. Towards grouping P2P flows and large-scale intrusion traffic, as well as other non-traffic data, experimental results confirm that ETSVC could be applied to resources constrained platform while achieving comparable accuracies with the state-of-the-art methods.

Copyright of Mugla Journal of Science & Technology is the property of Mugla Journal of Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A robust multi-product inventory optimization approach is developed with an uncertainty set constructed from the available data using support vector clustering (SVC). The multi-product inventory problem is subject to demand uncertainties in a newsvendor setting with the historical demand data as the only available information. By using SVC, the uncertainty set to which the uncertain demands belong is constructed with a certain confidence in a data-driven approach. The associated robust counterpart model is then developed using the absolute robustness criterion. Through mathematical deduction, the proposed counterpart model is transformed into a tractable linear programming model which can be solved efficiently. The transformed and the original models are proved to be mathematically equivalent. Numerical studies are conducted to illustrate the effectiveness and practicality of the proposed SVC-based data-driven robust optimization approach for dealing with demand uncertainties. The results show that the robust optimization approach under the proposed SVC-based uncertainty set outperforms those under the traditional, i.e., the box and the ellipsoid, uncertainty sets. These results provide evidences that the proposed data-driven robust optimization approach can better hedge against demand uncertainties in multi-product inventory problems. [ABSTRACT FROM AUTHOR]

• A forecasting method for the arrival times of electric vehicles is proposed. • An optimal parameter modification model is established based on the feedback. • Historical data is used to obtain the optimization range of optimal parameter values. • The optimization range is dynamically adjusted considering the robustness. Affected by weather conditions, traffic conditions, and driver behavior, the arrival characteristics of electric vehicles (EVs) vary significantly from day to day. This study proposes a feedback-driven real-time forecasting approach that combines historical data to improve the forecasting accuracy of arrival times of EVs. For model-based forecasting methods that sample from probability density functions (PDFs), the related parameter values are dynamically optimized. Compared with sampling from PDFs with empirical parameter values, the dynamic optimal parameter values can track the characteristics of EV arrivals by fully using the continuously updated EV feedback. Considering robustness, a historical data-based support vector clustering technology is utilized to obtain the optimization range of optimal parameter values. As a key of this study, the conservativeness of the optimization range is dynamically adjusted with the periodic updates of EV feedback. The experimental results indicate that, by making full utilization of EV feedback, the proposed method can effectively reduce the forecasting errors of EV arrival times. [ABSTRACT FROM AUTHOR]

Support vector clustering (SVC) is a boundary-based algorithm, which has several advantages over other clustering methods, including identifying clusters of arbitrary shapes and numbers. Leveraged by the high generalization ability of the large margin distribution machine (LDM) and the optimal margin distribution clustering (ODMC), we propose a new clustering method: minimum distribution for support vector clustering (MDSVC), for improving the robustness of boundary point recognition, which characterizes the optimal hypersphere by the first-order and second-order statistics and tries to minimize the mean and variance simultaneously. In addition, we further prove, theoretically, that our algorithm can obtain better generalization performance. Some instructive insights for adjusting the number of support vector points are gained. For the optimization problem of MDSVC, we propose a double coordinate descent algorithm for small and medium samples. The experimental results on both artificial and real datasets indicate that our MDSVC has a significant improvement in generalization performance compared to SVC.

The precipitation of LED phosphor glue is not only related to the physical properties of phosphor powder and silica gel, but also influenced by the uncertainties in the production process. In this paper, support vector clustering (SVC) is combined with T-S neuro-fuzzy network to build the neuro-fuzzy network prediction model of phosphor powder precipitation. The structure identification of the predictive model and the neuro-fuzzy network parameter learning algorithm are derived. Finally, the flow chart of the modeling of predictive model is given. The test results show that the training time of the new TSFNN proposed in this paper is 56% less than the standard TSFNN model and the average error of the new TSFNN is 33.33% less than the standard one. LED phosphor powder mixing system test shows that the new TSFNN model control system effectively enhances the LED light color consistency comparing with the traditional method. [ABSTRACT FROM AUTHOR]

Community structure, one of the most attention attracting properties in complex net-works, has been a cornerstone in advances of various scientific branches. A number of tools have been involved in recent studies concentrating on the community detection algorithms. In this paper, we propose a support vector clustering method based on a proximity graph, owing to which the introduced algorithm surpasses the traditional support vector approach both in accuracy and complexity. Results of extensive experiments undertaken on computer generated networks and real world data sets illustrate competent performances in comparison with the other counterparts. [ABSTRACT FROM AUTHOR]

The portfolio optimization (PO) model with semi-mean absolute deviation (SMAD) risk measure has been commonly applied to construct optimal portfolios due to the ease of solving the corresponding linear programming model. We propose a robust PO model with SMAD that considers the uncertainty associated with asset expected returns. This uncertainty is dealt by adopting a data-driven approach that captures the uncertain asset returns in a convex uncertainty set through support vector clustering. The proposed model involves solving a quadratic programming problem to identify support vectors and a robust linear PO model. The ability of the proposed technique to control the conservatism and the computational ease associated with it makes it a practical approach to yield robust optimal portfolios. The effectiveness of the model is demonstrated by constructing optimal portfolios with the constituents of four global market indices, namely Dow Jones Industrial Average (USA), DAX 30 (Germany), Nifty 50 (India), and EURO STOXX 50 (Europe). The out-of-sample statistics generated from the robust portfolios are compared with the optimal portfolios obtained from its nominal counterpart, naive 1 / n strategy, and other robust technique available in the literature. We find that the proposed model consistently performs well in most data sets over several performance measures like average returns, risk measured by standard deviation, value at risk, conditional value at risk and various reward-risk ratios. Comparative analysis of these models in different market phases of EURO STOXX 50 demonstrates the effectiveness of the developed robust technique, especially during the bearish phase. [ABSTRACT FROM AUTHOR]

Active frequency response (AFR) control is needed in current power systems. To solve the over-frequency problems of generators connected to non-disturbed buses during the AFR control period, the generators should be clustered into coherent groups. Thus, the control efficiency can be improved on the premise of ensuring control accuracy. Since the influencing factors (such as the model parameters, operation modes, and disturbance locations, etc.) of power system operation can be comprehensively reflected by the generator frequency, which is easily collected and calculated, the generator frequency can be used as the coherency identification input. In this paper, we propose a coherency identification method of AFR control based on support vector clustering for a bulk power system. By mapping data samples from the initial space to the high-dimensional feature space, the radius of the minimal enclosing sphere that can envelop all the data samples is obtained. Moreover, the coherency identification of generators is determined for AFR control according to the evaluating method of AFR clustering control effects and the evaluating index of cluster compactness and separation. The simulation results for the modified New England IEEE 10-generator 39-bus system and Henan power grid show that the proposed method is feasible and effective.

We propose piecewise linear kernel-based support vector clustering (SVC) as a new approach tailored to data-driven robust optimization. By solving a quadratic program, the distributional geometry of massive uncertain data can be effectively captured as a compact convex uncertainty set, which considerably reduces conservatism of robust optimization problems. The induced robust counterpart problem retains the same type as the deterministic problem, which provides significant computational benefits. In addition, by exploiting statistical properties of SVC, the fraction of data coverage of the data-driven uncertainty set can be easily selected by adjusting only one parameter, which furnishes an interpretable and pragmatic way to control conservatism and exclude outliers. Numerical studies and an industrial application of process network planning demonstrate that, the proposed data-driven approach can effectively utilize useful information with massive data, and better hedge against uncertainties and yield less conservative solutions. [ABSTRACT FROM AUTHOR]

In spite of with great advantage of discovering arbitrary shapes of clusters, support vector clustering (SVC) is frustrated by large-scale data, especially on resource limited platform. It is due to pricey storage and computation consumptions from solving dual problem and labeling clusters upon the pre-computed kernel matrix and sampling point pairs, respectively. Towards on it, we first present a dual coordinate descent method to reformulate the solver that leads to a flexible training phase carried out on any runtime platform with/without sufficient memory. Then, a novel labeling phase who does connectivity analysis between two nearest neighboring decomposed convex hulls referring to clusters is proposed, in which a new designed strategy namely sample once connected checking first tries to reduces the scope of sampling analysis. By integrating them together, a faster and reformulated SVC (FRSVC) is created with less consumption achieved according to comparative analysis of time and space complexities. Furthermore, experimental results confirm a significant improvement on flexibility of selective efficiency without losing accuracy, with which a balance can be easily reached on the basis of resources a platform equipped. [ABSTRACT FROM AUTHOR]

Support-based clustering has recently absorbed plenty of attention because of its applications in solving the difficult and diverse clustering or outlier detection problem. Support-based clustering method perambulates two phases: finding the domain of novelty and performing the clustering assignment. To find the domain of novelty, the training time given by the current solvers is typically over-quadratic in the training size. This fact impedes the application of support-based clustering method to the large-scale datasets. In this paper, we propose applying stochastic gradient descent framework to the first phase of support-based clustering for finding the domain of novelty in the form of a half-space and a new strategy to perform the clustering assignment. We validate our proposed method on several well-known datasets for clustering task to show that the proposed method renders a comparable clustering quality to the baselines while being faster than them. [ABSTRACT FROM AUTHOR]

A two-phase online anomaly detection method based on support vector clustering (SVC) in the presence of non-stationary data is developed in this paper which permits arbitrary-shaped data clusters to be precisely treated. In the first step, offline learning is performed to achieve an appropriate detection model. Then the current model dynamically evolves to match the rapidly changing real-world data. To reduce the dimension of the quadratic programming (QP) problem emerging in the SVC, self-organizing map (SOM) and a replacement mechanism are used to summarize the incoming data. Thus, the proposed method can be efficiently and effectively useable in real time applications. The performance of the proposed method is evaluated by a simulated dataset, three subsets extracted from the KDD Cup 99 dataset, and the keystroke dynamics dataset. Results illustrate capabilities of the proposed method in detection of new attacks as well as normal pattern changes over the time.

This study presents an approach for developing the dynamic equivalent model of an active distribution network (ADN), consisting of several micro‐grids, for frequency stability studies. The proposed grey‐box equivalent model relies on Prony analysis to establish stop time and load damping as the required modelling parameters. Support vector clustering (SVC) and grouping procedure are employed for aggregation and order‐reduction of ADN. This significantly decreases the sensitivity of the estimated parameters to operating point changes which, in turn, guarantees the model robustness. This is done through representing the SVC output, that is, clusters, by cluster substitutes. The final ADN dynamic equivalent model is represented by several groups, in which their mutual interactions are taken into account by a new developed mathematical‐based criterion. Simulation results reveal that the proposed model is robust which could successfully take into account the continuous and discontinuous uncertainties. [ABSTRACT FROM AUTHOR]

Various data driven soft sensor models have been established for online prediction of the silicon content in blast furnace ironmaking processes. However, two main disadvantages still remain in these empirical models. First, most of traditional outlier detection methods for preprocessing the data samples assume that they (approximately) follow a Gaussian distribution and thus may be invalid for some situations. To address this problem, a support vector clustering (SVC) based efficient outlier detection method is proposed whereby the process nonlinearity and non-Gaussianity can be better handled. Second, only using a single global model is insufficient to capture all the process characteristics, especially for those complicated regions. In this paper, a reliable just-in-time modelling method is proposed. The SVC outlier detection is integrated into the just-in-time-based local modelling method to enhance the reliability of quality prediction. A healthier relevant data set is constructed to build a more reliable local prediction model. Moreover, the historical data set is updated repetitively in a reasonable way. The superiority of the proposed method is demonstrated and compared with other soft sensors in terms of online prediction of the silicon content in an industrial blast furnace in China. [ABSTRACT FROM AUTHOR]

As an important boundary-based clustering algorithm, support vector clustering (SVC) benefits multiple applications for its capability of handling arbitrary cluster shapes. However, its popularity is degraded by both its highly intensive pricey computation and poor label performance which are due to redundant kernel function matrix required by estimating a support function and ineffectively checking segmers between all pairs of data points, respectively. To address these two problems, a fast and scalable SVC (FSSVC) method is proposed in this paper to achieve significant improvement on efficiency while guarantees a comparable accuracy with the state-of-the-art methods. The heart of our approach includes (1) constructing the hypersphere and support function by cluster boundaries which prunes unnecessary computation and storage of kernel functions and (2) presenting an adaptive labeling strategy which decomposes clusters into convex hulls and then employs a convex-decomposition-based cluster labeling algorithm or cone cluster labeling algorithm on the basis of whether the radius of the hypersphere is greater than 1. Both theoretical analysis and experimental results (e.g., the first rank of a nonparametric statistical test) show the superiority of our method over the others, especially for large-scale data analysis under limited memory requirements. [ABSTRACT FROM AUTHOR]

Clustering algorithms have evolved to handle more and more complex structures. However, the measures that allow to qualify the quality of such clustering partitions are rare and have been developed only for specific algorithms. In this work, we propose a new cluster validity measure (CVM) to quantify the clustering performance of hierarchical algorithms that handle overlapping clusters of any shape and in the presence of outliers. This work also introduces a cluster merging system (CMS) to group clusters that share outliers. When located in regions of cluster overlap, these outliers may be issued by a mixture of nearby cores. The proposed CVM and CMS are applied to hierarchical extensions of the Support Vector and Gaussian Process Clustering algorithms both in synthetic and real experiments. These results show that the proposed metrics help to select the appropriate level of hierarchy and the appropriate hyperparameters. [ABSTRACT FROM AUTHOR]