Search

Information system is a database that shows relationships between objects and attributes. A real-valued information system is an information system whose information values are real numbers. A real-valued information system with decision attributes is referred to as a real-valued decision information system. If objects, conditional attributes and information values in a real-valued decision information system are cells, genes and gene expression values, respectively, then this information system is said to be a gene decision space. In a gene decision space, people are faced with gene expression data. If gene expression data in a gene decision space changes to single cell RNA-seq data, then this space is referred to as a single cell gene decision space. This paper explores gene selection in a single cell gene decision space in terms of Gaussian kernel. In the first place, the distance between two cells in each subspace of a single cell gene decision space is constructed. Next, the fuzzy T cos -equivalence relation on the cell set is obtained in terms of Gaussian kernel. After that, measures of uncertainty for a single cell gene decision space are investigated. Lastly, gene selection algorithms in a single cell gene decision space are presented in terms of the proposed information entropy and information granularity. The presented algorithms are testified in several publicly open single cell RNA-seq data sets. Experimental results reveal that the presented algorithms can select appropriate genes related to classification, and significantly improve classification performances. [ABSTRACT FROM AUTHOR]

In this paper, a new method is proposed for estimating fuzzy regression models based on a novel robust support vector machines with exact predictors and fuzzy responses. For this purpose, a three-stage support vector machine algorithm was introduced based on a modified robust loss function. Some common goodness-of-fit criteria and a popular kernel were also employed to examine the performance of the proposed method in cases where the outliers occur in the data set. The effectiveness of the proposed method was illustrated through three numerical cases including a simulation study and two applied examples. The proposed method was also compared with several common fuzzy linear/nonlinear/nonparametric regression models. The numerical results clearly indicated that the proposed model is capable of providing accurate results in the cases involving data sets with or without outliers. Thus, the proposed fuzzy regression model can be successfully applied to improve the prediction accuracy and interpretability of the fuzzy regression models for real-life applications in the intelligence systems. [ABSTRACT FROM AUTHOR]

In this paper, a novel non-parametric Bayesian compressive sensing algorithm is proposed to enhance reconstruction performance of sparse entries with a continuous structure by exploiting the location dependence of entries. An approach is proposed which involves the logistic model and location-dependent Gaussian kernel. The variational Bayesian inference scheme is used to perform the posterior distributions and acquire an approximately analytical solution. Compared to the conventional clustered based methods, which only exploit the information of neighboring pixels, the proposed approach takes the relationship between the pixels of the entire image into account to enable the utilization of the underlying sparse signal structure. It significantly reduces the required number of observations for sparse reconstruction. Both real-valued signal applications, including one-dimension signal and two-dimension image, and complex-valued signal applications, including single-snapshot direction-of-arrival (DOA) estimation of distributed sources and inverse synthetic aperture radar (ISAR) imaging with a limited number of pluses, demonstrate the superiority of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Gaussian kernel Support Vector Machines (SVMs) deliver state-of-the-art generalization performance for non-linear classification, but the time complexity of their training process is at least quadratic w.r.t. the size of training set, preventing them from scaling up on large datasets. To address this issue, we propose a novel approach to large-scale kernel SVMs in sublinear time w.r.t. the size of training set, which combines three well-known and efficient techniques with theoretical guarantees. First, we subsample massive samples to reduce the sample size. Then, we use the random Fourier feature mapping on the subsamples to construct explicit random feature space where we can train a linear SVM to approximate the corresponding Gaussian kernel SVM. Finally, we use parallel algorithms to make our approach more scalable. Deriving the upper bounds of kernel matrix approximation error, hypothesis error and excess risk w.r.t. the size of training set and the dimension of random feature space, we establish the theoretical foundation of our proposed approach. In this way, we can reduce the time complexity of training kernel SVMs without sacrificing much accuracy. Our proposed approach achieves high accuracy and has sublinear training time complexity, which exhibits good scalability theoretically and empirically. [ABSTRACT FROM AUTHOR]

We investigate the connection between the asymptotic bias of the well-known Nadaraya–Watson kernel regression and the mean shift (MS) vector with the Gaussian kernel. We first show that the asymptotic bias for the univariate Nadaraya–Watson kernel regression can be estimated using the MS scalar. Then, we investigate the general D -dimensional case ( D > 1) and derive a formula for the asymptotic bias as a function of the MS vector. We show that when the regression function is a linear function of its entries, then the asymptotic bias can be represented as a linear function of the MS vector. The MS algorithm for the univariate and the linear cases can be used to find points where the Nadaraya–Watson kernel regression gives an unbiased estimate. Furthermore, this connection suggests that exploiting the theoretical properties of the asymptotic bias of the Nadaraya–Watson kernel regression may be helpful to show the convergence of the MS algorithm. Through the simulations, we show that how the given theoretical results can be used to estimate the bias of the estimated clean signal by just observing the noisy signal. Having access to the bias of the estimator, enables us to evaluate the accuracy of the estimated clean data, which later will be used to design a quantizer. [ABSTRACT FROM AUTHOR]

The success of kernel-based learning methods is heavily dependent on the choice of a kernel function and proper setting of its parameters. In this paper, we optimize the Gaussian kernel for binary-class problems by using centered kernel polarization criterion. This criterion is an extension of kernel polarization and a simplified style of centered kernel alignment. Compared with formulated kernel polarization criterion, the proposed criterion has a defined geometrical significance, and it can locate the global optimal point with less influence of threshold selection. Furthermore, the approximate criterion function can be proved to have a determined global minimum point by adopting the Euler–Maclaurin formula under weaker conditions. In addition, taking the preservation of within-class local structure into account, we present an evaluation criterion named local multiclass centered kernel polarization in multiclass classification scenario. Comparative experiments are conducted on some benchmark examples with three Gaussian kernel based learning methods and the results well demonstrate the effectiveness and efficiency of the proposed quality measures. [ABSTRACT FROM AUTHOR]

The use of quaternion data can provide benefit in applications like robotics and image recognition, and particularly for performing transforms in 3-D space. Here, we describe a kernel adaptive algorithm for quaternions. A least mean square (LMS)-based method was used, resulting in the derivation of the quaternion kernel LMS (Quat-KLMS) algorithm. Deriving this algorithm required describing the idea of a quaternion reproducing kernel Hilbert space (RKHS), as well as kernel functions suitable with quaternions. A modified HR calculus for Hilbert spaces was used to find the gradient of cost functions defined on a quaternion RKHS. In addition, the use of widely linear (or augmented) filtering is proposed to improve performance. The benefit of the Quat-KLMS and widely linear forms in learning nonlinear transformations of quaternion data are illustrated with simulations. [ABSTRACT FROM PUBLISHER]

An extreme learning machine (ELM) can be regarded as a two-stage feed-forward neural network (FNN) learning system that randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of this paper is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part of this topic, we pointed out that via appropriately selected activation functions, ELM does not degrade the generalization capability in the sense of expectation. In this paper, we launch the study in a different direction and show that the randomness of ELM also leads to certain negative consequences. On one hand, we find that the randomness causes an additional uncertainty problem of ELM, both in approximation and learning. On the other hand, we theoretically justify that there also exist activation functions such that the corresponding ELM degrades the generalization capability. In particular, we prove that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN with Gaussian kernel. To facilitate the use of ELM, we also provide a remedy to such a degradation. We find that the well-developed coefficient regularization technique can essentially improve the generalization capability. The obtained results reveal the essential characteristic of ELM in a certain sense and give theoretical guidance concerning how to use ELM. [ABSTRACT FROM AUTHOR]

The application of fuzzy clustering in image segmentation is a research hotspot nowadays. Existing robust fuzzy clustering have some problems such as the inability to adaptively select spatial constraint parameters, the inability to accurately segment images corrupted by high noise, and the inability to achieve a balance between noise suppression and detail preservation. In the fuzzy clustering based on objective function optimization, the choice of distance measure is very important. Gaussian kernel function is defined by Euclidean distance and has been widely used in many fields such as pattern recognition, machine learning, etc. However, Euclidean distance in fuzzy clustering is very sensitive to outliers or noise, it is difficult to obtain satisfactory results for some special data sets, which will affect the performance of clustering algorithm. In this paper, a non local information self-integration optimization algorithm based on kernel-based fuzzy local information clustering algorithm is proposed. The algorithm uses the self-integration method on the basis of the local information of the image and introduces non-local information at the same time, which solves the common problems of current clustering algorithm. Firstly, the self-integrating method solves the problem of selecting spatial constraint parameters, and the algorithm continues self-learning and iteratively calculates the parameter values; secondly, the distance measure uses Gaussian kernel induced distance to further enhance the robustness against noise and the adaptability of processing data sets; Finally, the local and non-local information are integrated at the same time to achieve a segmentation effect, which can effectively suppress most of the noise and retain the details of original image. Experimental results show that the proposed algorithm is superior to existing state-of-the-art fuzzy clustering-related algorithm in the presence of high noise. [ABSTRACT FROM AUTHOR]

The complex kernel least mean square (CKLMS) algorithm is recently derived and allows for online kernel adaptive learning for complex data. Kernel adaptive methods can be used in finding solutions for neural network and machine learning applications. The derivation of CKLMS involved the development of a modified Wirtinger calculus for Hilbert spaces to obtain the cost function gradient. We analyze the convergence of the CKLMS with different kernel forms for complex data. The expressions obtained enable us to generate theory-predicted mean-square error curves considering the circularity of the complex input signals and their effect on nonlinear learning. Simulations are used for verifying the analysis results. [ABSTRACT FROM AUTHOR]

Abstract: We develop a test for log-concavity of multivariate densities. The method uses kernel density estimation, where the test statistic is the smallest bandwidth for which the estimate is log-concave. We examine the properties of this technique through numerical studies. [ABSTRACT FROM AUTHOR]

Abstract: Kernel methods and rough sets are two general pursuits in the domain of machine learning and intelligent systems. Kernel methods map data into a higher dimensional feature space, where the resulting structure of the classification task is linearly separable; while rough sets granulate the universe with the use of relations and employ the induced knowledge granules to approximate arbitrary concepts existing in the problem at hand. Although it seems there is no connection between these two methodologies, both kernel methods and rough sets explicitly or implicitly dwell on relation matrices to represent the structure of sample information. Based on this observation, we combine these methodologies by incorporating Gaussian kernel with fuzzy rough sets and propose a Gaussian kernel approximation based fuzzy rough set model. Fuzzy T-equivalence relations constitute the fundamentals of most fuzzy rough set models. It is proven that fuzzy relations with Gaussian kernel are reflexive, symmetric and transitive. Gaussian kernels are introduced to acquire fuzzy relations between samples described by fuzzy or numeric attributes in order to carry out fuzzy rough data analysis. Moreover, we discuss information entropy to evaluate the kernel matrix and calculate the uncertainty of the approximation. Several functions are constructed for evaluating the significance of features based on kernel approximation and fuzzy entropy. Algorithms for feature ranking and reduction based on the proposed functions are designed. Results of experimental analysis are included to quantify the effectiveness of the proposed methods. [Copyright &y& Elsevier]

Abstract: Redundancy analysis (RA) is a versatile technique used to predict multivariate criterion variables from multivariate predictor variables. The reduced-rank feature of RA captures redundant information in the criterion variables in a most parsimonious way. A ridge type of regularization was introduced in RA to deal with the multicollinearity problem among the predictor variables. The regularized linear RA was extended to nonlinear RA using a kernel method to enhance the predictability. The usefulness of the proposed procedures was demonstrated by a Monte Carlo study and through the analysis of two real data sets. [Copyright &y& Elsevier]

Abstract: The problem of choosing a good parameter setting for a better generalization performance in a learning task is the so-called model selection. A nested uniform design (UD) methodology is proposed for efficient, robust and automatic model selection for support vector machines (SVMs). The proposed method is applied to select the candidate set of parameter combinations and carry out a k-fold cross-validation to evaluate the generalization performance of each parameter combination. In contrast to conventional exhaustive grid search, this method can be treated as a deterministic analog of random search. It can dramatically cut down the number of parameter trials and also provide the flexibility to adjust the candidate set size under computational time constraint. The key theoretic advantage of the UD model selection over the grid search is that the UD points are “far more uniform”and “far more space filling” than lattice grid points. The better uniformity and space-filling phenomena make the UD selection scheme more efficient by avoiding wasteful function evaluations of close-by patterns. The proposed method is evaluated on different learning tasks, different data sets as well as different SVM algorithms. [Copyright &y& Elsevier]

A jump process approach is proposed for the trend estimation of time series. The proposed jump process estimator can locally minimize two important features of a trend, the smoothness and fidelity, and explicitly balance the fundamental tradeoff between them. A weighted average form of the jump process estimator is derived. The connection of the proposed approach to the Hanning filter, Gaussian kernel regression, the heat equation and the Wiener process is discussed. It is found that the weight function of the jump process approaches the Gaussian kernel, as the smoothing parameter increases. The proposed method is validated through numerical applications to both real data analysis and simulation study, and a comparison with the Henderson filter. [Copyright &y& Elsevier]

We use the ideas of maximum mean discrepancy and ranks of nearest neighbors to propose some tests of independence among multiple random vectors of arbitrary dimensions. Numerical studies demonstrate that proposed tests can outperform the existing tests in various examples. [ABSTRACT FROM AUTHOR]