Search

The recently proposed two-dimensional locality preserving projection (2DLPP) is an excellent matrix-based dimensionality reduction method. However, the formulation of 2DLPP may encounter several drawbacks in real-world applications, such as sensitiveness to outliers, non-orthogonal projections, and singularity problem. To alleviate these issues, in this paper, we propose a novel robust two-dimensional locality preserving projection (2DRLPP) for noisy image recognition. The proposed 2DRLPP preserves the local manifold structure of two-dimensional image space under the robust L 1 -norm criterion. Different from the existing 2DLPP, our 2DRLPP enjoys several incomparable advantages: (i) 2DRLPP extracts projections from matrix-based image space via the L 1 -norm locality preserving criterion rather than the L 2 -norm one, and hence it is more robust to outliers. (ii) The orthogonality constraint is further imposed on 2DRLPP to ensure its orthogonal projections. (iii) The introduced regularization term can not only control the model complexity but also guarantee the stability of solution. (iv) A simple but efficient iterative algorithm is presented to solve the corresponding L 1 -norm minimization problem, whose convergence is guaranteed theoretically. Extensive experimental results on three noisy face image datasets confirm the feasibility and robustness of the proposed approach.

In this paper, by introducing the statistics of training data into support vector regression (SVR), we propose a minimum deviation distribution regression (MDR). Rather than just minimizing the structural risk, MDR also minimizes both the regression deviation mean and the regression deviation variance, which is able to deal with the different distribution of boundary data and noises. The formulation of minimizing the first and second order statistics in MDR leads to a strongly convex quadratic programming problem (QPP). An efficient dual coordinate descend algorithm is adopted for small sample problem, and an average stochastic gradient algorithm for large scale one. Both theoretical analysis and experimental results illustrate the efficiency and effectiveness of the proposed method.

In this paper, we formulate a twin-type support vector machine for large-scale classification problems, called weighted linear loss twin support vector machine (WLTSVM). By introducing the weighted linear loss, our WLTSVM only needs to solve simple linear equations with lower computational cost, and meanwhile, maintains the generalization ability. So, it is able to deal with large-scale problems efficiently without any extra external optimizers. The experimental results on several benchmark datasets indicate that, comparing to TWSVM, our WLTSVM has comparable classification accuracy but with less computational time.

Note that for GEPSVM proposed in [1], the predicted class of a testing point is determined by comparing two distances between the testing point and two hyperplanes, while the optimization problems are based on comparing two distances between a hyperplane and two kinds of the training points. So there exists some inconformity between the decision process and the training process. In this paper, we propose a new proximal classifier, called PCC for short, with consistency, which is always based on comparing two distances between a point (the testing point in the decision process and the training point in the training process) and two hyperplanes. This consistency not only makes our PCC to be more reasonable logically, but also naturally leads to a simpler decision function with less computation cost. Furthermore, in our PCC two general eigenvalue problems in GEPSVM are replaced by two simple eigenvalues problems with a parameter @d. In addition, different regularization terms are introduced in the formulation of our PCC, avoiding the singular problems possibly appeared in GEPSVM. Experimental results on several benchmark data sets show that our PCC is not only faster, but also has better generalization.

For the recently proposed projection twin support vector machine (PTSVM) [1], we propose a simpler and reasonable variant from theoretical point of view, called projection twin support vector machine with regularization term, RPTSVM for short. Note that only the empirical risk minimization is introduced in primal problems of PTSVM, incurring the possible singularity problems. Our RPTSVM reformulates the primal problems by adding a maximum margin regularization term, and, therefore, the singularity is avoided and the regularized risk principle is implemented. In addition, the nonlinear classification ignored in PTSVM is also considered in our RPTSVM. Further, a successive overrelaxation technique and a genetic algorithm are introduced to solve our optimization problems and to do the parameter selection, respectively. Computational comparisons of our RPTSVM against original PTSVM, TWSVM and MVSVM indicate that our RPTSVM obtains better generalization than others.

In many cases, the output of a classifier should be a calibrated posterior probability to enable post-processing. However, twin support vector machines (TWSVM) do not provide such probabilities. In this paper, we propose a TWSVM probability model, called PTWSVM, to estimate the posterior probability. Note that our model is quite different from the SVM probability model because of the different mechanism of TWSVM and SVM. In fact, for TWSVM, we first define a new ranking continues output by comparing the distances between the sample and the two non-parallel hyperplanes, and then map this ranking continues output into probability by training the parameters of an additional sigmoid function. Our PTWSVM has been tested on both artificial datasets and several data-mining-style datasets, and the numerical experiments indicate that PTWSVM yields nice results.