Search

In this article, a novel ensemble model, called Multiple Kernel Ensemble Learning (MKEL), is developed by introducing a unified ensemble loss. Different from the previous multiple kernel learning (MKL) methods, which attempt to seek a linear combination of basis kernels as a unified kernel, our MKEL model aims to find multiple solutions in corresponding Reproducing Kernel Hilbert Spaces (RKHSs) simultaneously. To achieve this goal, multiple individual kernel losses are integrated into a unified ensemble loss. Therefore, each model can co-optimize to learn its optimal parameters by minimizing a unified ensemble loss in multiple RKHSs. Furthermore, we apply our proposed ensemble loss into the deep network paradigm and take the sub-network as a kernel mapping from the original input space into a feature space, named Deep-MKEL (D-MKEL). Our D-MKEL model can utilize the diversified deep individual sub-networks into a whole unified network to improve the classification performance. With this unified loss design, our D-MKEL model can make our network much wider than other traditional deep kernel networks and more parameters are learned and optimized. Experimental results on several mediate UCI classification and computer vision datasets demonstrate that our MKEL model can achieve the best classification performance among comparative MKL methods, such as Simple MKL, GMKL, Spicy MKL, and Matrix-Regularized MKL. On the contrary, experimental results on large-scale CIFAR-10 and SVHN datasets concretely show the advantages and potentialities of the proposed D-MKEL approach compared to state-of-the-art deep kernel methods. [ABSTRACT FROM AUTHOR]

Support Vector Machine (SVM) is widely applied in classification and regression tasks where support vectors are pursued through convex quadratic programming technique due to its effectiveness and efficiency. However, existing studies ignore the importance of training samples when they are fed into the model. In this paper, we propose a novel Iterative Factoring Support Vector Machine (If-SVM) method. Sample factoring is introduced in our proposed model to measure the significance of each data point, where it can effectively reduce the negative impact of trivial or noisy data points. In this way, our proposed model is concentrates on the critical data points falling around the hyperplane. By introducing this iterative factoring of data points into SVM, the classification accuracy of our proposed method is above that of 1.45% than other comparative methods in image recognition datasets. Experimental results on a variety of UCI demonstrate that, our proposed method has superior performances in decreasing the total number of support vectors than the other state-of-the-art SVM methods. More importantly, our further experiments also illustrate that, the classification performance of the state-of-the-art SVM methods can be improved 1.29% by incorporating our sample factoring idea into their models, which demonstrate our idea is a useful tool to improve the state-of-art SVM models. [ABSTRACT FROM AUTHOR]