Search

$k$ k -means method using Lloyd heuristic is a traditional clustering method which has played a key role in multiple downstream tasks of machine learning because of its simplicity. However, Lloyd heuristic always finds a bad local minimum, i.e., the bad local minimum makes objective function value not small enough, which limits the performance of $k$ k -means. In this paper, we use coordinate descent (CD) method to solve the problem. First, we show that the $k$ k -means minimization problem can be reformulated as a trace maximization problem, then a simple and efficient coordinate descent scheme is proposed to solve the maximization problem. Two interesting findings through theory are that Lloyd cannot decrease the objective function value of $k$ k -means produced by our CD further, and our proposed method CD to solve $k$ k -means problem can avoid produce empty clusters. In addition, according to the computational complexity analysis, it is verified CD has the same time complexity with original $k$ k -means method. Extensive experiments including statistical hypothesis testing, on several real-world datasets with varying number of clusters, varying number of samples and varying number of dimensions show that CD performs better compared to Lloyd, i.e., lower objective value, better local minimum and fewer iterations. And CD is more robust to initialization than Lloyd whether the initialization strategy is random or initialization of $k$ k -means++. [ABSTRACT FROM AUTHOR]

Due to the exponential growth of multimedia data, multi-modal hashing as a promising technique to make cross-view retrieval scalable is attracting more and more attention. However, most of the existing multi-modal hashing methods either divide the learning process unnaturally into two separate stages or treat the discrete optimization problem simplistically as a continuous one, which leads to suboptimal results. Recently, a few discrete multi-modal hashing methods that try to address such issues have emerged, but they still ignore several important discrete constraints (such as the balance and decorrelation of hash bits). In this paper, we overcome those limitations by proposing a novel method named “Enhanced Discrete Multi-modal Hashing (EDMH)” which learns binary codes and hashing functions simultaneously from the pairwise similarity matrix of data, under the aforementioned discrete constraints. Although the model of EDMH looks a lot more complex than the other models for multi-modal hashing, we are actually able to develop a fast iterative learning algorithm for it, since the subproblems of its optimization all have closed-form solutions after introducing a couple of auxiliary variables. Our experimental results on three real-world datasets have revealed the usefulness of those previously ignored discrete constraints and demonstrated that EDMH not only performs much better than state-of-the-art competitors according to several retrieval metrics but also runs much faster than most of them. [ABSTRACT FROM AUTHOR]

Dictionary learning plays a significant role in the field of machine learning. Existing works mainly focus on learning dictionary from a single domain. In this paper, we propose a novel projective double reconstructions (PDR) based dictionary learning algorithm for cross-domain recognition. Owing the distribution discrepancy between different domains, the label information is hard utilized for improving discriminability of dictionary fully. Thus, we propose a more flexible label consistent term and associate it with each dictionary item, which makes the reconstruction coefficients have more discriminability as much as possible. Due to the intrinsic correlation between cross-domain data, the data should be reconstructed with each other. Based on this consideration, we further propose a projective double reconstructions scheme to guarantee that the learned dictionary has the abilities of data itself reconstruction and data cross-reconstruction. This also guarantees that the data from different domains can be boosted mutually for obtaining a good data alignment, making the learned dictionary have more transferability. We integrate the double reconstructions, label consistency constraint and classifier learning into a unified objective and its solution can be obtained by proposed optimization algorithm that is more efficient than the conventional $\ell _{1}$ optimization based dictionary learning methods. The experiments show that the proposed PDR not only greatly reduces the time complexity for both training and testing, but also outperforms over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the explosive growth of data, the multi-view data is widely used in many fields, such as data mining, machine learning, computer vision, and so on. Because such data always has a complex structure, i.e., many categories, many perspectives of description and high dimension, how to formulate an accurate and reliable framework for the multi-view classification is a very challenging task. In this paper, we propose a novel multi-view classification method by using multiple multi-class Support Vector Machines (SVMs) with a novel collaborative strategy. Here, each multi-class SVM embeds the scaling factor to renewedly adjust the weight allocation of all features, which is beneficial to highlight more important and discriminative features. Furthermore, we adopt the decision function values to integrate multiple multi-class learners and introduce the confidence score across multiple classes to determine the final classification result. In addition, through a series of the mathematical deduction, we bridge the proposed model with the solvable problem and solve it through an alternating iteration optimization method. We evaluate the proposed method on several image and face datasets, and the experimental results demonstrate that our proposed method performs better than other state-of-the-art learning algorithms. [ABSTRACT FROM AUTHOR]

This paper addresses two fundamental problems: 1) learning discriminative model parameters and 2) avoiding over-fitting, which often occurs in regression-based classification tasks. We formulate these two problems in terms of relaxing both the strict binary label matrix and graph regularization term into more flexible forms so that the margins between different classes are enlarged as much as possible and the problem of over-fitting is avoided to some extent. This task is accomplished by the proposed double relaxed regression (DRR) method. The convex problem of DRR is solved efficiently with an iterative procedure. Extensive experiments on synthetic and real world image data sets demonstrate the effectiveness of the proposed method in terms of both classification accuracy and running time. [ABSTRACT FROM AUTHOR]

Sparse unmixing has become a popular tool for hyperspectral imagery interpretation. It refers to finding the optimal subset of a spectral library to reconstruct the image data and further estimate the proportions of different materials. Recently, multi-objective based sparse unmixing methods have presented promising performance because of their advantages in addressing combinatorial problems. A spectral and multi-objective based sparse unmixing (SMoSU) algorithm was proposed in our previous work, which solves the decision-making problem well. However, it does not show outstanding advantages in strong noise cases. To solve the problem, in this paper, SMoSU is improved based on the estimation of distribution algorithms (EDAs). The machine learning based EDAs have been a reliable approach in solving multi-objective problems. However, most of them are for special problems and relatively weak in theoretical foundations. Thus, it is unreliable to extend it directly to sparse unmixing. Here, we improve EDA on the basis of classification and propose a classification-based model for individual generating under the framework of SMoSU (CM-MoSU). In CM-MoSU, the whole population is divided to be positive and negative. Then, the macroinformation of positive individuals is used to guide the generation of new individuals. Therefore, the optimization task could pay more attention to the feasible space with high quality. Moreover, some theoretical analyses are presented to prove the reliability of CM-MoSU. In experiments, several state-of-the-art sparse unmixing algorithms are compared. Both synthetic and real-world experiments demonstrate the effectiveness of CM-MoSU. [ABSTRACT FROM AUTHOR]