Search

In this article, we focus on the unsupervised multiview feature selection, which tries to handle high-dimensional data in the field of multiview learning. Although some graph-based methods have achieved satisfactory performance, they ignore the underlying data structure across different views. Besides, their predefined Laplacian graphs are sensitive to the noises in the original data space and fail to obtain the optimal neighbor assignment. To address the above problems, we propose a novel unsupervised multiview feature selection model based on graph learning, and the contributions are three-fold: 1) during the feature selection procedure, the consensus similarity graph shared by different views is learned. Therefore, the proposed model can reveal the data relationship from the feature subset; 2) a reasonable rank constraint is added to optimize the similarity matrix to obtain more accurate information; and 3) an autoweighted framework is presented to assign view weights adaptively, and an effective alternative iterative algorithm is proposed to optimize the problem. Experiments on various datasets demonstrate the superiority of the proposed method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Binarized neural networks (BNNs) have drawn significant attention in recent years, owing to great potential in reducing computation and storage consumption. While it is attractive, traditional BNNs usually suffer from slow convergence speed and dramatical accuracy-degradation on large-scale classification datasets. To minimize the gap between BNNs and deep neural networks (DNNs), we propose a new framework of designing BNNs, dubbed Hyper-BinaryNet, from the aspect of enhanced information-flow. Our contributions are threefold: 1) Considering the capacity-limitation in the backward pass, we propose an 1-bit convolution module named HyperConv. By exploiting the capacity of auxiliary neural networks, BNNs gain better performance on large-scale image classification task. 2) Considering the slow convergence speed in BNNs, we rethink the gradient accumulation mechanism and propose a hyper accumulation technique. By accumulating gradients in multiple variables rather than one as before, the gradient paths for each weight increase, which escapes BNNs from the gradient bottleneck problem during training. 3) Considering the ill-posed optimization problem, a novel gradient estimation warmup strategy, dubbed STE-Warmup, is developed. This strategy prevents BNNs from the unstable optimization process by progressively transferring neural networks from 32-bit to 1-bit. We conduct evaluations with variant architectures on three public datasets: CIFAR-10/100 and ImageNet. Compared with state-of-the-art BNNs, Hyper-BinaryNet shows faster convergence speed and outperforms existing BNNs by a large margin. [ABSTRACT FROM AUTHOR]

These days, clustering is one of the most classical themes to analyze data structures in machine learning and pattern recognition. Recently, the anchor-based graph has been widely adopted to promote the clustering accuracy of plentiful graph-based clustering techniques. In order to achieve more satisfying clustering performance, we propose a novel clustering approach referred to as the progressive self-supervised clustering method with novel category discovery (PSSCNCD), which consists of three separate procedures specifically. First, we propose a new semisupervised framework with novel category discovery to guide label propagation processing, which is reinforced by the parameter-insensitive anchor-based graph obtained from balanced $K$ -means and hierarchical $K$ -means (BKHK). Second, we design a novel representative point selected strategy based on our semisupervised framework to discover each representative point and endow pseudolabel progressively, where every pseudolabel hypothetically corresponds to a real category in each self-supervised label propagation. Third, when sufficient representative points have been found, the labels of all samples will be finally predicted to obtain terminal clustering results. In addition, the experimental results on several toy examples and benchmark data sets comprehensively demonstrate that our method outperforms other clustering approaches. [ABSTRACT FROM AUTHOR]

Graph-based clustering aims to partition the data according to a similarity graph, which has shown impressive performance on various kinds of tasks. The quality of similarity graph largely determines the clustering results, but it is difficult to produce a high-quality one, especially when data contain noises and outliers. To solve this problem, we propose a robust rank constrained sparse learning (RRCSL) method in this article. The $L_{2,1}$ -norm is adopted into the objective function of sparse representation to learn the optimal graph with robustness. To preserve the data structure, we construct an initial graph and search the graph within its neighborhood. By incorporating a rank constraint, the learned graph can be directly used as the cluster indicator, and the final results are obtained without additional postprocessing. In addition, the proposed method cannot only be applied to single-view clustering but also extended to multiview clustering. Plenty of experiments on synthetic and real-world datasets have demonstrated the superiority and robustness of the proposed framework. [ABSTRACT FROM AUTHOR]

The multiple kernel $k$ -means (MKKM) and its variants utilize complementary information from different sources, achieving better performance than kernel $k$ -means (KKM). However, the optimization procedures of most previous works comprise two stages, learning the continuous relaxation matrix and obtaining the discrete one by extra discretization procedures. Such a two-stage strategy gives rise to a mismatched problem and severe information loss. Even worse, most existing MKKM methods overlook the correlation among prespecified kernels, which leads to the fusion of mutually redundant kernels and bad effects on the diversity of information sources, finally resulting in unsatisfying results. To address these issues, we elaborate a novel Discrete and Parameter-free Multiple Kernel $k$ -means (DPMKKM) model solved by an alternative optimization method, which can directly obtain the cluster assignment results without subsequent discretization procedure. Moreover, DPMKKM can measure the correlation among kernels by implicitly introducing a regularization term, which is able to enhance kernel fusion by reducing redundancy and improving diversity. Noteworthily, the time complexity of optimization algorithm is successfully reduced, through masterly utilizing of coordinate descent technique, which contributes to higher algorithm efficiency and broader applications. What’s more, our proposed model is parameter-free avoiding intractable hyperparameter tuning, which makes it feasible in practical applications. Lastly, extensive experiments conducted on a number of real-world datasets illustrated the effectiveness and superiority of the proposed DPMKKM model. [ABSTRACT FROM AUTHOR]

Fuzzy clustering is one of the most popular clustering approaches and has attracted considerable attention in many fields. However, high computational cost has become a bottleneck which limits its applications in large-scale problems. Moreover, most fuzzy clustering algorithms are sensitive to noise. To address these issues, a novel fuzzy clustering algorithm, called fast fuzzy clustering based on anchor graph (FFCAG), is proposed. The FFCAG algorithm integrates anchor-based similarity graph construction and membership matrix learning into a unified framework, such that the prior knowledge of anchors can be further utilized to improve clustering performance. Specifically, FFCAG first constructs an anchor-based similarity graph with a parameter-free neighbor assignment strategy. Then, it designs a quadratic programming model to learn the membership matrix of anchors, which is very different from traditional fuzzy clustering algorithms. More importantly, a novel balanced regularization term is introduced into the objective function to produce more accurate clustering results. Finally, we adopt an alternating optimization algorithm with guaranteed convergence to solve the proposed method. Experimental results performed on synthetic and real-world datasets demonstrate the proposed FFCAG can significantly reduce the computational time with comparable, even superior, clustering performance, compared with state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Sparse unmixing (SU) algorithms use the existing spectral library as prior knowledge to analyze the endmembers and estimate abundance maps. The majority of SU algorithms use loss functions based on the $L_{2,1}$ -norm or $F$ -norm to minimize reconstruction error. They have different advantages and shortcomings. In short, $F$ -norm has a differentiable characteristic, and it is easy to minimize as a loss function. However, it is very sensitive to heavy noise and outliers. While the $L_{2,1}$ -norm emphasizes the reconstruction error on each band and is robust to noise with different intensities in different bands. But the $L_{2,1}$ -norm is nondifferentiable at zero-point. This article introduces an adaptive loss function based on the $\sigma $ -norm for SU, which combines the advantages of the $L_{2,1}$ -norm and $F$ -norm. The adaptive loss function is related to a nonnegative parameter $\sigma $. By adjusting the parameter $\sigma $ , the adaptive loss function can approach the $F$ -norm or $L_{2,1}$ -norm. To the best of our knowledge, it is the first time to apply an adaptive loss function to SU. Moreover, the adaptive loss function is globally differentiable, and we propose an optimization algorithm for the adaptive loss function and verify its convergence. Experiments on real-world and synthetic HSIs show that the adaptive loss function effectively enhances the performance of the SU algorithms. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection (UFS) aims to remove the redundant information and select the most representative feature subset from the original data, so it occupies a core position for high-dimensional data preprocessing. Many proposed approaches use self-expression to explore the correlation between the data samples or use pseudolabel matrix learning to learn the mapping between the data and labels. Furthermore, the existing methods have tried to add constraints to either of these two modules to reduce the redundancy, but no prior literature embeds them into a joint model to select the most representative features by the computed top ranking scores. To address the aforementioned issue, this article presents a novel UFS method via a convex non-negative matrix factorization with an adaptive graph constraint (CNAFS). Through convex matrix factorization with adaptive graph constraint, it can dig up the correlation between the data and keep the local manifold structure of the data. To our knowledge, it is the first work that integrates pseudo label matrix learning into the self-expression module and optimizes them simultaneously for the UFS solution. Besides, two different manifold regularizations are constructed for the pseudolabel matrix and the encoding matrix to keep the local geometrical structure. Eventually, extensive experiments on the benchmark datasets are conducted to prove the effectiveness of our method. The source code is available at: https://github.com/misteru/CNAFS. [ABSTRACT FROM AUTHOR]

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging $\ell _{2,1}$ -norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the $\ell _{2,0}$ -norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S2DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks. [ABSTRACT FROM AUTHOR]

$k$ k -means method using Lloyd heuristic is a traditional clustering method which has played a key role in multiple downstream tasks of machine learning because of its simplicity. However, Lloyd heuristic always finds a bad local minimum, i.e., the bad local minimum makes objective function value not small enough, which limits the performance of $k$ k -means. In this paper, we use coordinate descent (CD) method to solve the problem. First, we show that the $k$ k -means minimization problem can be reformulated as a trace maximization problem, then a simple and efficient coordinate descent scheme is proposed to solve the maximization problem. Two interesting findings through theory are that Lloyd cannot decrease the objective function value of $k$ k -means produced by our CD further, and our proposed method CD to solve $k$ k -means problem can avoid produce empty clusters. In addition, according to the computational complexity analysis, it is verified CD has the same time complexity with original $k$ k -means method. Extensive experiments including statistical hypothesis testing, on several real-world datasets with varying number of clusters, varying number of samples and varying number of dimensions show that CD performs better compared to Lloyd, i.e., lower objective value, better local minimum and fewer iterations. And CD is more robust to initialization than Lloyd whether the initialization strategy is random or initialization of $k$ k -means++. [ABSTRACT FROM AUTHOR]

Due to the exponential growth of multimedia data, multi-modal hashing as a promising technique to make cross-view retrieval scalable is attracting more and more attention. However, most of the existing multi-modal hashing methods either divide the learning process unnaturally into two separate stages or treat the discrete optimization problem simplistically as a continuous one, which leads to suboptimal results. Recently, a few discrete multi-modal hashing methods that try to address such issues have emerged, but they still ignore several important discrete constraints (such as the balance and decorrelation of hash bits). In this paper, we overcome those limitations by proposing a novel method named “Enhanced Discrete Multi-modal Hashing (EDMH)” which learns binary codes and hashing functions simultaneously from the pairwise similarity matrix of data, under the aforementioned discrete constraints. Although the model of EDMH looks a lot more complex than the other models for multi-modal hashing, we are actually able to develop a fast iterative learning algorithm for it, since the subproblems of its optimization all have closed-form solutions after introducing a couple of auxiliary variables. Our experimental results on three real-world datasets have revealed the usefulness of those previously ignored discrete constraints and demonstrated that EDMH not only performs much better than state-of-the-art competitors according to several retrieval metrics but also runs much faster than most of them. [ABSTRACT FROM AUTHOR]

Recently, several robust principle component analysis (RPCA) models have been proposed to improve the robustness of principle component analysis (PCA). But an important problem that the robustness to outliers affects the discrimination of correct samples has not been solved yet. To solve this problem, we propose a truncated robust principle component analysis (T-RPCA) model which treats correct samples and outliers separately. In fact, the proposed model performs an implicitly truncated weighted learning scheme which is more reasonable for robustness learning respective to previous works. Moreover, we propose a re-weighted (RW) optimization framework to solve a general problem and generalize two sub-frameworks upon it. To be specific, the first sub-framework orients a general truncated loss optimization problem which contains the objective problem of T-RPCA, and the second one focuses on a general singular-value based optimization problem. Besides, we provide rigorously theoretical guarantees for the proposed model, RW framework and sub-frameworks. Empirical studies demonstrate that the proposed T-RPCA model outperforms previous RPCA models on reconstruction and classification tasks. [ABSTRACT FROM AUTHOR]

Semisupervised band selection (BS) technique plays an important role in processing hyperspectral images (HSIs) because of its superiority of using the limited labeled data and plentiful unlabeled data to select the discriminative and informative feature, which copes with the problem of high-dimensional and scare labeled samples of HSIs. Among semisupervised BS models, graph-based methods are superior to others in many situations and have received more and more attention. However, traditional graph-based models construct a similarity matrix and select valuable bands independently. The similarity matrix remains constant, which will damage the local manifold structure and lead to a suboptimal result. To solve this problem, we propose a semisupervised band selection with an optimal graph (BSOG) approach, which performs BS and local structure learning simultaneously. Instead of fixing the input similarity matrix, the similarity matrix is updated constantly to learn a better local structure. Besides, the learned similarity matrix is adaptive. Then, the optimal band subset can be selected by analyzing the obtained projection matrix $\boldsymbol W$. An efficient and simple optimization algorithm is proposed to solve this model. Experiments on four real HSIs data validate the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Spectral clustering methods are gaining more and more interests and successfully applied in many fields because of their superior performance. However, there still exist two main problems to be solved: 1) spectral clustering methods consist of two successive optimization stages—spectral embedding and spectral rotation, which may not lead to globally optimal solutions, 2) and it is known that spectral methods are time-consuming with very high computational complexity. There are methods proposed to reduce the complexity for data vectors but not for graphs that only have information about similarity matrices. In this paper, we propose a new method to solve these two challenging problems for graph clustering. In the new method, a new framework is established to perform spectral embedding and spectral rotation simultaneously. The newly designed objective function consists of both terms of embedding and rotation, and we use an improved spectral rotation method to make it mathematically rigorous for the optimization. To further accelerate the algorithm, we derive a low-dimensional representation matrix from a graph by using label propagation, with which, in return, we can reconstruct a double-stochastic and positive semidefinite similarity matrix. Experimental results demonstrate that our method has excellent performance in time cost and accuracy. [ABSTRACT FROM AUTHOR]

Recently, remote sensing image captioning (RSIC) has drawn an increasing attention. In this field, the encoder–decoder-based methods have become the mainstream due to their excellent performance. In the encoder–decoder framework, the convolutional neural network (CNN) is used to encode a remote sensing image into a semantic feature vector, and a sequence model such as long short-term memory (LSTM) is subsequently adopted to generate a content-related caption based on the feature vector. During the traditional training stage, the probability of the target word at each time step is forcibly optimized to 1 by the cross entropy (CE) loss. However, because of the variability and ambiguity of possible image captions, the target word could be replaced by other words like its synonyms, and therefore, such an optimization strategy would result in the overfitting of the network. In this article, we explore the overfitting phenomenon in the RSIC caused by CE loss and correspondingly propose a new truncation cross entropy (TCE) loss, aiming to alleviate the overfitting problem. In order to verify the effectiveness of the proposed approach, extensive comparison experiments are performed on three public RSIC data sets, including UCM-captions, Sydney-captions, and RSICD. The state-of-the-art result of Sydney-captions and RSICD and the competitive results of UCM-captions achieved by TCE loss demonstrate that the proposed method is beneficial to RSIC. [ABSTRACT FROM AUTHOR]

Linear Discriminant Analysis (LDA) is one of the most successful supervised dimensionality reduction methods and has been widely used in many real-world applications. However, ℓ2-norm is employed as the distance metric in the objective of LDA, which is sensitive to outliers. Many previous works improve the robustness of LDA by using ℓ1-norm distance. However, the robustness against outliers is limited and the solver of ℓ1-norm is mostly based on the greedy search strategy, which is time-consuming and easy to get stuck in a local optimum. In this paper, we propose a novel robust LDA measured by ℓ2,1-norm to learn robust discriminative projections. The proposed model is challenging to solve since it needs to minimize and maximize (minmax) ℓ2,1-norm terms simultaneously. As a result, we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem, and then rigorously prove its convergence. More importantly, an alternately non-greedy iterative re-weighted optimization algorithm is developed based on the preceding approach for solving proposed ℓ2,1-norm minmax problem. Besides, an optimal weighted mean mechanism is driven according to the designed objective and solver, which can be applied to other approaches for robustness improvement. Experimental results on several real-world datasets show the effectiveness of proposed method. [ABSTRACT FROM AUTHOR]

Dimensionality reduction has attracted much research interest in the past few decades. Existing dimensionality reduction methods like linear discriminant analysis and principal component analysis have achieved promising performance, but the single and linear projection properties limit further improvements of performance. A novel convolutional two-dimensional nonlinear discriminant analysis method is proposed for dimensionality reduction in this article. In order to handle nonlinear data properly, we present a newly designed structure with convolutional neural networks (CNNs) to realize an equivalent objective function with classical two-dimensional linear discriminant analysis (2DLDA) and thus embed the original 2DLDA into an end-to-end network. In this way, the proposed dimensionality reduction network can utilize the nonlinearity of the CNN and benefit from the learning ability. The results of experiment on different image-related applications demonstrate that our method outperforms other comparable approaches, and its effectiveness is proved. [ABSTRACT FROM AUTHOR]

Band selection is one of the most important technique in the reduction of hyperspectral image (HSI). Different from traditional feature selection problem, an important characteristic of it is that there is usually strong correlation between neighboring bands, that is, bands with close indexes. Aiming to fully exploit this prior information, a novel band selection method called optimal neighborhood reconstruction (ONR) is proposed. In ONR, band selection is considered as a combinatorial optimization problem. It evaluates a band combination by assessing its ability to reconstruct the original data, and applies a noise reducer to minimize the influence of noisy bands. Instead of using some approximate algorithms, ONR exploits a recurrence relation that underlies the optimization target to obtain the optimal solution in an efficient way. Besides, we develop a parameter selection approach to automatically determine the parameter of ONR, ensuring it is adaptable to different data sets. In experiments, ONR is compared with some state-of-the-art methods on six HSI data sets. The results demonstrate that ONR is more effective and robust than the others in most of the cases. [ABSTRACT FROM AUTHOR]

Semi-supervised learning models for multiview data are important in image classification tasks, since heterogeneous features are easy to obtain and semi-supervised schemes are economical and effective. To model the view importance, conventional graph-based multiview learning models learn a linear combination of views while assuming a priori weights distribution. In this paper, we present a novel structural regularized semi-supervised model for multiview data, termed Adaptive MUltiview SEmi-supervised model (AMUSE). Our new model learns weights from a priori graph structure, which is more reasonable than weight regularization. Theoretical analysis reveals the significant difference between AMUSE and the prior arts. An efficient optimization algorithm is provided to solve the new model. Experimental results on six real-world data sets demonstrate the effectiveness of the structural regularized weights learning scheme. [ABSTRACT FROM AUTHOR]

Neighborhood preserving embedding (NPE) has been proposed to encode overall geometry manifold embedding information. However, the class-special structure of the data is destroyed by noise or outliers existing in the data. To address this problem, in this article, we propose a novel embedding approach called robust flexible preserving embedding (RFPE). First, RFPE recovers the noisy data by low-rank learning and obtains clean data. Then, the clean data are used to learn the projection matrix. In this way, the projective learning is totally unaffected by noise or outliers. By encoding a flexible regularization term, RFPE can keep the property of the data points with a nonlinear manifold and be more flexible. RFPE searches the optimal projective subspace for feature extraction. In addition, we also extend the proposed RFPE to a kernel case and propose kernel RFPE (KRFPE). Extensive experiments on six public image databases show the superiority of the proposed methods over other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Due to the multimodality of non-Gaussian data, traditional globality-preserved dimensionality reduction (DR) methods, such as linear discriminant analysis (LDA) and principal component analysis (PCA) are difficult to deal with. In this paper, we present a novel local DR framework via auto-optimized graph embedding to extract the intrinsic submanifold structure of multimodal data. Specifically, the proposed model seeks to learn an embedding space which can preserve the local neighborhood structure by constructing a ${k}$ -nearest neighbors (${k}$ NNs) graph on data points. Different than previous works, our model employs the $\boldsymbol {\ell }_{\boldsymbol {0}}$ -norm constraint and binary constraint on the similarity matrix to impose that there only be a ${k}$ nonzero value in each row of the similarity matrix, which can ensure the ${k}$ -connectivity in graph. More important, as the high-dimensional data probably contains some noises and redundant features, calculating the similarity matrix in the original space by using a kernel function is inaccurate. As a result, a mechanism of an auto-optimized graph is derived in the proposed model. Concretely, we learn the embedding space and similarity matrix simultaneously. In other words, the selection of neighbors is automatically executed in the optimal subspace rather than in the original space when the algorithm reaches convergence, which can alleviate the affect of noises and improve the robustness of the proposed model. In addition, four supervised and semisupervised local DR methods are derived by the proposed framework which can extract the discriminative features while preserving the submanifold structure of data. Last but not least, since two variables need to be optimized simultaneously in the proposed methods, and the constraints on the similarity matrix are difficult to satisfy, which is an NP-hard problem. Consequently, an efficient iterative optimization algorithm is introduced to solve the proposed problems. Extensive experiments conducted on synthetic data and several real-world datasets have demonstrated the advantages of the proposed methods in robustness and recognition accuracy. [ABSTRACT FROM AUTHOR]

The existing unsupervised feature extraction methods frequently explore low-redundant features by an uncorrelated constraint. However, the constrained models might incur trivial solutions, due to the singularity of scatter matrix triggered by high-dimensional data. In this paper, we propose a regularized regression model with a generalized uncorrelated constraint for feature selection, which leads to three merits: 1) exploring the low-redundant and discriminative features; 2) avoiding the trivial solutions and 3) simplifying the optimization. Besides that, the local cluster structure is achieved via a novel constrained spectral analysis for the unsupervised learning, where Must-Links and Cannot-Links are transformed into a intrinsic graph and a penalty graph respectively, rather than incorporated into a mixed affinity graph. Accordingly, a discriminative and uncorrelated feature selection with constrained spectral analysis (DUCFS) is proposed with adopting $\sigma $ -norm regularization for interpolating between F-norm and $\ell _{2,1}$ -norm. Due to the flexible gradient and global differentiability, our model converges fast. Extensive experiments on benchmark datasets among several state-of-the-art approaches verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Dictionary learning plays a significant role in the field of machine learning. Existing works mainly focus on learning dictionary from a single domain. In this paper, we propose a novel projective double reconstructions (PDR) based dictionary learning algorithm for cross-domain recognition. Owing the distribution discrepancy between different domains, the label information is hard utilized for improving discriminability of dictionary fully. Thus, we propose a more flexible label consistent term and associate it with each dictionary item, which makes the reconstruction coefficients have more discriminability as much as possible. Due to the intrinsic correlation between cross-domain data, the data should be reconstructed with each other. Based on this consideration, we further propose a projective double reconstructions scheme to guarantee that the learned dictionary has the abilities of data itself reconstruction and data cross-reconstruction. This also guarantees that the data from different domains can be boosted mutually for obtaining a good data alignment, making the learned dictionary have more transferability. We integrate the double reconstructions, label consistency constraint and classifier learning into a unified objective and its solution can be obtained by proposed optimization algorithm that is more efficient than the conventional $\ell _{1}$ optimization based dictionary learning methods. The experiments show that the proposed PDR not only greatly reduces the time complexity for both training and testing, but also outperforms over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the explosive growth of data, the multi-view data is widely used in many fields, such as data mining, machine learning, computer vision, and so on. Because such data always has a complex structure, i.e., many categories, many perspectives of description and high dimension, how to formulate an accurate and reliable framework for the multi-view classification is a very challenging task. In this paper, we propose a novel multi-view classification method by using multiple multi-class Support Vector Machines (SVMs) with a novel collaborative strategy. Here, each multi-class SVM embeds the scaling factor to renewedly adjust the weight allocation of all features, which is beneficial to highlight more important and discriminative features. Furthermore, we adopt the decision function values to integrate multiple multi-class learners and introduce the confidence score across multiple classes to determine the final classification result. In addition, through a series of the mathematical deduction, we bridge the proposed model with the solvable problem and solve it through an alternating iteration optimization method. We evaluate the proposed method on several image and face datasets, and the experimental results demonstrate that our proposed method performs better than other state-of-the-art learning algorithms. [ABSTRACT FROM AUTHOR]

Graph clustering plays an important role in data mining. Based on an input data graph, data points are partitioned into clusters. However, most existing methods keep the data graph fixed during the clustering procedure, so they are limited to exploit the implied data manifold and highly dependent on the initial graph construction. Inspired by the recent development on manifold learning, this paper proposes an Adaptive Consistency Propagation (ACP) method for graph clustering. In order to utilize the features captured from different perspectives, we further put forward the Multi-view version of the ACP model (MACP). The main contributions are threefold: (1) the manifold structure of input data is sufficiently exploited by propagating the topological connectivities between data points from near to far; (2) the optimal graph for clustering is learned by taking graph learning as a part of the optimization procedure; and (3) the negotiation among the heterogeneous features is captured by the multi-view clustering model. Extensive experiments on real-world datasets validate the effectiveness of the proposed methods on both single- and multi-view clustering, and show their superior performance over the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ${\ell _{2,1}}$ ℓ 2 , 1 -norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper addresses two fundamental problems: 1) learning discriminative model parameters and 2) avoiding over-fitting, which often occurs in regression-based classification tasks. We formulate these two problems in terms of relaxing both the strict binary label matrix and graph regularization term into more flexible forms so that the margins between different classes are enlarged as much as possible and the problem of over-fitting is avoided to some extent. This task is accomplished by the proposed double relaxed regression (DRR) method. The convex problem of DRR is solved efficiently with an iterative procedure. Extensive experiments on synthetic and real world image data sets demonstrate the effectiveness of the proposed method in terms of both classification accuracy and running time. [ABSTRACT FROM AUTHOR]

Spectral clustering is an important clustering method widely used for pattern recognition and image segmentation. Classical spectral clustering algorithms consist of two separate stages: 1) solving a relaxed continuous optimization problem to obtain a real matrix followed by 2) applying ${K}$ -means or spectral rotation to round the real matrix (i.e., continuous clustering result) into a binary matrix called the cluster indicator matrix. Such a separate scheme is not guaranteed to achieve jointly optimal result because of the loss of useful information. To obtain a better clustering result, in this paper, we propose a joint model to simultaneously compute the optimal real matrix and binary matrix. The existing joint model adopts an orthonormal real matrix to approximate the orthogonal but nonorthonormal cluster indicator matrix. It is noted that only in a very special case (i.e., all clusters have the same number of samples), the cluster indicator matrix is an orthonormal matrix multiplied by a real number. The error of approximating a nonorthonormal matrix is inevitably large. To overcome the drawback, we propose replacing the nonorthonormal cluster indicator matrix with a scaled cluster indicator matrix which is an orthonormal matrix. Our method is capable of obtaining better performance because it is easy to minimize the difference between two orthonormal matrices. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method (called JSESR). [ABSTRACT FROM AUTHOR]

Sparse unmixing has become a popular tool for hyperspectral imagery interpretation. It refers to finding the optimal subset of a spectral library to reconstruct the image data and further estimate the proportions of different materials. Recently, multi-objective based sparse unmixing methods have presented promising performance because of their advantages in addressing combinatorial problems. A spectral and multi-objective based sparse unmixing (SMoSU) algorithm was proposed in our previous work, which solves the decision-making problem well. However, it does not show outstanding advantages in strong noise cases. To solve the problem, in this paper, SMoSU is improved based on the estimation of distribution algorithms (EDAs). The machine learning based EDAs have been a reliable approach in solving multi-objective problems. However, most of them are for special problems and relatively weak in theoretical foundations. Thus, it is unreliable to extend it directly to sparse unmixing. Here, we improve EDA on the basis of classification and propose a classification-based model for individual generating under the framework of SMoSU (CM-MoSU). In CM-MoSU, the whole population is divided to be positive and negative. Then, the macroinformation of positive individuals is used to guide the generation of new individuals. Therefore, the optimization task could pay more attention to the feasible space with high quality. Moreover, some theoretical analyses are presented to prove the reliability of CM-MoSU. In experiments, several state-of-the-art sparse unmixing algorithms are compared. Both synthetic and real-world experiments demonstrate the effectiveness of CM-MoSU. [ABSTRACT FROM AUTHOR]

2-D neighborhood preserving projection (2DNPP) uses 2-D images as feature input instead of 1-D vectors used by neighborhood preserving projection (NPP). 2DNPP requires less computation time than NPP. However, both NPP and 2DNPP use the ${L}_{ {{2}}}$ norm as a metric, which is sensitive to noise in data. In this paper, we proposed a novel NPP method called low-rank 2DNPP (LR-2DNPP). This method divided the input data into a component part that encoded low-rank features, and an error part that ensured the noise was sparse. Then, a nearest neighbor graph was learned from the clean data using the same procedure as 2DNPP. To ensure that the features learned by LR-2DNPP were optimal for classification, we combined the structurally incoherent learning and low-rank learning with NPP to form a unified model called discriminative LR-2DNPP (DLR-2DNPP). By encoding the structural incoherence of the learned clean data, DLR-2DNPP could enhance the discriminative ability for feature extraction. Theoretical analyses on the convergence and computational complexity of LR-2DNPP and DLR-2DNPP were presented in details. We used seven public image databases to verify the performance of the proposed methods. The experimental results showed the effectiveness of our methods for robust image representation. [ABSTRACT FROM AUTHOR]

2-D linear discriminant analysis (2DLDA) has been widely used in pattern recognition and image classification. 2DLDA selects discriminative features from the up and left corner of images. However, 2DLDA uses the Frobenius norm (F-norm), which is sensitive to noise or outliers in data, as a metric. In this paper, we propose a novel framework, called horizontal and vertical nuclear norm-based 2DLDA (HVNN-2DLDA) for image representation. In the proposed framework, HVNN-2DLDA methods (i.e., HNN-2DLDA and VNN-2DLDA) are proposed, and both use the nuclear norm as a criterion. The nuclear norm can provide more structure and global information for the reconstruction of noisy images. HNN-2DLDA and VNN-2DLDA represent images in the row and column directions, respectively. In addition, by combining the row and column directions, we propose a bilateral nuclear norm-based 2DLDA method called BNN-2DLDA. The advantage of BNN-2DLDA over HNN-2DLDA and VNN-2DLDA is that an image sample can be represented by both the row and the column directions instead of only the row or column direction. HVNN-2DLDA learns a set of local optimal projection vectors by maximizing the ratio of the nuclear norm of the between-class scatter matrix and the nuclear norm of the within-class scatter matrix. To verify the robustness and recognition performance in image classification of HVNN-2DLDA, six public image databases are used for experiments. The experimental results demonstrate the effectiveness and the feasibility of the proposed framework. [ABSTRACT FROM AUTHOR]

Spectral clustering has been playing a vital role in various research areas. Most traditional spectral clustering algorithms comprise two independent stages (e.g., first learning continuous labels and then rounding the learned labels into discrete ones), which may cause unpredictable deviation of resultant cluster labels from genuine ones, thereby leading to severe information loss and performance degradation. In this work, we study how to achieve discrete clustering as well as reliably generalize to unseen data. We propose a novel spectral clustering scheme which deeply explores cluster label properties, including discreteness, nonnegativity, and discrimination, as well as learns robust out-of-sample prediction functions. Specifically, we explicitly enforce a discrete transformation on the intermediate continuous labels, which leads to a tractable optimization problem with a discrete solution. Besides, we preserve the natural nonnegative characteristic of the clustering labels to enhance the interpretability of the results. Moreover, to further compensate the unreliability of the learned clustering labels, we integrate an adaptive robust module with \ell 2,p loss to learn prediction function for grouping unseen data. We also show that the out-of-sample component can inject discriminative knowledge into the learning of cluster labels under certain conditions. Extensive experiments conducted on various data sets have demonstrated the superiority of our proposal as compared to several existing clustering approaches. [ABSTRACT FROM PUBLISHER]

Finding overlapping communities from multimedia social networks is an interesting and important problem in data mining and recommender systems. However, extant overlapping community discovery with swarm intelligence often generates overlapping community structures with superfluous small communities. To deal with the problem, in this paper, an efficient algorithm (LEPSO) is proposed for overlapping communities discovery, which is based on line graph theory, ensemble learning, and particle swarm optimization (PSO). Specifically, a discrete PSO, consisting of an encoding scheme with ordered neighbors and a particle updating strategy with ensemble clustering, is devised for improving the optimization ability to search communities hidden in social networks. Then, a postprocessing strategy is presented for merging the finer-grained and suboptimal overlapping communities. Experiments on some real-world and synthetic datasets show that our approach is superior in terms of robustness, effectiveness, and automatically determination of the number of clusters, which can discover overlapping communities that have better quality than those computed by state-of-the-art algorithms for overlapping communities detection. [ABSTRACT FROM PUBLISHER]