Search

These days, clustering is one of the most classical themes to analyze data structures in machine learning and pattern recognition. Recently, the anchor-based graph has been widely adopted to promote the clustering accuracy of plentiful graph-based clustering techniques. In order to achieve more satisfying clustering performance, we propose a novel clustering approach referred to as the progressive self-supervised clustering method with novel category discovery (PSSCNCD), which consists of three separate procedures specifically. First, we propose a new semisupervised framework with novel category discovery to guide label propagation processing, which is reinforced by the parameter-insensitive anchor-based graph obtained from balanced K -means and hierarchical K -means (BKHK). Second, we design a novel representative point selected strategy based on our semisupervised framework to discover each representative point and endow pseudolabel progressively, where every pseudolabel hypothetically corresponds to a real category in each self-supervised label propagation. Third, when sufficient representative points have been found, the labels of all samples will be finally predicted to obtain terminal clustering results. In addition, the experimental results on several toy examples and benchmark data sets comprehensively demonstrate that our method outperforms other clustering approaches.

Due to the inconsistent distributions and representations of different modalities (e.g., images and texts), it is very challenging to correlate such heterogeneous data. A standard solution is to construct one common subspace, where the common representations of different modalities are generated to bridge the heterogeneity gap. Existing methods based on common representation learning mostly adopt a less effective two-stage paradigm: first, generating separate representations for each modality by exploiting the modality-specific properties as the complementary information, and then capturing the cross-modal correlation in the separate representations for common representation learning. Moreover, these methods usually neglect that there may exist interference in the modality-specific properties, that is, the unrelated objects and background regions in images or the noisy words and incorrect sentences in the text. In this article, we hypothesize that explicitly modeling the interference within each modality can improve the quality of common representation learning. To this end, we propose a novel model private-shared subspaces separation (P3S) to explicitly learn different representations that are partitioned into two kinds of subspaces: 1) the common representations that capture the cross-modal correlation in a shared subspace and 2) the private representations that model the interference within each modality in two private subspaces. By employing the orthogonality constraints between the shared subspace and the private subspaces during the one-stage joint learning procedure, our model is able to learn more effective common representations for different modalities in the shared subspace by fully excluding the interference within each modality. Extensive experiments conducted on cross-modal retrieval verify the advantages of our P3S method compared with 15 state-of-the-art methods on four widely used cross-modal datasets.

Linear discriminant analysis (LDA) has been proven to be effective in dimensionality reduction. However, the performance of LDA depends on the consistency assumption of the global structure and the local structure. Some work extended LDA along this line of research and proposed local formulations of LDA. Unfortunately, the learning scheme of these algorithms is suboptimal in that the intrinsic relationship between data points is pre-learned in the original space, which is usually affected by the noise and redundant features. Besides, the time cost is relatively high. To alleviate these drawbacks, we propose a Fast Locality Discriminant Analysis framework (FLDA), which has three advantages: (1) It can divide a non-Gaussian distribution class into many sub-blocks that obey Gaussian distributions by using the anchor-based strategy. (2) It captures the manifold structure of data by learning the fuzzy membership relationship between data points and the corresponding anchor points, which can reduce computation time. (3) The weights between data points and anchor points are adaptively updated in the subspace where the irrelevant information and the noise in high-dimensional space have been effectively suppressed. Extensive experiments on toy data sets, UCI benchmark data sets and imbalanced data sets demonstrate the efficiency and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this article, we focus on the unsupervised multiview feature selection, which tries to handle high-dimensional data in the field of multiview learning. Although some graph-based methods have achieved satisfactory performance, they ignore the underlying data structure across different views. Besides, their predefined Laplacian graphs are sensitive to the noises in the original data space and fail to obtain the optimal neighbor assignment. To address the above problems, we propose a novel unsupervised multiview feature selection model based on graph learning, and the contributions are three-fold: 1) during the feature selection procedure, the consensus similarity graph shared by different views is learned. Therefore, the proposed model can reveal the data relationship from the feature subset; 2) a reasonable rank constraint is added to optimize the similarity matrix to obtain more accurate information; and 3) an autoweighted framework is presented to assign view weights adaptively, and an effective alternative iterative algorithm is proposed to optimize the problem. Experiments on various datasets demonstrate the superiority of the proposed method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Graph-based clustering aims to partition the data according to a similarity graph, which has shown impressive performance on various kinds of tasks. The quality of similarity graph largely determines the clustering results, but it is difficult to produce a high-quality one, especially when data contain noises and outliers. To solve this problem, we propose a robust rank constrained sparse learning (RRCSL) method in this article. The $L_{2,1}$ -norm is adopted into the objective function of sparse representation to learn the optimal graph with robustness. To preserve the data structure, we construct an initial graph and search the graph within its neighborhood. By incorporating a rank constraint, the learned graph can be directly used as the cluster indicator, and the final results are obtained without additional postprocessing. In addition, the proposed method cannot only be applied to single-view clustering but also extended to multiview clustering. Plenty of experiments on synthetic and real-world datasets have demonstrated the superiority and robustness of the proposed framework. [ABSTRACT FROM AUTHOR]

Semi-supervised learning as one of most attractive problems in machine learning research field has aroused broad attentions in recent years. In this paper, we propose a novel locality preserved dimensionality reduction framework, named Semi-supervised Adaptive Local Embedding learning (SALE), which learns a local discriminative embedding by constructing a $k_1$ k 1 Nearest Neighbors ($k_1$ k 1 NN) graph on labeled data, so as to explore the intrinsic structure, i.e., sub-manifolds from non-Gaussian labeled data. Then, mapping all samples into learned embedding and constructing another $k_2$ k 2 NN graph on all embedded data to explore the global structure of all samples. Therefore, the unlabeled data and their corresponding labeled neighbors can be clustered into same sub-manifold, so as to improve the discriminative power of embedded data. Furthermore, we propose two semi-supervised dimensionality reduction methods with orthogonal and whitening constraints based on proposed SALE framework. An efficient alternatively iterative optimization algorithm is developed to solve the NP-hard problem in our models. Extensive experiments conducted on several synthetic and real-world data sets demonstrate the superiorities of our methods on local structure exploration and classification task. [ABSTRACT FROM AUTHOR]

Sparse unmixing (SU) is a semisupervised learning problem, which performs abundance estimation when a spectral library is given. In this way, the essence of SU is to select the most suitable subset from the spectral library for representing all mixed pixels. Many SU methods adopt joint-sparse and low-rank constraints to guide the abundance estimation. However, the spatial correlation learning in these algorithms is not accurate enough, which seriously affects the unmixing performance. Besides, most pruning-based unmixing methods suffer from complicated pruning strategies and ignore the relationship between the spectral library and mixed pixels. This article proposes a reweighted low-rank and joint-sparse unmixing approach, which combines an effective pruning strategy (RLSU-LP). The RLSU-LP approach consists of rough unmixing stage, library pruning, and fine-tuning unmixing stage. First, the proposed method utilizes image segmentation to obtain different homogeneous regions, i.e., superpixels. A confidence index is introduced to describe the superpixel homogeneity, which is conducive to learning the meticulous spatial correlation. The RLSU-LP method reasonably relaxes or tightens the sparse and low-rank constraints of the abundance matrix by using the confidence index. Furthermore, a supervised library pruning strategy is proposed, which aims to eliminate the inactive endmembers by considering the contribution of representing mixed pixels. Experiments on the synthesized dataset and authentic hyperspectral images verify the effectiveness of our proposed algorithm. [ABSTRACT FROM AUTHOR]

$k$ k -means method using Lloyd heuristic is a traditional clustering method which has played a key role in multiple downstream tasks of machine learning because of its simplicity. However, Lloyd heuristic always finds a bad local minimum, i.e., the bad local minimum makes objective function value not small enough, which limits the performance of $k$ k -means. In this paper, we use coordinate descent (CD) method to solve the problem. First, we show that the $k$ k -means minimization problem can be reformulated as a trace maximization problem, then a simple and efficient coordinate descent scheme is proposed to solve the maximization problem. Two interesting findings through theory are that Lloyd cannot decrease the objective function value of $k$ k -means produced by our CD further, and our proposed method CD to solve $k$ k -means problem can avoid produce empty clusters. In addition, according to the computational complexity analysis, it is verified CD has the same time complexity with original $k$ k -means method. Extensive experiments including statistical hypothesis testing, on several real-world datasets with varying number of clusters, varying number of samples and varying number of dimensions show that CD performs better compared to Lloyd, i.e., lower objective value, better local minimum and fewer iterations. And CD is more robust to initialization than Lloyd whether the initialization strategy is random or initialization of $k$ k -means++. [ABSTRACT FROM AUTHOR]

Multimodal emotion recognition in conversational videos (ERC) develops rapidly in recent years. To fully extract the relative context from video clips, most studies build their models on the entire dialogues which make them lack of real-time ERC ability. Different from related researches, a novel multimodal emotion recognition model for conversational videos based on reinforcement learning and domain knowledge (ERLDK) is proposed in this paper. In ERLDK, the reinforcement learning algorithm is introduced to conduct real-time ERC with the occurrence of conversations. The collection of history utterances is composed as an emotion-pair which represents the multimodal context of the following utterance to be recognized. Dueling deep-Q-network (DDQN) based on gated recurrent unit (GRU) layers is designed to learn the correct action from the alternative emotion categories. Domain knowledge is extracted from public dataset based on the former information of emotion-pairs. The extracted domain knowledge is used to revise the results from the RL module and is transformed into other dataset to examine the rationality. The experimental results on datasets show that ERLDK achieves the state-of-the-art results on weighted average and most of the specific emotion categories. [ABSTRACT FROM AUTHOR]

Due to the exponential growth of multimedia data, multi-modal hashing as a promising technique to make cross-view retrieval scalable is attracting more and more attention. However, most of the existing multi-modal hashing methods either divide the learning process unnaturally into two separate stages or treat the discrete optimization problem simplistically as a continuous one, which leads to suboptimal results. Recently, a few discrete multi-modal hashing methods that try to address such issues have emerged, but they still ignore several important discrete constraints (such as the balance and decorrelation of hash bits). In this paper, we overcome those limitations by proposing a novel method named “Enhanced Discrete Multi-modal Hashing (EDMH)” which learns binary codes and hashing functions simultaneously from the pairwise similarity matrix of data, under the aforementioned discrete constraints. Although the model of EDMH looks a lot more complex than the other models for multi-modal hashing, we are actually able to develop a fast iterative learning algorithm for it, since the subproblems of its optimization all have closed-form solutions after introducing a couple of auxiliary variables. Our experimental results on three real-world datasets have revealed the usefulness of those previously ignored discrete constraints and demonstrated that EDMH not only performs much better than state-of-the-art competitors according to several retrieval metrics but also runs much faster than most of them. [ABSTRACT FROM AUTHOR]

Feature selection is one of the most important techniques to deal with the high-dimensional data for a variety of machine learning and data mining tasks, such clustering, classification, and retrieval, etc. Fuzziness is a widespread nature of data in nature human society. However, most existing feature selection methods ignore the existence of fuzziness in the data, resulting in sub-optimal feature subsets. To address the problem, we propose a novel unsupervised feature selection method, called Unsupervised Discriminative Projection for Feature Selection (UDPFS) to select discriminative features by conducting fuzziness learning and sparse learning, simultaneously. Specifically, we use projection matrix transform data as its low-dimensional representation, which are partitioned into clusters by using membership matrix with sparse constraint. In addition, $\ell _{2, 1}$ ℓ 2 , 1 -norm regularization is applied to the projection matrix. Then, a discriminative projection matrix with row sparse is obtained by perform fuzziness learning and sparse learning, simultaneously. An effective alternative optimization algorithm is proposed to solve the objective function. Evaluate experimental results on several real-world datasets show the effectiveness and superiority of the proposed unsupervised feature selection method. [ABSTRACT FROM AUTHOR]

Effectively recognizing different sceneries with complex backgrounds and varied lighting conditions plays an important role in modern AI systems. Competitive performance has recently been achieved by the deep scene categorization models. However, these models implicitly hypothesize that the image-level labels are 100% correct, which is too restrictive. Practically, the image-level labels for massive-scale scenery sets are usually calculated by external predictors such as ImageNet-CN. These labels can easily become contaminated because no predictors are completely accurate. This article proposes a new deep architecture that calculates scene categories by hierarchically deriving stable templates, which are discovered using a generative model. Specifically, we first construct a semantic space by incorporating image-level labels using subspace embedding. Afterward, it is noticeable that in the semantic space, the superpixel distributions from identically labeled images remain unchanged, regardless of the image-level label noises. On the basis of this observation, a probabilistic generative model learns the stable templates for each scene category. To deeply represent each scenery category, a novel aggregation network is developed to statistically concatenate the CNN features learned from scene annotations predicted by HSA. Finally, the learned deep representations are integrated into an image kernel, which is subsequently incorporated into a multiclass SVM for distinguishing scene categories. Thorough experiments have shown the performance of our method. As a byproduct, an empirical study of 33 SIFT-flow categories shows that the learned stable templates remain almost unchanged under a nearly 36% image label contamination rate. [ABSTRACT FROM AUTHOR]

The fusion of a low-spatial-resolution hyperspectral image (HSI) (LR-HSI) with its corresponding high-spatial-resolution multispectral image (MSI) (HR-MSI) to reconstruct a high-spatial-resolution HSI (HR-HSI) has been a significant subject in recent years. Nevertheless, it is still difficult to achieve the cross-mode information fusion of spatial mode and spectral mode when reconstructing HR-HSI for the existing methods. In this article, based on a convolutional neural network (CNN), an interpretable spatial–spectral reconstruction network (SSR-NET) is proposed for more efficient HSI and MSI fusion. More specifically, the proposed SSR-NET is a physical straightforward model that consists of three components: 1) cross-mode message inserting (CMMI); this operation can produce the preliminary fused HR-HSI, preserving the most valuable information of LR-HSI and HR-MSI; 2) spatial reconstruction network (SpatRN); the SpatRN concentrates on reconstructing the lost spatial information of LR-HSI with the guidance of spatial edge loss (Lspat); and 3) spectral reconstruction network (SpecRN); the SpecRN pays attention to reconstruct the lost spectral information of HR-MSI under the constraint of spatial edge loss ({Lspec). Comparative experiments are conducted on six HSI data sets of Urban, Pavia University (PU), Pavia Center (PC), Botswana, Indian Pines (IP), and Washington DC Mall (WDCM), and the proposed SSR-NET achieves the superior or competitive results in comparison with seven state-of-the-art methods. The code of SSR-NET is available at https://github.com/hw2hwei/SSRNET. [ABSTRACT FROM AUTHOR]

Graph clustering has achieved promising performance in various real-world applications, and attracted sufficient attention in machine learning. Generally, it encodes the samples' relationship with an affinity graph, and then conducts graph-theoretic optimization to partition the samples into clusters. The performance of the graph clustering methods may be affected by many factors, i.e., the graph quality, the loss measurement and the ad hoc post-processing. In this paper, a new Robust Doubly Stochastic graph clustering method (RDS) is presented, which has the following advantages: (1) it learns a doubly stochastic graph with the self-expression strategy automatically, and does not need the graph normalization step to improve the graph quality; (2) it utilizes a new loss function to calculate the approximation error, which is robust to the outliers that far from the normal samples; (3) it generates the cluster indicator according to the learned graph directly, such that the uncertainty caused by the post-processing procedure can be avoided. Extensive experiments on eight benchmarks demonstrate the effectiveness of RDS on data clustering, and show its advantages over the previous graph clustering methods are also verified. [ABSTRACT FROM AUTHOR]

The stope dynamic disaster, which occurs around the mining space and is represented by stress-type rock burst and fracture-type rock burst, seriously affects the safety production of coal mine. How to effectively forewarn rock burst risk to reduce the disaster caused by rock burst is an urgent problem to be solved in stope. In this paper, the spatiotemporal parameters that affect the occurrence of stope dynamic disaster are collected, and the big data of stope state is established. Due to the complex temporal and spatial parameters of rock bursts with different intensities occurrence and their varying degrees, it is difficult for some machine learning methods to excavate their internal relationships and make accurate warning. In this paper, (1) a big data platform is used to record and integrate the multiple parameters of stope dynamic disaster, and data preprocessing technology is used to de-noise and standardize them. (2) Based on the fused historical data, a strong classifier is iteratively found by the classification algorithm AdaBoost to identify the existence of the rock burst risk, so as to achieve the purpose of accurately and timely warning of rock burst risk. The research of this paper uses big data mining technology and machine learning method to carry out intelligent perception warning of the rock burst risk in real time. The experimental results show that the proposed method has a good effect and is of great significance to the prevention and control of rock burst disaster in stope. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection is a hot topic in the field of machine learning, which is more convenient and important because it does not require labeled data. Currently, unsupervised feature selection algorithms based on ℓ 2 , 1 -norm are very mature and have certain robustness to outliers, but they still have some problems that cannot be ignored. For example, the sparsity of ℓ 2 , 1 -norm is not ideal, they cannot achieve row sparsity but only element sparsity. Second, fine-tuning meaningless regularization parameters increases cost and makes it easy to fall into suboptimal solutions. To release the above problems, we propose an unsupervised feature selection algorithm via Sparse Robust Subspace Learning (SRSL), which combines reconstruction term and variance term so that the model simultaneously preserves reconstruction information and enhances separability. What is more, using the ℓ 2 , 0 -norm constraint on transformation matrix makes our model have a row-sparse property. In addition, we design the boolean weight so that the model not only eliminates outliers fundamentally to enhance robustness, but also achieves the effect of anomaly detection. To solve this NP-hard problem, we carefully design an optimization algorithm, which has strict convergence guarantees and obtains a closed-form solution. Experimental results on several real-world datasets demonstrate that our algorithm outperforms other comparison algorithms in both clustering and anomaly detection applications. [Display omitted] • A robust feature selection method based on reconstruction and variance is proposed. • Designing boolean weight to fundamentally eliminate outliers. • Designing an algorithm to solve ℓ 2 , 0 -norm constrained problem of feature selection. • Our algorithm has significant advantages in practical applications. [ABSTRACT FROM AUTHOR]

Cooperative behavior is essential for conflicts between collective and individual benefits, and evolutionary game theory provides a key framework to solve this problem. Decision-making of human or automata agent occurs not only in a static environment, but also in the dynamic interactive environment. Since the reinforcement learning algorithm is well performed at explaining the problem in regard to state, action, and environment, we propose a game model with individual state transition which is influenced by the self-regarding Q -learning algorithm. In detail, we at the first time investigate a two-state two-action game, where agents can choose either to participate in the network game (i.e., active agent) or to cut off all the links based on its Q -table (i.e., inactive agent), involving in a dynamic interactive environment. Through numerical simulations, it is shown that cooperation can reach the maximal level in the moderate value space of fixed reward obtained by inactive agents. In particular, long-term expectations and large learning rates are more productive in promoting cooperation. Furthermore, when the dynamic interactive environment reaches a stable state, the number of active neighbors of active cooperators is larger than that of active defectors, which is further larger than the number of active neighbors of inactive agents. Finally, we testify the results of theoretical analysis from the perspective of state transition. [ABSTRACT FROM AUTHOR]

• A novel nonlinear feature selective networks is designed. • Proposed method is applied for continuous values regression. • An optimizer joins back propagation and re-weighted is designed. The curse of dimensionality is a long-standing intractable issue in many machine learning and computer vision tasks. Feature selection, a data preprocessing technique, aims to select most discriminative feature subsets for improving performance of downstream machine learning tasks. However, most existing feature selection methods concentrate upon learning a linear relationship between data points and their labels, which causes that they are incapable of handling nonlinear complex data in real-world applications. In this paper, we first propose a novel end-to-end Nonlinear Feature Selective Networks (NFSN) that be able to select discriminative feature subsets while preserving their nonlinear structure by embedding a ℓ 2 , p -norm regularized hidden layer into designed continuous values regression networks. In addition, we propose an efficient optimization algorithm that joins back propagation algorithm and re-weighted optimization strategy to acquire derivative of all weights accurately. Experimental results on the nonlinear analog pulse signal and real-world datasets demonstrate the superiority of proposed method compared to some related methods on feature selection. Our source code available on: https://github.com/StevenWangNPU/NFSN. [ABSTRACT FROM AUTHOR]

• This paper proposes a method to replace the linear mapping-based spectral analysis with neural networks, which is more appropriate to learn the nonlinear mapping relationship between the original data and the pseudo cluster label. • Through constructing two different adaptive graph constraints, the mutual interaction between the NNs-based pseudo label learning module and the self-expression module is facilitated to explore the feature selection priority for the raw data. • Through a lot of well-designed experiments, the effectiveness of the method proposed in this paper has been fully proven. Unsupervised feature selection (UFS), which selects the most important feature subset and eliminates the unnecessary information for the upcoming data analysis, is a significant problem in machine learning and has been explored for years. Most UFS methods map features into a pseudo label space by multiplying a projection matrix constrained with sparsity to learn the mapping from the features to the labels. However, the mapping relationship is usually not linear, and linear regression may result in a suboptimal selection. To address this issue, we propose a novel UFS method, called neural networks embedded self-expression (NNSE). NNSE replaces the linear regression of traditional spectral analysis methods with neural networks to learn the pseudo label space. Besides, we embed neural networks into the self-expression model to improve the representative ability by preserving the local structure with an adaptive graph regularization module. Then we propose an efficient alternative iterative algorithm to solve the proposed model. Experimental results on 8 public datasets show NNSE outperforms the other state-of-the-art methods. Moreover, experimental results are also presented to show the convergence of the proposed method. The source code is available at: https://github.com/misteru/NNSE. [ABSTRACT FROM AUTHOR]