Search

Contrastive learning predicts whether two images belong to the same category by training a model to make their feature representations as close (positive samples) or as far away as possible (negative samples). Selecting appropriate samples is critical to effectively train a model, however, existing methods suffer from false or uninformative sample problems. This paper rethinks how to mine samples in contrastive learning and the proposed method is more comprehensive. It takes into account both positive and negative samples, and mines potential samples from two aspects. First, for positive samples, this paper incorporates both the augmented sample views and the mined sample views. A weighted combination of these positive samples is achieved by using both hard and soft weighting strategies simultaneously. Second, considering the existence of false and uninformative negative samples, this paper analyzes the negative samples from the perspective of gradient and mines negative samples that are neither too difficult nor too easy as potential negative samples, i.e., those negative samples that are close to positive samples. Compared with previous state-of-the-art self-supervised methods, experiments show the obvious advantages of the proposed method, and the corresponding top-1 accuracies of linear classification are improved by 0.77%, 2.39%, and 1.01% on CIFAR10, CIFAR100, and TinyImageNet, respectively. Source code and pretrained models are available at https://github.com/dhkdhk/PSM. [ABSTRACT FROM AUTHOR]

Self-supervised methods based on contrastive learning have achieved great success in unsupervised visual representation learning. However, most methods under this framework suffer from the problem of false negative samples. Inspired by the mean shift for self-supervised learning, we propose a new simple framework, namely Multiple Sample Views and Queues (MSVQ). We jointly construct three soft labels on-the-fly by utilizing two complementary and symmetric approaches: multiple augmented positive views and two momentum encoders that generate various semantic features for negative samples. Two teacher networks perform similarity relationship calculations with negative samples and then transfer this knowledge to the student network. Let the student network mimic the similarity relationships between the samples, thus giving the student network a more flexible ability to identify false negative samples in the dataset. The classification results on four benchmark image datasets demonstrate the high effectiveness and efficiency of our approach compared to some classical methods. Source code and pretrained models are available here. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) has been proposed as a new algorithm for training single hidden layer feed forward neural networks. The main merit of ELM lies in the fact that the input weights as well as hidden layer bias are randomly generated and thus the output weights can be obtained analytically, which can overcome the drawbacks incurred by gradient-based training algorithms such as local optima, improper learning rate and low learning speed. Based on the consistency property of data, which enforces similar samples to share similar properties, we propose a discriminative graph regularized Extreme Learning Machine (GELM) for further enhancing its classification performance in this paper. In the proposed GELM model, the label information of training samples are used to construct an adjacent graph and correspondingly the graph regularization term is formulated to constrain the output weights to learn similar outputs for samples from the same class. The proposed GELM model also has a closed form solution as the standard ELM and thus the output weights can be obtained efficiently. Experiments on several widely used face databases show that our proposed GELM can achieve much performance gain over standard ELM and regularized ELM. Moreover, GELM also performs well when compared with the state-of-the-art classification methods for face recognition. [ABSTRACT FROM AUTHOR]

Reservoir computing (RC) is a novel class of recurrent neural networks (RNN) models. Nonlinear spiking neural P (NSNP) systems are neural-like computing models with nonlinear spiking mechanisms. By introducing NSNP systems as the reservoir, we propose a new RC model for time series classification task, termed TSC-NSNP model. However, due to the high-dimensional nature of the reservoir state space, the TSC-NSNP model, like existing RC models, will encounter some challenges. To address the challenges. we utilize the reservoir model space representation and dimensionality reduction method to propose two improved models, termed TSC-DR-NSNP model and TSC-RMS-NSNP model. The three RC models can be easily realized and learnt in the RC framework. The proposed three RC models are evaluated on 21 benchmark time series classification data sets, and are compared with 20 classification models. The comparisons demonstrate the effectiveness of the presented three RC models for time series classification tasks. [ABSTRACT FROM AUTHOR]

• The matrix completion is firstly applied to weakly supervised LDL problems. • Smoothness assumption is utilized to capture the correlation among samples. • Our transductive LDL algorithm outperforms better than existing algorithms. Label distribution learning (LDL) is one of the paradigms for dealing with label ambiguity, and it can learn the relative importance of each label to a particular instance. Most of the existing LDL approaches require strong supervision information, which always involves high cost of the data-labeling process. To this end, we propose a novel transductive weakly supervised LDL algorithm based on matrix completion, which simultaneously explores the discriminant knowledge of both training instances and testing instances. Moreover, the manifold regularization is exploited to capture the samples' relevance to enhance the performance of matrix completion. Experimental results on real-world data sets with various missing percentages validate the effectiveness of our method. [ABSTRACT FROM AUTHOR]