Search

The scarcity of labels is common and great challenge in traditional supervised learning. Semi-supervised learning (SSL) leverages unlabeled samples to alleviate the absence of label information. Similar with annotation, label proportion is another type of prior information and plays a significant role in classification tasks. Compared with the acquisition of labels, label proportion can be obtained more easily. For example, only a small number of patients have been diagnosed with or not with cancers in hospital database, while the proportion with cancer can be generally estimated by historical records. How to incorporate such prior information of label proportion is crucial but rarely studied in literature. Traditional SSL methods often ignore this prior information and will lead to performance degradation inevitably. To solve this problem, we propose a novel SSL with Label Proportion (SSLLP). Our approach encourages to preserve label consistency and label proportion by imposing the cardinality bound constraints. Our formulated problem equals to a mixed-integer constrained submodular minimization and it is difficult to be solved directly. Therefore, we transformed the original problem into a convex one by Lov $\acute{\text{a}}$ sz extension and designed an efficient solving algorithm. Extensive experimental results present the improved performance of our method over several state-of-the-art methods.

Multi-instance learning (MIL) has been extensively applied to various real tasks involving objects with bags of instances, such as in drugs and images. Previous studies on MIL assume that data are entirely complete. However, in many real tasks, the instance is fragmentary. In this article, we present probably the first study on multi-instance classification with fragmentary data. In our proposed framework, called fragmentary multi-instance classification (FIC), the fragmentary data are completed and the multi-instance classifier is learned jointly. To facilitate the integration between the completion and classifier learning, FIC establishes the weighting mechanism to measure the importance levels of different instances. To validate the compatibility of our framework, four typical MIL methods, including multi-instance support vector machine (MI-SVM), expectation maximization diverse density (EM-DD), citation-K nearest neighbors (Citation-KNNs), and MIL with discriminative bag mapping (MILDM), are embedded into the framework to obtain the corresponding FIC versions. As an illustration, an efficient solving algorithm is developed to address the problem for MI-SVM, together with the proof of convergence behavior. The experimental results on various types of real-world datasets demonstrate the effectiveness.

Traditional multi-view learning usually assumes each instance appears in all views. However, in real-world applications, it is not an uncommon case that a number of instances suffer from some view samples missing. How to effectively cluster this kind of partial multi-view data has attracted much attention. In this paper, we propose an incomplete multi-view clustering method, namely Multi-view Spectral Clustering with Incomplete Graphs (MSCIG), which connects processes of spectral embedding and similarity matrix completion to achieve better clustering performance. Specifically, MSCIG recovers missing entries of each similarity matrix based on multiplications of a common representation matrix and corresponding view-specific representation matrix, and in turn learns these representation matrices based on the complete similarity matrices. Besides, MSCIG adopts the p-th root integration strategy to incorporate losses of multiple views, which characterizes the contributions of different views. Moreover, we develop an iterative algorithm with proved convergence to solve the resultant problem of MSCIG, which updates the common representation matrix, view-specific representation matrices, similarity matrices, and view weights alternatively. We conduct extensive experiments on 9 benchmark datasets to compare the proposed algorithm with existing state-of-the-art incomplete multi-view clustering methods. Experimental results validate the effectiveness of the proposed algorithm.

Many feature extraction methods reduce the dimensionality of data based on the input graph matrix. The graph construction which reflects relationships among raw data points is crucial to the quality of resulting low-dimensional representations. To improve the quality of graph and make it more suitable for feature extraction tasks, we incorporate a new graph learning mechanism into feature extraction and add an interaction between the learned graph and the low-dimensional representations. Based on this learning mechanism, we propose a novel framework, termed as unsupervised single view feature extraction with structured graph (FESG), which learns both a transformation matrix and an ideal structured graph containing the clustering information. Moreover, we propose a novel way to extend FESG framework for multi-view learning tasks. The extension is named as unsupervised multiple views feature extraction with structured graph (MFESG), which learns an optimal weight for each view automatically without requiring an additional parameter. To show the effectiveness of the framework, we design two concrete formulations within FESG and MFESG, together with two efficient solving algorithms. Promising experimental results on plenty of real-world datasets have validated the effectiveness of our proposed algorithms.

Data recycling, which reuses the historical data to assist the present data to achieve better performance, is an emerging and important research topic. A common case is that historical examples only have features from one source while presently have more data collection ways and extract different types of features simultaneously for new examples. Previous studies assume that either historical data appear in all sources, or at least there is one type of representations for all data. In this paper, we study the challenging problem in the above common case and propose a novel semisupervised approach by leveraging nonoverlapping historical features (NHFs). It learns full representations of both historical features and present features in a latent subspace. We utilize the intrinsic geometrical structure of all data and add the label information of historical data as a hard constraint to discover a latent subspace. Then, the classification will be performed with these new representations. Moreover, we provide an efficient algorithm to solve the formulated optimization problem with proved convergence behavior, together with some insightful discussions about parameter determination. Experimental results on real-world data sets are provided to examine the effectiveness of our algorithm. Furthermore, we have also evaluated our method in face recognition. They all demonstrate the effectiveness of our proposed approach on recycling NHFs.

In many computer vision and machine learning applications, the data sets distribute on certain low-dimensional subspaces. Subspace clustering is a powerful technology to find the underlying subspaces and cluster data points correctly. However, traditional subspace clustering methods can only be applied on data from one source, and how to extend these methods and enable the extensions to combine information from various data sources has become a hot area of research. Previous multi-view subspace methods aim to learn multiple subspace representation matrices simultaneously and these learning task for different views are treated equally. After obtaining representation matrices, they stack up the learned representation matrices as the common underlying subspace structure. However, for many problems, the importance of sources and the importance of features in one source both can be varied, which makes the previous approaches ineffective. In this paper, we propose a novel method called Robust Auto-weighted Multi-view Subspace Clustering (RAMSC). In our method, the weight for both the sources and features can be learned automatically via utilizing a novel trick and introducing a sparse norm. More importantly, the objective of our method is a common representation matrix which directly reflects the common underlying subspace structure. A new efficient algorithm is derived to solve the formulated objective with rigorous theoretical proof on its convergency. Extensive experimental results on five benchmark multi-view datasets well demonstrate that the proposed method consistently outperforms the state-of-the-art methods.

Feature extraction, one kind of dimensionality reduction methodology, has aroused considerable research interests during the last few decades. Traditional graph embedding methods construct a fixed graph with original data to fulfill the aim of feature extraction. The lack of the graph learning mechanism leaves room for the improvement of their performances. In this paper, we propose a novel framework, termed as unsupervised feature extraction using a learned graph with clustering structure (LGCS), in which a graph learning mechanism has been presented. To be specific, the proposed LGCS learns both a transformation matrix and an ideal structured graph which incorporates clustering information. To show the effectiveness of the framework, we present a concrete method within our framework, and an iteration algorithm has been designed to solve the corresponding optimizing problem. Promising experimental results on real-world datasets have validated the effectiveness of our proposed algorithm.