Search

Conventional machine learning algorithms suffer the problem that the model trained on existing data fails to generalize well to the data sampled from other distributions. To tackle this issue, unsupervised domain adaptation (UDA) transfers the knowledge learned from a well-labeled source domain to a different but related target domain where labeled data is unavailable. The majority of existing UDA methods assume that data from the source domain and the target domain are available and complete during training. Thus, the divergence between the two domains can be formulated and minimized. In this paper, we consider a more practical yet challenging UDA setting where either the source domain data or the target domain data are unknown. Conventional UDA methods would fail this setting since the domain divergence is agnostic due to the absence of the source data or the target data. Technically, we investigate UDA from a novel view—adversarial attack—and tackle the divergence-agnostic adaptive learning problem in a unified framework. Specifically, we first report the motivation of our approach by investigating the inherent relationship between UDA and adversarial attacks. Then we elaborately design adversarial examples to attack the training model and harness these adversarial examples. We argue that the generalization ability of the model would be significantly improved if it can defend against our attack, so as to improve the performance on the target domain. Theoretically, we analyze the generalization bound for our method based on domain adaptation theories. Extensive experimental results on multiple UDA benchmarks under conventional, source-absent and target-absent UDA settings verify that our method is able to achieve a favorable performance compared with previous ones. Notably, this work extends the scope of both domain adaptation and adversarial attack, and expected to inspire more ideas in the community. [ABSTRACT FROM AUTHOR]

Spectral clustering (SC) has become one of the most widely-adopted clustering algorithms, and been successfully applied into various applications. We in this work explore the problem of spectral clustering in a lifelong learning framework termed as Generalized Lifelong Spectral Clustering (GL $^2$ 2 SC). Different from most current studies, which concentrate on a fixed spectral clustering task set and cannot efficiently incorporate a new clustering task, the goal of our work is to establish a generalized model for new spectral clustering tasks by “What” and “How” to lifelong learn from past tasks. In respect of “what to lifelong learn”, our GL $^2$ 2 SC framework contains a dual memory mechanism with a deep orthogonal factorization manner: an orthogonal basis memory stores hidden and hierarchical clustering centers among learned tasks, and a feature embedding memory captures deep manifold representation common across multiple related tasks. When learning a new clustering task, the intuition here for “how to lifelong learn” is that GL $^2$ 2 SC can transfer intrinsic knowledge from dual memory mechanism to obtain task-specific encoding matrix. Then the encoding matrix can redefine the dual memory over time to provide maximal benefits when learning future tasks, and reversely maximize performance for past tasks. To achieve this, we propose an alternative optimization formulation with convergence guarantee for solving our GL $^2$ 2 SC model. To the end, empirical comparisons on several benchmark datasets show the effectiveness of our GL $^2$ 2 SC, in comparison with several state-of-the-art clustering models. [ABSTRACT FROM AUTHOR]

Domain adaptation aims to exploit domain-invariant features by aligning the cross-domain distributions in the manifold subspace for applying the classifier trained on the source domain to the target domain. However, two limitations may still deteriorate their performances: (1) the influences of noisy or irrelevant features in the original feature space are ignored, which may unexpectedly hurt the classification of target samples; (2) the graph constructed directly in the original data space cannot accurately capture the inherent local manifold structures of high-dimensional data due to the curse of dimensionality, which may seriously mislead the transferable features learning. In this paper, we propose a novel approach to address these problems, referred to as joint Adaptive Dual Graph and Feature Selection for domain adaptation (ADGFS). Specifically, feature selection can characterize the relative importance of different features through a scaling factor, which enables ADGFS to not only reduce the impacts of noisy or irrelevant features on knowledge transfer but also learn informative domain-invariant features. Meanwhile, ADGFS adaptively optimizes the dual graph by learning the similarity matrices of both instance-level and feature-level graphs in the projected low-dimensional manifold subspace rather than the original high-dimensional space, such that the intrinsic local manifold structures of data can be captured precisely. Moreover, ADGFS simultaneously aligns the marginal and conditional probability distributions in the nonnegative matrix factorization framework to narrow the distribution discrepancies between the two different domains, which can adequately transfer knowledge from the source domain to the target domain. Comprehensive experiments on four benchmark datasets can demonstrate that the effectiveness of the proposed approach in cross-domain image classification. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation addresses the problem of transferring knowledge from a well-labeled source domain to an unlabeled target domain where the two domains have distinctive data distributions. Thus, the essence of domain adaptation is to mitigate the distribution divergence between the two domains. The state-of-the-art methods practice this very idea by either conducting adversarial training or minimizing a metric which defines the distribution gaps. In this paper, we propose a new domain adaptation method named adversarial tight match (ATM) which enjoys the benefits of both adversarial training and metric learning. Specifically, at first, we propose a novel distance loss, named maximum density divergence (MDD), to quantify the distribution divergence. MDD minimizes the inter-domain divergence (“match” in ATM) and maximizes the intra-class density (“tight” in ATM). Then, to address the equilibrium challenge issue in adversarial domain adaptation, we consider leveraging the proposed MDD into adversarial domain adaptation framework. At last, we tailor the proposed MDD as a practical learning loss and report our ATM. Both empirical evaluation and theoretical analysis are reported to verify the effectiveness of the proposed method. The experimental results on four benchmarks, both classical and large-scale, show that our method is able to achieve new state-of-the-art performance on most evaluations. [ABSTRACT FROM AUTHOR]

A real-world recommender usually adopts heterogeneous types of user feedbacks, for example, numerical ratings such as 5-star grades and binary ratings such as likes and dislikes. In this work, we focus on transferring knowledge from binary ratings to numerical ratings, facing a more serious data sparsity problem. Conventional Collective Factorization methods usually assume that there are shared user and item latent factors across multiple related domains, but may ignore the shared common knowledge of rating patterns. Furthermore, existing works may also fail to consider the hierarchical structures in the heterogeneous recommendation scenario (i.e., genre, sub-genre, detailed-category). To address these challenges, in this paper, we propose a novel Deep Low-rank Sparse Collective Factorization (DLSCF) framework for heterogeneous recommendation. Specifically, we adopt low-rank sparse decomposition to capture the common rating patterns in related domains while splitting the domain-specific patterns. We also factorize the model in multiple layers to capture the affiliation relation between latent categories and sub-categories. We propose both batch and Stochastic Gradient Descent (SGD) based optimization algorithms for solving DLSCF. Experimental results on MoviePilot, Netfilx, Flixter, MovieLens10M and MovieLens20M datasets demonstrate the effectiveness of the proposed algorithms, by comparing them with several state-of-the-art batch and SGD based approaches. [ABSTRACT FROM AUTHOR]

• Firstly, we consider a new yet practical DA scenario, called sparsely-labeled source assisted domain adaptation. • Secondly, we propose a unified framework to jointly seek cluster centroids, source and target labels, and domain-invariant features. • Thirdly, we construct an optimization strategy to solve the objective function efficiently. Domain Adaptation (DA) aims to generalize the classifier learned from a well-labeled source domain to an unlabeled target domain. Existing DA methods usually assume that rich labels could be available in the source domain. However, we usually confront the source domain with a large number of unlabeled data but only a few labeled data, and thus, how to transfer knowledge from this sparsely-labeled source domain to the target domain is still a challenge, which greatly limits its application in the wild. This paper proposes a novel Sparsely-Labeled Source Assisted Domain Adaptation (SLSA-DA) algorithm to address the challenge with limited labeled source domain samples. Specifically, due to the label scarcity problem, the projected clustering is first conducted on both the source and target domains, so that the discriminative structures of data could be exploited elegantly. Then label propagation is adopted to propagate the labels from those limited labeled source samples to the whole unlabeled data progressively, so that the cluster labels are revealed correctly. Finally, we jointly align the marginal and conditional distributions to mitigate the cross-domain mismatching problem, and optimize those three procedures iteratively. However, it is nontrivial to incorporate the above three procedures into a unified optimization framework seamlessly since some variables to be optimized are implicitly involved in their formulas, thus they could not benefit to each other. Remarkably, we prove that the projected clustering and conditional distribution alignment could be reformulated into other formulations, thus the implicit variables are embedded in different optimization steps. As such, the variables related to those three quantities could be optimized in a unified optimization framework and benefit to each other, and improve the recognition performance obviously. Extensive experiments have verified that our approach could deal with the challenge in the SLSA-DA setting, and achieve the best performances across different real-world cross-domain visual recognition tasks. Our preliminary Matlab code is available at https://github.com/WWLoveTransfer/SLSA-DA/. [ABSTRACT FROM AUTHOR]