Search

In this paper, we propose two GAN-based deep subspace clustering approaches: deep subspace clustering via dual adversarial generative networks (DSC-DAG) and self-supervised deep subspace clustering with adversarial generative networks (S $^2$ 2 DSC-AG). In DSC-DAG, the distributions of both the inputs and corresponding latent representations are learnt via adversarial training simultaneously. Besides, there are two kinds of synthetic representations to facilitate the fine-tuning of the encoder: the combinations of latent representations with random combination coefficients and representations of real-like inputs derived from noise variables. In S $^2$ 2 DSC-AG, a self-supervised information learning module substitutes for adversarial learning in the latent space, since both of them play the same role in learning discriminative latent representations. We analyze connections between these methods and demonstrate their equivalences. We conduct extensive experiments on multiple real-world data sets against state-of-the-art subspace clustering methods in terms of accuracy, normalized mutual information and purity. Experimental results demonstrate the effectiveness and superiority of our proposed methods. [ABSTRACT FROM AUTHOR]

Deep learning demonstrates its impressive success across various machine learning problems. However, its performance often suffers in the case where the training and test data sets follow different distributions, due to the domain shift. Most current domain adaptation methods minimize the discrepancy between the source and target domains by enforcing the alignment of their marginal distributions without considering the class-level matching. Consequently, data from different classes may become close together after mapping. To address this issue, we propose an unsupervised domain adaptation method by generating image-label pairs in the target domain, in which the model is augmented with the generated target pairs and achieve class-level transfer. Specifically, we integrate generative adversarial networks (GAN) into the model predictor, where the generator fed with labels aims to produce corresponding target domain images with a well-designed semantic loss. Meanwhile, compared to previous methods which focus on discrepancy reduction across domains, i.e., image to image translation, our model focuses on semantic preservation during image generation. Our model is straightforward yet effective for unsupervised domain adaptation problems. Without any labels in the target domain in all the experiments, we demonstrate the validity of our approach by presenting the plausible generated target image-label pairs. In addition, our proposed method achieves the best or comparable performance on multiple unsupervised domain adaptation datasets which include image classification and semantic segmentation. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) receive extensive attention due to its capability of high-fidelity image synthesis. The synthesized content can be edited by imposing transformations on the latent codes. To edit real images, the GAN inversion methods mainly focus on learning an inverse mapping from the data space to the latent space of a well-trained GAN through a separate encoder. In this paper, we propose a Discriminator Feature-based Progressive Inversion (DFPI) model for GAN-based image reconstruction and enhancement. We find that the expressiveness of the generator feature is richer than that of the latent code for a given input image. To achieve high-quality reconstruction, we estimate the generator features in a light-weight optimization process, conditioned on off-the-shelf discriminator features. Specifically, a recomposition matrix is learnt to map high-level discriminator features to the high-level generator features due to their reverse structure. To progressively improve the reconstruction quality, we adopt a convolutional correction module to alleviate the estimation error at a lower generator layer, conditioned on the features of a lower discriminator layer. By performing the progressive correction process, the input images can be accurately reconstructed. Note that the resulting recomposition matrix and correction module can be generalized to unseen images. The experiments demonstrate the superior performance of DFPI over competing methods on GAN inversion as well as a variety of image enhancement tasks. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) have brought great progress in image-to-image translation. The problem that we focus on is how to accurately extract and transfer the makeup style from a reference facial image to a target face. We propose a GAN-based generative model with Target-aware makeup Style Encoding and Verification, which is referred to as TSEV-GAN. This design is due to the following two insights: (a) When directly encoding the reference image, the encoder may focus on regions which are not necessarily important or desirable. To precisely capture the style, we encode the difference map between the reference and corresponding de-makeup images, and then inject the obtained style code into a generator. (b) A generic real-fake discriminator cannot ensure the correctness of the rendered makeup pattern. In view of this, we impose style representation learning on a conditional discriminator. By identifying style consistency between the reference and synthesized images, the generator is induced to precisely replicate the desirable makeup. We perform extensive experiments on the existing makeup benchmarks to verify the effectiveness of our improvement strategies in transferring a variety of makeup styles. Moreover, the proposed model is able to outperform other existing state-of-the-art makeup transfer methods in terms of makeup similarity and irrelevant content preservation. [ABSTRACT FROM AUTHOR]

• We exploit the Siamese Network to find a clustering-friendly embedding space to mine highly-reliable pseudo-supervised information for the application of VAT and Conditional-GAN to synthesize cluster-specific samples in the setting of unsupervised learning. • We proposed adopting VAT to synthesize samples with different levels of perturbations that can enhance the robustness of Feature Extractor to noise and improve the lower-dimensional latent coding space discovered by the Feature Extractor. • We conducted experiments to verify that the latent space discovered by the Feature Extractor can facilitate the Siamese Network to find a clustering-friendly embedding space and extract pseudo-supervised information for VAT and Conditional-GAN. • The training of our EDCN involves the adversarial gaming between three players, which not only boosts performance improvement of the clustering but also preserves the cluster-specific information from the Siamese Network in synthesizing samples. In this paper, we propose an enhanced deep clustering network (EDCN), which is composed of a Feature Extractor, a Conditional Generator, a Discriminator and a Siamese Network. Specifically, we will utilize two kinds of generated data based on adversarial training, as well as the original data, to train the Feature Extractor for learning effective latent representations. In addition, we adopt the Siamese network to find an embedding space, where a better affinity similarity matrix is obtained as the key to success of spectral clustering in providing reliable pseudo-labels. Particularly, the obtained pseudo-labels will be used to generate realistic data by the Generator. Finally, the discriminator is used to model the real joint distribution of data and corresponding latent representations for Feature Extractor enhancement. To evaluate our proposed EDCN, we conduct extensive experiments on multiple data sets including MNIST, USPS, FRGC, CIFAR-10, STL-10, and Fashion-MNIST by comparing our method with a number of state-of-the-art deep clustering methods, and experimental results demonstrate its effectiveness and superiority. [ABSTRACT FROM AUTHOR]

• We propose a residual generator (Rg-GAN) served as a better approximation divergence minimization framework for GAN, and prove that residual generator for standard and least-squares GAN are equivalent to the minimization of reverse-KL and a new instance of f-divergence, respectively. • We prove that Rg-GAN can be reduced to IPMs based GAN and bridge the gap between IPMs and f-divergence. • We propose a new loss function for the discriminator of Rg-GAN that manifests a better discriminative property and therefore improved on Rg-GAN generalisation ability. • We conduct experiments on multiple benchmark data sets and demonstrate that our proposed framework can mitigate the mode collapse issue and facilitate GAN to generate higher-quality images with negligible additional computation cost. GAN is a generative modelling framework which has been proven as able to minimise various types of divergence measures under an optimal discriminator. However, there is a gap between the loss function of GAN used in theory and in practice. In theory, the proof of the Jensen divergence minimisation involves the min-max criterion, but in practice the non-saturating criterion is instead used to avoid gradient vanishing. We argue that the formulation of divergence minimization via GAN is biased and may yield a poor convergence of the algorithm. In this paper, we propose the Residual Generator for GAN (Rg-GAN), which is inspired by the closed-loop control theory, to bridge the gap between theory and practice. Rg-GAN minimizes the residual between the loss of the generated data to be real and the loss of the generated data to be fake from the perspective of the discriminator. In this setting, the loss terms of the generator depend only on the generated data and therefore contribute to the optimisation of the model. We formulate the residual generator for standard GAN and least-squares GAN and show that they are equivalent to the minimisation of reverse-KL divergence and a novel instance of f-divergence, respectively. Furthermore, we prove that Rg-GAN can be reduced to Integral Probability Metrics (IPMs) GANs (e.g., Wasserstein GAN) and bridge the gap between IPMs and f-divergence. Additionally, we further improve on Rg-GAN by proposing a loss function for the discriminator that has a better discrimination ability. Experiments on synthetic and natural images data sets show that Rg-GAN is robust to mode collapse, and improves the generation quality of GAN in terms of FID and IS scores. [ABSTRACT FROM AUTHOR]