Search

Semantic segmentation, which aims to acquire pixel-level understanding about images, is among the key components in computer vision. To train a good segmentation model for real-world images, it usually requires a huge amount of time and labor effort to obtain sufficient pixel-level annotations of real-world images beforehand. To get rid of such a nontrivial burden, one can use simulators to automatically generate synthetic images that inherently contain full pixel-level annotations and use them to train a segmentation model for the real-world images. However, training with synthetic images usually cannot lead to good performance due to the domain difference between the synthetic images (i.e., source domain) and the real-world images (i.e., target domain). To deal with this issue, a number of unsupervised domain adaptation (UDA) approaches have been proposed, where no labeled real-world images are available. Different from those methods, in this work, we conduct a pioneer attempt by using easy-to-collect image-level annotations for target images to improve the performance of cross-domain segmentation. Specifically, we leverage those image-level annotations to construct curriculums for the domain adaptation problem. The curriculums describe multi-level properties of the target domain, including label distributions over full images, local regions and single pixels. Since image annotations are “weak” labels compared to pixel annotations for segmentation, we coin this new problem as weakly-supervised cross-domain segmentation. Comprehensive experiments on the $\texttt {GTA5}\rightarrow \texttt {Cityscapes}$ and $\texttt {SYNTHIA}\rightarrow \texttt {Cityscapes}$ settings demonstrate the effectiveness of our method over the existing state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

• We propose to address ZSL as a standard domain adaptation task. • A novel domain-specific batch normalization component is proposed to align domain gaps in semantic-visual fusions. • Extensive experiments show the effectiveness and robustness of our approach. Zero-Shot Learning (ZSL) learns models for recognizing new classes. One of the main challenges in ZSL is the domain discrepancy caused by the category inconsistency between training and testing data. Domain adaptation is the most intuitive way to address this challenge. However, existing domain adaptation techniques cannot be directly applied into ZSL due to the disjoint label space between source and target domains. This work proposes the Transferrable Semantic-Visual Relation (TSVR) approach towards transductive ZSL. TSVR redefines image recognition as predicting the similarity/dissimilarity labels for semantic-visual fusions consisting of class attributes and visual features. After the above transformation, the source and target domains can have the same label space, which hence enables to quantify domain discrepancy. For the redefined problem, the number of similar semantic-visual pairs is significantly smaller than that of dissimilar ones. To this end, we further propose to use Domain-Specific Batch Normalization to align the domain discrepancy. [ABSTRACT FROM AUTHOR]

Abstract The recent successes of deep neural networks heavily rely on massive amounts of data annotations. For a certain task that is short of labeled data, the cost of large-scale data annotation is usually prohibitively expensive. Through transferring the already learnt knowledge from a different but related source domain that is sufficiently labeled, domain adaptation aims to relieve such a burden for the target domain. The recent studies reveal that learning domain-invariant features has gained great successes in unsupervised domain adaptation. However, domain invariance does not necessarily imply discriminative representations in the target domain due to the lack of labeled target data. To address this issue, we propose a new unsupervised domain adaptation method named TarGAN, which can successfully disentangle the class and the style codes of the target generator and generate Tar get samples with given class labels based on the recent advances of G enerative A dversarial N etworks. This is achieved by tying the high-level layers of the source generator and the target generator, and concurrently enforcing high mutual information between the class code and the generated images. Accompanied with given class labels, these generated target images can significantly improve the classifier's discriminative ability on the target domain. Experiments on several standard benchmarks demonstrate that our proposed TarGAN can outperform the existing state-of-the-art domain adaptation methods. [ABSTRACT FROM AUTHOR]