Search

Despite the considerable advances in deep learning for object recognition, there are still several factors that hinder the performance of deep learning models. One of these factors is domain shift, which occurs due to variations in the distribution of the testing and training data. This paper addresses the issue of compact feature clustering in domain generalization, with the aim of optimizing the embedding space from multi-domain data. Specifically, we propose a domain-aware triplet loss for domain generalization, which not only facilitates clustering of similar semantic features but also disperses features that arise from the domain. Unlike previous methods that focus on aligning distributions, our algorithm disperses domain information in the embedding space. Our approach is based on the assumption that embedding features can be clustered based on domain information, which is supported mathematically and empirically in this paper. Furthermore, in our investigation of feature clustering in domain generalization, we observe that the factors that influence the convergence of metric learning loss in domain generalization are more significant than the pre-defined domains. To address this issue, we utilize two methods to normalize the embedding space and reduce the internal covariate shift of the embedding features. Our ablation study illustrates the effectiveness of our algorithm. Additionally, our experiments on benchmark datasets, including PACS, VLCS, and Office-Home, demonstrate that our method outperforms related approaches that focus on domain discrepancy. Notably, our results on RegnetY-16GF are substantially better than state-of-the-art methods on the benchmark datasets. Our code is available at https://github.com/workerbcd/DCT. • We find that the feature embeddings may be clustered due to the domain gaps. • We visualize this domain clustering phenomenon and support it theoretically. • We propose a domain-aware triplet (D-Triplet) loss to generalize the model • D-Triplet loss can efficiently mitigate the domain-cluster issue on feature level. • We utilize various normalization techniques to prompt the convergence of D-Triplet. • The experimental results validate the effectiveness of our method. [Display omitted] [ABSTRACT FROM AUTHOR]

Generalizing to out-of-distribution (OOD) data is a challenging task for existing deep learning approaches. This problem largely comes from the common but often incorrect assumption of statistical learning algorithms that the source and target data come from the same i.i.d. distribution. To tackle the limited variability of domains available during training, as well as domain shifts at test time, numerous approaches for domain generalization have focused on generating samples from new domains. Recent studies on this topic suggest that feature statistics from instances of different domains can be mixed to simulate synthesized images from a novel domain. While this simple idea achieves state-of-art results on various domain generalization benchmarks, it ignores structural information which is key to transferring knowledge across different domains. In this paper, we leverage the ability of humans to recognize objects using solely their structural information (prominent region contours) to design a Structural-Aware Feature Stylization method for domain generalization. Our method improves feature stylization based on mixing instance statistics by enforcing structural consistency across the different style-augmented samples. This is achieved via a multi-task learning model which classifies original and augmented images while also reconstructing their edges in a secondary task. The edge reconstruction task helps the network preserve image structure during feature stylization, while also acting as a regularizer for the classification task. Through quantitative comparisons, we verify the effectiveness of our method upon existing state-of-the-art methods on PACS, VLCS, OfficeHome, DomainNet and Digits-DG. The implementation is available at this repository. • Feature stylization alone led to losing structural domain-invariant information. • Image contours contain important domain-agnostic structural information. • Enforcing structural consistency while stylization boost model's generalization [ABSTRACT FROM AUTHOR]

In this paper, we tackle the problem of detailed hand-object 3D reconstruction from monocular video with unknown objects , for applications where the required accuracy and level of detail is important, e.g. object hand-over in human–robot collaboration, or manipulation and contact point analysis. While the recent literature on this topic is promising, the accuracy and generalization abilities of existing methods are still lacking. This is due to several limitations, such as the assumption of known object class or model for a small number of instances, or over-reliance on off-the-shelf keypoint and structure-from-motion methods for object-relative viewpoint estimation, prone to complete failure with previously unobserved, poorly textured objects or hand-object occlusions. To address previous method shortcomings, we present a 2-stage pipeline superseding state-of-the-art (SotA) performance on several metrics. First, we robustly retrieve viewpoints relying on a learned pairwise camera pose estimator trainable with a low data regime, followed by a globalized Shonan pose averaging. Second, we simultaneously estimate detailed 3D hand-object shapes and refine camera poses using a differential renderer-based optimizer. To better assess the out-of-distribution abilities of existing methods, and to showcase our methodological contributions, we introduce the new SHOWMe benchmark dataset with 96 sequences annotated with poses, millimetric textured 3D shape scans, and parametric hand models, introducing new object and hand diversity. Remarkably, we show that our method is able to reconstruct 100% of these sequences as opposed to SotA Structure-from-Motion (SfM) or hand-keypoint-based pipelines, and obtains reconstructions of equivalent or better precision when existing methods do succeed in providing a result. We hope these contributions lead to further research under harder input assumptions. The dataset can be downloaded at https://download.europe.naverlabs.com/showme. • Object-agnostic hand-object 3D reconstruction from monocular hand-object motion video • Robust rigid-transformation estimation network that leverages large pre-trained model • Two-stage pipeline for 3D hand-object reconsruction • New hand-object dataset to benchmark hand-object 3D reconstruction • Rigid-transformation estimation performance analysis v/s object sizes and textures [ABSTRACT FROM AUTHOR]

Deep metric learning aims to create a feature space where the projected samples have maximised inter-class and minimised intra-class distance. Most approaches employ only distance-based metrics to achieve this objective, but neglect other properties of the projections in the embedding space, such as their density, sparsity and presence of outliers. In this paper, we propose a novel density-based regulariser, LOFReg, designed to be used as a complement to previously proposed distance-based metric learning loss functions for the re-identification (ReID) and few-shot classification (FSC) tasks. Our method is based on the well-known, in anomaly detection literature, local outlier factor (LOF) algorithm, which estimates the local density deviation of a data point with respect to its neighbours. These measurements are used in our regularisation methodology to achieve an embedding space of evenly distributed samples and to increase the generalisation ability of the model compared to solely distance-based learning. Comprehensive experiments on four publicly available datasets for ReID and FSC, demonstrate consistent improvement against previously proposed metric learning loss functions. Particularly, our experiments show up to 5% improvement in ReID settings, up to 13% in FSC settings and up to 6% improvement against other state-of-art regularisers. • A novel density regularisation loss function for deep metric learning is proposed. • For the first time outlier detection is used in deep metric learning context. • LOFReg reduces over-fitting and increase generalisation in ReID and FSC settings. • Consistent improvement is achieved against 6 SoA loss functions and 4 datasets. • LOFReg improves against other recent state-of-art DML regularisers. [ABSTRACT FROM AUTHOR]

Scene text erasing seeks to erase text contents from scene images and current state-of-the-art text erasing models are trained on large-scale synthetic data. Although data synthetic engines can provide vast amounts of annotated training samples, there are differences between synthetic and real-world data. In this paper, we employ self-supervision for feature representation on unlabeled real-world scene text images. A novel pretext task is designed to keep consistent among text stroke masks of image variants. We design the Progressive Erasing Network in order to remove residual texts. The scene text is erased progressively by leveraging the intermediate generated results which provide the foundation for subsequent higher quality results. Experiments show that our method significantly improves the generalization of the text erasing task and achieves state-of-the-art performance on public benchmarks. • A novel framework with self-supervision for text erasing. • A new pretext task for text erasing based on text stroke masks from image variants. • The proposed method performs in a progressive manner and clearly boosts the results. • Superiority of the framework on public benchmarks compared with previous methods. [ABSTRACT FROM AUTHOR]

This paper proposes a tree-structured structure-from-motion (SfM) method that recovers 3D scene structures and estimates camera poses from unordered image sets. Starting from atomic structures spanning the scene, we build well-connected structure groups, and propose RANSAC generalized Procrustes analysis (RGPA) to glue structures in the same group. The grouping-aligning operations hierarchically proceed until the full scene is reconstructed. Our work is the first attempt of using GPA for modern 3D reconstruction tasks. RGPA is able to merge multiple structures at a time and automatically identify outliers. The reconstruction tree is much more compact and balanced than previous hierarchical SfM methods and has a very shallow depth. These advantages, along with the resulting removal of intermediate bundle adjustments, lead to significantly improved computational efficiency over state-of-the-art SfM methods. The cameras and 3D scene can be robustly recovered in the presence of moderate noise. We verify the efficacy of our method on a variety of datasets, and demonstrate that our method is able to produce metric reconstructions efficiently and robustly. [ABSTRACT FROM AUTHOR]

Intrinsic image decomposition is the decomposition of an image into its reflectance and shading components. The intrinsic image decomposition problem is inherently ill-posed, since there can be multiple solutions to compute the intrinsic components forming the same image. In this paper, we explore the use of physics-based priors. We also propose a new architecture that separates the learning components in a stacked manner. We explore various ways of integrating such priors into a deep learning system. Our method is trained and tested on a large synthetic garden dataset to assess its performance. It is evaluated and compared to state-of-the-art methods using two standard intrinsic datasets. Finally, the pre-trained network is tested on real world images to show the generalisation capabilities of the network. • A (generalised) physics-based cue to steer a CNN model by an integrated IID pipeline is proposed. • Reflectance cues are exploited for shading computation through a stacked approach. • Separating the intrinsic components for the learning phase is shown to be beneficial. • Proposed method has generalization capacity to unseen domains & can cope with dataset bias problem. [ABSTRACT FROM AUTHOR]

LP relaxation based message passing and flow-based algorithms are two of the popular techniques for performing MAP inference in graphical models. Generic Cuts (GC) (Arora et al., 2015) combines the two approaches to generalize the traditional max-flow min-cut based algorithms for binary models with higher order clique potentials. The algorithm has been shown to be significantly faster than the state of the art algorithms. The time and memory complexities of Generic Cuts are linear in the number of constraints, which in turn is exponential in the clique size. This limits the applicability of the approach to small cliques only. In this paper, we propose a lazy version of Generic Cuts exploiting the property that in most of such inference problems a large fraction of the constraints are never used during the course of minimization. We start with a small set of constraints (called the active constraints) which are expected to play a role during the minimization process. GC is then run with this reduced set allowing it to be efficient in time and memory. The set of active constraints is adaptively learnt over multiple iterations while guaranteeing convergence to the optimum for submodular clique potentials. Our experiments show that the number of constraints required by the algorithm is typically less than 3% of the total number of constraints. Experiments on computer vision datasets show that our approach can significantly outperform the state of the art both in terms of time and memory and is scalable to clique sizes that could not be handled by existing approaches. [ABSTRACT FROM AUTHOR]

Domain generalization has recently generated increasing attention in machine learning in that it tackles the challenging out-of-distribution problem. The huge domain shift from source domains to target domains induces great uncertainty in making predictions on the target domains to which no data is accessible during learning. In this paper, we propose meta conditional variational auto-encoder (Meta-CVAE), a new meta probabilistic latent variable framework for domain generalization. The Meta-CVAE can better model the uncertainty across domains by inheriting the strong ability of probabilistic modeling from VAE. By leveraging the meta-learning framework to mimic the generalization from source to target domains during learning, our Meta-CVAE learns to acquire the capability of generalization by episodically transferring knowledge across domains. Meta-CVAE is optimized with a variational objective based on a newly derived evidence lower bound under the meta-learning setting. To further enhance prediction performance, we develop the Wasserstein Meta-CVAE by imposing a Wasserstein distance based discriminative constraint on the latent representations, which essentially separate different classes in the semantic space. Extensive experiments on diverse benchmarks demonstrate that our methods outperforms previous approaches consistently, and comprehensive ablation studies further validate its effectiveness on domain generalization. [Display omitted] • We propose a new meta conditional variational auto-encoder model for domain generalization. • We develop a meta evidence lower bound for optimization to learn a space invariant to domain. • We further introduce Wasserstein distance into our Meta-CVAE model to improve performance. [ABSTRACT FROM AUTHOR]

The Bag of Words paradigm has been the baseline from which several successful image classification solutions were developed in the last decade. These represent images by quantizing local descriptors and summarizing their distribution. The quantization step introduces a dependency on the dataset, that even if in some contexts significantly boosts the performance, severely limits its generalization capabilities. Differently, in this paper, we propose to model the local features distribution with a multivariate Gaussian, without any quantization. The full rank covariance matrix, which lies on a Riemannian manifold, is projected on the tangent Euclidean space and concatenated to the mean vector. The resulting representation, a Gaussian of Local Descriptors (GOLD), allows to use the dot product to closely approximate a distance between distributions without the need for expensive kernel computations. We describe an image by an improved spatial pyramid, which avoids boundary effects with soft assignment: local descriptors contribute to neighboring Gaussians, forming a weighted spatial pyramid of GOLD descriptors. In addition, we extend the model leveraging dataset characteristics in a mixture of Gaussian formulation further improving the classification accuracy. To deal with large scale datasets and high dimensional feature spaces the Stochastic Gradient Descent solver is adopted. Experimental results on several publicly available datasets show that the proposed method obtains state-of-the-art performance. [ABSTRACT FROM AUTHOR]

In the object recognition community, much effort has been spent on devising expressive object representations and powerful learning strategies for designing effective classifiers, capable of achieving high accuracy and generalization. In this scenario, the focus on the training sets has been historically weak; by and large, training sets have been generated with a substantial human intervention, requiring considerable time. In this paper, we present a strategy for automatic training set generation. The strategy uses semantic knowledge coming from WordNet, coupled with the statistical power provided by Google Ngram, to select a set of meaningful text strings related to the text class-label ( e.g. , “cat”), that are subsequently fed into the Google Images search engine, producing sets of images with high training value. Focusing on the classes of different object recognition benchmarks (PASCAL VOC 2012, Caltech-256, ImageNet, GRAZ and OxfordPet), our approach collects novel training images, compared to the ones obtained by exploiting Google Images with the simple text class-label. In particular, we show that the gathered images are better able to capture the different visual facets of a concept, thus encoding in a more successful manner the intra-class variance. As a consequence, training standard classifiers with this data produces performances not too distant from those obtained from the classical hand-crafted training sets. In addition, our datasets generalize well and are stable, that is, they provide similar performances on diverse test datasets. This process does not require manual intervention and is completed in a few hours. [ABSTRACT FROM AUTHOR]

Abstract: The median graph has been shown to be a good choice to obtain a representative of a set of graphs. However, its computation is a complex problem. Recently, graph embedding into vector spaces has been proposed to obtain approximations of the median graph. The problem with such an approach is how to go from a point in the vector space back to a graph in the graph space. The main contribution of this paper is the generalization of this previous method, proposing a generic recursive procedure that permits to recover the graph corresponding to a point in the vector space, introducing only the amount of approximation inherent to the use of graph matching algorithms. In order to evaluate the proposed method, we compare it with the set median and with the other state-of-the-art embedding-based methods for the median graph computation. The experiments are carried out using four different databases (one semi-artificial and three containing real-world data). Results show that with the proposed approach we can obtain better medians, in terms of the sum of distances to the training graphs, than with the previous existing methods. [Copyright &y& Elsevier]

Abstract: This paper presents some new approaches for computing graph prototypes in the context of the design of a structural nearest prototype classifier. Four kinds of prototypes are investigated and compared: set median graphs, generalized median graphs, set discriminative graphs and generalized discriminative graphs. They differ according to (i) the graph space where they are searched for and (ii) the objective function which is used for their computation. The first criterion allows to distinguish set prototypes which are selected in the initial graph training set from generalized prototypes which are generated in an infinite set of graphs. The second criterion allows to distinguish median graphs which minimize the sum of distances to all input graphs of a given class from discriminative graphs, which are computed using classification performance as criterion, taking into account the inter-class distribution. For each kind of prototype, the proposed approach allows to identify one or many prototypes per class, in order to manage the trade-off between the classification accuracy and the classification time. Each graph prototype generation/selection is performed through a genetic algorithm which can be specialized to each case by setting the appropriate encoding scheme, fitness and genetic operators. An experimental study performed on several graph databases shows the superiority of the generation approach over the selection one. On the other hand, discriminative prototypes outperform the generative ones. Moreover, we show that the classification rates are improved while the number of prototypes increases. Finally, we show that discriminative prototypes give better results than the median graph based classifier. [Copyright &y& Elsevier]

Abstract: AdaBoost has been applied to object detection to construct the detectors with high performance of discrimination and generalization by single-feature learner. However, the poor discriminative power of extremely weak single-feature learners limits its application for general object detection. In this paper, we propose a novel comprehensive learner design mechanism toward effective object detection in terms of both discrimination and generalization abilities. Firstly, the part-sense multi-feature learners are designed to linearly combine the multiple local features to improve the descriptive and discriminative capacity of the learner. Secondly, we formulate the feature selection in part-sense multi-feature learner as a weighted LASSO regression. Using Least Angle Regression (LARS) method, our approach can choose features adaptively, efficiently and as few as possible to guarantee generalization performance. Finally, a robust L1-regularized gradient boosting is proposed to integrate our part-sense sparse features learner into an object classifier. Extensive experiments and comparisons on the face dataset and the human dataset show the proposed approach outperforms the traditional single-feature learner and other multi-feature learners in discriminative and generalization abilities. [Copyright &y& Elsevier]

Training deep neural networks with only a few labeled samples can lead to overfitting. This is problematic in semi-supervised learning where only a few labeled samples are available. In this paper, we show that a consequence of overfitting in SSL is feature distribution misalignment between labeled and unlabeled samples. Hence, we propose a new feature distribution alignment method. Our method is particularly effective when using only a small amount of labeled samples. We test our method on CIFAR-10, SVHN and LSUN. On SVHN we achieve a test error of 3.88% (250 labeled samples) and 3.39% (1000 labeled samples), which is close to the fully supervised model 2.89% (73k labeled samples). In comparison, the current SOTA achieves only 4.29% and 3.74%. On LSUN we achieve superior results than a state-of-the- art method even when using 100 × less unlabeled samples (500 labeled samples). Finally, we provide a theoretical insight why feature distribution misalignment occurs and show that our method reduces it. • Feature distribution misalignment and poor generalization are related. • Consistency regularization and adversarial alignment lead to mutual benefits for SSL. • Our method achieves the SOTA on SVHN, CIFAR-10 and LSUN. • On LSUN we outperform a state-of-the-art SSL method even when using 100× less unlabeled samples. • Adversarial alignment always improves when combined with traditional SSL methods. • Unsupervised domain adaptation can be helpful for semi-supervised learning. [ABSTRACT FROM AUTHOR]