Search

Deep point cloud registration methods face challenges to partial overlaps and rely on labeled data To address these issues we propose UDPReg an unsupervised deep probabilistic registration framework for point clouds with partial overlaps Specifically we first adopt a network to learn posterior probability distributions of Gaussian mixture models GMMs from point clouds To handle partial point cloud registration we apply the Sinkhorn algorithm to predict the distribution level correspondences under the constraint of the mixing weights of GMMs To enable unsupervised learning we design three distribution consistency based losses self consistency cross consistency and local contrastive The self consistency loss is formulated by encouraging GMMs in Euclidean and feature spaces to share identical posterior distributions The cross consistency loss derives from the fact that the points of two partially overlapping point clouds belonging to the same clusters share the cluster centroids The cross consistency loss allows the network to flexibly learn a transformation invariant posterior distribution of two aligned point clouds The local contrastive loss facilitates the network to extract discriminative local features Our UDPReg achieves competitive performance on the 3DMatch 3DLoMatch and ModelNet ModelLoNet benchmarks

This paper reviews the third biennial challenge on spectral reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image. This challenge presents the "ARAD_1K" data set: a new, larger-than-ever natural hyperspectral image data set containing 1,000 images. Challenge participants were required to recover hyper-spectral information from synthetically generated JPEG-compressed RGB images simulating capture by a known calibrated camera, operating under partially known parameters, in a setting which includes acquisition noise. The challenge was attended by 241 teams, with 60 teams com-peting in the final testing phase, 12 of which provided de-tailed descriptions of their methodology which are included in this report. The performance of these submissions is re-viewed and provided here as a gauge for the current state-of-the-art in spectral reconstruction from natural RGB images.

Since the rise of vision-language navigation (VLN), great progress has been made in instruction following - building a follower to navigate environments under the guidance of instructions. However, far less attention has been paid to the inverse task: instruction generation - learning a speaker to generate grounded descriptions for navigation routes. Existing VLN methods train a speaker independently and often treat it as a data augmentation tool to strengthen the follower, while ignoring rich cross-task relations. Here we describe an approach that learns the two tasks simultaneously and exploits their intrinsic correlations to boost the training of each: the follower judges whether the speaker-created instruction explains the original navigation route correctly, and vice versa. Without the need of aligned instruction-path pairs, such cycle-consistent learning scheme is complementary to task-specific training targets defined on labeled data, and can also be applied over unlabeled paths (sampled without paired instructions). Another agent, called creator is added to generate counterfactual environments. It greatly changes current scenes yet leaves novel items - which are vital for the execution of original instructions - unchanged. Thus more informative training scenes are synthesized and the three agents compose a powerful VLN learning system. Extensive experiments on a standard benchmark show that our approach improves the performance of various follower models and produces accurate navigation instructions.

This article studies the problem of learning weakly supervised semantic segmentation (WSSS) from image-level supervision only. Rather than previous efforts that primarily focus on intra-image information, we address the value of cross-image semantic relations for comprehensive object pattern mining. To achieve this, two neural co-attentions are incorporated into the classifier to complimentarily capture cross-image semantic similarities and differences. In particular, given a pair of training images, one co-attention enforces the classifier to recognize the common semantics from co-attentive objects, while the other one, called contrastive co-attention, drives the classifier to identify the unique semantics from the rest, unshared objects. This helps the classifier discover more object patterns and better ground semantics in image regions. More importantly, our algorithm provides a unified framework that handles well different WSSS settings, i.e., learning WSSS with (1) precise image-level supervision only, (2) extra simple single-label data, and (3) extra noisy web data. Without bells and whistles, it sets new state-of-the-arts on all these settings. Moreover, our approach ranked 1 st place in the WSSS Track of CVPR2020 LID Challenge. The extensive experimental results demonstrate well the efficacy and high utility of our method.

This paper reviews the second NTIRE challenge on image dehazing (restoration of rich details in hazy image) with focus on proposed solutions and results. The training data consists from 55 hazy images (with dense haze generated in an indoor or outdoor environment) and their corresponding ground truth (haze-free) images of the same scene. The dense haze has been produced using a professional haze/fog generator that imitates the real conditions of haze scenes. The evaluation consists from the comparison of the dehazed images with the ground truth images. The dehazing process was learnable through provided pairs of haze-free and hazy train images. There were ~ 270 registered participants and 23 teams competed in the final testing phase. They gauge the state-of-the-art in image dehazing.

Software tools known as language workbenches are used to define and deploy custom (domain-specific) languages for the purpose of modeling (specific parts) of a system of interest. Because system modeling is a practice that stands to benefit from real-time collaboration, technologies offering real-time collaborative mechanisms for language workbenches are starting to make an appearance. However, these collaboration technologies are generally limited to providing collaboration among clients of a single designated workbench. If the collaborating engineers wish to use different workbenches to work on the model, cross-platform support for collaborative modeling becomes a necessity. In this paper we propose Parsafix, a tool-based approach for achieving real-time collaboration between different language workbenches for users collaborating on models conforming to the same domain-specific language. We propose the main components and mechanisms that make up Parsafix, as well as the implementation of a prototype tool supporting those mechanisms. The prototype tool allows for collaboration between users of JetBrains MPS and Spoofax (within the Eclipse IDE), by making use of the IDEs' respective real-time collaboration technologies Modelix and Saros. A hands-on session is proposed to showcase the feasibility of having collaborative modeling across different language workbenches through Parsafix.

Vision Meets Drone: Multiple Object Tracking (VisDrone-MOT2021) challenge - the forth annual activity organized by the VisDrone team - focuses on benchmarking UAV MOT algorithms in realistic challenging environments. It is held in conjunction with ICCV 2021. VisDrone-MOT2021 contains 96 video sequences in total, including 56 sequences (~24K frames) for training, 7 sequences (~3K frames) for validation and 33 sequences (~13K frames) for testing. Bounding-box annotations for novel object categories are provided every frame and temporally consistent instance IDs are also given. Additionally, occlusion ratio and truncation ratio are provided as extra useful annotations. The results of eight state-of-the-art MOT algorithms are reported and discussed. We hope that our VisDrone-MOT2021 challenge will facilitate future research and applications in the field of UAV vision. The website of our challenge can be found at http://www.aiskyeye.com/.

To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner. It is a compact, efficient and powerful framework that exploits structural information over different human granularities and eases the difficulty of person partitioning. Specifically, a dense-to-sparse projection field, which allows explicitly associating dense human semantics with sparse keypoints, is learnt and progressively improved over the network feature pyramid for robustness. Then, the difficult pixel grouping problem is cast as an easier, multi-person joint assembling task. By formulating joint association as maximum-weight bipartite matching, a differentiable solution is developed to exploit projected gradient descent and Dykstra’s cyclic projection algorithm. This makes our method end-to-end trainable and allows back-propagating the grouping error to directly supervise multi-granularity human representation learning. This is distinguished from current bottom-up human parsers or pose estimators which require sophisticated post-processing or heuristic greedy algorithms. Experiments on three instance-aware human parsing datasets show that our model outperforms other bottom-up alternatives with much more efficient inference.

The Visual Object Tracking challenge VOT2020 is the eighth annual tracker benchmarking activity organized by the VOT initiative. Results of 58 trackers are presented; many are state-of-the-art trackers published at major computer vision conferences or in journals in the recent years. The VOT2020 challenge was composed of five sub-challenges focusing on different tracking domains: (i) VOT-ST2020 challenge focused on short-term tracking in RGB, (ii) VOT-RT2020 challenge focused on “real-time” short-term tracking in RGB, (iii) VOT-LT2020 focused on long-term tracking namely coping with target disappearance and reappearance, (iv) VOT-RGBT2020 challenge focused on short-term tracking in RGB and thermal imagery and (v) VOT-RGBD2020 challenge focused on long-term tracking in RGB and depth imagery. Only the VOT-ST2020 datasets were refreshed. A significant novelty is introduction of a new VOT short-term tracking evaluation methodology, and introduction of segmentation ground truth in the VOT-ST2020 challenge – bounding boxes will no longer be used in the VOT-ST challenges. A new VOT Python toolkit that implements all these novelites was introduced. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The dataset, the evaluation kit and the results are publicly available at the challenge website (http://votchallenge.net ).