Search

In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of the rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset., Comment: An extension of our CVPR 2022 paper (RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior)

The goal of 3D pose transfer is to transfer the pose from the source mesh to the target mesh while preserving the identity information (e.g., face, body shape) of the target mesh. Deep learning-based methods improved the efficiency and performance of 3D pose transfer. However, most of them are trained under the supervision of the ground truth, whose availability is limited in real-world scenarios. In this work, we present X-DualNet, a simple yet effective approach that enables unsupervised 3D pose transfer. In X-DualNet, we introduce a generator $G$ which contains correspondence learning and pose transfer modules to achieve 3D pose transfer. We learn the shape correspondence by solving an optimal transport problem without any key point annotations and generate high-quality meshes with our elastic instance normalization (ElaIN) in the pose transfer module. With $G$ as the basic component, we propose a cross consistency learning scheme and a dual reconstruction objective to learn the pose transfer without supervision. Besides that, we also adopt an as-rigid-as-possible deformer in the training process to fine-tune the body shape of the generated results. Extensive experiments on human and animal data demonstrate that our framework can successfully achieve comparable performance as the state-of-the-art supervised approaches., Comment: arXiv admin note: text overlap with arXiv:2109.15025

Instance segmentation on 3D point clouds has been attracting increasing attention due to its wide applications, especially in scene understanding areas. However, most existing methods operate on fully annotated data while manually preparing ground-truth labels at point-level is very cumbersome and labor-intensive. To address this issue, we propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information respectively to unknown regions using self-attention and a cross-graph random walk method. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs to resolve ambiguities in closely placed objects, improving instance assignment accuracy. RWSeg generates high-quality instance-level pseudo labels. Experimental results on ScanNet-v2 and S3DIS datasets show that our approach achieves comparable performance with fully-supervised methods and outperforms previous weakly-supervised methods by a substantial margin.

Few-shot object detection (FSOD), which aims at learning a generic detector that can adapt to unseen tasks with scarce training samples, has witnessed consistent improvement recently. However, most existing methods ignore the efficiency issues, e.g., high computational complexity and slow adaptation speed. Notably, efficiency has become an increasingly important evaluation metric for few-shot techniques due to an emerging trend toward embedded AI. To this end, we present an efficient pretrain-transfer framework (PTF) baseline with no computational increment, which achieves comparable results with previous state-of-the-art (SOTA) methods. Upon this baseline, we devise an initializer named knowledge inheritance (KI) to reliably initialize the novel weights for the box classifier, which effectively facilitates the knowledge transfer process and boosts the adaptation speed. Within the KI initializer, we propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights. Finally, our approach not only achieves the SOTA results across three public benchmarks, i.e., PASCAL VOC, COCO and LVIS, but also exhibits high efficiency with 1.8-100x faster adaptation speed against the other methods on COCO/LVIS benchmark during few-shot transfer. To our best knowledge, this is the first work to consider the efficiency problem in FSOD. We hope to motivate a trend toward powerful yet efficient few-shot technique development. The codes are publicly available at https://github.com/Ze-Yang/Efficient-FSOD., Comment: Accepted to IEEE Transactions on Image Processing (TIP). Codes are available at https://github.com/Ze-Yang/Efficient-FSOD

3D pose transfer is one of the most challenging 3D generation tasks. It aims to transfer the pose of a source mesh to a target mesh and keep the identity (e.g., body shape) of the target mesh. Some previous works require key point annotations to build reliable correspondence between the source and target meshes, while other methods do not consider any shape correspondence between sources and targets, which leads to limited generation quality. In this work, we propose a correspondence-refinement network to achieve the 3D pose transfer for both human and animal meshes. The correspondence between source and target meshes is first established by solving an optimal transport problem. Then, we warp the source mesh according to the dense correspondence and obtain a coarse warped mesh. The warped mesh will be better refined with our proposed Elastic Instance Normalization, which is a conditional normalization layer and can help to generate high-quality meshes. Extensive experimental results show that the proposed architecture can effectively transfer the poses from source to target meshes and produce better results with satisfied visual performance than state-of-the-art methods.

Few-shot learning aims to adapt knowledge learned from previous tasks to novel tasks with only a limited amount of labeled data. Research literature on few-shot learning exhibits great diversity, while different algorithms often excel at different few-shot learning scenarios. It is therefore tricky to decide which learning strategies to use under different task conditions. Inspired by the recent success in Automated Machine Learning literature (AutoML), in this paper, we present Meta Navigator, a framework that attempts to solve the aforementioned limitation in few-shot learning by seeking a higher-level strategy and proffer to automate the selection from various few-shot learning designs. The goal of our work is to search for good parameter adaptation policies that are applied to different stages in the network for few-shot classification. We present a search space that covers many popular few-shot learning algorithms in the literature and develop a differentiable searching and decoding algorithm based on meta-learning that supports gradient-based optimization. We demonstrate the effectiveness of our searching-based method on multiple benchmark datasets. Extensive experiments show that our approach significantly outperforms baselines and demonstrates performance advantages over many state-of-the-art methods. Code and models will be made publicly available., Comment: Accepted by ICCV2021

Due to the scarcity of annotated scene flow data, self-supervised scene flow learning in point clouds has attracted increasing attention. In the self-supervised manner, establishing correspondences between two point clouds to approximate scene flow is an effective approach. Previous methods often obtain correspondences by applying point-wise matching that only takes the distance on 3D point coordinates into account, introducing two critical issues: (1) it overlooks other discriminative measures, such as color and surface normal, which often bring fruitful clues for accurate matching; and (2) it often generates sub-par performance, as the matching is operated in an unconstrained situation, where multiple points can be ended up with the same corresponding point. To address the issues, we formulate this matching task as an optimal transport problem. The output optimal assignment matrix can be utilized to guide the generation of pseudo ground truth. In this optimal transport, we design the transport cost by considering multiple descriptors and encourage one-to-one matching by mass equality constraints. Also, constructing a graph on the points, a random walk module is introduced to encourage the local consistency of the pseudo labels. Comprehensive experiments on FlyingThings3D and KITTI show that our method achieves state-of-the-art performance among self-supervised learning methods. Our self-supervised method even performs on par with some supervised learning approaches, although we do not need any ground truth flow for training., Comment: Accepted to CVPR2021 (oral)

Scene flow in 3D point clouds plays an important role in understanding dynamic environments. Although significant advances have been made by deep neural networks, the performance is far from satisfactory as only per-point translational motion is considered, neglecting the constraints of the rigid motion in local regions. To address the issue, we propose to introduce the motion consistency to force the smoothness among neighboring points. In addition, constraints on the rigidity of the local transformation are also added by sharing unique rigid motion parameters for all points within each local region. To this end, a high-order CRFs based relation module (Con-HCRFs) is deployed to explore both point-wise smoothness and region-wise rigidity. To empower the CRFs to have a discriminative unary term, we also introduce a position-aware flow estimation module to be incorporated into the Con-HCRFs. Comprehensive experiments on FlyingThings3D and KITTI show that our proposed framework (HCRF-Flow) achieves state-of-the-art performance and significantly outperforms previous approaches substantially., Comment: Accepted to CVPR2021

The determination of the Higgs self coupling is one of the key ingredients for understanding the mechanism behind the electroweak symmetry breaking. An indirect method for constraining the Higgs trilinear self coupling via single Higgs production at next-to-leading order (NLO) has been proposed in order to avoid the drawbacks of studies with double Higgs production. In this paper we study the Higgs self interaction through the vector boson fusion (VBF) process $e^{-} p \to \nu_{e} h j$ at the future LHeC. At NLO level, we compute analytically the scattering amplitudes for relevant processes, in particular those induced by the Higgs self interaction. A Monte Carlo simulation and a statistical analysis utilizing the analytic results are then carried out for Higgs production through VBF and decay to $b\bar{b}$, which yield for the trilinear Higgs self-coupling rescaling parameter $\kappa_{\lambda}$ the limit [-0.57, 2.98] with $2~\text{ab}^{-1}$ integrated luminosity. If we assume about 10% of the signal survives the event selection cuts, and include all the background, the constraint will be broadened to [-2.11, 4.63]., Comment: 19 pages, 7 figures, 2 tables

As one category of vector boson fusion, photo-production is one important production mechanism at $e$-$p$ colliders. A future $e$-$p$ collider -- Large Hadron-electron Collider (LHeC) has been discussed as a "Higgs factory" candidate where the Higgs boson produced via weak boson fusion (WBF) at the LHeC plays an important role in precision measurement of Yukawa couplings. On the other hand, a measurement of photo-production of the Higgs boson, if possible, might be complementary to the measurement of Higgs to di-photon partial decay width $\Gamma({h\to\gamma\gamma})$. In this paper, we study the possibility of measuring this production process at the LHeC with the help of the photon PDFs published in recent years. This process has a clean final state without additional colored particles in the detectable region other than the decay products of the Higgs. We compute the cross sections of all related processes and find that the production rate is at the same order as the neutral current WBF production of Higgs boson with missing forward jets. However, a detailed phenomenological study of various Higgs decay channels shows that even in the most promising semi-leptonic $WW$ channel, the feasibility of identifying such photo-production is negative due to an irreducible photo-production of $W^{+}W^{-}$., Comment: 19pages, 6figures

Large Hadron Collider (LHC) has provided direct evidence of Yukawa couplings between the third generation charged fermions and the 125 GeV Higgs boson. Whether the first two generation charged fermions arise from exactly the same mechanism becomes the next interesting question. Therefore, direct measurements of charm or muon Yukawa couplings will be crucial to answering this puzzle. The charm Yukawa measurement at the LHC suffers from severe QCD background and it is extremely difficult to reach the sensitivity. In this paper, we compare the potential of probing charm Yukawa coupling at the two proposed future "Higgs Factory" experiments, the Large Hadron electron Collider (LHeC) and Circular electron positron collider (CEPC). At the LHeC, Higgs bosons will be produced via weak boson fusion and the energetic forward jet may suppress the background significantly. However, due to huge $\gamma-g$ scattering background, the potential of LHeC search is still limited. With $-80\%$ polarized electron beam of 60 GeV, the signal significance can only reach $2\sigma$ for $ \kappa_{c}\simeq 1$ with a 3 ab$^{-1}$ integrated luminosity. In comparison, measurement at the CEPC can then reach $8.0\sigma$ for $\kappa_{c}\simeq 1$ with a 2 ab$^{-1}$ of data., Comment: 28 pages, 9 figures and 11 tables

In this paper, we present our deep attention-based classification (DABC) network for robust single image depth prediction, in the context of the Robust Vision Challenge 2018 (ROB 2018). Unlike conventional depth prediction, our goal is to design a model that can perform well in both indoor and outdoor scenes with a single parameter set. However, robust depth prediction suffers from two challenging problems: a) How to extract more discriminative features for different scenes (compared to a single scene)? b) How to handle the large differences of depth ranges between indoor and outdoor datasets? To address these two problems, we first formulate depth prediction as a multi-class classification task and apply a softmax classifier to classify the depth label of each pixel. We then introduce a global pooling layer and a channel-wise attention mechanism to adaptively select the discriminative channels of features and to update the original features by assigning important channels with higher weights. Further, to reduce the influence of quantization errors, we employ a soft-weighted sum inference strategy for the final prediction. Experimental results on both indoor and outdoor datasets demonstrate the effectiveness of our method. It is worth mentioning that we won the 2-nd place in single image depth prediction entry of ROB 2018, in conjunction with IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018.

The search of the light Higgsino {in the Minimal Supersymmetric Standard Model (MSSM)} is a crucial test for the criteria of the naturalness in supersymmetry. On the other hand, the direct production of light Higgsino is also known as one of the most challenging SUSY searches at the current CERN Large Hadron collider (LHC). The lack of visible leptons due to the compressed spectrum and their small production rates limits their discovery potential in both mono-jet plus MET as well as the weak boson fusion (WBF) production. The signal $S/B$ ratio prediction is usually within the background systematic uncertainties at the LHC. Without color exchange between the beams, the $e-p$ colliders are well-known to have the WBF feature. Therefore, we study the search of the light Higgsinos at two future $e-p$ colliders at CERN, LHeC and FCC-eh.~The light Higgsinos will be produced in pair through weak boson fusion with controlled background at these colliders. We find the Higgsino of 95/145~GeV can be reached at 2$\sigma$ level at the future LHeC/FCC-eh with a luminosity $3 ~\text{ab}^{-1}$ respectively., Comment: 18 pages, 7 tables, 5 figures. We updated the results at the LHeC and FCC-eh after detector simulation. The relevant Higgs mass spectrum and cuts criteria are added by new tables. The significance is also updated due to new reducible backgrounds have been considered

The precision measurement of the $WW\gamma$ vertex at the future Large Hadron electron Collider (LHeC) at CERN is discussed in this paper. We propose to measure this vertex in the $e^{-} p\to e^{-}W^{\pm}j$ channel as a complement to the conventional charged current $\nu_{e}\gamma j$ channel. In addition to the cross section measurement, $\chi^{2}$ method studies of angular variables provide powerful tools to probe the anomalous structure of triple gauge boson couplings. We study the distribution of the well-known azimuthal angle between the final state forward electron and jet in this vector-boson fusion (VBF) process. On the other hand, full reconstruction of leptonic $W$ decay opens a new opportunity to measure $W$ polarization that is also sensitive to the anomalous triple gauge boson couplings. Taking into consideration the superior determination of parton distribution functions~(PDFs) based on future LHeC data, the constraints of $\lambda_{\gamma}$ and $\Delta\kappa_{\gamma}$ might reach up to $\mathcal{O}(10^{-3})$ level in the most ideal case with the 2--3~ab$^{-1}$ data set, which shows a potential advantage compared to those from LHC and LEP data., Comment: 15 pages, 8 figures