Search

Object-centric geometric perception aims at extracting the geometric attributes of 3D objects. These attributes include shape, pose, and motion of the target objects, which enable fine-grained object-level understanding for various tasks in graphics, computer vision, and robotics. With the growth of 3D geometry data and 3D deep learning methods, it becomes more and more likely to achieve such tasks directly using 3D input data. Among different 3D representations, a 3D point cloud is a simple, common, and memory-efficient representation that could be directly retrieved from multi-view images, depth scans, or LiDAR range images. Different challenges exist in achieving object-centric geometric perception, such as achieving a fine-grained geometric understanding of common articulated objects with multiple rigid parts, learning disentangled shape and pose representations with fewer labels, or tackling dynamic and sequential geometric input in an end-to-end fashion. Here we identify and solve these challenges from a 3D deep learning perspective by designing effective and generalizable 3D representations, architectures, and pipelines. We propose the first deep pose estimation for common articulated objects by designing a novel hierarchical invariant representation. To push the boundary of 6D pose estimation for common rigid objects, a simple yet effective self-supervised framework is designed to handle unlabeled partial segmented scans. We further contribute a novel 4D convolutional neural network called PointMotionNet to learn spatio-temporal features for 3D point cloud sequences. All these works advance the domain of object-centric geometric perception from a unique 3D deep learning perspective.

Equivariance allows learning a representation that disentangles an entity or a feature from it's meta-properties. Spatially-equivariant representations lead to more detailed representations that can capture greater information from the image space in comparison to spatially-invariant representations. Convolutional Neural Networks, the current work-horses for image based analysis are built with baked-in spatial-invariance which helps in tasks like object detection. However, tasks like image synthesis that require learning an accurate manifold in order to generate visually accurate and diverse images would suffer due to the incorporated invariance. Equivariant architectures like Capsule Networks prove to be better critics for Generative Adversarial Networks as they learn disentangled representations of the meta-properties of the entities they represent. This helps the GANs to learn the data manifold much faster and therefore, synthesize visually accurate images in significantly lesser number of training samples and training epochs in comparison to GAN variants that use CNNs. Apart from proposing architectures that incorporate Capsule Networks into GANs, the thesis also assesses the effects of varying amounts of invariance over the quality and diversity of the images generated.