Search

The Kinematic Structure (KS) is a compact and structured representation that fully discloses the motion capabilities of an articulated rigid object. Estimating KSs is thus an active topic in the fields of computer vision and robotics, with applications in robot manipulation tasks, unsupervised motion retargeting, and robot-assisted dressing, to name a few possibilities. While previous approaches are typically offline or computationally demanding, in this thesis, novel KS estimation methods from vision-based data that are suitable for real-time applications will be developed. The thesis starts by providing empirical evidence that initially representing the object by semi-dense three-dimensional (3D) points is a valid compromise between accuracy and computational processing costs. The problem of motion segmentation of articulated rigid bodies from semi-dense 3D points is then cast as a subspace clustering problem. Online processing is explored to handle incomplete point trajectories and partial occlusions during KS estimation. A suitable incremental metric representation of the tracked semi-dense 3D points is proposed, based on the observation that the distance between points belonging to the same rigid part is constant. This representation allows mitigating noise and points' tracking errors while implicitly encoding motion information, which is combined with the object's topological distances to build more plausible KSs. The use of event cameras to estimate KSs is considered for the first time in this thesis. A novel framework for event-based motion estimation is proposed, which can estimate the parameters of several motion models. The framework does not rely on any intermediate image-based representation and can thus handle augmented events from additional sensors. An incremental version of this framework is then used to perform joint shape-motion segmentation for event-based KS estimation without having to track feature points, which represents a paradigm shift on vision-based KS estimation. New challenging sequences for KS estimation are also made available. Experimental results corroborate that event cameras outperform frame-based cameras on motion-related tasks, and specifically on KS estimation. This thesis advances the state-of-the-art on vision-based KS estimation by proposing new frame-based KS estimation methods and taking the first steps towards considering event cameras for KS estimation. Its contributions are likely to bolster real-time applications that rely on KSs. Open Access