Search

Multi-step forceful manipulation tasks, such as opening a push-and-twist childproof bottle, require a robot to make various planning choices that are substantially impacted by the requirement to exert force during the task. The robot must reason over discrete and continuous choices relating to the sequence of actions, such as whether to pick up an object, and the parameters of each of those actions, such how to grasp the object. To enable planning and executing forceful manipulation, we augment an existing task and motion planner with constraints that explicitly consider torque and frictional limits, captured through the proposed forceful kinematic chain constraint. In three domains, opening a childproof bottle, twisting a nut and cutting a vegetable, we demonstrate how the system selects from among a combinatorial set of strategies. We also show how cost-sensitive planning can be used to find strategies and parameters that are robust to uncertainty in the physical parameters. [ABSTRACT FROM AUTHOR]

In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB (red, green, blue) tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, multiple contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We extend and test Tac2Pose in multi-contact scenarios where two tactile sensors are simultaneously in contact with the object, as during a grasp with a parallel jaw gripper. We further show that when the output pose distribution is filtered with a prior on the object pose, Tac2Pose is often able to improve significantly on the prior. This suggests synergistic use of Tac2Pose with additional sensing modalities (e.g., vision) even in cases where the tactile observation from a grasp is not sufficiently discriminative. Given a coarse estimate of an object's pose, even ambiguous contacts can be used to determine an object's pose precisely. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. We show that even in the presence of model uncertainty, Tac2Pose is able to achieve fine accuracy comparable to when the object model is the manufacturer's CAD (computer-aided design) model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: mcube.mit.edu/research/tac2pose.html [ABSTRACT FROM AUTHOR]

This paper studies the robustness of grasping in the frictionless plane from a geometric perspective. By treating grasping as a process that shapes the free-space object over time, we define three types of certificates to guarantee success of a grasp: (a) invariance under an initial set, (b) convergence toward a goal grasp, and (c) observability over the final object pose. We develop convex-combinatorial models for each of these certificates, which can be expressed as simple semi-algebraic relations under mild-modeling assumptions, such as point-fingers and frictionless contact. By leveraging these models to synthesize certificates, we optimize certifiable grasps of planar objects composed as a union of convex polygons, using manipulators described as point-fingers. We validate this approach in simulations by grasping random polygons, and with real sensorless grasps of several objects. [ABSTRACT FROM AUTHOR]

Cables are complex, high-dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and incremental deformations, or various mechanical fixtures such as clamps, pins, or rings. We are interested in manipulating freely moving cables, in real time, with a pair of robotic grippers, and with no added mechanical constraints. The main contribution of this paper is a perception and control framework that moves in that direction, and uses real-time tactile feedback to accomplish the task of following a dangling cable. The approach relies on a vision-based tactile sensor, GelSight, that estimates the pose of the cable in the grip, and the friction forces during cable sliding. We achieve the behavior by combining two tactile-based controllers: (1) cable grip controller, where a PD controller combined with a leaky integrator regulates the gripping force to maintain the frictional sliding forces close to a suitable value; and (2) cable pose controller, where an linear–quadratic regulator controller based on a learned linear model of the cable sliding dynamics keeps the cable centered and aligned on the fingertips to prevent the cable from falling from the grip. This behavior is possible with the use of reactive gripper fitted with GelSight-based high-resolution tactile sensors. The robot can follow 1 m of cable in random configurations within two to three hand regrasps, adapting to cables of different materials and thicknesses. We demonstrate a robot grasping a headphone cable, sliding the fingers to the jack connector, and inserting it. To the best of the authors' knowledge, this is the first implementation of real-time cable following without the aid of mechanical fixtures. Videos are available at http://gelsight.csail.mit.edu/cable/ [ABSTRACT FROM AUTHOR]