Search

Multi-step forceful manipulation tasks, such as opening a push-and-twist childproof bottle, require a robot to make various planning choices that are substantially impacted by the requirement to exert force during the task. The robot must reason over discrete and continuous choices relating to the sequence of actions, such as whether to pick up an object, and the parameters of each of those actions, such how to grasp the object. To enable planning and executing forceful manipulation, we augment an existing task and motion planner with constraints that explicitly consider torque and frictional limits, captured through the proposed forceful kinematic chain constraint. In three domains, opening a childproof bottle, twisting a nut and cutting a vegetable, we demonstrate how the system selects from among a combinatorial set of strategies. We also show how cost-sensitive planning can be used to find strategies and parameters that are robust to uncertainty in the physical parameters. [ABSTRACT FROM AUTHOR]

In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB (red, green, blue) tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, multiple contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We extend and test Tac2Pose in multi-contact scenarios where two tactile sensors are simultaneously in contact with the object, as during a grasp with a parallel jaw gripper. We further show that when the output pose distribution is filtered with a prior on the object pose, Tac2Pose is often able to improve significantly on the prior. This suggests synergistic use of Tac2Pose with additional sensing modalities (e.g., vision) even in cases where the tactile observation from a grasp is not sufficiently discriminative. Given a coarse estimate of an object's pose, even ambiguous contacts can be used to determine an object's pose precisely. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. We show that even in the presence of model uncertainty, Tac2Pose is able to achieve fine accuracy comparable to when the object model is the manufacturer's CAD (computer-aided design) model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: mcube.mit.edu/research/tac2pose.html [ABSTRACT FROM AUTHOR]

This paper studies the robustness of grasping in the frictionless plane from a geometric perspective. By treating grasping as a process that shapes the free-space object over time, we define three types of certificates to guarantee success of a grasp: (a) invariance under an initial set, (b) convergence toward a goal grasp, and (c) observability over the final object pose. We develop convex-combinatorial models for each of these certificates, which can be expressed as simple semi-algebraic relations under mild-modeling assumptions, such as point-fingers and frictionless contact. By leveraging these models to synthesize certificates, we optimize certifiable grasps of planar objects composed as a union of convex polygons, using manipulators described as point-fingers. We validate this approach in simulations by grasping random polygons, and with real sensorless grasps of several objects. [ABSTRACT FROM AUTHOR]

Cables are complex, high-dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and incremental deformations, or various mechanical fixtures such as clamps, pins, or rings. We are interested in manipulating freely moving cables, in real time, with a pair of robotic grippers, and with no added mechanical constraints. The main contribution of this paper is a perception and control framework that moves in that direction, and uses real-time tactile feedback to accomplish the task of following a dangling cable. The approach relies on a vision-based tactile sensor, GelSight, that estimates the pose of the cable in the grip, and the friction forces during cable sliding. We achieve the behavior by combining two tactile-based controllers: (1) cable grip controller, where a PD controller combined with a leaky integrator regulates the gripping force to maintain the frictional sliding forces close to a suitable value; and (2) cable pose controller, where an linear–quadratic regulator controller based on a learned linear model of the cable sliding dynamics keeps the cable centered and aligned on the fingertips to prevent the cable from falling from the grip. This behavior is possible with the use of reactive gripper fitted with GelSight-based high-resolution tactile sensors. The robot can follow 1 m of cable in random configurations within two to three hand regrasps, adapting to cables of different materials and thicknesses. We demonstrate a robot grasping a headphone cable, sliding the fingers to the jack connector, and inserting it. To the best of the authors' knowledge, this is the first implementation of real-time cable following without the aid of mechanical fixtures. Videos are available at http://gelsight.csail.mit.edu/cable/ [ABSTRACT FROM AUTHOR]

This article presents an offline solution and online approximation to the hybrid control problem of planar non-prehensile manipulation. Hybrid dynamics and underactuation are key characteristics of this task that complicate the design of feedback controllers. We show that a model predictive control approach used in tandem with integer programming offers a powerful solution to capture the dynamic constraints associated with the friction cone as well as the hybrid nature of contact. We introduce the Model Predictive Controller with Learned Mode Scheduling (MPC-LMS), which leverages integer programming and machine learning techniques to effectively deal with the combinatorial complexity associated with determining sequences of contact modes. We validate the controller design through a numerical simulation study and with experiments on a planar manipulation setup using an industrial ABB IRB 120 robotic arm. Results show that the proposed algorithm achieves closed-loop tracking of a nominal trajectory by reasoning in real-time across multiple contact modalities. [ABSTRACT FROM AUTHOR]

In this article, we present the mechanics and algorithms to compute the set of feasible motions of an object pushed in a plane. This set is known as the motion cone and was previously described for non-prehensile manipulation tasks in the horizontal plane. We generalize its construction to a broader set of planar tasks, such as those where external forces including gravity influence the dynamics of pushing, or prehensile tasks, where there are complex frictional interactions between the gripper, object, and pusher. We show that the motion cone is defined by a set of low-curvature surfaces and approximate it by a polyhedral cone. We verify its validity with thousands of pushing experiments recorded with a motion tracking system. Motion cones abstract the algebra involved in the dynamics of frictional pushing and can be used for simulation, planning, and control. In this article, we demonstrate their use for the dynamic propagation step in a sampling-based planning algorithm. By constraining the planner to explore only through the interior of motion cones, we obtain manipulation strategies that are robust against bounded uncertainties in the frictional parameters of the system. Our planner generates in-hand manipulation trajectories that involve sequences of continuous pushes, from different sides of the object when necessary, with 5–1,000 times speed improvements to equivalent algorithms. [ABSTRACT FROM AUTHOR]

This paper addresses the identification of the inertial parameters and the contact forces associated with objects making and breaking frictional contact with the environment. Our goal is to explore under what conditions, and to what degree, the observation of physical interaction, in the form of motions and/or applied external forces, is indicative of the underlying dynamics that governs it. In this study we consider the cases of passive interaction, where an object free-falls under gravity, and active interaction, where known external perturbations act on an object at contact. We assume that both object and environment are planar and rigid, and exploit the well-known complementarity formulation for contact resolution to establish a constrained optimization-based problem to estimate inertial parameters and contact forces. We also show that when contact modes are known, or guessed, the formulation provides a closed-form relationship between inertial parameters, contact forces, and observed motions, that turns into a least squares problem.Consistent with intuition, the analysis indicates that without the application of known external forces, the identifiable set of parameters remains coupled, i.e. the ratio of mass moment of inertia to mass and the ratio of contact forces to the mass. Interestingly the analysis also shows that known external forces can lead to decoupling and identifiability of mass, mass moment of inertia, and normal and tangential contact forces. We evaluate the proposed algorithms both in simulation and with real experiments for the cases of a free falling square, ellipse, and rimless wheel interacting with the ground, as well as a disk interacting with a manipulator. [ABSTRACT FROM AUTHOR]