Search

Sensory perception for dexterous robotic hands is an active research area and recent progress in robotics. Effective dexterous manipulation requires robotic hands to accurately feedback their state or perceive the surrounding environment. This article reviews the state-of-the-art of sensory perception for dexterous robotic manipulation. Two types of sensors, such as intrinsic and extrinsic sensors, are introduced according to their function and layout in robotic hands. These sensors provide rich information to a robotic hand, which contains the posture, the contact information of objects, and the physical information of the environment. Then, a comprehensive analysis of perception methods including planning-level, control-level, and learning-level perceptions is presented. The information obtained from sensory perception can help robotic hands to make decisions effectively. Previously issued reviews mainly focus on the design of tactile senor, while we analyze and discuss the relationship among sensing, perception, and dexterous manipulation. Some potential research topics on sensory perception are also summarized and discussed.

The brain–computer interface-based rehabilitation robot has quickly become a very important research area due to its natural interaction. One of the most important problems in brain–computer interface is that large-scale annotated electroencephalography data sets required by advanced classifiers are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed with the test data. It can be considered a powerful tool for solving the problem of insufficient training data. There are two basic issues with transfer learning, under transfer and negative transfer. We proposed a novel brain–computer interface framework by using autoencoder-based transfer learning, which includes three main components: an autoencoder framework, a joint adversarial network, and a regularized manifold constraint. The autoencoder framework automatically encodes and reconstructs data from source and target domains and forces the neural network to learn to represent these domains reliably. The joint adversarial network aims to force the network to learn to encode more appropriately for the source domain and target domain simultaneously, thereby overcoming the problem of under transfer. The regularized manifold constraint aims to avoid the problem of negative transfer by avoiding geometric manifold structure in the target domain being destroyed by the source domain. Experiments show that the brain–computer interface framework proposed by us can achieve better results than state-of-the-art approaches in electroencephalography signal classification tasks. This is helpful in aiding our rehabilitation robot to understand the intention of patients and can help patients to carry out rehabilitation exercises effectively.

This article introduces a visual–tactile multimodal grasp data set, aiming to further the research on robotic manipulation. The data set was built by the novel designed dexterous robot hand, the Intel’s Eagle Shoal robot hand (Intel Labs China, Beijing, China). The data set contains 2550 sets data, including tactile, joint, time label, image, and RGB and depth video. With the integration of visual and tactile data, researchers could be able to better understand the grasping process and analyze the deeper grasping issues. In this article, the building process of the data set was introduced, as well as the data set composition. In order to evaluate the quality of data set, the tactile data were analyzed by short-time Fourier transform. The tactile data–based slip detection was realized by long short-term memory and contrasted with visual data. The experiments compared the long short-term memory with the traditional classifiers, and generalization ability on different grasp directions and different objects is implemented. The results have proved that the data set’s value in promoting research on robotic manipulation area showed the effective slip detection and generalization ability of long short-term memory. Further work on visual and tactile data will be devoted to in the future.

Compared with traditional rigid gripper with joint-linkage structure, novel soft robotic gripper gives rise to continuous concern for the advantages of no-damage grasping, convenient manufacture, easy control, and low cost. In this study, we design and built two kinds of soft robotic grippers with four fiber-reinforced soft actuators which are distributed in circular and rectangle shapes for single and twin contacts grasping. A novel hybrid valve pneumatic control scheme combining proportional and solenoid valves is proposed. Also, a mode controllable hybrid valve pressure control method is proposed to adjust internal pressure of soft robotic grippers to adapt to different grasping tasks. The experiment results verify that the performances of hybrid valve outperform those of individual proportional valve or solenoid valve in the aspects of response time and steady-state accuracy. The hybrid valve has wide range of pressure regulation, result in that the soft robotic grippers are qualified to grasp various objects with different shapes, sizes, and weights.

Conventional visual perception technology is subject to many restrictions, such as illumination, background clutter, and occlusion. Many intrinsic properties of objects, like stiffness, hardness, and internal state, cannot be effectively perceived by visual sensors. For robots, tactile perception is a key approach to obtain environmental and object information. Different from vision sensors, tactile sensors can directly measure some physical properties of objects and environment. At the same time, humans also utilize touch sensory receptors as an important means to perceive and interact with the environment. In this article, we present a detailed discussion on tactile object recognition problem. We divide the current studies on the tactile object recognition into three subcategories and detailed analysis has been put forward on them. In addition, we also discuss some advanced topics such as visual–tactile fusion, exploratory procedure, and data sets.

Teleoperation is of great importance in the area of robotics especially when people’s presence at the robot working space is unavailable. It provides an alternative to employ human intelligence in the control of the robot remotely. We establish robotic teleoperation systems with a wearable multimodal fusion device. The device is integrated with 18 low-cost inertial and magnetic measurement units, which cover all segments of the arm and hand. The multimodal fusion algorithm based on extended Kalman filter is deduced to determine the orientations and positions of each segment. Then, the robotic teleoperation systems using the proposed device are designed. The novel teleoperation schemes can be applied for 11DOF robotic arm–hand system and 10DOF robotic arm–hand system, in which the operator’s fingers are used for robotic hand teleoperation, and the arms with palm are used for robotic arm teleoperation. Meanwhile, the proposed robotic teleoperation systems are fully realized with a user-friendly human–machine interaction interface. Finally, a series of experiments are conducted with our robotic teleoperation system successfully.

Grasping has always been a great challenge for robots due to its lack of the ability to well understand the perceived sensing data. In this work, we propose an end-to-end deep vision network model to predict possible good grasps from real-world images in real time. In order to accelerate the speed of the grasp detection, reference rectangles are designed to suggest potential grasp locations and then refined to indicate robotic grasps in the image. With the proposed model, the graspable scores for each location in the image and the corresponding predicted grasp rectangles can be obtained in real time at a rate of 80 frames per second on a graphic processing unit. The model is evaluated on a real robot-collected data set and different reference rectangle settings are compared to yield the best detection performance. The experimental results demonstrate that the proposed approach can assist the robot to learn the graspable part of the object from the image in a fast manner.

Grasping has always been a great challenge for robots due to its lack of the ability to well understand the perceived sensing data. In this work, we propose an end-to-end deep vision network model to predict possible good grasps from real-world images in real time. In order to accelerate the speed of the grasp detection, reference rectangles are designed to suggest potential grasp locations and then refined to indicate robotic grasps in the image. With the proposed model, the graspable scores for each location in the image and the corresponding predicted grasp rectangles can be obtained in real time at a rate of 80 frames per second on a graphic processing unit. The model is evaluated on a real robot-collected data set and different reference rectangle settings are compared to yield the best detection performance. The experimental results demonstrate that the proposed approach can assist the robot to learn the graspable part of the object from the image in a fast manner. [ABSTRACT FROM AUTHOR]