Search

Three dimensional (3D) robot measurement systems, with binocular planar structured light cameras (3D cameras) installed at the terminal flange of robots, are widely used to measure complete workpieces by stitching point clouds obtained from various sampled poses. To ensure accurate stitching, hand–eye calibration is necessary. However, 3D cameras often demonstrate low precision in measuring jumping edges and vertices, resulting in challenges when selecting appropriate calibrators and calibration methods for hand–eye calibration. We propose a simple and high‐precision hand–eye calibration method for 3D robot measurement systems. Initially, a single sphere is utilized as the calibrator, and its poses are observed using specific robotic motions. Subsequently, the hand–eye calibration problem is formulated as a well‐known equation, AX=XB$A X = X B$, relying solely on the high‐precision relative pose of the robot. Finally, a novel simultaneous solution for AX=XB$A X = X B$ and the corresponding closed‐form initial solution are introduced. Simulations and experiments confirm that the proposed method exhibits high noise resistance and achieves high‐precision calibration, even with a limited calibration data quantity. Compared with traditional methods, the fitting error of the proposed method can be reduced from over 0.9 mm to less than 0.6 mm. [ABSTRACT FROM AUTHOR]

Hand-eye calibration is crucial for vision-based robots using open-loop visual control. Based on the product of exponentials model, this study introduces a novel hand-eye calibration algorithm to improve calibration accuracy. As opposed to the traditional method of solving the AX = XB type in the hand-eye calibration using the marker, we transform the solution into an AX = B type without requiring accurate camera parameters. Firstly, the nominal marker pose in the robot base is described by the nominal kinematic parameters of the robot, camera, and end-effector-to-camera. The actual marker pose is obtained by physically contacting the marker with the probe. Subsequently, the error parameter of the hand-eye pose is defined as linear to the error between the nominal and actual marker poses and can be solved by an iterative least-squares algorithm. Finally, the calibrated hand-eye pose is acquired by multiplying its nominal pose and the exponential mapping of the error parameter. The simulations and real experiments illustrate the effectiveness and stability of the proposed algorithm. Noteworthy, the mean position accuracy of the vision-based robot is improved nearly 20 times after calibration. Furthermore, the comparison experiment demonstrates that the proposed algorithm is preferable for improving hand-eye calibration accuracy. [ABSTRACT FROM AUTHOR]

The vision system provides an important way for construction robots to obtain the type and spatial location information of the object. The characteristics of the construction environment, construction object, and robot structure are jointly examined in this paper to propose an approach of object segmentation by spraying the robot based on multi-layer perceptron. Firstly, the hand-eye system experimental platform is built through establishing the mathematical model of the system and calibrating the parameters of the model. Secondly, effort is made to carry out research on image preprocessing algorithms and related experiments, and compare the effects of different binocular stereo-matching algorithms in the actual engineering environment. Finally, research and an experiment are conducted to identify the applicability and effect of the depth image object segmentation algorithm based on multi-layer perceptron. The experimental results prove that the application of multi-layer perceptron to object segmentation by spraying robots can meet the requirement on solution accuracy and is suitable for the object segmentation of complex projects in real life. This approach not only overcomes the shortcomings of the existing recognition methods that are poor in accuracy and difficult to be used widely, but also provides basic data for the subsequent three-dimensional reconstruction, thus making a significant contribution to the research of image processing by spraying robots. [ABSTRACT FROM AUTHOR]

In many measurement applications using multicameras, there is often a nonoverlapping field of view (nFOV) between multicameras. It benefits greatly from having all of the cameras unified in a single coordinate frame by means of common spatial constraints. However, it is very difficult to establish common spatial constraints for accurate and quick calibration of the extrinsic parameters between multicameras in the case of long working distance, asymmetric working angle, or limited working space. To overcome these issues, we propose a flexible calibration method using a camera rig, which can be easily modeled as a hand–eye calibration problem and solved by only several sets of calibration images. To further improve the calibration accuracy, the camera intrinsic parameters, lens distortion coefficients, and extrinsic parameters are optimized simultaneously, similar to binocular calibration. However, this causes calculation instability of multiparameters. We extend the epipolar constraint to two images with an nFOV and add it into the optimization objective function together with the reprojection error, which ensures the basic consistency of intrinsic parameters before and after optimization, and reduce the fluctuation caused by the number of images involved in calibration. Experiments and application have proved that the proposed new methods are feasible and effective. [ABSTRACT FROM AUTHOR]

In the visual robot system, the calibration of the hand-eye system is very important, which has a great influence on the positioning accuracy of the robot. Traditional methods are either complicated or require advanced external equipment. This paper proposes a new flexible method for hand-eye calibration, which is simple and flexible. Firstly, the robot carries the target to perform two transformation motions to solve the rotation relationship, and then the robot tool coordinate system performs several rotation motions to solve the transformation relationship. The paper provides empirical insights about how the robot hand-eye system is calibrated by controlling the robot to perform the specified motion without expensive and complicated 3D measurement equipment. The experiment and analysis indicate that the developed hand-eye calibration has high precision in 6-DOF industrial robot assembly application. [ABSTRACT FROM AUTHOR]

Hand-eye calibration is a fundamental step for a robot equipped with vision systems. However, this problem usually interacts with robot calibration because robot geometric parameters are not very precise. In this article, a new calibration method considering the rotation parameters of the robot pose is proposed. First, a constraint least square model is established assuming that each spherical center measurement of standard ball is equal in the robot base frame, which provides an initial solution. To further improve the solution accuracy, a nonlinear calibration model in the sensor frame is established. Since it can reduce one error accumulation process, a more accurate reference point can be used for optimization. Then, the rotation parameters of the robot pose whose slight errors cause large disturbance to the solution are selected by analyzing the coefficient matrices of the error items. Finally, the hand-eye transformation parameters are refined together with the rotation parameters in the nonlinear optimization solution. Some comparative simulations are performed between the modified least square method, constrained least square method, and the proposed method. The experiments are conducted on a 5-axis hybrid robot named TriMule to demonstrate the superior accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

For visually aided industrial robots, the transformation between the mounted visual information system and the end-effector must be calibrated prior to the practical use. However, during the data acquisition stage, measurement uncertainties will lead to inevitable mismatched data. This article proposes a novel correspondence matching method for the hand-eye calibration system of the type $\boldsymbol {AX} = \boldsymbol {XB}$. The correspondence matching refers to the problem of finding out the well-matched $\boldsymbol {A}$ and $\boldsymbol {B}$ and the relative time delay between them, from a series of measurements. A neat Lie algebra formulation has been obtained to cast the original problem into a high-dimensional point-cloud registration problem. This solution allows for the simultaneous solution of the hand-eye calibration, correspondence matching, and time-delay estimation as well as the uncertainty description of the obtained results. Synthetic simulations verify the correctness of the proposed method. Experimental studies have also confirmed its practical feasibility on an automatic welding platform. [ABSTRACT FROM AUTHOR]

Due to motion constraint of 4-R(2-SS) parallel robot, it is difficult to calculate the translation component of hand–eye calibration based on the existing model solving method accurately. Additionally, the camera calibration error, robot motion error, and invalid calibration motion poses make it difficult to achieve fast and accurate online hand–eye calibration. Therefore, we propose a hand–eye calibration method with motion error compensation and vertical-component correction for 4-R(2-SS) parallel robot by improving the existing eye-to-hand model and solving method. Firstly, the eye-to-hand model of single camera is improved and the robot motion error in the improved model is compensated to reduce the influence of camera calibration error and robot motion error on model accuracy. Secondly, the vertical-component of hand–eye calibration is corrected based on vertical constraint between calibration plate and end effector in parallel robot to calculate the pose and motion error in calibration of 4-R(2-SS) parallel robot accurately. Thirdly, the nontrivial solution constraint of eye-to-hand model is constructed and adopted to remove invalid calibration motion poses and plan calibration motion. Finally, the proposed method was verified by experiments with a fruit sorting system based on 4-R(2-SS) parallel robot. Compared with random motion, the existing model, and solving method, the average time of online calibration based on planned motion decreases by 29.773 s and the average error of calibration based on the improved model and solving method decreases by 151.293. The proposed method can improve the accuracy and efficiency of hand–eye calibration of 4-R(2-SS) parallel robot effectively and further realize accurate and fast grasping. [ABSTRACT FROM AUTHOR]

• A novel robot hand-eye calibration method is proposed without complicated error identification. • Robot hand-eye can be effectively calibrated without calculating the complex matrix equations. • Robot relocalization is implemented to reduce the introduction of robot kinematic error. • The IRLS spherical fitting algorithm is used to suppress the influence of measurement noise. • The error of the proposed method in the criterion sphere splicing verification is only 0.0323 mm. The accuracy of hand-eye calibration is easily affected by robot kinematic error, measurement error, random error, etc., and can be to a certain extent enhanced by the complicated error identification. To address this problem, we propose a novel errors-unidentified hand-eye calibration method. A relocalization-based hand-eye calibration is implemented to overcome the challenge from robot kinematics at first owing to the high robot relocalization accuracy. The tool center point (TCP) coordinates are then obtained based on an iterative reweighted least squares (IRLS) spherical fitting algorithm. An iterative combinatorial refinement algorithm is finally presented to search the solutions in the form of permutation and combination when solving the rigid transformation matrix. Experiments on criterion sphere and blade demonstrate that by converting the hand-eye matrix solving problem into a point set matching problem, the accuracy of the proposed method is enhanced by 60% on average compared with three state-of-the-art hand-eye calibration methods. [ABSTRACT FROM AUTHOR]

Featured Application: This study aims to improve the absolute position error of robot manipulators for vehicle assembly line without using an expensive external apparatus. It is well known that most of the industrial robots have excellent repeatability in positioning. However, the absolute position errors of industrial robots are relatively poor, and in some cases the error may reach even several millimeters, which make it difficult to apply the robot system to vehicle assembly lines that need small position errors. In this paper, we have studied a method to reduce the absolute position error of robots using machine vision and neural network. The position/orientation of robot tool-end is compensated using a vision-based approach combined with a neural network, where a novel indirect calibration approach is presented in order to gather information for training the neural network. In the simulation, the proposed compensation algorithm was found to reduce the positional error to 98%. On average, the absolute position error was 0.029 mm. The application of the proposed algorithm in the actual robot experiment reduced the error to 50.3%, averaging 1.79 mm. [ABSTRACT FROM AUTHOR]

EAST Optical Diagnostic Beam Orientation Robot (EODBOR) is designed for the calibration of the visible optical diagnostic system before the plasma experiment, which can orientate light beams with high precision. At present, the motion of EODBOR is purely controlled by the operators during the whole calibration process since planning algorithms based on prior knowledge of the environment cannot be adopted due to the intrinsically unstructured property of EAST's vacuum vessel. To automate the calibration, we upgraded our previous system and propose a method for environment-aware motion planning based on Robot Operating System (ROS). In this method, the probabilistic octree is used to represent unknown objects in the environment, which is updated with point clouds from an RGB-D camera mounted at the end of the manipulator. We verify the feasibility and stability of the proposed method through simulations and experiments. • An environment-aware motion planning system based on Robot Operating System (ROS) is developed. • A hand–eye calibration method for RGB-D cameras using a calibration sphere is proposed. • Simulations are carried out in a virtual scene based on the real size of the vacuum vessel of EAST and the installation position of the robotic arm. • Experiments using an eye-in-hand system consisting of a UR5 robotic arm and a RealSense D435 validate the feasibility of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract In many industrial sectors, Non Destructive Testing (NDT) methods are used for the thermomechanical analysis of parts in assemblies of engines or reactors or for the control of metal forming processes. This article suggests an automated multi-view approach for the thermal 3 D reconstruction required in order to compute 3 D surface temperature models. This approach is based only on infrared cameras mounted on a Cartesian robot. The low resolution of these cameras associated to a lack of texture to infrared images require to use a global approach based first on an uncalibrated rectification and then on the simultaneous execution, in a single step, of the dense 3 D reconstruction and of an extended self-calibration. The uncalibrated rectification is based on an optimization process under constraints which calculates the homographies without prior calculation of the Fundamental Matrix and which minimizes the projective deformations between the initial images and the rectified ones. The extended self-calibration estimates both the parameters of virtual cameras that could provide the rectified images directly, and the parameters of the robot. It is based on two criteria evaluated according to the noise level of the infrared images. This global approach is validated through the reconstruction of a hot object against a reference reconstruction acquired by a 3 D scanner. [ABSTRACT FROM AUTHOR]

Multirobot comanipulation shows great potential in surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand–eye calibration) and tools (tool–flange calibration), as well as the relationships among the base frames (robot–robot calibration) have to be determined timely to enable accurate robotic cooperation for the constantly changing configuration of the systems. We formulated the problem of hand–eye, tool–flange, and robot–robot calibration to a matrix equation \mathbfAXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two nonsimultaneous methods were compared through thorough simulations under various robot movements and noise levels. Comprehensive experiments on real robots were also performed to further validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method. [ABSTRACT FROM PUBLISHER]

Accurate estimation of robot-world and hand–eye transformations is crucial for robotic vision system. Although as existing methods provide accurate calibration results, which method is best for uncalibrated robot remains a challenging question. This study applied the motion tensor to the dual equations as an operator that performs coordinate frame change, resulting in the most efficient and elegant representation of dual equations. The analytical solution and the numerical solution were given, respectively. The analytical solution requires only four independent variables involving the screw axis, since the pitch and magnitude of a general screw motion are coordinate invariants. The numerical solution used the closed-form rotation solution based on the Kronecker product as an initial value, and followed the optimization with Gauss–Newton algorithm. The superiority with applications in uncalibrated robot was further tested through simulated and experimental tests compared to the existing methods. • The motion-tensor-based dual equations was proposed. • The analytical and numerical separable solutions were given, respectively. • The superiority with applications in uncalibrated robot was further tested. [ABSTRACT FROM AUTHOR]

A robotic manipulator using a stereo camera mounted on one of its links requires a precise kinematic transformation calibration between the manipulator and the camera coordinate frames, the so-called hand–eye calibration, to achieve high-accuracy end-effector positioning. This paper introduces a new method that performs simultaneous joint angle and hand–eye calibration, based on a traditional method that uses a sequence of pure rotations of the manipulator links. The new method considers an additional joint angle constraint, which improves the calibration accuracy when the circular arc that can be measured by the stereo camera is very limited. Experimental results using a manipulator developed for humanitarian demining demonstrate that relative errors between the end effector and the external points mapped by the stereo camera are greatly reduced compared to traditional methods. [ABSTRACT FROM AUTHOR]