Search

With the rapid development of artificial intelligence and computer vision, numerous technologies have been introduced to automate manufacturing in the industry. Typical metal workpieces in the industry often have highly reflective surfaces, come in various sizes, and are positioned irregularly. The motor rotor presented in this paper is one such representative workpiece. Traditional grasping methods for workpiece loading and unloading are pre-programmed and often struggle to cope with complex and disordered situations. In this paper, we introduce a structured light (SL) sensor as the visual guide for the triaxial robot. Furthermore, we propose a high-precision hand-eye calibration method for the non-orthogonal coordinate system of the triaxial robot. Additionally, a motor rotor center localization method based on U-Net image segmentation is proposed. By combining the high-precision hand-eye calibration and localization, we can accurately and automatically locate and grasp the rotor. We have conducted sufficient experiments to verify the effectiveness and accuracy of our system. [ABSTRACT FROM AUTHOR]

Hand-eye calibration, a fundamental task in vision-based robotic systems, is commonly equipped with collaborative robots, especially for robotic applications in small and medium-sized enterprises (SMEs). Most approaches to hand-eye calibration rely on external markers or human assistance. We proposed a novel methodology that addresses the hand-eye calibration problem using the robot base as a reference, eliminating the need for external calibration objects or human intervention. Using point clouds of the robot base, a transformation matrix from the coordinate frame of the camera to the robot base is established as “I=AXB.” To this end, we exploit learning-based 3D detection and registration algorithms to estimate the location and orientation of the robot base. The robustness and accuracy of the method are quantified by ground-truth-based evaluation, and the accuracy result is compared with other 3D vision-based calibration methods. To assess the feasibility of our methodology, we carried out experiments utilizing a low-cost structured light scanner across varying joint configurations and groups of experiments. The proposed hand-eye calibration method achieved a translation deviation of 0.930 mm and a rotation deviation of 0.265 degrees according to the experimental results. Additionally, the 3D reconstruction experiments demonstrated a rotation error of 0.994 degrees and a position error of 1.697 mm. Moreover, our method offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods. We conduct indoor 3D reconstruction and robotic grasping experiments based on our hand-eye calibration method. Related code is released at . [ABSTRACT FROM AUTHOR]

In industrial environments, cameras are heavily influenced by light and perspectives, making traditional vision-based parallel robots struggle with object-sorting tasks. Two-dimensional vision lacks depth information, which limits parallel robots to pick up objects at various heights. This paper proposes a 3D pickup estimation method for parallel robots based on point cloud simplification and registration for multi-objective pickup tasks. Firstly, a point cloud segmentation method based on the spatial domain is proposed to separate incomplete point clouds of object from the robot's workspace. The object dataset is generated by scanning complete point clouds of object using a 3D scanner. Secondly, according to the fast point feature histogram (FPFH) and the weight locally optimal projection (WLOP) algorithms, fusing the FPFH and WLOP (FF-WLOP) method is proposed to simplify the incomplete point cloud and extract more distinctive edge features of objects. The complete point cloud from the dataset is aligned with the simplified incomplete point cloud, and the calculated barycenter's coordinate information is given to the incomplete point cloud. Next, a dynamic weight singular value decomposition (D-SVD) hand-eye calibration method and an optimal projection point strategy are proposed to transform the barycenter coordinates of the object to the optimal pickup coordinates. Experimental results show that the point cloud registration error is 0.38 mm, the pickup rate is 92%, and the robot positioning error is 4.67 mm, which meets the basic pickup requirements. [ABSTRACT FROM AUTHOR]

Marker-less hand–eye calibration permits the acquisition of an accurate transformation between an optical sensor and a robot in unstructured environments. Single monocular cameras, despite their low cost and modest computation requirements, present difficulties for this purpose due to their incomplete correspondence of projected coordinates. In this work, we introduce a hand–eye calibration procedure based on the rotation representations inferred by an augmented autoencoder neural network. Learning-based models that attempt to directly regress the spatial transform of objects such as the links of robotic manipulators perform poorly in the orientation domain, but this can be overcome through the analysis of the latent space vectors constructed in the autoencoding process. This technique is computationally inexpensive and can be run in real time in markedly varied lighting and occlusion conditions. To evaluate the procedure, we use a color-depth camera and perform a registration step between the predicted and the captured point clouds to measure translation and orientation errors and compare the results to a baseline based on traditional checkerboard markers. [ABSTRACT FROM AUTHOR]

This study addresses the problem of hand–eye calibration in robotic systems by developing Continual Learning (CL)-based approaches. Traditionally, robots require explicit models to transfer knowledge from camera observations to their hands or base. However, this poses limitations, as the hand–eye calibration parameters are typically valid only for the current camera configuration. We, therefore, propose a flexible and autonomous hand–eye calibration system that can adapt to changes in camera pose over time. Three CL-based approaches are introduced: the naive CL approach, the reservoir rehearsal approach, and the hybrid approach combining reservoir sampling with new data evaluation. The naive CL approach suffers from catastrophic forgetting, while the reservoir rehearsal approach mitigates this issue by sampling uniformly from past data. The hybrid approach further enhances performance by incorporating reservoir sampling and assessing new data for novelty. Experiments conducted in simulated and real-world environments demonstrate that the CL-based approaches, except for the naive approach, achieve competitive performance compared to traditional batch learning-based methods. This suggests that treating hand–eye calibration as a time sequence problem enables the extension of the learned space without complete retraining. The adaptability of the CL-based approaches facilitates accommodating changes in camera pose, leading to an improved hand–eye calibration system. [ABSTRACT FROM AUTHOR]

Copyright of Food & Machinery is the property of Food & Machinery Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In order to enable rescue robots to accurately detect human joint points in dangerous building environments to replace rescuers into dangerous scenes. A machine learning based human joint point detection method was proposed. Firstly, human joint detection data was obtained through Kinect cameras, and normalized methods were used to smooth and preprocess the joint data. Secondly, human joint length constraints and joint rotation angle constraints were added to the preprocessed data to generate the spatial positions of each human joint point. Finally, the Tsai method was used for hand eye calibration to obtain the distance between the robotic arm and the injured person. The experimental results show that the method accurately measures the position of the casualty joint point in the rescue environment of a dangerous building, which verifies the effectiveness of the proposed detection method. [ABSTRACT FROM AUTHOR]

[Objective] Designing a apace grid-based hand-eye calibration method to reduce robot positioning errors and calibration object Recognition errors in automated food production lines. [Methods] Based on the composition of the attitude matrix, the attitude and position of the end of the robot relative to the camera were obtained by translating and rotating the end of the robot, and a registration method using known corresponding point pairs was proposed to reduce the influence of the robot positioning error on the attitude solution in the translation method. Based on the space circle fitting method, a two-point vector method was proposed to solve the end position, which reduced the amount of data collection. A method of fitting the center of the sphere was proposed to compensate for the result of hand-eye calibration. [Results] The average measured error after calibration is 0.8 mm. [Conclusion] The hand-eye system based on the Eye-In-Hand type, can realize high-precision object positioning.

Aiming at solving the problems such as long calibration time, low precision, and complex operation in hand-eye calibration of welding robot, an automatic hand-eye calibration algorithm based on linear structured light was proposed to solve the calibration matrix X by using AX=ZB calibration equation. Firstly, a square calibration plate is customized to effectively constrain the structured light. The α-shape algorithm was adopted to extract the contour of the 3D point cloud model of the calibration plate. Secondly, an improved random sampling consistency algorithm which could determine the optimal iterative number was proposed to fit the contour point cloud, the contour point cloud model fitted was obtained. Finally, the 3D coordinates of the target points were determined with the linear structured light to complete the hand-eye calibration. In order to prevent the calibration plate from deviating from the acquisition range of the vision sensor during the calibration process, the distance between the linear structural light and the inner circle in the calibration plate was set to limit the motion range of the robot. In order to eliminate the error transfer of the robot body, an optimal solution of the rotation matrix R and the translation vector t of the calibration data was calculated with the singular value decomposition (SVD) and the least square rigid transpose method. The experimental results show that the calibration accuracy reaches 0.3 mm without compensating the robot body error, and the calibration speed is improved by 36% than the existing automatic calibration method. Therefore, the algorithm proposed can automatically complete the calibration only by establishing the user coordinates in advance, which improves the working accuracy and efficiency of the welding robots greatly. [ABSTRACT FROM AUTHOR]

The hand-eye calibration problem is an important application problem in robot research. Based on the 2-norm of dual quaternion vectors, we propose a new dual quaternion optimization method for the hand-eye calibration problem. The dual quaternion optimization problem is decomposed to two quaternion optimization subproblems. The first quaternion optimization subproblem governs the rotation of the robot hand. It can be solved efficiently by the eigenvalue decomposition or singular value decomposition. If the optimal value of the first quaternion optimization subproblem is zero, then the system is rotationwise noiseless, i.e., there exists a "perfect" robot hand motion which meets all the testing poses rotationwise exactly. In this case, we apply the regularization technique for solving the second subproblem to minimize the distance of the translation. Otherwise we apply the patching technique to solve the second quaternion optimization subproblem. Then solving the second quaternion optimization subproblem turns out to be solving a quadratically constrained quadratic program. In this way, we give a complete description for the solution set of hand-eye calibration problems. This is new in the hand-eye calibration literature. The numerical results are also presented to show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

At present, the HUD calibration of the whole vehicle enterprise adopts the traditional physical Master Gauge to calibrate the camera, and the scheme has low flexibility, occupies a large space, and is unsuitable for the mixed production of different types of vehicles. A flexible calibration approach is presented based on collaborative robots for HUD inspection of passenger cars. They are considering that the general HUD (Head-Up Display) virtual image distance is 2–3 m. A 2.3 m hand-eye calibration method is investigated by mounting a monocular camera on a collaborative robot, considering the effect of ambient light on long-distance hand-eye calibration, and employing customized alumina calibration plates and lighting through appropriate angles. Using the area_scan_polynomial model for camera calibration, the camera’s internal reference is solved, and the average error of the resulting camera’s internal reference is about 0.15. Compared with the master gauge camera calibration, it shows higher flexibility. The hand-eye calibration results show that the Translation part’s maximum error is about 1mm, the maximum error of the Rotational position is about 0.1 degree, and the camera calibration error is within 0.08 pixels. For repeatability, the median of Translation RMS was 0.5 with an interquartile range of 0.171, and the median of Translation Maximum was 0.972 with an interquartile range of 0.216. The median of Rotation RMS was 0.0325 with an interquartile range of 0.0125. The median of the Rotation Maximum was 0.059, with an interquartile range of 0.016. The data of translation is relatively scattered, with an extensive range of changes, but it still maintains a certain repeatability. The Rotation data is relatively stable, with a small degree of change and good repeatability, further meeting the use requirements in the final inspection HUD scene of the whole vehicle.

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot's work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand–eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs. [ABSTRACT FROM AUTHOR]

Marker-less hand–eye calibration permits the acquisition of an accurate transformation between an optical sensor and a robot in unstructured environments. Single monocular cameras, despite their low cost and modest computation requirements, present difficulties for this purpose due to their incomplete correspondence of projected coordinates. In this work, we introduce a hand–eye calibration procedure based on the rotation representations inferred by an augmented autoencoder neural network. Learning-based models that attempt to directly regress the spatial transform of objects such as the links of robotic manipulators perform poorly in the orientation domain, but this can be overcome through the analysis of the latent space vectors constructed in the autoencoding process. This technique is computationally inexpensive and can be run in real time in markedly varied lighting and occlusion conditions. To evaluate the procedure, we use a color-depth camera and perform a registration step between the predicted and the captured point clouds to measure translation and orientation errors and compare the results to a baseline based on traditional checkerboard markers.

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot’s work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand–eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs.

This paper investigates the direct application of standardized designs on the robot for conducting robot hand–eye calibration by employing 3D scanners with collaborative robots. The well-established geometric features of the robot flange are exploited by directly capturing its point cloud data. In particular, an iterative method is proposed to facilitate point cloud processing towards a refined calibration outcome. Several extensive experiments are conducted over a range of collaborative robots, including Universal Robots UR5 & UR10 e-series, Franka Emika, and AUBO i5 using an industrial-grade 3D scanner Photoneo Phoxi S & M and a commercial-grade 3D scanner Microsoft Azure Kinect DK. Experimental results show that translational and rotational errors converge efficiently to less than 0.28 mm and 0.25 degrees, respectively, achieving a hand–eye calibration accuracy as high as the camera's resolution, probing the hardware limit. A welding seam tracking system is presented, combining the flange-based calibration method with soft tactile sensing. The experiment results show that the system enables the robot to adjust its motion in real-time, ensuring consistent weld quality and paving the way for more efficient and adaptable manufacturing processes. [Display omitted] • A novel hand–eye calibration method with intrinsic design features of robot. • An iterative algorithm for effective optimization of hand–eye calibration. • Quantitative evaluation of the calibration method on various cobots and 3D scanners. • A safe and adaptable weld seam tracking system with the calibration method. [ABSTRACT FROM AUTHOR]

• A cylinder instead of sphere is used for calibration, which is easy to manufacture and can cover a larger robot workspace. • An unconstrained dimensionality reduction method is proposed to accurately fit the incomplete conic profiles as ellipses. • The calibration error of hand eye parameters is reduced by about 50% compared to feature points col-linearity based method. This paper proposed a hand-eye calibration method for a line structured-light (LSL) scanner mounted on industrial robot. Unlike widely used spherical targets, the target used in our work is a cylinder, which is inexpensive and easy to be manufactured in larger size to cover a wider calibration space. In order to address the profile self-occlusion and drastically shape changing of conic section during calibration, an unconstrained dimensionality reduction method is proposed to fit the scanned profiles. Based on fitted ellipse centers under various robot poses, the hand-eye calibration model is developed based on the difference between cylinder radius and the distance from the profile point to the cylinder axis. Simulation and actual experiments have shown that the proposed ellipse fitting method is more accurate and robust, and the calibration method achieves higher reconstruction accuracy under the same robot positioning error, need less robot operations to obtain more accurate hand-eye parameters. [ABSTRACT FROM AUTHOR]

In order to solve the problem of noise interference in camera calibration and robot forward kinematics solution, this study proposes a dual quaternion hand-eye calibration algorithm based on twice opposition-learning and random differential variation (ODHPO). The hand-eye calibration equation was innovatively rewritten in the form of dual quaternion, and the F-norm minimization model of the rotation and translation error function was constructed for optimization. At the same time, the penalty function method is introduced to effectively transform the constrained optimization problem in hand-eye calibration into an unconstrained problem, which is further solved by the ODHPO algorithm for global optimization. Compared with the singular value decomposition algorithm based on the traditional dual quaternion method, the ODHPO algorithm performs better in global optimization ability and convergence stability. Through numerical simulation and real robot hand-eye calibration experiments, it is proved that the proposed algorithm is superior to the traditional dual quaternion (CDQ) algorithm, the classical algorithms Tsai method and Navy method in terms of solution accuracy, sensitivity to the number of pose transformations and stability, demonstrating its potential for application and practical significance in robot vision system. ● An improved hunter-prey optimization algorithm with twice opposition-learning and random difference variation is proposed. ● A dual quaternion hand-eye calibration algorithm based on improved hunter-prey optimization is proposed. ● The effectiveness of the improved algorithm is verified by 12 benchmark test functions ● The proposed calibration algorithm is superior to other traditional calibration algorithms [ABSTRACT FROM AUTHOR]

The dual quaternion plays an advantageous role in the field of robotic kinematics and dynamics due to its compact expression. It is well known that quaternion can be easily implemented as algebraic operations on vectors through matrix operators. Similarly, the dual matrix operators can convert dual quaternion operations to matrix operations. However, it has not received adequate attention, and thus there are very few applications at present. With the dual matrix operators of dual quaternion, this paper re-verifies the equivalency between the conjugate formula of unit dual quaternion and dual Euler–Rodrigues formula. To combine this equivalence with the homomorphic mapping of Lie groups, a theoretical correlation of the current hand-eye calibration methods is established. Motivated by the attempt to establish a more complete scheme for hand-eye calibration, a new simultaneous method based on the conjugate formula of dual quaternion is proposed. The method verifies that the rotation problem can be extended to the rigid-body transformation problem via the dual algebra. Finally, the simulation and experimental validations for the hand–eye calibration method are given. • The conjugate formula of unit dual quaternion is re-verified via the dual operations. • A new simultaneous method is defined for hand-eye calibration. • The theoretical correlation with the existing methods is established. [ABSTRACT FROM AUTHOR]

• A novel robot hand-eye calibration method is proposed without complicated error identification. • Robot hand-eye can be effectively calibrated without calculating the complex matrix equations. • Robot relocalization is implemented to reduce the introduction of robot kinematic error. • The IRLS spherical fitting algorithm is used to suppress the influence of measurement noise. • The error of the proposed method in the criterion sphere splicing verification is only 0.0323 mm. The accuracy of hand-eye calibration is easily affected by robot kinematic error, measurement error, random error, etc., and can be to a certain extent enhanced by the complicated error identification. To address this problem, we propose a novel errors-unidentified hand-eye calibration method. A relocalization-based hand-eye calibration is implemented to overcome the challenge from robot kinematics at first owing to the high robot relocalization accuracy. The tool center point (TCP) coordinates are then obtained based on an iterative reweighted least squares (IRLS) spherical fitting algorithm. An iterative combinatorial refinement algorithm is finally presented to search the solutions in the form of permutation and combination when solving the rigid transformation matrix. Experiments on criterion sphere and blade demonstrate that by converting the hand-eye matrix solving problem into a point set matching problem, the accuracy of the proposed method is enhanced by 60% on average compared with three state-of-the-art hand-eye calibration methods. [ABSTRACT FROM AUTHOR]