Search

With the rapid development of artificial intelligence and computer vision, numerous technologies have been introduced to automate manufacturing in the industry. Typical metal workpieces in the industry often have highly reflective surfaces, come in various sizes, and are positioned irregularly. The motor rotor presented in this paper is one such representative workpiece. Traditional grasping methods for workpiece loading and unloading are pre-programmed and often struggle to cope with complex and disordered situations. In this paper, we introduce a structured light (SL) sensor as the visual guide for the triaxial robot. Furthermore, we propose a high-precision hand-eye calibration method for the non-orthogonal coordinate system of the triaxial robot. Additionally, a motor rotor center localization method based on U-Net image segmentation is proposed. By combining the high-precision hand-eye calibration and localization, we can accurately and automatically locate and grasp the rotor. We have conducted sufficient experiments to verify the effectiveness and accuracy of our system. [ABSTRACT FROM AUTHOR]

Hand-eye calibration, a fundamental task in vision-based robotic systems, is commonly equipped with collaborative robots, especially for robotic applications in small and medium-sized enterprises (SMEs). Most approaches to hand-eye calibration rely on external markers or human assistance. We proposed a novel methodology that addresses the hand-eye calibration problem using the robot base as a reference, eliminating the need for external calibration objects or human intervention. Using point clouds of the robot base, a transformation matrix from the coordinate frame of the camera to the robot base is established as “I=AXB.” To this end, we exploit learning-based 3D detection and registration algorithms to estimate the location and orientation of the robot base. The robustness and accuracy of the method are quantified by ground-truth-based evaluation, and the accuracy result is compared with other 3D vision-based calibration methods. To assess the feasibility of our methodology, we carried out experiments utilizing a low-cost structured light scanner across varying joint configurations and groups of experiments. The proposed hand-eye calibration method achieved a translation deviation of 0.930 mm and a rotation deviation of 0.265 degrees according to the experimental results. Additionally, the 3D reconstruction experiments demonstrated a rotation error of 0.994 degrees and a position error of 1.697 mm. Moreover, our method offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods. We conduct indoor 3D reconstruction and robotic grasping experiments based on our hand-eye calibration method. Related code is released at . [ABSTRACT FROM AUTHOR]

In industrial environments, cameras are heavily influenced by light and perspectives, making traditional vision-based parallel robots struggle with object-sorting tasks. Two-dimensional vision lacks depth information, which limits parallel robots to pick up objects at various heights. This paper proposes a 3D pickup estimation method for parallel robots based on point cloud simplification and registration for multi-objective pickup tasks. Firstly, a point cloud segmentation method based on the spatial domain is proposed to separate incomplete point clouds of object from the robot's workspace. The object dataset is generated by scanning complete point clouds of object using a 3D scanner. Secondly, according to the fast point feature histogram (FPFH) and the weight locally optimal projection (WLOP) algorithms, fusing the FPFH and WLOP (FF-WLOP) method is proposed to simplify the incomplete point cloud and extract more distinctive edge features of objects. The complete point cloud from the dataset is aligned with the simplified incomplete point cloud, and the calculated barycenter's coordinate information is given to the incomplete point cloud. Next, a dynamic weight singular value decomposition (D-SVD) hand-eye calibration method and an optimal projection point strategy are proposed to transform the barycenter coordinates of the object to the optimal pickup coordinates. Experimental results show that the point cloud registration error is 0.38 mm, the pickup rate is 92%, and the robot positioning error is 4.67 mm, which meets the basic pickup requirements. [ABSTRACT FROM AUTHOR]

Marker-less hand–eye calibration permits the acquisition of an accurate transformation between an optical sensor and a robot in unstructured environments. Single monocular cameras, despite their low cost and modest computation requirements, present difficulties for this purpose due to their incomplete correspondence of projected coordinates. In this work, we introduce a hand–eye calibration procedure based on the rotation representations inferred by an augmented autoencoder neural network. Learning-based models that attempt to directly regress the spatial transform of objects such as the links of robotic manipulators perform poorly in the orientation domain, but this can be overcome through the analysis of the latent space vectors constructed in the autoencoding process. This technique is computationally inexpensive and can be run in real time in markedly varied lighting and occlusion conditions. To evaluate the procedure, we use a color-depth camera and perform a registration step between the predicted and the captured point clouds to measure translation and orientation errors and compare the results to a baseline based on traditional checkerboard markers. [ABSTRACT FROM AUTHOR]

This study addresses the problem of hand–eye calibration in robotic systems by developing Continual Learning (CL)-based approaches. Traditionally, robots require explicit models to transfer knowledge from camera observations to their hands or base. However, this poses limitations, as the hand–eye calibration parameters are typically valid only for the current camera configuration. We, therefore, propose a flexible and autonomous hand–eye calibration system that can adapt to changes in camera pose over time. Three CL-based approaches are introduced: the naive CL approach, the reservoir rehearsal approach, and the hybrid approach combining reservoir sampling with new data evaluation. The naive CL approach suffers from catastrophic forgetting, while the reservoir rehearsal approach mitigates this issue by sampling uniformly from past data. The hybrid approach further enhances performance by incorporating reservoir sampling and assessing new data for novelty. Experiments conducted in simulated and real-world environments demonstrate that the CL-based approaches, except for the naive approach, achieve competitive performance compared to traditional batch learning-based methods. This suggests that treating hand–eye calibration as a time sequence problem enables the extension of the learned space without complete retraining. The adaptability of the CL-based approaches facilitates accommodating changes in camera pose, leading to an improved hand–eye calibration system. [ABSTRACT FROM AUTHOR]

Copyright of Food & Machinery is the property of Food & Machinery Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In order to enable rescue robots to accurately detect human joint points in dangerous building environments to replace rescuers into dangerous scenes. A machine learning based human joint point detection method was proposed. Firstly, human joint detection data was obtained through Kinect cameras, and normalized methods were used to smooth and preprocess the joint data. Secondly, human joint length constraints and joint rotation angle constraints were added to the preprocessed data to generate the spatial positions of each human joint point. Finally, the Tsai method was used for hand eye calibration to obtain the distance between the robotic arm and the injured person. The experimental results show that the method accurately measures the position of the casualty joint point in the rescue environment of a dangerous building, which verifies the effectiveness of the proposed detection method. [ABSTRACT FROM AUTHOR]

[Objective] Designing a apace grid-based hand-eye calibration method to reduce robot positioning errors and calibration object Recognition errors in automated food production lines. [Methods] Based on the composition of the attitude matrix, the attitude and position of the end of the robot relative to the camera were obtained by translating and rotating the end of the robot, and a registration method using known corresponding point pairs was proposed to reduce the influence of the robot positioning error on the attitude solution in the translation method. Based on the space circle fitting method, a two-point vector method was proposed to solve the end position, which reduced the amount of data collection. A method of fitting the center of the sphere was proposed to compensate for the result of hand-eye calibration. [Results] The average measured error after calibration is 0.8 mm. [Conclusion] The hand-eye system based on the Eye-In-Hand type, can realize high-precision object positioning.

Aiming at solving the problems such as long calibration time, low precision, and complex operation in hand-eye calibration of welding robot, an automatic hand-eye calibration algorithm based on linear structured light was proposed to solve the calibration matrix X by using AX=ZB calibration equation. Firstly, a square calibration plate is customized to effectively constrain the structured light. The α-shape algorithm was adopted to extract the contour of the 3D point cloud model of the calibration plate. Secondly, an improved random sampling consistency algorithm which could determine the optimal iterative number was proposed to fit the contour point cloud, the contour point cloud model fitted was obtained. Finally, the 3D coordinates of the target points were determined with the linear structured light to complete the hand-eye calibration. In order to prevent the calibration plate from deviating from the acquisition range of the vision sensor during the calibration process, the distance between the linear structural light and the inner circle in the calibration plate was set to limit the motion range of the robot. In order to eliminate the error transfer of the robot body, an optimal solution of the rotation matrix R and the translation vector t of the calibration data was calculated with the singular value decomposition (SVD) and the least square rigid transpose method. The experimental results show that the calibration accuracy reaches 0.3 mm without compensating the robot body error, and the calibration speed is improved by 36% than the existing automatic calibration method. Therefore, the algorithm proposed can automatically complete the calibration only by establishing the user coordinates in advance, which improves the working accuracy and efficiency of the welding robots greatly. [ABSTRACT FROM AUTHOR]

The hand-eye calibration problem is an important application problem in robot research. Based on the 2-norm of dual quaternion vectors, we propose a new dual quaternion optimization method for the hand-eye calibration problem. The dual quaternion optimization problem is decomposed to two quaternion optimization subproblems. The first quaternion optimization subproblem governs the rotation of the robot hand. It can be solved efficiently by the eigenvalue decomposition or singular value decomposition. If the optimal value of the first quaternion optimization subproblem is zero, then the system is rotationwise noiseless, i.e., there exists a "perfect" robot hand motion which meets all the testing poses rotationwise exactly. In this case, we apply the regularization technique for solving the second subproblem to minimize the distance of the translation. Otherwise we apply the patching technique to solve the second quaternion optimization subproblem. Then solving the second quaternion optimization subproblem turns out to be solving a quadratically constrained quadratic program. In this way, we give a complete description for the solution set of hand-eye calibration problems. This is new in the hand-eye calibration literature. The numerical results are also presented to show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

At present, the HUD calibration of the whole vehicle enterprise adopts the traditional physical Master Gauge to calibrate the camera, and the scheme has low flexibility, occupies a large space, and is unsuitable for the mixed production of different types of vehicles. A flexible calibration approach is presented based on collaborative robots for HUD inspection of passenger cars. They are considering that the general HUD (Head-Up Display) virtual image distance is 2–3 m. A 2.3 m hand-eye calibration method is investigated by mounting a monocular camera on a collaborative robot, considering the effect of ambient light on long-distance hand-eye calibration, and employing customized alumina calibration plates and lighting through appropriate angles. Using the area_scan_polynomial model for camera calibration, the camera’s internal reference is solved, and the average error of the resulting camera’s internal reference is about 0.15. Compared with the master gauge camera calibration, it shows higher flexibility. The hand-eye calibration results show that the Translation part’s maximum error is about 1mm, the maximum error of the Rotational position is about 0.1 degree, and the camera calibration error is within 0.08 pixels. For repeatability, the median of Translation RMS was 0.5 with an interquartile range of 0.171, and the median of Translation Maximum was 0.972 with an interquartile range of 0.216. The median of Rotation RMS was 0.0325 with an interquartile range of 0.0125. The median of the Rotation Maximum was 0.059, with an interquartile range of 0.016. The data of translation is relatively scattered, with an extensive range of changes, but it still maintains a certain repeatability. The Rotation data is relatively stable, with a small degree of change and good repeatability, further meeting the use requirements in the final inspection HUD scene of the whole vehicle.

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot's work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand–eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs. [ABSTRACT FROM AUTHOR]

Three dimensional (3D) robot measurement systems, with binocular planar structured light cameras (3D cameras) installed at the terminal flange of robots, are widely used to measure complete workpieces by stitching point clouds obtained from various sampled poses. To ensure accurate stitching, hand–eye calibration is necessary. However, 3D cameras often demonstrate low precision in measuring jumping edges and vertices, resulting in challenges when selecting appropriate calibrators and calibration methods for hand–eye calibration. We propose a simple and high‐precision hand–eye calibration method for 3D robot measurement systems. Initially, a single sphere is utilized as the calibrator, and its poses are observed using specific robotic motions. Subsequently, the hand–eye calibration problem is formulated as a well‐known equation, AX=XB$A X = X B$, relying solely on the high‐precision relative pose of the robot. Finally, a novel simultaneous solution for AX=XB$A X = X B$ and the corresponding closed‐form initial solution are introduced. Simulations and experiments confirm that the proposed method exhibits high noise resistance and achieves high‐precision calibration, even with a limited calibration data quantity. Compared with traditional methods, the fitting error of the proposed method can be reduced from over 0.9 mm to less than 0.6 mm. [ABSTRACT FROM AUTHOR]

Marker-less hand–eye calibration permits the acquisition of an accurate transformation between an optical sensor and a robot in unstructured environments. Single monocular cameras, despite their low cost and modest computation requirements, present difficulties for this purpose due to their incomplete correspondence of projected coordinates. In this work, we introduce a hand–eye calibration procedure based on the rotation representations inferred by an augmented autoencoder neural network. Learning-based models that attempt to directly regress the spatial transform of objects such as the links of robotic manipulators perform poorly in the orientation domain, but this can be overcome through the analysis of the latent space vectors constructed in the autoencoding process. This technique is computationally inexpensive and can be run in real time in markedly varied lighting and occlusion conditions. To evaluate the procedure, we use a color-depth camera and perform a registration step between the predicted and the captured point clouds to measure translation and orientation errors and compare the results to a baseline based on traditional checkerboard markers.

Currently, coal gangue pre-sorting is still mostly done manually, with high labor intensity, low sorting efficiency, and safety hazards. Using coal gangue sorting robots to replace manual coal gangue pre-sorting is an effective way to ensure the health and safety of workers and improve work efficiency. However, the existing coal gangue sorting robots have poor performance in situations such as low light intensity and coal gangue surface covered with coal powder. To solve the above problems, an experimental platform for coal gangue sorting robot based on image detection is proposed. This experimental platform collects coal gangue images through industrial cameras. The platform uses ResNet18-YOLOv3 deep learning algorithm to identify the coal gangue in the images. The platform uses TCP communication to provide the position information of the gangue to the coal gangue sorting module for trajectory planning, then controls the manipulator to clamp the gangue and completes the gangue sorting operation. The platform uses the Halcon calibration method for hand-eye calibration of the experimental platform, in order to achieve the conversion of camera pixel coordinates and manipulator spatial coordinates. The positioning error of the experimental platform is calibrated. For coal gangue samples with sizes above 50 mm, the positioning error should not exceed 9 mm. The experimental results show that the recognition accuracy of the experimental platform for coal gangue under strong lighting conditions is 99%. The recognition accuracy of coal gangue under weak lighting conditions is 95%. The recognition accuracy of coal gangue under pulverized coal adhesion conditions is not less than 82%. The accuracy of coal gangue sorting is 82%.

Three dimensional (3D) robot measurement systems, with binocular planar structured light cameras (3D cameras) installed at the terminal flange of robots, are widely used to measure complete workpieces by stitching point clouds obtained from various sampled poses. To ensure accurate stitching, hand–eye calibration is necessary. However, 3D cameras often demonstrate low precision in measuring jumping edges and vertices, resulting in challenges when selecting appropriate calibrators and calibration methods for hand–eye calibration. We propose a simple and high‐precision hand–eye calibration method for 3D robot measurement systems. Initially, a single sphere is utilized as the calibrator, and its poses are observed using specific robotic motions. Subsequently, the hand–eye calibration problem is formulated as a well‐known equation, AX=XB, relying solely on the high‐precision relative pose of the robot. Finally, a novel simultaneous solution for AX=XB and the corresponding closed‐form initial solution are introduced. Simulations and experiments confirm that the proposed method exhibits high noise resistance and achieves high‐precision calibration, even with a limited calibration data quantity. Compared with traditional methods, the fitting error of the proposed method can be reduced from over 0.9 mm to less than 0.6 mm.

Hand-eye calibration is one of the important problems in the field of robot vision and control, aiming to determine the pose transformation between the robot end-effector and the visual system. The analytical solution of this problem has a more explicit numerical computation process and a more stable solution time compared with iterative or deep learning methods, and theoretically provides more accurate results. When using quaternion parameterization for rigid body motion rotation, the hand-eye calibration problem can be transformed into solving the eigenvector corresponding to the maximum eigenvalue of a symmetric matrix. This paper proposes an analytical solution for hand-eye calibration based on quaternion parameterization. Compared with previous methods, the advantage of this method is that the solution for the required eigenvector is transformed into solving cubic equations through the eigenvector-eigenvalue identity, without SVD or Eigendecomposition, thus making the proposed method a complete analytical solution, and avoids solving irrelevant information. The performance of the proposed method was evaluated through multiple experiments conducted on a synthetic dataset as well as two real-world datasets, and compared to representative analytical methods. The experimental results unequivocally demonstrate that our proposed method achieves comparable accuracy with significantly shorter solution times. Therefore, our complete analytical solution offers an efficient and accurate alternative for addressing the hand-eye calibration problem without SVD or Eigendecomposition. [ABSTRACT FROM AUTHOR]

With the promotion of Industry 4.0 reform, the trend of intelligent and precise production in the production workshop is gradually highlighted. This directly leads to higher requirements for robot hand eye coordination accuracy in automated workshops. In order to achieve more precise robot hand eye coordination control, this study designed a new mean calculation method based on the probability density theory, and designed a new mean robot hand eye calibration algorithm based on this. After the test, it is found that the translation error and rotation error calculated by the new mean algorithm are 0.26 and 0.92 respectively, which are significantly lower than other comparison algorithms when using all test samples of normal distribution. And the calculation time of the algorithm when using all the test samples is 2115 ms, which is also significantly lower than the comparison algorithm. The simulation results show that the new mean hand eye calibration method designed in this study can achieve more accurate hand eye coordination control of robots, and has certain application potential in high-precision industrial production scenarios. [ABSTRACT FROM AUTHOR]

In the functioning of the hand–eye collaboration of an apple picking robot, the accuracy of the hand–eye relationship is a key factor affecting the efficiency and accuracy of the robot's operation. In order to enhance the low accuracy of traditional hand–eye calibration methods, linear and nonlinear solving methods based on mathematical tools such as quaternions are commonly adopted. To solve the loss of accuracy in decoupling during the linearization solution and to reduce the cumulative error that occurs during nonlinear solutions, a hand–eye calibration method, based on the ICP algorithm, is proposed in this paper. The method initializes the ICP matching algorithm with a solution derived from Tsai–Lenz, and substitutes it for iterative computation, thereby ascertaining a precise hand–eye conversion relationship by optimizing the error threshold and iteration count in the ICP matching process. Experimental results demonstrate that the ICP-based hand–eye calibration optimization algorithm not only circumvents the issues pertaining to accuracy loss and significant errors during solving, but also enhances the rotation accuracy by 13.6% and the translation accuracy by 2.47% compared with the work presented by Tsai–Lenz. [ABSTRACT FROM AUTHOR]

For a robot to pick up an object viewed by a camera, the object’s position in the image coordinate system must be converted to the robot coordinate system. Recently, a neural network-based method was proposed to achieve this task. This methodology can accurately convert the object’s position despite errors and disturbances that arise in a real-world environment, such as the deflection of a robot arm triggered by changes in the robot’s posture. However, this method has some drawbacks, such as the need for significant effort in model selection, hyperparameter tuning, and lack of stability and interpretability in the learning results. To address these issues, a method involving linear and nonlinear regressions is proposed. First, linear regression is employed to convert the object’s position from the image coordinate system to the robot base coordinate system. Next, B-splines-based nonlinear regression is applied to address the errors and disturbances that occur in a real-world environment. Since this approach is more stable and has better calibration performance with interpretability as opposed to the recent method, it is more practical. In the experiment, calibration results were incorporated into a robot, and its performance was evaluated quantitatively. The proposed method achieved a mean position error of 0.5 mm, while the neural network-based method achieved an error of 1.1 mm.

Hand-eye calibration is crucial for vision-based robots using open-loop visual control. Based on the product of exponentials model, this study introduces a novel hand-eye calibration algorithm to improve calibration accuracy. As opposed to the traditional method of solving the AX = XB type in the hand-eye calibration using the marker, we transform the solution into an AX = B type without requiring accurate camera parameters. Firstly, the nominal marker pose in the robot base is described by the nominal kinematic parameters of the robot, camera, and end-effector-to-camera. The actual marker pose is obtained by physically contacting the marker with the probe. Subsequently, the error parameter of the hand-eye pose is defined as linear to the error between the nominal and actual marker poses and can be solved by an iterative least-squares algorithm. Finally, the calibrated hand-eye pose is acquired by multiplying its nominal pose and the exponential mapping of the error parameter. The simulations and real experiments illustrate the effectiveness and stability of the proposed algorithm. Noteworthy, the mean position accuracy of the vision-based robot is improved nearly 20 times after calibration. Furthermore, the comparison experiment demonstrates that the proposed algorithm is preferable for improving hand-eye calibration accuracy. [ABSTRACT FROM AUTHOR]

With the increasing automation of mobile phone assembly, industrial robots are gradually being used in production lines for loading and unloading operations. At present, industrial robots are mainly used in online teaching mode, in which the robot's movement and path are set by teaching in advance and then repeat the point-to-point operation. This mode of operation is less flexible and requires high professionalism in teaching and offline programming. When positioning and grasping different materials, the adjustment time is long, which affects the efficiency of production changeover. To solve the problem of poor adaptability of loading robots to differentiated products in mobile phone automatic assembly lines, it is necessary to quickly adjust the positioning and grasping of different models of mobile phone middle frames. Therefore, this paper proposes a highly adaptive grasping and positioning method for vision-guided right-angle robots. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Mine Automation is the property of Industry & Mine Automation Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

For a robot to pick up an object viewed by a camera, the object's position in the image coordinate system must be converted to the robot coordinate system. Recently, a neural network-based method was proposed to achieve this task. This methodology can accurately convert the object's position despite errors and disturbances that arise in a real-world environment, such as the deflection of a robot arm triggered by changes in the robot's posture. However, this method has some drawbacks, such as the need for significant effort in model selection, hyperparameter tuning, and lack of stability and interpretability in the learning results. To address these issues, a method involving linear and nonlinear regressions is proposed. First, linear regression is employed to convert the object's position from the image coordinate system to the robot base coordinate system. Next, B-splines-based nonlinear regression is applied to address the errors and disturbances that occur in a real-world environment. Since this approach is more stable and has better calibration performance with interpretability as opposed to the recent method, it is more practical. In the experiment, calibration results were incorporated into a robot, and its performance was evaluated quantitatively. The proposed method achieved a mean position error of 0.5 mm, while the neural network-based method achieved an error of 1.1 mm. [ABSTRACT FROM AUTHOR]

This paper studies the effect of reference frame selection in sensor-to-sensor extrinsic calibration when formulated as a motion-based hand–eye calibration problem. As the sensor trajectories typically contain some composition of noise, the aim is to determine which selection strategies work best under which noise conditions. Different reference selection options are tested under varying noise conditions in simulations, and the findings are validated with real data from the KITTI dataset. The study is conducted for four state-of-the-art methods, as well as two proposed cost functions for nonlinear optimization. One of the proposed cost functions incorporates outlier rejection to improve calibration performance and was shown to significantly improve performance in the presence of outliers, and either match or outperform the other algorithms in other noise conditions. However, the performance gain from reference frame selection was deemed larger than that from algorithm selection. In addition, we show that with realistic noise, the reference frame selection method commonly used in the literature, is inferior to other tested options, and that relative error metrics are not reliable for telling which method achieves best calibration performance. [ABSTRACT FROM AUTHOR]

The measurement system proposed in this paper, using a measuring arm and line structured light, has a wide range of applications. To improve the scanning efficiency, the system outlined in this paper uses two single-line structured lights to form crosshair structured light, which we combine with a measuring arm to form a comprehensive scanning measurement system. The calibration method of Zhengyou Zhang and a calibration board are used to complete parameter calibration of the sensors and cameras, as well as hand–eye calibration of the measuring arm. For complex curved-surface objects, this system extracts the cross-line structured light optical center location, which suffers from ambiguity. Therefore, we introduce the use of periodic control of the two line structured light sources in order to resolve the light extraction polysemy. Our experimental results indicate that the proposed system can effectively satisfy the function of crosshair structured light scanning of large, complex surfaces. [ABSTRACT FROM AUTHOR]

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot’s work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand–eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs.

The Euclidean distance error of calibration results cannot be calculated during the hand–eye calibration process of a manipulator because the true values of the hand–eye conversion matrix cannot be obtained. In this study, a new method for error analysis and algorithm optimization is presented. An error analysis of the method is carried out using a priori knowledge that the location of the augmented reality markers is fixed during the calibration process. The coordinates of the AR marker center point are reprojected onto the pixel coordinate system and then compared with the true pixel coordinates of the AR marker center point obtained by corner detection or manual labeling to obtain the Euclidean distance between the two coordinates as the basis for the error analysis. We then fine-tune the results of the hand–eye calibration algorithm to obtain the smallest reprojection error, thereby obtaining higher-precision calibration results. The experimental results show that, compared with the Tsai–Lenz algorithm, the optimized algorithm in this study reduces the average reprojection error by 44.43% and the average visual positioning error by 50.63%. Therefore, the proposed optimization method can significantly improve the accuracy of hand–eye calibration results.

Multi-line LiDAR and GPS/IMU are widely used in autonomous driving and robotics, such as simultaneous localization and mapping (SLAM). Calibrating the extrinsic parameters of each sensor is a necessary condition for multi-sensor fusion. The calibration of each sensor directly affects the accurate positioning control and perception performance of the vehicle. Through the algorithm, accurate extrinsic parameters and a symmetric covariance matrix of extrinsic parameters can be obtained as a measure of the confidence of the extrinsic parameters. As for the calibration of LiDAR-GPS/IMU, many calibration methods require specific vehicle motion or manual calibration marking scenes to ensure good constraint of the problem, resulting in high costs and a low degree of automation. To solve this problem, we propose a new two-step self-calibration method, which includes extrinsic parameter initialization and refinement. The initialization part decouples the extrinsic parameters from the rotation and translation part, first calculating the reliable initial rotation through the rotation constraints, then calculating the initial translation after obtaining a reliable initial rotation, and eliminating the accumulated drift of LiDAR odometry by loop closure to complete the map construction. In the refinement part, the LiDAR odometry is obtained through scan-to-map registration and is tightly coupled with the IMU. The constraints of the absolute pose in the map refined the extrinsic parameters. Our method is validated in the simulation and real environments, and the results show that the proposed method has high accuracy and robustness. [ABSTRACT FROM AUTHOR]

Many approaches have been developed to solve the hand–eye calibration problem. The traditional approach involves a precise mathematical model, which has advantages and disadvantages. For example, mathematical representations can provide numerical and quantitative results to users and researchers. Thus, it is possible to explain and understand the calibration results. However, information about the end-effector, such as the position attached to the robot and its dimensions, is not considered in the calibration process. If there is no CAD model, additional calibration is required for accurate manipulation, especially for a handmade end-effector. A neural network-based method is used as the solution to this problem. By training a neural network model using data created via the attached end-effector, additional calibration can be avoided. Moreover, it is not necessary to develop a precise and complex mathematical model. However, it is difficult to provide quantitative information because a neural network is a black box. Hence, a method with both advantages is proposed in this study. A mathematical model was developed and optimized using the data created by the attached end-effector. To acquire accurate data and evaluate the calibration results, a tablet computer was utilized. The established method achieved a mean positioning error of 1.0 mm. [ABSTRACT FROM AUTHOR]

Copyright of Machine Tool & Hydraulics is the property of Guangzhou Mechanical Engineering Research Institute (GMERI) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Aiming at the stability of hand-eye calibration in fruit picking scene, a simple hand-eye calibration method for picking robot based on optimization combined with TOF (Time of Flight) camera is proposed. This method needs to fix the TOF depth camera at actual and calculated coordinates of the peach the end of the robot, operate the robot to take pictures of the calibration board from different poses, and record the current photographing poses to ensure that each group of pictures is clear and complete, so as to use the TOF depth camera to image the calibration board. Obtain multiple sets of calibration board depth maps and corresponding point cloud data, that is, “eye” data. Through the circle center extraction and positioning algorithm, the circle center points on each group of calibration plates are extracted, and a circle center sorting method based on the vector angle and the center of mass coordinates is designed to solve the circle center caused by factors such as mirror distortion, uneven illumination and different photographing poses. And through the tool center point of the actuator, the coordinate value of the circle center point on the four corners of each group of calibration plates in the robot end coordinate system is located in turn, and the “hand” data is obtained. Combined with the SVD method, And according to the obtained point residuals, the weight coefficients of the marker points are redistributed, and the hand-eye parameters are iteratively optimized, which improves the accuracy and stability of the hand-eye calibration. the method proposed in this paper has a better ability to locate the gross error under the environment of large gross errors. In order to verify the feasibility of the hand-eye calibration method, the indoor picking experiment was simulated, and the peaches were identified and positioned by combining deep learning and 3D vision to verify the proposed hand-eye calibration method. The JAKA six-axis robot and TuYang depth camera are used to build the experimental platform. The experimental results show that the method is simple to operate, has good stability, and the calibration plate is easy to manufacture and low in cost. work accuracy requirements. [ABSTRACT FROM AUTHOR]

The vision system provides an important way for construction robots to obtain the type and spatial location information of the object. The characteristics of the construction environment, construction object, and robot structure are jointly examined in this paper to propose an approach of object segmentation by spraying the robot based on multi-layer perceptron. Firstly, the hand-eye system experimental platform is built through establishing the mathematical model of the system and calibrating the parameters of the model. Secondly, effort is made to carry out research on image preprocessing algorithms and related experiments, and compare the effects of different binocular stereo-matching algorithms in the actual engineering environment. Finally, research and an experiment are conducted to identify the applicability and effect of the depth image object segmentation algorithm based on multi-layer perceptron. The experimental results prove that the application of multi-layer perceptron to object segmentation by spraying robots can meet the requirement on solution accuracy and is suitable for the object segmentation of complex projects in real life. This approach not only overcomes the shortcomings of the existing recognition methods that are poor in accuracy and difficult to be used widely, but also provides basic data for the subsequent three-dimensional reconstruction, thus making a significant contribution to the research of image processing by spraying robots. [ABSTRACT FROM AUTHOR]