Search

Image Jacobian matrix is core portion of robot vision servo system. During estimating image Jacobian matrix with traditional methodology of KF (Kalman Filter) and UKF (Unscented Kalman Filter), the system noises of unknown statistical properties is determined according to priori knowledge, which is remained unchanged in system. However, this methodology of dealing system noises would make image Jacobian matrix’s estimation inaccuracy because of time-varying system noises. Based on traditional methodology of KF and UKF, in this paper, we structured propagator of system noises estimation with the difference value of system state predicted value and updated value, and raised Adoption Unscented Kalman Filter to estimate image Jacobian matrix. Two points on the tomato string’s stem, clamping point and picking point, were chosen as feature points in the image feature space. The center of mass of fruit stem was extracted as the holding point. According to the vertical growth characteristics of tomato clusters and the distance between the holding point on the end and the cutting point on the image, a point above the holding point was extracted as the cutting point. Image Jacobian matrix was used to transform information of pixel difference in image feature space to change in pose of end-effector in manipulator motion space. First, results of image feature space localization experiment showed that six image feature errors were smaller and the average image feature error of visual servo system based on AUKF was 4.978 pixes/mm in condition of unknown system noises. Compared with visual servo system based on KF and UKF, average image feature error reduced 54.44% and 24.78%, respectively. Second, three dimensional space positioning experiment between end-effector and target picking point showed that the final distance between end-effector and target picking point was 4.19 mm, reduced by 79.74% and 80.38% compared to methods of KF and UKF. Third, the response time of visual servo control system based on KF method, UKF method and AUKF method to estimate image Jacobian matrix was 2, 1.5 and 1.2 s, respectively. The picking experiment was carried out on the visual servo test platform of the seven-degree-of-freedom picking manipulator. The visual system based on each visual servo control method was tested for 30 times. For the visual servo system based on AUKF method estimation, the success rate of picking the static target was 90%, and the success rate of picking the dynamic target was 83%. Compared with KF methodology and UKF methodology, the success rate of static picking test was increased by 17 and 10 percentage points, respectively, and the success rate of dynamic picking test was increased by 16 and 10 percentage points, respectively. The results showed that the static picking time based on AUKF methodology was the 18 s, compared with the visual servo system based on KF methodology and UKF methodology, the time of static picking was reduced by 35.71% and 25%, respectively. The average dynamic picking time of visual servo system based on AUKF was 22 s and the test time of dynamic picking was reduced by 35.29% and 26.67%, respectively compared with the visual servo system based on KF methodology and UKF methodology. The image Jacobian matrix based on AUKF was suitable for the dynamic picking of tomato clusters, and the accuracy of visual servo control under dynamic condition was lower than that under static condition. Test results suggested that the proposed AUKF method with KF method and the UKF method compared to estimating by image Jacobin matrix, and AUKF method to estimate the image Jacobin matrix, reduced the process of picking robot visual servo control system noise interference, which made the picking robot visual servo control system of processing noise adapted to the change of the visual servo system, resulting in higher positioning precision for picking robot visual servo control system . [ABSTRACT FROM AUTHOR]

Automatic harvesting of fruits and vegetables is a key process to realize the automation and intellectualization of agricultural production, which can reduce production costs and improve the production efficiency. Tomato is one of the most fruits and vegetables planted in China. The researches on tomato harvesting robots are focused on the single fruit harvesting. The end-effector of the harvesting robot touches the tomato fruit directly and picks one tomato once during harvesting, so the tomato fruits are easy to be damaged and the harvesting efficiency is very low. To overcome the above shortages of the single fruit harvesting, the method of the tomato cluster picking was presented, and the determination method of picking point was developed based on the corner detection on the tomato stem skeleton for the binocular vision system of the tomato cluster picking robot in the paper. The test platform of the vision system was set up and the experiments on the determination of the picking points have been conducted. In the paper, the color images of the tomato clusters were collected in greenhouse in the agricultural production base. The target tomato cluster was segmented from background by OTSU method according to the color features of the tomato clusters and denoised by the morphologic method and threshold method. The centroid of tomato fruit cluster was obtained by enclosing rectangle method. Furthermore, the rectangle region of interest of the target stem was recognized according to constraint relationship between the fruit cluster and the fruit stem, which included the whole stem of the target tomato cluster. Converting the RGB (red, green, blue) color channel map to the HSV (hue, saturation, value) channel map, there are different grayscales between the stem and the background. The target stem of the tomato cluster was separated from background by using the gray difference of H channel and denoised by morphologic method, median filtering method and extracting the maximum connected region. To avoid the effects of the irregular stem on the picking point determination, we abstracted the stem skeletons by Zhang’s thinning algorithm to simplify the stem original shape and extract the kernel information of the stem, and then, the Harris algorithm was applied to extract the corner points of the skeleton, and the corner points were sorted from large to small according to the vertical axis coordinates. The picking feature points on the target stem were obtained by averaging pixel coordinates of the 5 corner points. Sixty groups of experiments on the determination of picking points of the tomato cluster were carried out on the binocular vision test platform. The results show that the success rate of position information obtaining of the picking point on the stem is 90%. Comparing X and Y coordinates data of the picking point and centroid of the tomato fruit clusters, the X coordinates of the picking feature points vary in small range, while Y coordinates vary within wide limits, which results from the main stem growing in longitudinal direction. The mean absolute value of the pixel deviation of the Y coordinates is 21. The research provides the basis for locating the picking point on the stem for the tomato fruit cluster picking robots. [ABSTRACT FROM AUTHOR]

Copyright of Transactions of the Chinese Society of Agricultural Engineering is the property of Chinese Society of Agricultural Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)