Search

Automatic navigation has improved the efficiency of farmland operation for the high crop yield. Visual navigation can be expected to realize the automatic navigation in the farmland operation machinery, because of its low hardware cost and wide application. However, the traditional algorithm of image processing cannot fully deal with the interference of light and shadow changes, occlusion, and weeds under the complex environment in an orchard. There is a high demand for better robustness and accuracy in the navigation path. In this study, an accurate and rapid detection of navigation lines was proposed for the orchard using the improved YOLOv7 deep learning model. Firstly, the attention mechanism module (Convolutional Block Attention Module, CBAM) containing the channel and spatial attention module was added to the detection head network of the original YOLOv7 structure. This attention mechanism enabled to more accurately capture the essential features of the target for better representation and generalization of the network. The efficient extraction and enhancement of the trunk key features were realized to weaken the environmental background interference. Secondly, the SPD-Conv (space-to-depth, nonstrided convolution, SPD-Conv, i.e., a convolution-free step or pooling) was introduced between the ELAN-H and Repconv modules. As such, the low-resolution images or small-size targets were also detected to reduce the missed and false detection. The data set was collected from Beijing Xingshou Agricultural Professional Cooperative in Changping District, Beijing, China. 28 videos of orchards and a total of 1588 images were obtained under different lighting conditions. The targets were labeled using Labelimg software, with a total of 11043 apple trunks. The detection accuracy of the improved YOLOv7 model was 95.21% after training on the dataset, where the detection speed was 42.07 frames/s. The improved model increased by 2.31 percentage points, and 4.85 frames/s, respectively, compared with the original. Therefore, the improved model can be expected to more accurately identify the trunks of fruit trees, suitable for the apple orchard and jujube garden with the dense tree trunks. The ablation experiments were performed on each improvement point. The model accuracy of 93.97% was achieved after the introduction of the CBAM attention mechanism module, which was 1.07 percentage points over the original. The CBAM attention mechanism and SPD-Conv modules were introduced with a precision of 95.21%, which was a 1.24 percentage point improvement than before, indicating the better effectiveness of each improvement module. The trunk root accurate extraction of midpoint coordinates was crucial for the fitting of the navigation line, particularly with the trunk root midpoint as the navigation positioning base point. The coordinates of the improved YOLOv7 training for the fruit tree trunk were set at the bottom of the rectangular frame, instead of the trunk root midpoint as the locating reference point. Finally, the locating reference points were fitted on both sides of the fruit tree line and navigation line using the least squares method. The average line error of 4.43 pixels was achieved in the error analysis of 769 locating reference points extracted from the randomly selected 100 images from the data set and the manual marking of the trunk midpoints. The internal and external parameter matrixes of the camera were calculated to convert the pixel coordinates into the camera coordinates using Matlab software. The average actual error of 8.85 cm was obtained, indicating the reasonable and effective midpoint at the bottom of the rectangular frame as the locating reference point. The average deviation of the fitting and manual observation navigation line was 2.45 pixels in the 100 images, while the actual deviation was 4.90 cm, which fully met the accuracy requirements of navigation in the orchard. Three videos were randomly selected for the speed detection analysis of navigation lines. The total average time of processing one frame image was 0.044 s, which fully met the speed needs of navigation in the orchard. [ABSTRACT FROM AUTHOR]

The counting of cotton rows is dynamically a premise of accurate positioning of a visual plant protection vehicle, which can ensure that the vehicle can traverse at the end of a field. In this study, cotton during plant protection periods was taken as the research object, which presented a dynamic counting method of cotton rows based on Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM). Hogs of cotton rows and inter-row background were extracted as feature descriptors of images, then they were classified by an SVM classifier. In this way, cotton rows could be identified through classifying cotton rows and inter-row backgrounds. 7 000 positive and negative training samples were respectively prepared for HOG characteristics of training samples. Among them, positive samples contained a complete outline of cotton lines in width, by contrast, negative samples contained soil among lines, plastic films and incomplete outline of cotton lines, etc. Besides, 1 000 positive and negative test samples were prepared to optimize parameters of the HOG-SVM model. By setting a sliding window to training samples, hog features of sample images were extracted, meanwhile positive and negative samples were marked for initial training on SVM classifier parameters. The initially trained SVM classifier was used to classify and test the samples, and those with wrong classifications were picked out for re-marking and re-training. Such retraining processes would continue until all test samples were correctly classified. Then the final HOG-SVM model was solidified for video detections. In recognizing processes, the model was used to slide the window in sequence images and to detect. In the sequence images, the HOG-SVM model built in the previous step was used by the sliding window. In this way, cotton rows in the video could be detected. The same rows of cotton could be detected for many times. To accurately identify cotton rows, Non-Maximum Suppression (NMS) was used to normalize target windows of detected cotton rows. All candidate target windows were scored with confidence and sorted from high to low, then the one with the highest score was retained. In two consecutive images, NCC (Normalized Cross-Correlation) values of target windows of front and back frames were solved to match the same rows of cotton, realized dynamic tracking of cotton rows, and further realized counting. Visual perspectives would have different effects on the upper and lower ends of images. There was an adhesion among cotton rows at the upper end of images, and those at the lower end of images were lack of seedlings and lodging. Such interferences had impacts on identifications of cotton rows. To solve this problem, the Region of Interest (ROI) was set for video sequence images in this study, and recognition processes of all cotton rows were within the ROI region with a very complex cotton field environment. This method had strong robustness to change in a camera's view angles and natural environment, such as to change in the light as well as shook of branches and leaves caused by winds, etc. In this algorithm, the recognition rate of cotton rows was higher than 90% under the condition in which winds were lower than level-2 and the average detection time per frame was lower than 32 ms, which met requirements of actual field operations. This dynamic counting method of cotton rows was used as a basis for determining transverse distances of visual plant protection vehicles at field ends, which could be used in other semi-structured rows to plant low and short crops. [ABSTRACT FROM AUTHOR]

The jujube industry occupies an important position in the social economy of Xinjiang. It is important to realize the automatic driving of the jujube harvester. This study proposes a visual navigation path detection algorithm for the jujube harvester which working above the jujube trees based on image processing, aiming at the harvest operation of the Jun-jujube and Hui-jujube orchards in Xinjiang. First, the variety of the jujube orchard was distinguished. Set the middle 1/3 area in the y-axis direction of the image as the processing area, according to the relationship between the standard deviation d and the minimum value f of the B-component vertical cumulative histogram of the processing area of the image, the jujube orchard variety was automatically determined. If the value of f/d was less than 5, it was the Jun-jujube orchard, and the rest was the Hui-jujube orchard. Secondly, navigation path was extracted based on the results of jujube orchard classification. For the Hui-jujube orchard, the cromatic aberration method and the OTSU method weare first used to transform the image into gray and binary, and then to denoise and fill the pixel hole that the black pixels inside white pixels or the white pixels inside black pixels in the binary image. Then, the pixel rows were scanned from the top to the bottom in the processing area, and then the coordinates average value of the pixel points with pixel value of 0 were taken as the candidate points on each pixel row, and the average value of all candidate points’ coordinates was used as the known point coordinates of Hough transform. Finally, the navigation path was fitted based on the Hough transform through the known points. For the Jun-jujube orchard, set the middle 1/3 of the X-axis direction of the image as the processing area. The scan interval was determined by vertically accumulating the R-component in the processing area. Then, in the processing area, the scanning interval was determined by accumulating R-component vertically, and then scanned line by line from top to bottom in the scanning area,,the pixel with the smallest R-component value in each row of pixels was taken as the candidate point of the line, and the average coordinate value of all candidate points was taken as the known point of Hough transform. Finally, the Hough transform of known points was used to fit the navigation path The test results showed that for the Hui-jujube orchard and the Jun-jujube orchard, the average path detection accuracy of the algorithm was 94% and 93%, and the average processing time of one frame image was 0.042 and 0.046 s respectively. The detection accuracy and real-time performance can meet the requirements of jujube harvester operation, and can automatically identify the types of jujube orchard for operation, which can provide theoretical basis for the realization of automatic driving of jujube harvester. [ABSTRACT FROM AUTHOR]

Directional seeding of corn seed can improve the light energy utilization of corn plants and the ventilated effect of field, which is of great significance to increase the corn yield per unit area. In order to realize directional seeding of corn seed, this study developed a kind of corn seed directional positioning machine based on image processing technology, and its main components included seed-metering device, conveying device, image acquisition and processing unit and directional positioning device. First, the compulsory clamp-type precision seed-metering device was chosen to complete seed feeding, and combined with the process of feeding seed when duckbilled seeding mouth was opened by roller, a guide-localization method was proposed for positioning process of seed onto conveyer belt. Based on this method, a kind of guide-localization tube was designed, and its structural dimension and installation position were determined through empirical analysis and theoretical calculation. And then the actual position distribution of corn seeds fed onto conveyer belt was acquired by the experiment. Also, locating notch of concave shape was designed to position seed by a mechanical way, and through speed and displacement analysis of each phase of seed positioning process, the conditions that seed could be positioned by locating notch effectively and continuously were obtained. At the same time, on the basis of previous studies, the image detection algorithm toward seed germ surface orientation and seed tip orientation of qualified seed was improved. In the end, the directional positioning device, the most important component of this machine, was developed with motors as rotary power. Location of seed was found according to image detection information as well as photoelectric positioning signal. After that, seed was grabbed by the way of suction to complete the following directional positioning work: On one hand, according to the image detection information about seed tip orientation, a specified motor drove a specified rotating mechanism of the machine to adjust seed tip orientation, which made seed towards the target direction; on the other hand, based on the image detection information about seed germ surface orientation, i.e. the front or the back, another specified motor drove another specified rotating mechanism of the machine to put seeds in 2 different positions located on each side of the machine. In addition, the required precision of revolving position was analyzed, and accordingly the positioning control mode was discussed and the corresponding operation parameters were set up. In the experiment, the results showed that the detection accuracy of germ surface orientation was 98%, the detection deviation of seed tip orientation was not more than 8°, the locating accuracy of concave notch was 97%, the locating and sucking accuracy of rotating arm was 94%, the locating accuracy of seed placing was 99%, and the angular deviation of seed placing was not more than 10°, which met the design requirements. If one viscous seed band is placed on each side of this machine, which is used to load seeds whose positioning and directing are completed and to make seeds stick on the seed bands evenly and orderly, it will provide condition and foundation for the subsequent directional packaging and seeding. [ABSTRACT FROM AUTHOR]

In a corn field with an unstructured environment, for the images collection by monocular vision, conventional path recognition algorithms are difficult to guarantee their robustness due to illumination variation, background reflection, shadow noise, and so on. In addition, the increase of the calculation amount of the algorithms caused by the complicated background information of the corn field environment affects the quickness and the real-timeliness of the corn harvester autonomous navigation, which leads to the difficulty of meeting the requirement for the operation efficiency of the corn field harvester and impedes the practical application of the mobile robot technology in agricultural production. The significant premise of visual navigation for a corn harvester is the precise and rapid detection of the navigation route. Unfortunately, some interferences will affect the extraction of the navigation route and the judgment for the edge of the corn field due to the corn shadow and weeds. The vision navigation has many technical advantages of adapting to the complicated field operating environment, has wide detection range, and has rich and complete information. This paper put forward the detection algorithms of the operation routes of the corn harvester and the judgement of the end of the corn field by analyzing the different color features of the visual navigation image. Here we studied the region of interest where lies the target straight line. To improve the rapidity of the navigation path recognition and meet the real-time requirements of autonomous navigation operations, for the color characteristic of the corn field environment, the clustering segmentation of the image was performed based on the analysis of red (R), green (G) and blue (B) components to achieve the respective clusters of the path and the corn field information. We used these color components to extract the target features of the inner and outside of the corn field respectively, and smooth the image using the moving average method with the set length. First, to remove the shadow interference of the corn columns, we compared the G component and the B component of a color image and smoothed the image using the moving average method. For the first frame, we used the distribution of the G component to find the alternate point; for other frames, we connected them with the previous frame to calculate the skew angles of the target line. Then, we extracted the navigation routes based on the Hough transform passing a known point. At last, it would be determined whether the corn harvester reached the end automatically according to the continuous mutation of the R component. The test proved that the processing time of detecting line was 50.1 ms per frame and the detection of the end was accurate and reliable, which could meet the demand of the real operation of a corn harvester in the field. It can be concluded that the navigation routes are extracted and the edge of the corn field is judged accurately, which satisfy the requirement of the practical application of the corn harvester in the field. In addition, the work can also provide the reference for the vision navigation routes' detection of sorghum and some high pole crops with mechanized harvesting. [ABSTRACT FROM AUTHOR]

It is difficult to locate the apple in fog and haze environment for apple harvest robot. This paper proposed a new method to apply the principle of DCP (dark channel prior) to remove fog and haze on images which were collected from apple orchard. We adopted a new route to achieve the value of atmospheric light coefficient. Scan the hazed image with a 15×15 window, and get the smallest value of the 225 pixels from every window. All the smallest values constituted a dark channel image. The values of dark channel were stored in a matrix at the first step, and then the 1/1000 largest elements and their locations were calculated and stored in a new matrix which had the same shape with the dark channel matrix. Extract the matrix of red (R) channel of the hazed image at the next step. At the third step, the corresponding values in the matrix of R channel were obtained according to the position information in the new matrix. Finally, the average value of these values was calculated as the value of atmospheric light coefficient. According to the requirement of apple harvest robot, we took the haze-removal strength parameter of DCP algorithm as 1. In order to speed up the running of the algorithm, we calculated the transmission radio with guided filter. Image segmentation method used in the study had 3 stages: binaryzation, de-noising and dilation. Firstly, the grey image was obtained by calculating a special linear combination with red (R) channel, green (G) channel and blue (B) channel. This method emphasized the value of red channel in color images, which was conducive to separate the apple better at the next step. Secondly, binary image was obtained by Otsu method based on the grey image. Finally, after the process of de-noising and dilation, a better segmentation results could be obtained. We developed an experimental software with Microsoft Vision Studio 2010 and OpenCV (Open Source Computer Vision Library) to test the haze-removal effects in apple harvest robot vision system. The graphical user interface of the program was developed based on MFC (Microsoft Foundation Classes) library. The software had achieved the following functions: reading the image, calculating the dark channel value, calculating the value of atmospheric light coefficient based on the R channel, calculating the transmission radio, and so on. Based on this software we compared some haze-removal methods including MSR (multiscale retinex), AHE (adaptive histogram equalization) and DCP with different parameters. Hardware platform of the experiment was X230, which is a notebook computer produced by Lenovo Inc. We took Nikon D7100 camera as the image acquisition equipment and fixed it with tripod when it worked. Experimental images were collected at the apple base in Changping District of Beijing. Image acquisition dates were some days with heavy fog and haze in November 2015. Twelve images were selected as the experimental materials. After the analysis of the experimental data, this paper got the following results: 1) The average contrast value of the images was 64.04 with our method; the AHE method was faster, but the contrast value was 35.46 with the AHE; the histogram obtained by our method had the characteristics of Gaussian distribution, which showed that our method could get better image quality; 2) Testing the 640×480 images pixels, our method required 36.46 ms computing time, the MSR method required 126.43 ms, and the AHE required 28.58 ms. The time performance of our method was not as good as the AHE, but it was better than the MSR; 3) The average location accuracy was 94.8% with our method, which was higher than other methods. The experiments show that our method can get better balance between efficiency and performance. It is a feasible method for the actual apple harvest operation. [ABSTRACT FROM AUTHOR]