Search

• The YOLOv3 model was trained on data acquired in 20 arable fields. • The generalization error on 5 different arable fields was evaluated. • The effect of the number of training images was investigated. • The effect of the number of training fields was investigated. • The generalization error was mitigated using incremental training. Lack of generalization in plant-detection models is one of the main challenges preventing the realization of autonomous weed-control systems. This paper investigates the effect of the train and test dataset distribution on the generalization error of a plant-detection model and uses incremental training to mitigate the said error. In this paper, we use the YOLOv3 object detector as plant-detection model. To train the model and test its generalization properties we used a broad dataset, consisting of 25 sub-datasets, sampled from multiple different geographic areas, soil types, cultivation conditions, containing variation in weeds, background vegetation, camera quality and variations in illumination. Using this dataset we evaluated the generalization error of a plant-detection model, assessed the effect of sampling training images from multiple arable fields on the generalization of our plant-detection model, we investigated the relation between the number of training images and the generalization of the plant-detection model and we applied incremental training to mitigate the generalization error of our plant-detection model on new arable fields. It was found that the average generalization error of our plant-detection model was 0.06 mAP. Increasing the number of sub-datasets for training, while keeping the total number of training images constant, increased the variation covered by the training set and improved the generalization of our plant-detection model. Adding more training images sampled from the same datasets increased the generalization further. However, this effect is limited and only holds when the new images cover new variation. Naively adding more images does not prepare the model for specific scenarios outside the training distribution. Using incremental training the model can be adapted to such scenarios and the generalization error can be mitigated. Depending on the discrepancy between the training set and the new field, finetuning on as little as 25 images can already mitigate the generalization error. [ABSTRACT FROM AUTHOR]

One of the problems in loose housing systems for laying hens is the laying of eggs on the floor, which need to be collected manually. In previous work, PoultryBot was presented to assist in this and other tasks. Here, probabilistic localisation with a particle filter is evaluated for use inside poultry houses. A poultry house is a challenging environment, because it is dense, with narrow static objects and many moving animals. Several methods and options were implemented and tested on data obtained with PoultryBot in a commercial poultry house. Although no animals were present, the localisation problem is still challenging here because of the repetitive nature of the poultry house interior, with its many narrow obstacles. Different parameter configurations were systematically evaluated, based on accuracy and applicability of the results. Estimated paths were quantitatively evaluated based on the Euclidian distance to a ground-truth determined with help of a total station. The presented system reached an accuracy of 0.37 m for 95% of the time, with a mean error of 0.2 m, making it suitable for localising PoultryBot in its future application. [ABSTRACT FROM AUTHOR]

In the current demand for automation in the agro-food industry, accurately detecting and localizing relevant objects in 3D is essential for successful robotic operations. However, this is a challenge due the presence of occlusions. Multi-view perception approaches allow robots to overcome occlusions, but a tracking component is needed to associate the objects detected by the robot over multiple viewpoints. Most multi-object tracking (MOT) algorithms are designed for high frame rate sequences and struggle with the occlusions generated by robots' motions and 3D environments. In this paper, we introduce MOT-DETR, a novel approach to detect and track objects in 3D over time using a combination of convolutional networks and transformers. Our method processes 2D and 3D data, and employs a transformer architecture to perform data fusion. We show that MOT-DETR outperforms state-of-the-art multi-object tracking methods. Furthermore, we prove that MOT-DETR can leverage 3D data to deal with long-term occlusions and large frame-to-frame distances better than state-of-the-art methods. Finally, we show how our method is resilient to camera pose noise that can affect the accuracy of point clouds. The implementation of MOT-DETR can be found here: https://github.com/drapado/mot-detr. • MOT-DETR: A novel method for single-shot 3D and tracking with transformers. • A novel implementation of transformers for 2D and 3D data fusion. • A use of FSPM for synthetic data generation. • Real world experiments in a challenging tomato greenhouse environment. • A tracking accuracy increase of 28.25 points against SOTA tracking algorithms. [ABSTRACT FROM AUTHOR]

Learning from Demonstration (LfD) can be used to make it easier for robots to learn to perform tasks such as apple harvesting. The required apple harvesting motions can be split into four steps: 1. Approaching, 2. Grasping, 3. Detaching, 4. Placing. So far, only parts of the apple harvesting task have been learned using LfD. In our work, we learned an abstracted version of the complete apple harvesting task using a cube. We used two methods and tested the sensitivity to model parameters and the number of demonstrations. We selected (1) the Gaussian Mixture Model (GMM) with Gaussian Mixture Regression (GMR) and (2) the Hidden Markov Model (HMM) with Linear Quadratic Regression (LQR). Both methods make use of Task Parameterization (TP), which allows models to combine multiple phases from the apple harvesting task. Combining multiple harvesting phases is important for effective LfD. We analysed the performance of these algorithms on four performance metrics, namely the grasp pose, the detachment motion, the place pose, and the success rate when using the real robot. A high grasp pose accuracy was achieved, specifically a position error of 0.004 and an orientation error of 0.004 for the GMM with GMR, trained with 100 demonstrations and 9 Gaussian components. The detachment motion was performed but with a reduced range. The model that performed the best was the GMM with GMR, trained with 60 demonstrations and 5 Gaussian components. A high place pose accuracy was achieved, specifically a position error of 0.003 and an orientation error of 0.003 for the GMM with GMR, trained with 100 demonstrations and 15 Gaussian components. The task was executed by a real robot successfully, resulting in a success rate of 67% for HMM with LQR, trained with 40 demonstrations, 5 states, and a scaling factor of 1.0. For the number of components, no clear relation was found. The same was true for the number of states in the HMM with LQR. For the scaling factor, a clear relation could be found. For this task, a value in the range of -1.0 to 1.0 should be used. For the number of demonstrations, there was an optimum of 40 demonstrations. Further improvements are needed to deal with challenges such as apple shift and detecting detachment of fruits. • Learning from Demonstration was used to learn the apple harvesting task. • Methods were trained on at most 100 demonstrations. • High accuracy at grasp pose and place pose. • Effect of training with fewer demonstrations investigated. • Sensitivity analysis of the model parameters was performed. [ABSTRACT FROM AUTHOR]

This study presents an automated lameness detection system that uses deep-learning image processing techniques to extract multiple locomotion traits associated with lameness. Using the T-LEAP pose estimation model, the motion of nine keypoints was extracted from videos of walking cows. The videos were recorded outdoors, with varying illumination conditions, and T-LEAP extracted 99.6% of correct keypoints. The trajectories of the keypoints were then used to compute six locomotion traits: back posture measurement, head bobbing, tracking distance, stride length, stance duration, and swing duration. The three most important traits were back posture measurement, head bobbing, and tracking distance. For the ground truth, we showed that a thoughtful merging of the scores of the observers could improve intra-observer reliability and agreement. We showed that including multiple locomotion traits improves the classification accuracy from 76.6% with only one trait to 79.9% with the three most important traits and to 80.1% with all six locomotion traits. • T-LEAP was used to track keypoints on the hooves, head and back of the cows. • Six locomotion traits associated with lameness were computed from the keypoints. • Lameness classification improves when including multiple locomotion traits. [ABSTRACT FROM AUTHOR]

• The performance of 40 combinations for vegetation segmentation was assessed. • A clear difference in performance was observed among various combinations under the given conditions. • CIVE + Kapur showed the best performance; while VEG + Kapur showed the worst. • When adapting the combination to the given conditions yielded a slightly higher performance. • However, the expected advantage of adapting the combination to the field condition is not large. Robust vegetation segmentation is required for a vision-based weed control robot in an agricultural field operation. The output of vegetation segmentation is a fundamental element in the subsequent process of weed/crop discrimination as well as weed control actuation. Given the abundance of colour indices and thresholding techniques, it is still far from clear how to choose a proper threshold technique in combination with a colour index for vegetation segmentation under agricultural field conditions. In this research, the performance of 40 combinations of eight colour indices and five thresholding techniques found in the literature was assessed to identify which combination works the best given varying field conditions in terms of illumination intensity, shadow presence and plant size. It was also assessed whether it was better to use one specific combination at all times or whether the combination should be adapted to the field conditions at hand. A clear difference in performance, represented in terms of MA (Modified Accuracy) which indicates the harmonic mean of relative vegetation area error and balanced accuracy, was observed among various combinations under the given conditions. On the image dataset that was used in this study, CIVE+Kapur (Colour Index of Vegetation Extraction+Max Entropy threshold) showed the best performance while VEG+Kapur (Vegetative Index+Max Entropy threshold) showed the worst. Adapting the combination to the given conditions yielded a slightly higher performance than when using a single combination for all (in this case CIVE+Kapur). Consistent results were obtained when validated on a different independent image dataset. Although a slightly higher performance was achieved when adapting the combination to the field conditions, this slight improvement seems not to outweigh the potential investment in sensor technology and software that are needed in practice to accurately determine the different conditions in the field. Therefore, the expected advantage of adapting the combination to the field condition is not large. [ABSTRACT FROM AUTHOR]

• MaskAL is software that can reduce the annotation effort for training Mask R-CNN. • MaskAL uses active learning to select images about which Mask R-CNN is uncertain. • On a broccoli dataset, MaskAL performed significantly better than random sampling. • By using MaskAL, 1400 image annotations were saved compared to random sampling. • The software is available on https://github.com/pieterblok/maskal. The generalisation performance of a convolutional neural network (CNN) is influenced by the quantity, quality, and variety of the training images. Training images must be annotated, and this is time consuming and expensive. The goal of our work was to reduce the number of annotated images needed to train a CNN while maintaining its performance. We hypothesised that the performance of a CNN can be improved faster by ensuring that the set of training images contains a large fraction of hard-to-classify images. The objective of our study was to test this hypothesis with an active learning method that can automatically select the hard-to-classify images. We developed an active learning method for Mask Region-based CNN (Mask R-CNN) and named this method MaskAL. MaskAL involved the iterative training of Mask R-CNN, after which the trained model was used to select a set of unlabelled images about which the model was most uncertain. The selected images were then annotated and used to retrain Mask R-CNN, and this was repeated for a number of sampling iterations. In our study, MaskAL was compared to a random sampling method on a broccoli dataset with five visually similar classes. MaskAL performed significantly better than the random sampling. In addition, MaskAL had the same performance after sampling 900 images as the random sampling had after 2300 images. Compared to a Mask R-CNN model that was trained on the entire training set (14,000 images), MaskAL achieved 93.9% of that model's performance with 17.9% of its training data. The random sampling achieved 81.9% of that model's performance with 16.4% of its training data. We conclude that by using MaskAL, the annotation effort can be reduced for training Mask R-CNN on a broccoli dataset with visually similar classes. Our software is available on https://github.com/pieterblok/maskal. [ABSTRACT FROM AUTHOR]