Search

• A comprehensive orchard dataset is collected that covers different apple varieties and lighting conditions. • A deep learning-based apple detection framework is developed with a novel suppression network. • The developed Suppression Mask R-CNN framework outperforms existing state-of-the-art DNNs. Robotic apple harvesting has received much research attention in the past few years due to growing shortage and rising cost in labor. One key enabling technology towards automated harvesting is accurate and robust apple detection, which poses great challenges as a result of the complex orchard environment that involves varying lighting conditions and foliage/branch occlusions. This letter reports on the development of a novel deep learning-based apple detection framework named Suppression Mask R-CNN. Specifically, we first collect a comprehensive apple orchard dataset for "Gala" and "Blondee" apples, using a color camera, under different lighting conditions (overcast and front lighting vs. back lighting). We then develop a novel suppression Mask R-CNN for apple detection, in which a suppression branch is added to the standard Mask R-CNN to suppress non-apple features generated by the original network. Comprehensive evaluations are performed, which show that the developed suppression Mask R-CNN network outperforms state-of-the-art models with a higher F1-score of 0.905 and a detection time of 0.25 second per frame on a standard desktop computer. [ABSTRACT FROM AUTHOR]

There is a growing need for robotic apple harvesting due to decreasing availability and rising cost in labor. Towards the goal of developing a viable robotic system for apple harvesting, this paper presents synergistic mechatronic design and motion control of a robotic apple harvesting prototype, which lays a critical foundation for future advancements. Specifically, we develop a deep learning-based fruit detection and localization system using a RGB-D camera. A three degree-of-freedom manipulator is designed with a hybrid pneumatic/motor actuation mechanism to achieve dexterous movements. A vacuum-based end-effector is used for apple detaching. These three components are integrated into a robotic apple harvesting prototype with simplicity, compactness, and robustness. Moreover, a nonlinear control scheme is developed for the manipulator to achieve accurate and agile motion control. Field experiments are conducted to demonstrate the performance of the developed apple harvesting robot. • The synergistic development of a robotic apple harvesting prototype is presented. • A control strategy is designed by fully exploiting the mechanical structure of the manipulator. • The experimental studies validate the design concept and underpin future research. [ABSTRACT FROM AUTHOR]