Search

The aim of this study is to develop a computer vision-based method to automatically detect aggressive behaviors among pigs. Ten repetitions of the same experiment were performed. In each of the experiment, 7 piglets were mixed from three litters in two pigpens and captured on video for a total of 6 h. From these videos, the first 3 h of video after mixing were recorded as a training set, and the 3 h of video after 24 h were recorded as a validation set. Connected area and adhesion index were used to locate aggressive pigs and to extract key frame sequences. The two pigs in aggression were regarded as a whole rectangle according to their characteristics of continuous and large-proportion adhesion. The acceleration feature was extracted by analyzing the displacement change of four sides of this rectangle between adjacent frames, and hierarchical clustering was used to calculate its threshold. Based on this feature, the recognition rules of medium and high aggression were designed. Testing 10 groups of pigs, the accuracy of recognizing medium aggression was 95.82% with a sensitivity of 90.57% and with a specificity of 96.95%, and the accuracy of recognizing high aggression was 97.04% with a sensitivity of 92.54% and with a specificity of 97.38%. These results indicate that the acceleration can be used to recognize pigs’ aggressive behaviors. [ABSTRACT FROM AUTHOR]

• Computer vision methods for livestock behaviour recognition were evaluated. • Journey from computer vision to deep learning was elaborated. • This paper involves with segmentation, identification and behaviour recognition. • Research trend of livestock behaviour recognition was illustrated from four aspects. • This review could provide researchers and producers with technical references. The increasing demand for sustainable livestock products also demands new considerations in animal breeding. Breeding programs are now seeking to integrate animal behavioural phenotypes, as these relate to the productivity, health and welfare of the animals and thereby can influence yield and economic benefits in the industry. Traditional manual observation of pig behaviour is time-consuming, laborious, subjective, and difficult to achieve in continuous and large-scale operations. It is not surprising that computer vision technology with the advantages of being objective, non-invasive and continuous has been widely researched for its use in the recognition of livestock behaviours over recent years. Nevertheless, in studies of livestock behaviour recognition, computer vision technology faces some challenges, e.g., complex scenes, variable illumination, occlusion, touching and overlapping between livestock, which has limited the fast translation of technology to industry. On the other hand, deep learning technology has proven to solve these difficulties to a certain extent and is being adopted to recognise livestock behaviours. This paper mainly evaluates the recent developments in computer vision methods for recognition of these behaviours in pigs and cattle. The focus on these species is made possible by the number of studies exist quantifying behaviours that are of importance for their health, welfare and productivity such as aggression, drinking, feeding, lameness, mounting, posture, tail-biting and nursing. This review paper especially analyses the development of image segmentation, identification and behaviour recognition using tradition computer vision and more recent deep learning methods, and evaluates the evolution of key research in the field. We elaborate the research trend of livestock behaviour recognition from four aspects, i.e., development of robust livestock identification algorithms, recognition of livestock behaviours for different growth stages, further quantification of the results of behaviour recognition, and building evaluation system of growth status, health and welfare. [ABSTRACT FROM AUTHOR]

• Feeding behaviour of multiple pigs at the nursery stage was recognised in this study. • Xception and LSTM were combined to extract discriminative spatial-temporal features. • These features can used to recognise feeding behaviour with an accuracy of 98.4%. • The identity and feeding time of each pig were determined in 4 feeding sub-regions. • The proposed algorithm could recognise 98.5% of feeding time of individual pigs. Monitoring the feeding behaviour of pigs and measuring their feeding time can help farmers to evaluate the pig health and welfare. The aim of this study was to develop a video-based deep learning algorithm to recognise feeding behaviour of nursery pigs and determine the feeding time of pigs on an individual level. In the experiment, two pens of pigs were video recorded for 3 days. In the video of feeding region of pen 1, 96,000 1 s feeding episodes and 96,000 1 s non-feeding episodes were generated. 70% of these data was randomly selected as training set and the remaining 30% as validation set. In the video of feeding region of pen 2, all the labelled 24,656 1 s feeding episodes and 61,744 1 s non-feeding episodes were used as test set. Firstly, the Convolutional Neural Network (CNN) architecture Xception was used to extract spatial features. These features were input into Long Short-term Memory (LSTM) framework to extract spatial-temporal features. Through the fully connected layer, the prediction function Softmax was finally used to classify these 1 s episodes as feeding or non-feeding. An image processing algorithm based on maximum entropy segmentation, HSV (Hue, Saturation and Value) colour space transformation and template matching was proposed to calculate the circularity of the head, the ratio of the head to the feeding sub-region, the accumulated pixels of the head motion, and the distance from the head to the number on pig back in order to determine the identity and feeding time of each pig. In the test set, the proposed algorithm could recognise feeding behaviour with an accuracy of 98.4%, a sensitivity of 98.8%, specificity of 98.3% and precision of 95.9%, and could correctly recognise 98.5% of feeding time of individual pigs from the total 45944 s feeding time of 8 pigs. The results indicate that the proposed method can be used to recognise feeding behavior of pigs and determine feeding time of each pig. [ABSTRACT FROM AUTHOR]

• It is the first time to use computer vision to recognise pig enrichment engagement. • A HSV colour space-based method was proposed to track the object region of interest. • InceptionV3 and LSTM were used to extract discriminative spatial-temporal features. • This algorithm has reference value for classifying similar behaviours of pigs. • Pigs preference to 3 enrichment objects was also determined in this study. As providing objects that pigs prefer can reduce the occurrence of tail-biting and aggression and consequently improve animal welfare, automatic recognition of pigs' engagement with different objects can have practical value. Therefore, aim of this study was to develop a computer vision based approach that utilised a recurrent neural network-based deep learning algorithm to recognise pig enrichment engagement (EE) behaviours and preliminarily determine the preference to objects. Two pig pens were studied. 1 day of video was recorded in pen 1, which generated 2400 1 s EE and 2400 1 s non-EE episodes. 80% of these data was randomly selected as training set and the remaining 20% as validation set. Moreover, 4 days of video were recorded and used as the test set in pen 2. Firstly, the HSV (Hue, Saturation, Value) colour space-based tracking algorithm was developed to locate object region of interest. Secondly, the convolutional neural network (CNN) architecture InceptionV3 was used to extract spatial features from each frame. These features were input into the long short-term memory (LSTM) framework to extract spatial-temporal features from each episode. Through the fully connected layer, the prediction function Softmax was finally used to classify these episodes as EE or non-EE behaviour. In the validation set, the proposed algorithm could recognise EE with blue ball, golden ball and wooden beam with an accuracy of 95.2%, 95.4% and 97.3%, respectively. By shortening the radius of the region of interest into a half of the average length of pig body, the corresponding accuracy could be improved into 96.9%, 97.1% and 97.9%, respectively. In the test set, the proposed algorithm could recognise EE with each of these 3 objects with an accuracy of 96.5%, 96.8% and 97.6%, respectively. The proportion of EE with each of these 3 objects was 75.8%, 6.0% and 18.2%, respectively. These results indicate that the proposed method can be used to recognise EE behaviours of pigs, and halving the radius of the region of interest can improve the recognition accuracy of EE behaviours. Moreover, the preference of pigs to objects based on EE duration were preliminarily determined as blue ball > wooden beam > golden ball. The obtained duration of EE behaviours can help farmers to evaluate the enrichment used and thereby to increase the health and welfare of the pigs in their care. Furthermore, the proposed algorithm has reference value for the classification of the behaviours with similar motion patterns. [ABSTRACT FROM AUTHOR]

• A deep learning method based on CNN and LSTM was used to recognise pig aggression. • This study directly process video episodes rather than individual frames. • CNN features were input into LSTM framework to extract spatial-temporal features. • The proposed method could recognise aggressive episodes with an accuracy of 97.2%. • A frame skipping approach can improve the accuracy to 98.4% and halve running time. Aggression is considered as a major animal welfare problem in commercial pig farming. The aim of this study is to develop a deep learning method based on convolutional neural network (CNN) and long short-term memory (LSTM) to recognise aggressive episodes of pigs. Compared to previous studies of pig behaviours based on deep learning, this study directly process video episodes rather than individual frames. In the experiment, nursery pigs (8/pen) were mixed for 3 days and then 8 h of video was recorded in each day. From these videos, 600 aggressive 2 s-episodes were manually selected and then augmented into 2400 episodes by using horizontal, vertical and diagonal mirroring. From the videos, 2400 non-aggressive 2 s-episodes were also manually selected. 80% of the data were randomly allocated as training set and the remaining 20% as validation set. Firstly, the CNN architecture VGG-16 was used to extract spatial features. These features were then input into LSTM framework to further extract temporal features. Through fully connected layer, the prediction function Softmax was finally used to determine if the current episode is aggression or non-aggression. Using the proposed method, aggressive episodes could be recognised with an accuracy of 97.2%. This result indicates that this method can be used to recognise aggressive episodes of pigs. Additionally, this paper further investigates the validity of this method under the conditions of skipping frames and reducing the episode length. The results show that a frame skipping approach whereby 30 fps is reduced into 15 fps within each 2 s-episode can improve the accuracy into 98.4% and halve the total running time. [ABSTRACT FROM AUTHOR]

• A new depth sensor of RealSense D435 was used for pig aggression detection. • Advantages of group analysis and 3D cameras were combined for the first time. • Connected area threshold was set to remove moving pixels without aggression. • SVM was used to classify the 4 features of motion shape index in each 3 s-unit. • The proposed algorithm could detect aggression with higher evaluation parameters. The aim of this study is to develop a depth image analysis method to automatically detect aggressive behaviours of group-housed pigs. In this work 2 experiments were performed. In each experiment, 8 pigs from 3 pens were mixed for 3 days and then 8 h of video was recorded in each day. In the 24 h data of the first experiment, all the 883 aggressive 3 s-units and manually selected 883 non-aggressive 3 s-units were used as training set. In the 24 h data of the second experiment, all the 856 aggressive 3 s-units and manually selected 856 non-aggressive 3 s-units were used as test set. Firstly, frame-to-frame distance was set and then frame difference method was used to obtain moving pixels. Secondly, moving pixels caused by non-aggressive behaviours were removed by setting threshold of connected area. Number of filtered moving pixels were summed and defined as motion shape index (MSI) in each frame. After dividing all videos into 3 s-units, the maximum, mean, variance and standard deviation of MSI in each unit were extracted as features. Finally, support vector machine (SVM) was used to classify these features in order to detect aggression. Using the proposed algorithm, aggressive behaviours could be detected with an accuracy of 97.5%, a sensitivity of 98.2%, specificity of 96.7% and precision of 96.8%. The results indicate that this algorithm can be used to detect aggressive behaviours of pigs. [ABSTRACT FROM AUTHOR]

• YOLOv5 was improved by fusing coordination attention for fruit detection. • A YOLOv5-lotus model is proposed to detect overripe lotus seedpods. • High accurate and fast speed lotus seedpods detection results were achieved. • Proposed model is robust against complex environment, long distance detection. • It provides an example for the detection of upward-growing conical fruit. Accurate detection of lotus seedpods in a nature environment is essential for agronomic applications for automated harvesting and yield mapping. Traditional detection methods are based on grower's experience, which is inefficient for the large-scale production. To improve the efficiency of harvesting lotus seedpods, this study proposes a YOLOv5-lotus method to effectively detect overripe lotus seedpods. The lotus seedpods image dataset is firstly created. An improved YOLOv5 network model based on coordinate attention (CA) module is then presented, namely YOLOv5-lotus model, where CA module is developed to strengthen the model inter-channel relationships and capture long-range dependencies with precise positional information, thus improving the detection accuracy of the algorithm. In order to reveal the feasibility and robustness of the proposed method, a number of case studies are presented on the detection of overripe lotus seedpods in various scenarios, including different poses, illuminations and degrees of occlusion. Compared with the classical YOLOv5s network, the average precision of YOLOv5-lotus model is increased by 0.7 % and average detection time is reduced by 0.7 ms. Compared to other state-of-the-art networks, our detection model is able to achieve the highest average precision value, faster efficient detection speed and higher F1 score, with the average precision being 98.3 %, the recall rate being 96.3 %, the precision rate being 97.3 %, F1 score being 0.968 and average detection time being 19.4 ms. Through case studies and comparisons, the effectiveness and superiority of the proposed approach are demonstrated. These research results can be applied to the detection of upwardly-growing conical fruit. It creates a prerequisite for the development of automatic harvesting equipment. [ABSTRACT FROM AUTHOR]

• This study proposes a model capable of identifying the phenological stages of wheat. • The camera combined with unmanned aerial vehicle are employed for biomass estimation. • The wheat biomass estimating accuracy is improved by identifying phenological stages. • The model demonstrates good adaptability under various treatments. Conventional models for estimating crop biomass are influenced by the wheat fertility process and often result in large errors, limiting the accurate crop biomass monitoring throughout the reproductive period. In this study, we developed a ResNet-Wheat model to extract phenological information from wheat and used this information to establish an aboveground biomass (AGB) estimation model. Using the spectral information obtained from unmanned aerial vehicles (UAVs), we inverted the coefficients (k, b) of the biomass estimation model to construct a new wheat biomass estimation model (FIWheat-AGB). The ResNet-Wheat model achieved overall recognition accuracy of 94.4 % at various phenological stages, proving it to be a dependable, accurate source of fertility period index (FI) for the FIWheat-AGB model. By combining the Lasso model with six vegetation indexes (VIs) with strong correlations with AGB, we could invert the biomass coefficients (k, b) with an R2 above 0.86 in all five phenological periods. The RMSE and MAE also remained stable. The FIWheat-AGB model utilized phenological information and VIs, leading to a high degree of precision in predicting AGB with an R2 range of 0.84–0.91, maintaining RMSE at 1.69–2.11 t/ha, and consistently maintaining an MAE of less than 1 t/ha over several periods. Thus, compared with the multi-period segmentation model (with an R2 range of 0.57–0.75) that is directly monitored by spectra, the suggested model is better suited for monitoring the entire wheat phenological period, and it provides a higher accuracy in estimating whole fertility biomass compared with the common spectral biomass estimation model and the crop simulation model. The proposed techniques can aid in estimating agronomic parameters in other crops throughout the fertility period. [ABSTRACT FROM AUTHOR]

• Multi-scale and dense wheat heads in a wild environment were counted in this study. • Contrast adjustment and Gaussian blur were used to increase image features. • AugFPN and cascaded IoU threshold were used to extract multi-dimensional features. • These features could be used to detect wheat heads with a precision of 95.17%. • The error rate was 3.7% so it could provide reliable data for yield estimation. Wheat head counting plays an important role in crop yield estimation, which also meets great challenges of high density, scale variation, and illumination difference. In this paper, an improved wheat head counting network (WHCnet) was proposed to count wheat heads from the top view in the wild, with a novel object counting by detection pipeline. Firstly, data enhancement such as contrast adjustment and Gaussian blur were conducted to improve the generalization ability. Secondly, the Augmented Feature Pyramid Networks (AugFPN) was used to pool the original information adaptively, which made full use of the underlying information and solved the problem of poor detection of small wheat heads. Finally, cascaded Intersection over Union (IoU) threshold was used to make the IoU of the training model as close as possible to the input target, which could effectively remove negative samples in the complex background, reduce the noise bounding box and improve the target positioning accuracy of occluded wheat heads. Experiments on the test set image showed that the proposed method had an average error rate of 3.7% and an AP of 95.17%, which was significantly better than other state-of-the-art methods. Besides, the average counting time for a single wheat field image (3968 × 2976) obtained by an unmanned aerial vehicle was 87 ms, and the counting accuracy was 95.8%, which effectively verified the generalization ability of the model. These results indicate that the proposed method can meet the requirements of wheat head counting in the field environment and provide reliable reference data for wheat yield estimation. [ABSTRACT FROM AUTHOR]

• Computer vision was developed to classify pig agonistic behaviors in feeding stalls. • This study utilized different validation strategies to evaluate model performance. • Random validation showed best performance but not comparable to real-world practices. Agonistic behavior at feeding spaces is associated with both welfare and feed intake issues in swine farming. Studying interactive social behaviors of group-housed pigs provides valuable information to improve their production and welfare. The aims of this study were to (1) develop a deep learning pipeline based on convolutional neural network (CNN) and long short-term memory (LSTM) to classify videos depicting four types of interactive behavior between pigs in a single-space feeding stall and (2) validate the CNN + LSTM pipeline through various blocked validation strategies. Four categories of behaviors were classified in this study: head-to-body contact (including gentle nosing, casual contact between head/ears of a pig with a feeding pig, head knocking, tail biting, and pushing); levering where the feeding pig was lifted from behind by another pig; mounting in which the feeding pig was mounted by another pig; and no-contact when a second pig entered the feeding stall without physical contact with the feeding pig. Behavior at the feeding stall was filmed twice, three weeks apart, for two consecutive days each week using six groups of grow-finish pigs (10 per group) housed in pens equipped with FIRE® feeders. This resulted in a total of 15,679 30-frame video episodes for classification. Random validation, blocking-by-time validation, and blocking-by-feeder validation were utilized for training-validation data split. The size of training sets was held constant (N = 7,500) through all validation scenarios. The average validation accuracies were 0.968(±0.001), 0.860(±0.033), 0.766(±0.026), and 0.860(±0.010) for random validation, blocking-by-time validation, and blocking-by-feeder validation (at Feeder 1 and at Feeder 2), respectively. The results indicate that the CNN + LSTM pipeline yielded acceptable predictive performance in random validation. However, performance was substantially worse in blocking-by-time and blocking-by-feeder validations. More work is needed for algorithm generalization to improve its robustness across a variety of real-life application scenarios. We provide public access to the dataset and the code. [ABSTRACT FROM AUTHOR]

• An NMR-based, non-destructive, accurate, and automated method to measure kernel moisture content (KMC) in maize. • Tracking regional variations in moisture during kernel development via MRI. • A strategy to select for low KMC at harvest. • Phenotypic responses after KMC selection. Kernel moisture content (KMC) is important for maize grain development, harvest, and storage. Breeders always consider KMC as a necessary indicator when conducting breeding programs. The fact that a large number of samples are required for the typical breeding process necessitates the development of accurate, high-throughput, and non-destructive detection techniques for KMC. Here, we proposed an effective method based on nuclear magnetic resonance (NMR) spectroscopy to track moisture variation in maize kernels and select offspring with low KMC at harvest. Both moisture mass and content could be accurately measured via regression analysis (R2 = 0.9924 and 0.9139, respectively). Magnetic resonance imaging technology was also applied to monitor the spatial distribution of moisture in individual kernels and ears. On the other hand, with two generations of KMC selection experiments, we demonstrated the effectiveness of the proposed method. KMC was measured for self-crossing kernels of two elite hybrids, namely Zhengdan958 and Xianyu335, and partial kernels were classified into a high-kernel-moisture group (HMG) or low-kernel-moisture group (LMG). The progenies were then selected for kernels having a high KMC in HMG and low KMC in LMG. After selection, KMCs in LMG were significantly lower than those in the HMG in both hybrids at generations F 2 and F 3. Finally, phenotypic response surveys revealed that KMC selection significantly affected the flowering time. Our results demonstrate the effectiveness of the NMR-based platform for optimizing KMC trait, with potential applications in maize breeding programs. [ABSTRACT FROM AUTHOR]

• Smartphone can be used to track and upload both flight path of plant-protection UAV and operation status of UAV pump. • Setting single-threshold (short-time average energy or zero-crossing rate) of collected pump voice, pump operation status cannot be determined. • Recorded flight path by smartphone cannot used directly for flight and spraying quality performance evaluation without pre-processing procedures including Adaptive Gaussian Filtering. • The increase of Adaptive Gaussian Filter parameter will decrease the measured positioning deviation, but it will decrease the measured total distance. We develop an online spraying quality assessment system of plant protection unmanned aerial vehicle (UAV) based on Android Client. The built-in location and motion system of mobile phone are used to acquire the GPS data, speed and height of the UAV placed with Android mobile phone. A pump operation endpoints extraction model based on double-threshold method is developed to determine the pump operation status (on/off) during UAV flight. Adaptive Gaussian Filtering is employed to improve the positioning accuracy. Some assessment indexes including overlap spray and blank spray rates indicating plant protection UAV spraying quality are proposed in this paper. The test results show that: ① the max deviation of the beginning/end point extracted by double-threshold model is 2 s, and the max deviation of total operation time is 4 s; ② adaptive Gaussian Filtering can effectively improve the accuracy of static and dynamic positioning. We take filter parameter σ = 1, to decrease the measured dynamic positioning deviation and avoid clear reduction of measured total distance (when σ > 1) by smartphone; ③ when σ = 1, the difference values of effective spray rate, the overlap spray rate and the blank spray rate between smartphone measuring and UAV recording are only 1.08％, 0.68％and 1.07％, respectively. [ABSTRACT FROM AUTHOR]