A computer vision approach for recognition of the engagement of pigs with different enrichment objects.

• It is the first time to use computer vision to recognise pig enrichment engagement. • A HSV colour space-based method was proposed to track the object region of interest. • InceptionV3 and LSTM were used to extract discriminative spatial-temporal features. • This algorithm has reference value for classifying similar behaviours of pigs. • Pigs preference to 3 enrichment objects was also determined in this study. As providing objects that pigs prefer can reduce the occurrence of tail-biting and aggression and consequently improve animal welfare, automatic recognition of pigs' engagement with different objects can have practical value. Therefore, aim of this study was to develop a computer vision based approach that utilised a recurrent neural network-based deep learning algorithm to recognise pig enrichment engagement (EE) behaviours and preliminarily determine the preference to objects. Two pig pens were studied. 1 day of video was recorded in pen 1, which generated 2400 1 s EE and 2400 1 s non-EE episodes. 80% of these data was randomly selected as training set and the remaining 20% as validation set. Moreover, 4 days of video were recorded and used as the test set in pen 2. Firstly, the HSV (Hue, Saturation, Value) colour space-based tracking algorithm was developed to locate object region of interest. Secondly, the convolutional neural network (CNN) architecture InceptionV3 was used to extract spatial features from each frame. These features were input into the long short-term memory (LSTM) framework to extract spatial-temporal features from each episode. Through the fully connected layer, the prediction function Softmax was finally used to classify these episodes as EE or non-EE behaviour. In the validation set, the proposed algorithm could recognise EE with blue ball, golden ball and wooden beam with an accuracy of 95.2%, 95.4% and 97.3%, respectively. By shortening the radius of the region of interest into a half of the average length of pig body, the corresponding accuracy could be improved into 96.9%, 97.1% and 97.9%, respectively. In the test set, the proposed algorithm could recognise EE with each of these 3 objects with an accuracy of 96.5%, 96.8% and 97.6%, respectively. The proportion of EE with each of these 3 objects was 75.8%, 6.0% and 18.2%, respectively. These results indicate that the proposed method can be used to recognise EE behaviours of pigs, and halving the radius of the region of interest can improve the recognition accuracy of EE behaviours. Moreover, the preference of pigs to objects based on EE duration were preliminarily determined as blue ball > wooden beam > golden ball. The obtained duration of EE behaviours can help farmers to evaluate the enrichment used and thereby to increase the health and welfare of the pigs in their care. Furthermore, the proposed algorithm has reference value for the classification of the behaviours with similar motion patterns. [ABSTRACT FROM AUTHOR]