Identification of rumen protozoa in ruminants is significant for species identification, ecological population structure survey, and protozoa behaviour analysis. At present, rumen protozoa identification still needs to be done manually, which is time-consuming and inefficient. To address this issue, this paper proposes a deep learning method for accurate segmentation and recognition of protozoa instances in rumen microscopic images for the first time. Firstly, a microscopic image dataset of protozoa was constructed, which consists of 2671 images of 17 species, 11 genera, and 2 orders, and the images were annotated in detail. Secondly, by comparing two instance segmentation models, an improved YOLACT was presented for efficient protozoan image segmentation. In the proposed method, the SE (Squeeze-and-Excitation) block is integrated into the shallowest layer of the backbone network to enhance the expression of features, and the ReLU activation function of all layers is replaced by FReLU activation function to eliminate the spatial insensitivity of ReLU activation function. The experimental results show that the segmentation accuracy of the improved YOLACT method is 89.45%, which is 0.55% higher than that of the original YOLACT and 2.63% higher than that of the Mask R-CNN method. Meanwhile, the detection speed of YOLACT is 1.7 times that of Mask R-CNN. This work reduces the labour cost of rumen protozoa identification and provides a new method for accurate identification of the microscopic images of protozoa. Besides, the microscopic image dataset of protozoa and the prediction models constructed in this paper will be publicly available. • We use deep neural network to recognise rumen protozoa in microscopic image. • YOLACT outperforms Mask R-CNN in the accuracy of protozoa segmentation. • Improved YOLACT among all methods achieves the best performance. • This work provides a technical reference to the related protozoa research. [ABSTRACT FROM AUTHOR]