Search

Markerless methods for animal posture tracking have been rapidly developing recently, but frameworks and benchmarks for tracking large animal groups in 3D are still lacking. To overcome this gap in the literature, we present 3D-MuPPET, a framework to estimate and track 3D poses of up to 10 pigeons at interactive speed using multiple camera views. We train a pose estimator to infer 2D keypoints and bounding boxes of multiple pigeons, then triangulate the keypoints to 3D. For identity matching of individuals in all views, we first dynamically match 2D detections to global identities in the first frame, then use a 2D tracker to maintain IDs across views in subsequent frames. We achieve comparable accuracy to a state of the art 3D pose estimator in terms of median error and Percentage of Correct Keypoints. Additionally, we benchmark the inference speed of 3D-MuPPET, with up to 9.45 fps in 2D and 1.89 fps in 3D, and perform quantitative tracking evaluation, which yields encouraging results. Finally, we showcase two novel applications for 3D-MuPPET. First, we train a model with data of single pigeons and achieve comparable results in 2D and 3D posture estimation for up to 5 pigeons. Second, we show that 3D-MuPPET also works in outdoors without additional annotations from natural environments. Both use cases simplify the domain shift to new species and environments, largely reducing annotation effort needed for 3D posture tracking. To the best of our knowledge we are the first to present a framework for 2D/3D animal posture and trajectory tracking that works in both indoor and outdoor environments for up to 10 individuals. We hope that the framework can open up new opportunities in studying animal collective behaviour and encourages further developments in 3D multi-animal posture tracking. [ABSTRACT FROM AUTHOR]

Measuring parental care behaviour in the wild is central to the study of animal ecology and evolution, but it is often labour‐ and time‐intensive. Efficient open‐source tools have recently emerged that allow animal behaviour to be quantified from videos using machine learning and computer vision techniques, but there is limited appraisal of how these tools perform compared to traditional methods. To gain insight into how different methods perform in extracting data from videos taken in the field, we compared estimates of the parental provisioning rate of wild house sparrows Passer domesticus from video recordings. We compared four methods: manual annotation by experts, crowd‐sourcing, automatic detection based on the open‐source software DeepMeerkat, and a hybrid annotation method. We found that the data collected by the automatic method correlated with expert annotation (r = 0.62) and further show that these data are biologically meaningful as they predict brood survival. However, the automatic method produced largely biased estimates due to the detection of non‐visitation events, while the crowd‐sourcing and hybrid annotation produced estimates that are equivalent to expert annotation. The hybrid annotation method takes approximately 20% of annotation time compared to manual annotation, making it a more cost‐effective way to collect data from videos. We provide a successful case study of how different approaches can be adopted and evaluated with a pre‐existing dataset, to make informed decisions on the best way to process video datasets. If pre‐existing frameworks produce biased estimates, we encourage researchers to adopt a hybrid approach of first using machine learning frameworks to preprocess videos, and then to do manual annotation to save annotation time. As open‐source machine learning tools are becoming more accessible, we encourage biologists to make use of these tools to cut annotation time but still get equally accurate results without the need to develop novel algorithms from scratch. [ABSTRACT FROM AUTHOR]

Animal sociality, an individual's propensity to associate with others, has fitness consequences through mate choice, for example, directly, by increasing the pool of prospective partners, and indirectly through increased survival, and individuals benefit from both. Annually, fitness consequences are realized through increased mating success and subsequent fecundity. However, it remains unknown whether these consequences translate to lifetime fitness. Here, we quantified social associations and their link to fitness annually and over lifetime, using a multi-generational, genetic pedigree. We used social network analysis to calculate variables representing different aspects of an individual's sociality. Sociality showed high within-individual repeatability. We found that birds with more opposite-sex associates had higher annual fitness than those with fewer, but this did not translate to lifetime fitness. Instead, for lifetime fitness, we found evidence for stabilizing selection on opposite-sex sociality, and sociality in general, suggesting that reported benefits are only short-lived in a wild population, and that selection favors an average sociality. [ABSTRACT FROM AUTHOR]