Search

Precision Livestock Farming relies on several technological approaches to acquire in the most efficient way precise and up-to-date data concerning individual animals. In dairy farming, particular attention is paid to the automatic cow detection and tracking, as such information is closely related to animal welfare and thus to possible health issues. Computer vision represents a suitable and promising method for this purpose. This paper describes the first step for the development of a computer vision system, based on deep learning, aiming to recognize in real-time the individual cows, detect their positions, actions and movements and record the time history outputs for each animal. Specifically, a neural network based on deep learning techniques has been trained and validated on a case study farm, for the automatic recognition of individual cows in videos recorded in the barn. Four cows were selected to train and validate a YOLO neural network able to recognize a cow starting from the coat pattern. Then, precision-recall curves of the identification of individual cows were elaborated for both the specific target classes and the whole dataset in order to assess the performances of the network. By means of data augmentation techniques, an enlarged dataset has been created and considered in order to improve the performance of the network and to provide indications to increase detection efficiency in those cases where data acquisition is not easy to be carried out for long periods. The mean average precision of the detection, ranging from 0.64 to 0.66, showed that it is possible to properly identify individual cows based on their morphological appearance and that the piebald spotting pattern of a cow’s coat represents a clearly distinguishable object for a computer vision network. The results also led to obtain indications about the quantity and the characteristics of the images to be used for the network training in order to achieve efficient detections when facing with applications involving animals.

As a result of increasing consumer demand for 3D content, content creation associated with this new modality has increased significantly in conjunction with some recent standardization activities on this data type. Hence, a scientific revisit is required particularly to some challenging problems associated with the conventional video, considering the fact that multi-view video has many promising solutions to such problems. Moreover, any 3D representation also produces its own requirements to be dealt with. This special issue is an effort to compile and review the current advances in 3D multimedia and multimodal information analysis and processing. Particularly, it deals with emerging 3D techniques for 3D data integration and object analysis based on multi-views, as well as 3D content protection.

Recent cost aggregation strategies that adapt their weights to image content enabled local algorithms to obtain results comparable to those of global algorithms based on more complex disparity optimization methods. Unfortunately, despite the potential advantages in terms of memory footprint and algorithmic simplicity compared to global algorithms, most of the state-of-the-art cost aggregation strategies deployed in local algorithms are extremely slow. In fact, their execution time is comparable and often worse than those of global approaches. In this paper we propose a framework for accurate and fast cost aggregation based on segmentation that allows us to obtain results comparable to state-of-the-art approaches much more efficiently (the execution time drops from minutes to seconds). A further speed-up is achieved taking advantage of multi-core capabilities available nowadays in almost any processor. The comparison with state-of-the-art cost aggregation strategies highlights the effectiveness of our proposal.

Depth estimation is crucial in several computer vision applications, and a recent trend in this field aims at inferring such a cue from a single camera. Unfortunately, despite the compelling results achieved, state-of-the-art monocular depth estimation methods are computationally demanding, thus precluding their practical deployment in several application contexts characterized by low-power constraints. Therefore, in this paper, we propose a lightweight Convolutional Neural Network based on a shallow pyramidal architecture, referred to as μPyD-Net, enabling monocular depth estimation on microcontrollers. The network is trained in a peculiar self-supervised manner leveraging proxy labels obtained through a traditional stereo algorithm. Moreover, we propose optimization strategies aimed at performing computations with quantized 8-bit data and map the high-level description of the network to low-level layers optimized for the target microcontroller architecture. Exhaustive experimental results on standard datasets and an in-depth evaluation with a device belonging to the popular Arm Cortex-M family confirm that obtaining sufficiently accurate monocular depth estimation on microcontrollers is feasible. To the best of our knowledge, our proposal is the first one enabling such remarkable achievement, paving the way for the deployment of monocular depth cues onto the tiny end-nodes of distributed sensor networks.

Monitoring streamflow velocity is of paramount importance for water resources management and in engineering practice. To this aim, image-based approaches have proved to be reliable systems to non-intrusively monitor water bodies in remote places at variable flow regimes. Nonetheless, to tackle their computational and energy requirements, offload processing and high-speed internet connections in the monitored environments, which are often difficult to access, is mandatory hence limiting the effective deployment of such techniques in several relevant circumstances. In this paper, we advance and simplify streamflow velocity monitoring by directly processing the image stream in situ with a low-power embedded system. By leveraging its standard parallel processing capability and exploiting functional simplifications, we achieve an accuracy comparable to state-of-the-art algorithms that typically require expensive computing devices and infrastructures. The advantage of monitoring streamflow velocity in situ with a lightweight and cost-effective embedded processing device is threefold. First, it circumvents the need for wideband internet connections, which are expensive and impractical in remote environments. Second, it massively reduces the overall energy consumption, bandwidth and deployment cost. Third, when monitoring more than one river section, processing “at the very edge” of the system efficiency improves scalability by a large margin, compared to offload solutions based on remote or cloud processing. Therefore, enabling streamflow velocity monitoring in situ with low-cost embedded devices would foster the widespread diffusion of gauge cameras even in developing countries where appropriate infrastructure might be not available or too expensive.

Nonintrusive image-based methods have the potential to advance hydrological streamflow observations by providing spatially distributed data at high temporal resolution. Due to their simplicity, correlation-based approaches have until recent been preferred to alternative image-based approaches, such as optical flow, for camera-based surface flow velocity estimate. In this work, we introduce a novel optical flow scheme, optical tracking velocimetry (OTV), that entails automated feature detection, tracking through the differential sparse Lucas-Kanade algorithm, and then a posteriori filtering to retain only realistic trajectories that pertain to the transit of actual objects in the field of view. The method requires minimal input on the flow direction and camera orientation. Tested on two image data sets collected in diverse natural conditions, the approach proved suitable for rapid and accurate surface flow velocity estimations. Five different feature detectors were compared and the features from accelerated segment test (FAST) resulted in the best balance between the number of features identified and successfully tracked as well as computational efficiency. OTV was relatively insensitive to reduced image resolution but was impacted by acquisition frequencies lower than 7–8 Hz. Compared to traditional correlation-based techniques, OTV was less affected by noise and surface seeding. In addition, the scheme is foreseen to be applicable to real-time gauge-cam implementations.

Obtaining accurate depth measurements out of a single image represents a fascinating solution to 3D sensing. CNNs led to considerable improvements in this field, and recent trends replaced the need for ground-truth labels with geometry-guided image reconstruction signals enabling unsupervised training. Currently, for this purpose, state-of-the-art techniques rely on images acquired with a binocular stereo rig to predict inverse depth (i.e., disparity) according to the aforementioned supervision principle. However, these methods suffer from well-known problems near occlusions, left image border, etc inherited from the stereo setup. Therefore, in this paper, we tackle these issues by moving to a trinocular domain for training. Assuming the central image as the reference, we train a CNN to infer disparity representations pairing such image with frames on its left and right side. This strategy allows obtaining depth maps not affected by typical stereo artifacts. Moreover, being trinocular datasets seldom available, we introduce a novel interleaved training procedure enabling to enforce the trinocular assumption outlined from current binocular datasets. Exhaustive experimental results on the KITTI dataset confirm that our proposal outperforms state-of-the-art methods for unsupervised monocular depth estimation trained on binocular stereo pairs as well as any known methods relying on other cues., 14 pages, 7 figures, 4 tables. Accepted to 3DV 2018

People tracking is a crucial task in most computer vision applications aimed at analyzing specific behaviors in the sensed area. Practical applications include vision analytics, people counting, etc. In order to properly follow the actions of a single subject, a people tracking framework needs to robustly recognize it from the rest of the surrounding environment, thus allowing proper management of changing positions, occlusions and so on. The recent widespread diffusion of deep learning techniques on almost any kind of computer vision application provides a powerful methodology to address recognition. On the other hand, a large amount of data is required to train state-of-the-art Convolutional Neural Networks (CNN) and this problem is solved, when possible, by means of transfer learning. In this paper, we propose a novel dataset made of nearly 26 thousand samples acquired with a custom stereo camera providing depth according to a fast and accurate stereo algorithm. The dataset includes sequences acquired in different environments with more than 20 different people moving across the sensed area. Once labeled the 26 K images and depth maps of the dataset, we train a head detection module based on state-of-the-art deep network on a portion of the dataset and validate it a different sequence. Finally, we include the head detection module within an existing 3D tracking framework showing that the proposed approach notably improves people detection and tracking accuracy.

Image distortion correction is a critical pre-processing step for a variety of computer vision and image processing algorithms. Standard real-time software implementations are generally not suited for direct hardware porting, so appropriated versions need to be designed in order to obtain implementations deployable on FPGAs. In this paper, hardware-compatible techniques for image distortion correction are introduced and analyzed in details. The considered solutions are compared in terms of output quality by using a geometrical-error-based approach, with particular emphasis on robustness with respect to increasing lens distortion. The required amount of hardware resources is also estimated for each considered approach., To be published in Proceedings of the International Conference on Reconfigurable Computing and FPGAs, Cancun, Mexico, 30 November, 2 December 2016

Stereo matching is a popular technique to infer depth from two or more images and wealth of methods have been proposed to deal with this problem. Despite these efforts, finding accurate stereo correspondences is still an open problem. The strengths and weaknesses of existing methods are often complementary and in this paper, motivated by recent trends in this field, we exploit this fact by proposing Deep Stereo Fusion, a Convolutional Neural Network capable of combining the output of multiple stereo algorithms in order to obtain more accurate result with respect to each input disparity map. Deep Stereo Fusion process a 3D features vector, encoding both spatial and cross-algorithm information, in order to select the best disparity hypothesis among those proposed by the single stereo matchers. To the best of our knowledge, our proposal is the first i) to leverage on deep learning and ii) able to predict the optimal disparity assignments by taking only as input cue the disparity maps. This second feature makes our method suitable for deployment even when other cues (e.g., confidence) are not available such as when dealing with disparity maps provided by off-the-shelf 3D sensors. We thoroughly evaluate our proposal on the KITTI stereo benchmark with respect state-of-the-art in this field.

In this paper we propose a framework for the fusion of depth data produced by a Time-of-Flight (ToF) camera and stereo vision system. Initially, depth data acquired by the ToF camera are upsampled by an ad-hoc algorithm based on image segmentation and bilateral filtering. In parallel a dense disparity map is obtained using the Semi-Global Matching stereo algorithm. Reliable confidence measures are extracted for both the ToF and stereo depth data. In particular, ToF confidence also accounts for the mixed-pixel effect and the stereo confidence accounts for the relationship between the pointwise matching costs and the cost obtained by the semi-global optimization. Finally, the two depth maps are synergically fused by enforcing the local consistency of depth data accounting for the confidence of the two data sources at each location. Experimental results clearly show that the proposed method produces accurate high resolution depth maps and outperforms the compared fusion algorithms.

In this paper we describe the strategy adopted to design, from scratch, an embedded RGBD sensor for accurate and dense depth perception on a low-cost FPGA. This device infers, at more than 30 Hz, dense depth maps according to a state-of-the-art stereo vision processing pipeline entirely mapped into the FPGA without buffering partial results on external memories. The strategy outlined in this paper enables accurate depth computation with a low latency and a simple hardware design. On the other hand, it poses major constraints to the computing structure of the algorithms that fit with this simplified architecture and thus, in this paper, we discuss the solutions devised to overcome these issues. We report experimental results concerned with practical application scenarios in which the proposed RGBD sensor provides accurate and real-time depth sensing suited for the embedded vision domain.