Search

Thermal imagery is emerging as a viable candidate for 24-7, all-weather pedestrian detection owning to thermal sensors’ robust performance for pedestrian detection under different weather and illumination conditions. Despite the promising results obtained from combining visible (RGB) and thermal cameras in multi-spectral fusion techniques, the complex synchronization requirements, including alignment and calibration of sensors, impede their deployment in real-world scenarios. In this paper, we introduce a novel approach for domain adaptation to enhance the performance of pedestrian detection based solely on thermal images. Our proposed approach involves several stages. Firstly, we use both thermal and visible images as input during the training phase. Secondly, we leverage a thermal-to-visible hallucination network to generate feature maps that are similar to those generated by the visible branch. Finally, we design a transformer-based multi-modal fusion module to integrate the hallucinated visible and thermal information more effectively. The thermal-to-visible hallucination network acts as domain adaptation, allowing us to obtain pseudo-visual and thermal features using solely thermal input. Based on the experimental results, it is observed the mean average precision (mAP) increases by 4.72% and the miss rate decreases by 7.56% on the KAIST dataset when compared to the baseline model.

Deep convolutional neural networks have been leveraged to achieve huge improvements in video understanding and human activity recognition performance in the past decade. However, most existing methods focus on activities that have similar time scales, leaving the task of action recognition on multiscale human behaviors less explored. In this study, a two-stream multiscale human activity recognition and anticipation (MS-HARA) network is proposed, which is jointly optimized using a multitask learning method. The MS-HARA network fuses the two streams of the network using an efficient temporal–channel attention (TCA)-based fusion approach to improve the model’s representational ability for both temporal and spatial features. We investigate the multiscale human activities from two basic categories, namely, midterm activities and long-term activities. The network is designed to function as part of a real-time processing framework to support interaction and mutual understanding between humans and intelligent machines. It achieves state-of-the-art results on several datasets for different tasks and different application domains. The midterm and long-term action recognition and anticipation performance, as well as the network fusion, are extensively tested to show the efficiency of the proposed network. The results show that the MS-HARA network can easily be extended to different application domains.

Extracting distinctive, robust, and general 3D local features is essential to downstream tasks such as point cloud registration. However, existing methods either rely on noise-sensitive handcrafted features, or depend on rotation-variant neural architectures. It remains challenging to learn robust and general local feature descriptors for surface matching. In this paper, we propose a new, simple yet effective neural network, termed SpinNet, to extract local surface descriptors which are rotation-invariant whilst sufficiently distinctive and general. A Spatial Point Transformer is first introduced to embed the input local surface into an elaborate cylindrical representation (SO(2) rotation-equivariant), further enabling end-to-end optimization of the entire framework. A Neural Feature Extractor, composed of point-based and 3D cylindrical convolutional layers, is then presented to learn representative and general geometric patterns. An invariant layer is finally used to generate rotation-invariant feature descriptors. Extensive experiments on both indoor and outdoor datasets demonstrate that SpinNet outperforms existing state-of-the-art techniques by a large margin. More critically, it has the best generalization ability across unseen scenarios with different sensor modalities.

Dynamical models estimate and predict the temporal evolution of physical systems. State-space models (SSMs) in particular represent the system dynamics with many desirable properties, such as being able to model uncertainty in both the model and measurements, and optimal (in the Bayesian sense) recursive formulations, e.g., the Kalman filter. However, they require significant domain knowledge to derive the parametric form and considerable hand tuning to correctly set all the parameters. Data-driven techniques, e.g., recurrent neural networks, have emerged as compelling alternatives to SSMs with wide success across a number of challenging tasks, in part due to their impressive capability to extract relevant features from rich inputs. They, however, lack interpretability and robustness to unseen conditions. Thus, data-driven models are hard to be applied in safety-critical applications, such as self-driving vehicles. In this work, we present DynaNet, a hybrid deep learning and time-varying SSM, which can be trained end-to-end. Our neural Kalman dynamical model allows us to exploit the relative merits of both SSM and deep neural networks. We demonstrate its effectiveness in the estimation and prediction on a number of physically challenging tasks, including visual odometry, sensor fusion for visual-inertial navigation, and motion prediction. In addition, we show how DynaNet can indicate failures through investigation of properties, such as the rate of innovation (Kalman gain).

ABSTRACTPrevious research has shown that African lions (Panthera leo) have the ability to discriminate between conspecific vocalisations, but little is known about how individual identity is conveyed in the spectral structure of roars. Using acoustic – accelerometer biologgers that allow vocalisations to be reliably associated with individual identity, we test for vocal individuality in the fundamental frequency (f0) of roars from 5 male lions, firstly by comparing simple f0 summary features and secondly by modelling the temporal pattern of the f0 contour. We then assess the application of this method for discriminating between individuals using passive acoustic monitoring. Results indicate that f0 summary features only allow for vocal discrimination with 70.7% accuracy. By comparison, vocal discrimination can be achieved with an accuracy of 91.5% based on individual differences in the temporal pattern of the f0 sequence. We further demonstrate that passively recorded lion roars can be localised and differentiated with similar accuracy. The existence of individually unique f0 contours in lion roars and their relatively lower attenuation indicates a likely mechanism enabling individual lions to identify conspecifics over long distances. These differences can be exploited by researchers to track individuals across the landscape and thereby supplement conventional lion monitoring approaches.

ABSTRACTAcoustic localisation technology has been widely tested and applied for passive acoustic monitoring and ecological research, however, hardware costs of commercially available devices limit scalability. Furthermore, few studies have explored its use with low-density arrays. We present a low-cost, custom-designed hardware and software system termed ‘CARACAL’ that is able to extract and localise weak acoustic signals. The key to this is the use of four microphones on each logger, allowing for phase-based measurements and the ability to enhance signal-to-noise ratio through beamforming. As a proof of concept, we test the functionality of the CARACAL system by conducting a gunshot localisation experiment and demonstrate animal call detection and localisation from a lion predation event. Results show the system could locate gunshots with an average accuracy of 33.2 ± 15.3 m within an array of 7 stations 500 m apart. When applied to animal call positioning, we show long range (> 1 km) localisation of three different species’ calls, Cape buffalo, chacma baboon and spotted hyaena. With a cost of approximately £150 per unit, the CARACAL system provides a cost-effective solution for acoustic localisation over large areas. The system is open source and can be customised to suit a variety of wildlife research applications.

Due to the sparse rewards and high degree of environmental variation, reinforcement learning approaches, such as deep deterministic policy gradient (DDPG), are plagued by issues of high variance when applied in complex real-world environments. We present a new framework for overcoming these issues by incorporating a stochastic switch, allowing an agent to choose between high- and low-variance policies. The stochastic switch can be jointly trained with the original DDPG in the same framework. In this article, we demonstrate the power of the framework in a navigation task, where the robot can dynamically choose to learn through exploration or to use the output of a heuristic controller as guidance. Instead of starting from completely random actions, the navigation capability of a robot can be quickly bootstrapped by several simple independent controllers. The experimental results show that with the aid of stochastic guidance, we are able to effectively and efficiently train DDPG navigation policies and achieve significantly better performance than state-of-the-art baseline models. [ABSTRACT FROM AUTHOR]

Letter to the Editor

Thanks for the excellent photo essay on the mud volcano disaster in Indonesia (Stuck in the mud, January 2008). I had been following the story in the newspapers, but your [...]

African elephants (Loxodonta africana) are sentient and intelligent animals that use a variety of vocalizations to greet, warn or communicate with each other. Their low-frequency rumbles propagate through the air as well as through the ground and the physical properties of both media cause differences in frequency filtering and propagation distances of the respective wave. However, it is not well understood how each mode contributes to the animals’ abilities to detect these rumbles and extract behavioural or spatial information. In this study, we recorded seismic and co-generated acoustic rumbles in Kenya and compared their potential use to localize the vocalizing animal using the same multi-lateration algorithms. For our experimental set-up, seismic localization has higher accuracy than acoustic, and bimodal localization does not improve results. We conclude that seismic rumbles can be used to remotely monitor and even decipher elephant social interactions, presenting us with a tool for far-reaching, non-intrusive and surprisingly informative wildlife monitoring.

Several letters to the editor are presented in response to articles in previous issues including "Cyprus at the Crossroads," in the December 2007 issue, "Struck in the Mud," in the January 2008 issue and "That Sinking Feeling," in the January 2008.