Search

Data augmentation is an effective way to overcome the overfitting problem of deep learning models. However, most existing studies on data augmentation work on framelike data (e.g., images), and few tackles with event-based data. Event-based data are different from framelike data, rendering the augmentation techniques designed for framelike data unsuitable for event-based data. This work deals with data augmentation for event-based object classification and semantic segmentation, which is important for self-driving and robot manipulation. Specifically, we introduce EventAugment, a new method to augment asynchronous event-based data by automatically learning augmentation policies. We first identify 13 types of operations for augmenting event-based data. Next, we formulate the problem of finding optimal augmentation policies as a hyperparameter optimization problem. To tackle this problem, we propose a random search-based framework. Finally, we evaluate the proposed method on six public datasets including N-Caltech101, N-Cars, ST-MNIST, N-MNIST, DVSGesture, and DDD17. Experimental results demonstrate that EventAugment exhibits substantial performance improvements for both deep neural network-based and spiking neural network-based models, with gains of up to approximately 4%. Notably, EventAugment outperform state-of-the-art methods in terms of overall performance.

Nowadays, wide-area action analysis systems are widely used in security and robotics communities, where multiobject simultaneous searching and action recognition play critical roles. However, these systems face challenges in balancing imaging resolution, searching efficiency, and localization precision. To tackle these problems, this research introduces a hybrid system that combines a wide-angle camera, a high-speed camera, and vibrating mirrors. Specifically, a high-probability region (HPR) generation method based on a wide-angle camera is introduced to estimate potential regions where target objects will appear, which enhances search efficiency by avoiding random searching. Then, based on the generated prior information, an uncertainty estimation module is introduced to approximate the best location and size of the object using tailored localization and size uncertainty estimation methods. Uncertainty measurement will narrow the searching range and increase the localization accuracy. Finally, by integrating the HPR generation and uncertainty estimation modules, an efficient hybrid vision system is implemented, which can search multiobject in wide areas and conduct action analysis on the detected objects. Comprehensive indoor and outdoor experiments demonstrate that the detection framework with HPRs can achieve a 12.5% improvement in recall, and with uncertainty estimation, there is at least 67.12% localization precision improvement. These results verify the efficiency and robustness of the system. Source code is available at https://github.com/longxianlei/Hybrid-Wide-Area-Human-Action-Analysis-System.