Search

As the monitor probes are used more and more widely these days, the task of detecting abnormal behaviors in surveillance videos has gained widespread attention. The generalization ability and parameter overhead of the model affect how accurate the detection result is. To deal with the poor generalization ability and high parameter overhead of the model in existing anomaly detection methods, we propose a three-dimensional multi-branch convolutional fusion network, named "Branch-Fusion Net". The network is designed with a multi-branch structure not only to significantly reduce parameter overhead but also to improve the generalization ability by understanding the input feature map from different perspectives. To ignore useless features during the model training, we propose a simple yet effective Channel Spatial Attention Module (CSAM), which sequentially focuses attention on key channels and spatial feature regions to suppress useless features and enhance important features. We combine the Branch-Fusion Net and the CSAM as a local feature extraction network and use the Bi-Directional Gated Recurrent Unit (Bi-GRU) to extract global feature information. The experiments are validated on a self-built Crimes-mini dataset, and the accuracy of anomaly detection in surveillance videos reaches 93.55% on the test set. The result shows that the model proposed in the paper significantly improves the accuracy of anomaly detection in surveillance videos with low parameter overhead. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN)-based autonomous driving object detection algorithms have excellent detection results on conventional datasets, but the detector performance can be severely degraded in low-light foggy weather environments. Existing methods have difficulty in achieving a balance between low-light image enhancement and object detection. To alleviate this problem, this paper proposes a foggy traffic environment object detection framework, IDOD-YOLOV7. This network is based on joint optimal learning of image defogging module IDOD (AOD + SAIP) and YOLOV7 detection modules. Specifically, for low-light foggy images, we propose to improve the image quality by joint optimization of image defogging (AOD) and image enhancement (SAIP), where the parameters of the SAIP module are predicted by a miniature CNN network and the AOD module performs image defogging by optimizing the atmospheric scattering model. The experimental results show that the IDOD module not only improves the image defogging quality for low-light fog images but also achieves better results in objective evaluation indexes such as PSNR and SSIM. The IDOD and YOLOV7 learn jointly in an end-to-end manner so that object detection can be performed while image enhancement is executed in a weakly supervised manner. Finally, a low-light fogged traffic image dataset (FTOD) was built by physical fogging in order to solve the domain transfer problem. The training of IDOD-YOLOV7 network by a real dataset (FTOD) improves the robustness of the model. We performed various experiments to visually and quantitatively compare our method with several state-of-the-art methods to demonstrate its superiority over the others. The IDOD-YOLOV7 algorithm not only suppresses the artifacts of low-light fog images and improves the visual effect of images but also improves the perception of autonomous driving in low-light foggy environments. [ABSTRACT FROM AUTHOR]

This document is a retraction note for an article titled "Lip segmentation using localized active contour model with automatic initial contour" published in the journal Neural Computing & Applications. The article has been retracted by the Editor-in-Chief and the publisher due to concerns regarding compromised editorial handling and peer review process, inappropriate or irrelevant references, and not being in scope of the journal or guest-edited issue. The authors have not responded to correspondence from the publisher regarding this retraction. Springer Nature, the publisher, remains neutral in regards to jurisdictional claims and institutional affiliations. [Extracted from the article]

Automatic visual lipreading, an efficient and convenient way of human–machine interaction, recognizes the content of the conversation from dynamic visual features of the speakers. Automatic lipreading based on acoustic speech alone can effectively prevent interference in complex environment, particularly under noisy conditions. In this paper, we propose novel visual extraction and HMM classification methods for automatic lipreading system, which reduce the dimension by using locally sensitive discriminant analysis algorithm and quantitative cluster by K-means algorithm. A model-based hybrid feature extraction method is proposed by optimizing the weight matrix of the LSDA algorithm. The effectiveness of the suggested approach is demonstrated by preliminary experiments on the English video database. Experimental results demonstrate that the proposed optimized algorithm can increase recognition rate up to 97%, which is 18% higher than the original algorithm. [ABSTRACT FROM AUTHOR]

In autonomous driving, lidar has become the main vehicle sensor due to its advantages such as long-range measurement and high accuracy. However, the collected point cloud data is sparse and unevenly distributed, and it lacks characterization capabilities when facing objects with missing or similar shapes, so that the detection accuracy is low while detecting long-distance small targets with similar shapes and a small number of point clouds. In order to improve the detection accuracy of small targets represented by point clouds, this paper adopts a method that fuses point cloud and RGB image to construct a 3D object detection network architecture based on two-stage complementary fusion. In the first stage of the fusion, we use the FPS method to select some points from the raw point cloud data as key points. Then, we voxelize the raw point cloud and use the 3D sparse convolutional neural network to extract multi-scale points cloud features, which would fuse features of different scales with key points. In the second stage of fusion, a 2D object detector is used to obtain the 2D bounding box and category information of the target in the image and take the camera as the origin to extend along the direction of the 2D bounding box to form a frustum; then the point cloud and target category information within the frustum are fused into the key points. This paper uses key points as a bridge to effectively combine the semantic information of the image such as texture and color with the point cloud features. The experimental results show that the network proposed in this paper has excellent performance on small objects. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has already been applied to English lip-reading. However, Chinese lip-reading starts late and lacks relevant dataset, and the recognition accuracy is not ideal. Therefore, this paper proposes a new hybrid neural network model to establish a Chinese lip-reading system. In this paper, we integrate the attention mechanism into both CNN and RNN. Specifically, we add the convolutional block attention module (CBAM) to the ResNet50 neural network, which enhances its ability to capture the small differences among the mouth patterns of similarly pronounced words in Chinese, improving the performance of feature extraction in the convolution process. We also add the time attention mechanism to the GRU neural network, which helps to extract the features among consecutive lip motion images. Considering the effects of the moments before and after on the current moment in the lip-reading process, we assign more weights to the key frames, which makes the features more representative. We further validate our model through experiments on our self-built dataset. Our experiments show that using convolutional block attention module (CBAM) in the Chinese lip-reading model can accurately recognize Chinese numbers 0–9 and some frequently used Chinese words. Compared with other lip-reading systems, our system has better performance and higher recognition accuracy. [ABSTRACT FROM AUTHOR]

The current field of autonomous driving has achieved superior object detection performance in good weather conditions. However, the environment sensing capability of autonomous vehicles is severely affected in rainfall traffic environments. Although deep-learning-based image derain algorithms have made significant progress, integrating them with high-level vision tasks, such as object detection, remains challenging due to the significant differences between the derain and object detection algorithms. Additionally, the accuracy of object detection in real rain traffic environments is significantly reduced due to the domain transfer problem between the training dataset and the actual rain environment. To address this domain-shifting problem, we propose an adaptive rain image enhancement object detection network for autonomous driving in adverse weather conditions (ARODNet). This network architecture consists of an image adaptive enhancement module, an image derain module, and an object detection module. The baseline detection module (CBAM-YOLOv7) is built by incorporating the YOLOv7 object detection network into a feed-forward convolutional neural network, and it includes an attention module (CBAM). We propose a domain adaptive rain image enhancement module, DRIP, for low-quality images acquired under heavy rainfall conditions. DRIP enhances low-quality images on rainy days by adaptively learning multiple preprocessing weights. To remove the effects of rain patterns and fog clouds on image detection, we introduce DRIP-enhanced images into the depth estimation derain module (DeRain) to prevent rain and fog from obscuring the objects to be detected. Finally, the multistage joint training strategy is adopted to improve the training efficiency, and the object detection is performed while the image is derained. The efficacy of the ARODNet network for object detection in rainy weather traffic environments has been demonstrated through numerous quantitative and qualitative studies. [ABSTRACT FROM AUTHOR]

• A method is proposed to analyze the radio spectral bias from fre- quency perspective, based on a multi-spectral attention mechanism with DCT for learning-based frequency components selection, termed as FLANs. This attention mechanism is the general case of conventional global aver- age pooling (GAP) and leverage identical structures of the popular neural networks. • We propose a simple yet effective learning-based dynamic frequency selection module in FLANs. The module consists of multiple learning-based switches to identify the trivial frequency components based on the DCT coefficients for static removal during inference. • To the best of our knowledge, this is the first work that compromises the merits of hand-crafted feature mechanism and deep learning in frequency domain for AMC. Extensive experiments have been conducted to validate the superiority of FLANs for automatic modulation classification over a wide variety of state-of-the-art methods on RADIOML 2018.01A dataset. Deep neural networks have been recently applied in automatic modulation classification task and achieved remarkable success. However, Existing neural networks mainly focus on the purely data-driven architecture design, and fail to explore the hand-crafted feature mechanisms which are particularly significant for radio signal presentation in wireless communication. Inspired by digital signal processing theories, we propose frequency learning attention networks (FLANs) to analyze the radio spectral bias from frequency perspective, based on a multi-spectral attention mechanism for learning-based frequency components selection. FLANs are the general case of classical global average pooling and leverage identical structures of the popular neural networks. Extensive experiments have been conducted to validate the superiority of FLANs for automatic modulation classification over a wide variety of state-of-the-art methods on RADIOML 2018.01A dataset. [ABSTRACT FROM AUTHOR]

Image quality is one of the most crucial influence factors when conducting biometric image-based human identification. However, it is not considered in most existing multi-biometric fusion algorithms. In this paper, a quality assessment-based dynamic weighted fusion algorithm is proposed, which applies a method of using image quality scores, and the scores are evaluated by integrating various image quality metrics to assess the quality of the feature matching process. According to the classification of the dual-modality feature matching quality, a dynamic weighted fusion strategy is proposed to increase the weight of biometric traits with better quality and weaken the impact of low-quality biometric traits on the recognition results, to achieve the adaptive optimization fusion of the biometric score level. Finally, the fused scores are used to make decisions. Experimental results reveal that the proposed algorithm is more robust and has better achievement than unimodal biometrics and traditional fusion algorithms. [ABSTRACT FROM AUTHOR]

Labanotation is a widely-used notation system for recording human dance movements. Automatically generating Labanotation scores from motion capture data can save significant manual effort and help the preservation of old folk dances in protecting intangible cultural heritages. Existing Labanotation generation methods have limited ability to capture the flexible limb movements as well as the rich periodic, symmetric, or repeated dance steps. In this paper, we present a novel LabanFormer model including a Multi-Scale Graph Attention network (MS-GAT) and a transformer model with Gated Recurrent Positional Encoding (GRPE) to achieve more effective Labanotation generation. First, the proposed MS-GAT can capture flexible limb movements by learning feature correlations between every two joints and aggregating features of neighboring joints over multiple scales. Second, we propose a new GRPE-based transformer to learn global temporal dependencies in the output feature sequences of MS-GAT. The novel GRPE module can encode position information with learnable parameters while handling various sequence lengths. As such, the periodic, symmetric, or repeated steps in dances can be accurately captured. Finally, the corresponding Laban symbols are generated by the decoder of the GRPE-based transformer. Extensive experiments on two real-world datasets show that the proposed LabanFormer model obtains remarkable performance compared with state-of-the-art approaches on the automatic Labanotation generation task. [ABSTRACT FROM AUTHOR]

To address the issue of high miss rates for distant small objects and the diminished system detection performance due to the influence of hazy when autonomous vehicles operate on mountain highways. We propose a framework for small object vehicle detection in hazy traffic environments (SHTDet). This framework aims to enhance small object detection for autonomous driving under hazy conditions on mountainous motorways. Specifically, to restore the clarity of hazy images, we designed an image enhancement (IE), and its parameters are predicted by a convolutional neural network [filter parameter estimation (FPE)]. In addition, to enhance the detection accuracy of small objects, we introduce a cascaded sparse query (CSQ) mechanism, which effectively utilizes high-resolution features while maintaining fast detection speed. We jointly optimize the IE and the detection network (CSQ-FCOS) in an end-to-end manner, ensuring that FPE module can learn a suitable IE. Our proposed SHTDet method is adept at adaptively handling sunny and hazy conditions. Extensive experiments demonstrate the efficacy of the SHTDet method in detecting small objects on hazy sections of mountain highways. [ABSTRACT FROM AUTHOR]