Search

Danmakus are user-generated comments that overlay on videos, enabling real-time interactions between viewers and video content. The emotional orientation of danmakus can reflect the attitudes and opinions of viewers on video segments, which can help video platforms optimize video content recommendation and evaluate users' abnormal emotion levels. Aiming at the problems of low transferability of traditional sentiment analysis methods in the danmaku domain, low accuracy of danmaku text segmentation, poor consistency of sentiment annotation, and insufficient semantic feature extraction, this paper proposes a video danmaku sentiment analysis method based on MIBE-RoBERTa-FF-BiLSTM. This paper constructs a "Bilibili Must-Watch List and Top Video Danmaku Sentiment Dataset" by ourselves, covering 10,000 positive and negative sentiment danmaku texts of 18 themes. A new word recognition algorithm based on mutual information (MI) and branch entropy (BE) is used to discover 2610 irregular network popular new words from trigrams to heptagrams in the dataset, forming a domain lexicon. The Maslow's hierarchy of needs theory is applied to guide the consistent sentiment annotation. The domain lexicon is integrated into the feature fusion layer of the RoBERTa-FF-BiLSTM model to fully learn the semantic features of word information, character information, and context information of danmaku texts and perform sentiment classification. Comparative experiments on the dataset show that the model proposed in this paper has the best comprehensive performance among the mainstream models for video danmaku text sentiment classification, with an F1 value of 94.06%, and its accuracy and robustness are also better than other models. The limitations of this paper are that the construction of the domain lexicon still requires manual participation and review, the semantic information of danmaku video content and the positive case preference are ignored. [ABSTRACT FROM AUTHOR]

The automation of hand hygiene monitoring is critical in healthcare for ensuring clean hands and preventing infectious disease spread. While advancements have been made, existing methods have limitations in accurately detecting and classifying handwashing actions. This paper addresses these limitations and introduces the Residual-Based Multilevel Fused Network (ResMFuse-Net) as a novel approach to automate the quality assurance of hand hygiene procedures. Our model integrates advanced techniques, including feature fusion, model compression, a feature fusion block (FFB), and a modified separable residual block (SE-ResB). The proposed model fused two networks into one trainable feature extraction pipeline, and applies model compression to retain the core blocks that are crucial for propagating strong and robust features while conserving a significant fraction of the computing resources. Additionally, we introduce a FFB that includes ConvLSTM and alpha dropout to learn spatial dependencies, establish correlations between frames in a video, and mitigate overfitting. This paper introduces a SE-ResB, which is a customized residual component composed of separable convolutions and LeakyReLU activation. The SE-ResB is incorporated to handle the fused features and generate a more diverse set of features, leading to considerable performance enhancements. This study also includes an ablation analysis that highlights the importance of each component. The proposed ResMFuse-Net is evaluated on two datasets: a newly created handwashing dataset (451 videos) and a publicly available dataset (656 videos). Achieving a recognition accuracy of 97.61% on the handwashing dataset and 98.69% on the other dataset, the ResMFuse-Net outperforms previous methods with fewer parameters and FLOPs, demonstrating its efficiency and cost-effectiveness. [ABSTRACT FROM AUTHOR]

Industrial defect detection is a hot topic in the field of computer vision. It is a challenging task due to complex features and many categories of industrial defects. In this paper, a deep learning model based on the multiscale feature extraction module is introduced for steel surface defect detection. The main focus on the feature extraction capability of the model and feature fusion capability to improve the accuracy of the model for steel surface defect detection. First, to improve the feature extraction ability of the model, a multiscale feature extraction (MSFE) module is introduced. The MSFE module can effectively extract multiscale features through three branches that have different convolution kernel sizes. Second, an efficient feature fusion (EFF) module is proposed to optimize feature fusion by adding features from the backbone network to the neck network. Third, this paper puts forward a new Bottleneck module by reducing the normalization layer and activation function in the original Bottleneck module. Finally, the backbone network is deepened to further enhance the feature extraction ability of the model. Extensive experiments are conducted on the public NEU-DET dataset. The experimental results validate the effectiveness of the designed modules and the proposed model. Compared with other state-of-the-art methods, the proposed model achieves optimal accuracy(73.08% mAP@0.5) while maintaining a small number of parameters. [ABSTRACT FROM AUTHOR]

Generally, the recognition performance of lightweight models is often lower than that of large models. Knowledge distillation, by teaching a student model using a teacher model, can further enhance the recognition accuracy of lightweight models. In this paper, we approach knowledge distillation from the perspective of intermediate feature-level knowledge distillation. We combine a cross-stage feature fusion symmetric framework, an attention mechanism to enhance the fused features, and a contrastive loss function for teacher and student models at the same stage to comprehensively implement a multistage feature fusion knowledge distillation method. This approach addresses the problem of significant differences in the intermediate feature distributions between teacher and student models, making it difficult to effectively learn implicit knowledge and thus improving the recognition accuracy of the student model. Compared to existing knowledge distillation methods, our method performs at a superior level. On the CIFAR100 dataset, it boosts the recognition accuracy of ResNet20 from 69.06% to 71.34%, and on the TinyImagenet dataset, it increases the recognition accuracy of ResNet18 from 66.54% to 68.03%, demonstrating the effectiveness and generalizability of our approach. Furthermore, there is room for further optimization of the overall distillation structure and feature extraction methods in this approach, which requires further research and exploration. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been the dominant architectures for feature extraction tasks, but CNNs do not look for and focus on some specific image features. Correlation operations play an important role in visual tracking. However, the correlation operation reserves a large amount of unfavorable background information. In this paper, we propose an effective feature recognizer including channel and spatial attention modules to focus on important object feature information. Thus, the representation power of the feature extraction network is improved. Further, we design a multi-scale feature fusion network. The fusion network performs feature fusion on template feature and encoded feature branches to establish connections between features at different scales. Experiments on six benchmarks demonstrate that the proposed tracker outperforms the state-of-the-art trackers. In particular, the proposed tracker achieves an 80.4% AUC on TrackingNet and a 68.4% AUC on GOT-10k while running at a real-time speed. [ABSTRACT FROM AUTHOR]

The presence of haze results in diminished image clarity, the loss of image details, color alterations, and a decline in image visibility. This paper proposes an efficient multi-stage feature fusion defogging network based on the attention mechanism for effective defogging in complex environments. To create a multi-branch defogging network, the model integrates several attention techniques to create various sorts of branching network architectures. The image's spatial details and contextual information are supplemented and merged based on the recovered image feature information of different network branches to increase the effectiveness of the network model. The stage attention fusion mechanism created among several network branches can lessen the loss of image data during feature extraction and improve the effectiveness of the image-defogging operation. The experimental results demonstrate that the proposed algorithm has superior defogging performance in both synthetic and real-world scene datasets and performs more admirably in terms of accuracy compared to other sophisticated algorithms, particularly in the RESIDE and O-Haze datasets. The PSNR metrics on the RESIDE and O-Haze datasets are improved by 1.58 dB and 1.61 dB, respectively, compared to the best-advanced technique suggested in this study, and the SSIM metrics on the O-Haze dataset are improved by 3.4%. [ABSTRACT FROM AUTHOR]

Siamese-based trackers have been widely studied for their high accuracy and speed. Both the feature extraction and feature fusion are two important components in Siamese-based trackers. Siamese-based trackers obtain fine local features by traditional convolution. However, some important channel information and global information are lost when enhancing local features. In the feature fusion process, cross-correlation-based feature fusion between the template and search region feature ignores the global spatial context information and does not make the best of the spatial information. In this paper, to solve the above problem, we design a novel feature extraction sub-network based on batch-free normalization re-parameterization convolution, which scales the features in the channel dimension and increases the receptive field. Richer channel information is obtained and powerful target features are extracted for the feature fusion. Furthermore, we learn a feature fusion network (FFN) based on feature filter. The FFN fuses the template and search region features in a global spatial context to obtain high-quality fused features by enhancing important features and filtering redundant features. By jointly learning the proposed feature extraction sub-network and FFN, the local and global information are fully exploited. Then, we propose a novel tracking algorithm based on the designed feature extraction sub-network and FFN with re-parameterization convolution and feature filter, referred to as RCFT. We evaluate the proposed RCFT tracker and some recent state-of-the-art (SOTA) trackers on OTB100, VOT2018, LaSOT, GOT-10k, UAV123 and the visual-thermal dataset VOT-RGBT2019 datasets, which achieves superior tracking performance with 45 FPS tracking speed. [ABSTRACT FROM AUTHOR]