Search

Abstract To address issues such as object‐background confusion and difficulties in multi‐scale object feature extraction in underwater scenarios, this article proposes an underwater organisms detection algorithm based on multi‐scale perception and representation enhancement. The key innovation of the proposed algorithm is the perception improvement of the deep learning model for underwater multi‐scale objects. First, for underwater large‐scale objects, omni‐dimensional dynamic convolution is embedded as an attention mechanism (AM) into the deep network to improve the network's sensitivity to large‐scale underwater objects. For underwater small‐scale objects, an information retention downsampling module is designed to reduce the effects of serious information loss. Then, a contextual transformer as an AM is introduced into shallow networks to strengthen the network's ability to extract features from small objects. The second innovation of the proposed algorithm is an underwater spatial pooling pyramid module which enhances the representation ability of the model. Furthermore, a lightweight decoupled head is designed to eliminate the conflict between classification and localization. The ablation experiment on the URPC dataset shows that the proposed models are effective for underwater object detection. The comparative experiments on the URPC and DUT‐USEG datasets demonstrate that the proposed algorithm achieves an advantage in detection performance compared with the mainstream detection algorithms and underwater detection algorithms.

Abstract Deep learning has made significant progress in the field of low‐light image enhancement. However, challenges remain, such as the substantial parameter consumption required for effective image enhancement. Inspired by multi‐scale geometric transformations in image detail enhancement, a novel model called the fixed‐directional filters network is proposed. Fixed‐directional filters network takes the original image as input and employs multiple branches for parallel processing. One branch uses conventional convolutional layers to extract features from the original image, while the other branches apply non‐linear mapping layers based on wavelet transforms. These wavelet transform branches capture the multi‐scale information of the image by combining different directions and convolutional kernels and utilize a trainable custom gamma mapping layer for non‐linear modulation to enhance specific regions of the image. The feature maps processed by each branch are merged through concatenation operations and then passed through convolutional layers to output the enhanced image. Using trainable mapping functions alone to enhance details significantly reduces the reliance on convolutional layers, effectively lowering the model's parameter count to only 13k parameters. Additionally, experiments demonstrate that fixed‐directional filters network significantly improves image quality, particularly in capturing image details and enhancing image contrast.

Abstract The surface quality of aluminium alloy castings is crucial to quality control. Aiming to address the challenges of limited samples and extensive computation in deep learning‐based surface defect detection for aluminium alloy castings, this paper proposes a surface defect detection method based on data enhancement and the Casting Real‐Time DEtection TRansformer. First, to tackle the issue of small sample sizes and uneven distribution in surface defect data sets of aluminium alloy castings, ECA‐MetaAconC Deep Convolution Generative Adversarial Networks is proposed for generating defects with fewer samples and employ the image augmentation (IMGAUG) library for sample enhancement. Second, building upon the Real‐Time DEtection TRansformer (RT‐DETR), a lightweight partial‐rep convolution is designed to decrease the network's parameter count. Simultaneously, the Deformable attention module and the DRBC3 module are introduced to enhance the neck network, thereby improving the model's capability to capture information and enhancing its detection performance. Compared to RT‐DETR, this method reduces the number of model parameters by 38.7%, increases mAP by 1.5%, and achieves a frame rate that is 1.58 times higher than the original model. The experimental results demonstrate that this method can effectively and accurately detect surface defects in aluminium alloy castings, satisfying industrial requirements.

Abstract To effectively address common issues such as cattle being obscured by fences and images prone to colour shifts and high brightness in a ranch setting, this paper proposes an algorithm for counting cows based on multi‐scale perception and image correlation. The algorithm first adjusts the model output scale to enhance cattle detection under current conditions. It incor‐porates efficient Partial Convolution (PConv) to replace 3 × 3 convolutions in the Neck segment of the YOLOv7 network, boost‐ing computational speed and reducing complexity. To streamline feature fusion, Dynamic Head (DyHead) unifies multiple at‐tentional operations in the Neck segment, enhancing efficiency. Additionally, it introduces a novel bounding box similarity metric Minimum Point DioU (MPDIoU) based on minimum point distance, encompassing factors from existing loss functions, while simplifying computations. Experimental results demonstrate the algorithm significantly improves detection, achieving 98.8% accuracy, 99.0% recall, and a 92.1% mAP value. Compared with mainstream SOTA models, Precision increases by 0.4%, Recall by 2.0%, and mAP value by 2.2%. Model size decreases by 23.9%, parameter count by 23.0%, and computational load by 6.1%. the algorithm shows improvements across all indices, meeting the challenge of real‐time cattle counting in ranches under complex conditions.

Abstract This study evaluates the effectiveness of three loss functions Asymmetric Unified Focal Loss (AUFL), Dice Similarity Coefficient Loss (DSCL), and Cross‐Entropy (CE) for segmenting COVID‐19 lung infections in computed tomography images. Detailed analyses using the intersection over union metric assessed each function's accuracy. AUFL achieved an average Dice Similarity Coefficient (DSC) of 85.18% ± 8.86%, outperforming DSCL 85.18% ± 8.86%, which had the same average DSC but less precise segmentation, and CE, which had an average DSC of 78.31% ± 11.93%. Segmentations using AUFL demonstrated more defined contours and better alignment with actual anatomical structures than those obtained with DSCL and CE. Observations revealed that AUFL‐generated segmentations had more precise boundaries and were more consistent with the expected anatomical regions of lung infections. This study is the first to quantitatively and qualitatively compare the effectiveness of AUFL, DSCL, and CE in segmenting COVID‐19 lung infections, providing concrete evidence of AUFL's superiority in segmentation performance and reliability for clinical applications. The findings underscore the importance of selecting appropriate loss functions to enhance segmentation in medical imaging, highlighting their crucial role in improving image‐based diagnostics and treatment. The study emphasizes the need for ongoing research to optimize these segmentation techniques further.

Abstract In pedestrian detection task, numerous predicted boxes and their corresponding scores are generated and these scores are used to filter these predicted boxes by non‐maximum suppression. This paper analysed the training process of the popular anchor‐based pedestrian detection models (e.g. YOLO and Faster RCNN), and found that the score of the predicted box reflects the overlap between the corresponding anchor and the ground truth, rather than the predicted box itself. Due to the many‐to‐one strategy adopted by anchor‐based methods, multiple predicted boxes could be generated around one predicted box. This study refers to the number of other predicted boxes around the target predicted box as its local density. When a predicted box has a higher local density, it should have a greater overlap with the ground truth. Therefore, this study proposed the fused score by introducing local density into the score. The experiments showed that replacing the score with the fused score can effectively improve the model's detection accuracy. The code and experiments will soon be open‐sourced at https://github.com/zefeichen/FusedScore.

Abstract Chinese stroke segmentation is a crucial and challenging task for various downstream applications such as font generation, aesthetic evaluation etc. Conventional semantic segmentation techniques typically face difficulties in accurately segmenting strokes, as intersection regions in Chinese characters can belong to multiple strokes simultaneously, and these approaches often lack a holistic understanding of character composition. This paper proposes a character stroke segmentation framework named Stroke‐Seg that integrates with various semantic segmentation architectures, demonstrating adaptability to different backbone networks to tackle the above tasks. A multi‐label output strategy is proposed to effectively classify strokes in intersection areas, overcoming the limitations of traditional semantic segmentation approaches. Additionally, a prior knowledge vector is incorporated into the input layer to provide character‐specific information on stroke composition, enhancing the ability of the framework to precisely identify and segment strokes. The effectiveness of the proposed framework is demonstrated through evaluations of a comprehensive dataset (brush calligraphy stroke segmentation dataset). Evaluations show that the proposed framework significantly improves the capability of semantic segmentation networks, achieving a remarkable improvement of up to 19.9% in the true stroke rate, compared to traditional stroke segmentation techniques. The Stroke‐Seg framework integrated with TransUNet demonstrates its high performance with an impressive 98.2% true stroke rate. Furthermore, the proposed framework combined with FCN still achieves good performance while consuming the least amount of computational resources and memory, demonstrating its potential for lightweight design.

Abstract Melanoma, a highly prevalent and lethal form of skin cancer, has a significant impact globally. The chances of recovery for melanoma patients substantially improve with early detection. Currently, deep learning (DL) methods are gaining popularity in assisting with the early identification of melanoma. Despite their high performance, relying solely on an image classifier undermines the credibility of the application and makes it difficult to understand the rationale behind the model's predictions highlighting the need for Explainable AI (XAI). This study provides a survey on skin cancer identification using DL techniques utilized in studies from 2017 to 2024. Compared to existing survey studies, the authors address the latest related studies covering several public skin cancer image datasets and focusing on segmentation, classification based on convolutional neural networks and vision transformers, and explainability. The analysis and the comparisons of the existing studies will be beneficial for the researchers and developers in this area, to identify the suitable techniques to be used for automated skin cancer image classification. Thereby, the survey findings can be used to implement support applications advancing the skin cancer diagnosis process.

Abstract Insulators, as typical equipment for distribution networks, provide good electrical insulation between live conductors and earth. Timely and accurate detection is essential for insulator detection issues. However, as the complexity of neural networks increases, the detection efficiency is often lower. Therefore, this paper proposes a fast insulator positioning and defect detection method. Firstly, for insulator target localization, the SqueezeNet network is improved using ECA attention mechanism. In addition, to address the issue of low defect detection accuracy, a joint algorithm has been proposed. The integration of convolutional variational autoencoder (CVAE) and generative adversarial network (GAN) solve their own shortcomings due to different image focus angles. The target localization accuracy reaches 94.30%, and the defect detection accuracy reaches 89.60%. It solves the problems of difficulty in locating small targets in a large field of view and inaccurate detection due to a small number of abnormal samples. This method has been tried and tested in practical distribution network systems.

Abstract License plate recognition is crucial in Intelligent Transportation Systems (ITS) for vehicle management, traffic monitoring, and security inspection. In highway scenarios, this task faces challenges such as diversity, blurriness, occlusion, and illumination variation of license plates. This article explores Recurrent Neural Networks based on Connectionist Temporal Classification (RNN‐CTC) for license plate recognition in challenging highway conditions. Four neural network models: ResNet50, ResNeXt, InceptionV3, and SENet, all combined with RNN‐CTC are comparatively evaluated. Furthermore, a novel architecture named ResNet50 Deep Fusion Network using Connectionist Temporal Classification (ResNet50‐DFN‐CTC) is proposed. Comparative and ablation experiments are conducted using the Highway License Plate Dataset of Southeast University (HLPD‐SU). Results demonstrate the superior performance of ResNet50‐DFN‐CTC in challenging highway conditions, achieving 93.158% accuracy with a processing time of 7.91 ms, outperforming other tested models. This research contributes to advancing license plate recognition technology for real‐world highway applications under adverse conditions.

Abstract With the advent of the digital era, millions of facial images are shared online daily, posing severe privacy threats. Generative face anonymization (GFA) methods generate virtual faces to conceal original identities, protecting sensitive information while preserving utility. However, deep learning based user identity linkage (UIL) methods can link similar faces to the same identity and leverage the linked profiles for malicious purposes, including localization and behaviour prediction. These UIL methods pose a significant challenge to the diversity of virtual faces, a challenge that existing GFA methods have not adequately addressed. To address this research gap, we propose Style Diversification‐based Generative Face Anonymization (SD‐GFA), a framework that generates virtual faces with diverse identities and high visual quality. SD‐GFA features an equalized control module to balance input faces and user‐specified keys, a face generation module with a re‐connection strategy for high‐quality synthesis, and a maximum probability simulation module to enhance diversity. Our experiments demonstrate that SD‐GFA effectively mitigates linkage risk by improving the diversity of virtual faces, while also enhancing their utility and visual quality. This study provides a robust solution to enhance the security of anonymized faces shared on the internet.

Abstract Unsupervised anomaly detection, often approached as a one‐class classification problem, is a critical task in computer vision. Knowledge distillation has emerged as a promising technique for enhancing anomaly detection accuracy, especially with the advent of reverse distillation networks that employ encoder–decoder architectures. This study introduces a novel reverse knowledge distillation framework known as RDMS, which incorporates a pretrained teacher encoding module, a multi‐level feature fusion connection module, and a student decoding module consisting of three independent decoders. RDMS is designed to distill distinct features from the teacher encoder, mitigating overfitting issues associated with similar or identical teacher–student structures. The model achieves an average of 99.3% image‐level AUROC and 98.34% pixel‐level AUROC on the MVTec‐AD dataset and demonstrates state‐of‐the‐art performance on the more challenging BTAD dataset. The RDMS model's high accuracy in anomaly detection and localization underscores the potential of multi‐student reverse distillation to advance unsupervised anomaly detection capabilities. The source code is available at https://github.com/zihengchen777/RDMS

Abstract Conventional computed tomography (CT) images often suffer from blurred edges and unclear details. Image super‐resolution methods can significantly enhance CT image quality, thereby improving diagnostic accuracy. To better extract detailed features and enhance the cascading effects of different feature levels, we propose a novel medical image super‐resolution algorithm that integrates discrete wavelet transform and multi‐scale adaptive feature selection. Our approach uses both the low‐resolution image and its high‐frequency component from the frequency domain as network inputs, with the high‐frequency component providing learning supervision, which enhances detail fidelity in reconstruction. Additionally, we introduce a multi‐scale adaptive feature selection module to learn from different layers of CT images and their inter‐layer correlations. Finally, the pixel information is efficiently integrated by a coordinate attention mechanism incorporating the concept of squeeze excitation. Experimental results show that our method outperforms state‐of‐the‐art methods, achieving superior reconstruction at scale factors of 2, 4, and 8, especially at scale factor 8, where it surpasses others by 1.12 in PSNR, 0.0145 in SSIM, and 0.0038 in LPIPS. Visually, our method also delivers more accurate details and better perceptual quality.

Abstract Fisheye cameras have been widely used in autonomous navigation, visual surveillance, and automatic driving. Due to severe geometric distortion, fisheye images cannot be processed effectively by conventional methods. The existing object detection algorithms cannot better detect the small targets or the objects with large distortion in the fisheye images. The size and scene of available fisheye datasets (such as WoodScape and VOC‐360) cannot satisfy the training of robust network models. Herein, the authors propose Swin‐Fisheye, an end‐to‐end object detection algorithm based on Swin Transformer. A feature pyramid module based on deformable convolution (DFPM) is designed to obtain richer contextual information from the multi‐scale feature maps. In addition, a projection transformation algorithm (PTA) is proposed, which can convert rectilinear images into fisheye images more accurately, and then create a fisheye image dataset (COCO‐Fish). The results of extensive experiments conducted on VOC‐360, WoodScape, and COCO‐Fish demonstrate that the proposed algorithm can achieve satisfactory results compared with state‐of‐the‐art methods.

Abstract Detecting taxi passengers is crucial for assessing taxi driver behavior, which plays a significant role in regulating the taxi industry. Despite the advancements in deep learning, object detection algorithms have not been extensively applied to this domain. In this article, an innovative taxi passenger detection algorithm is introduced based on YOLOv8, a lightweight and highly accurate method designed to automatically monitor driver behavior and regulate the taxi industry. To address the challenge of deploying complex object detection models on mobile devices, the ghost module is incorporated in place of standard convolutions within the C2f module, thereby making the model more lightweight. Furthermore, the model's performance is enhanced by integrating an improved version of Frequency Channel Attention (FCA), termed Parallel Frequency Channel Attention (PFCA), which boosts detection accuracy with minimal additional parameters and computational overhead. Experimental results on a specific taxi passenger dataset demonstrate that the proposed method significantly outperforms the baseline YOLOv8n model. Specifically, the model reduces the number of parameters and floating point operations by 12.96% and 8.18%, respectively, while achieving increases in mAP50 and mAP50‐95 by 0.27 and 0.73 percentage points, respectively.

Abstract Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications.

Abstract The rapid and repeatable characterization of individual morphology has advanced automated taxonomic classification. The most direct study of evolutionary processes is, however, not from taxonomic description, but rather of the evolution of the traits that comprise individuals and define species. Repeatable signatures of individual morphology are crucial for analysing the response to selection at scale, and thus tracking evolutionary trajectories through time and across species boundaries. Here, we introduce our 3DKMI—an open‐source MATLAB package designed for the study of morphology using three‐dimensional (3D) Krawtchouk moment invariants. The volumetric features derived from the 3D images remain stable under translation, scaling and rotation and, for an image of size 128 × 128 × 128 can be computed in less than 0.1 s. We applied our package as a case study on a collection of 300 X‐ray computed tomography scans of planktonic foraminifera specimens across five species to (1) assess the invariance of the features under different transformations and (2) analyse morphological differences among species based on the extracted characteristics. We show that 3DKMI has the capacity to efficiently and repeatedly characterize the signatures of individual morphology. In the future, we hope that the 3D feature extraction technique 3DKMI will be widely applied to digital collections to advance research in ecology and evolution.

Abstract Counting high‐density objects quickly and accurately is a popular area of research. Crowd counting has significant social and economic value and is a major focus in artificial intelligence. Despite many advancements in this field, many of them are not widely known, especially in terms of research data. The authors proposed a three‐tier standardised dataset taxonomy (TSDT). The Taxonomy divides datasets into small‐scale, large‐scale and hyper‐scale, according to different application scenarios. This theory can help researchers make more efficient use of datasets and improve the performance of AI algorithms in specific fields. Additionally, the authors proposed a new evaluation index for the clarity of the dataset: average pixel occupied by each object (APO). This new evaluation index is more suitable for evaluating the clarity of the dataset in the object counting task than the image resolution. Moreover, the authors classified the crowd counting methods from a data‐driven perspective: multi‐scale networks, single‐column networks, multi‐column networks, multi‐task networks, attention networks and weak‐supervised networks and introduced the classic crowd counting methods of each class. The authors classified the existing 36 datasets according to the theory of three‐tier standardised dataset taxonomy and discussed and evaluated these datasets. The authors evaluated the performance of more than 100 methods in the past five years on different levels of popular datasets. Recently, progress in research on small‐scale datasets has slowed down. There are few new datasets and algorithms on small‐scale datasets. The studies focused on large or hyper‐scale datasets appear to be reaching a saturation point. The combined use of multiple approaches began to be a major research direction. The authors discussed the theoretical and practical challenges of crowd counting from the perspective of data, algorithms and computing resources. The field of crowd counting is moving towards combining multiple methods and requires fresh, targeted datasets. Despite advancements, the field still faces challenges such as handling real‐world scenarios and processing large crowds in real‐time. Researchers are exploring transfer learning to overcome the limitations of small datasets. The development of effective algorithms for crowd counting remains a challenging and important task in computer vision and AI, with many opportunities for future research.

Abstract Underwater object detection (UOD) is crucial for marine economic development, environmental protection, and the planet's sustainable development. The main challenges of this task arise from low‐contrast, small objects, and mimicry of aquatic organisms. The key to addressing these challenges is to focus the model on obtaining more discriminative information. The authors observe that the edges of underwater objects are highly unique and can be distinguished from low‐contrast or mimicry environments based on their edges. Motivated by this observation, an Edge‐guided Representation Learning Network, termed ERL‐Net is proposed, that aims to achieve discriminative representation learning and aggregation under the guidance of edge cues. Firstly, an edge‐guided attention module is introduced to model the explicit boundary information, which generates more discriminative features. Secondly, a hierarchical feature aggregation module is proposed to aggregate the multi‐scale discriminative features by regrouping them into three levels, effectively aggregating global and local information for locating and recognising underwater objects. Finally, a wide and asymmetric receptive field block is proposed to enable features to have a wider receptive field, allowing the model to focus on smaller object information. Comprehensive experiments on three challenging underwater datasets show that our method achieves superior performance on the UOD task.

Abstract Image classification algorithms are commonly based on the Independent and Identically Distribution (i.i.d.) assumption, but in practice, the Out‐Of‐Distribution (OOD) problem widely exists, that is, the contexts of images in the model predicting are usually unseen during training. In this case, existing models trained under the i.i.d. assumption are limiting generalisation. Causal inference is an important method to learn the causal associations which are invariant across different environments, thus improving the generalisation ability of the model. However, existing methods usually require partitioning of the environment to learn invariant features, which mostly have imbalance problems due to the lack of constraints. In this paper, we propose a balanced causal learning framework (BCL), starting from how to divide the dataset in a balanced way and the balance of training after the division, which automatically generates fine‐grained balanced data partitions in an unsupervised manner and balances the training difficulty of different classes, thereby enhancing the generalisation ability of models in different environments. Experiments on the OOD datasets NICO and NICO++ demonstrate that BCL achieves stable predictions on OOD data, and we also find that models using BCL focus more accurately on the foreground of images compared with the existing causal inference method, which effectively improves the generalisation ability.

Abstract Recovering 3D human meshes from monocular images is an inherently ill‐posed and challenging task due to depth ambiguity, joint occlusion, and truncation. However, most existing approaches do not model such uncertainties, typically yielding a single reconstruction for one input. In contrast, the ambiguity of the reconstruction is embraced and the problem is considered as an inverse problem for which multiple feasible solutions exist. To address these issues, the authors propose a multi‐hypothesis approach, multi‐hypothesis human mesh recovery (MH‐HMR), to efficiently model the multi‐hypothesis representation and build strong relationships among the hypothetical features. Specifically, the task is decomposed into three stages: (1) generating a reasonable set of initial recovery results (i.e., multiple hypotheses) given a single colour image; (2) modelling intra‐hypothesis refinement to enhance every single‐hypothesis feature; and (3) establishing inter‐hypothesis communication and regressing the final human meshes. Meanwhile, the authors take further advantage of multiple hypotheses and the recovery process to achieve human mesh recovery from multiple uncalibrated views. Compared with state‐of‐the‐art methods, the MH‐HMR approach achieves superior performance and recovers more accurate human meshes on challenging benchmark datasets, such as Human3.6M and 3DPW, while demonstrating the effectiveness across a variety of settings. The code will be publicly available at https://cic.tju.edu.cn/faculty/likun/projects/MH‐HMR.

Abstract Image captioning aims to automatically generate a natural language description of a given image, and most state‐of‐the‐art models have adopted an encoder–decoder transformer framework. Such transformer structures, however, show two main limitations in the task of image captioning. Firstly, the traditional transformer obtains high‐level fusion features to decode while ignoring other‐level features, resulting in losses of image content. Secondly, the transformer is weak in modelling the natural order characteristics of language. To address theseissues, the authors propose a HIerarchical and Sequential Transformer (HIST) structure, which forces each layer of the encoder and decoder to focus on features of different granularities, and strengthen the sequentially semantic information. Specifically, to capture the details of different levels of features in the image, the authors combine the visual features of multiple regions and divide them into multiple levels differently. In addition, to enhance the sequential information, the sequential enhancement module in each decoder layer block extracts different levels of features for sequentially semantic extraction and expression. Extensive experiments on the public datasets MS‐COCO and Flickr30k have demonstrated the effectiveness of our proposed method, and show that the authors’ method outperforms most of previous state of the arts.

Abstract Siamese networks excel at comparing two images, serving as an effective class verification technique for a single‐per‐class reference image. However, when multiple reference images are present, Siamese verification necessitates multiple comparisons and aggregation, often unpractical at inference. The Centre‐Loss approach, proposed in this research, solves a class verification task more efficiently, using a single forward‐pass during inference, than sample‐to‐sample approaches. Optimising a Centre‐Loss function learns class centres and minimises intra‐class distances in latent space. The authors compared verification accuracy using Centre‐Loss against aggregated Siamese when other hyperparameters (such as neural network backbone and distance type) are the same. Experiments were performed to contrast the ubiquitous Euclidean against other distance types to discover the optimum Centre‐Loss layer, its size, and Centre‐Loss weight. In optimal architecture, the Centre‐Loss layer is connected to the penultimate layer, calculates Euclidean distance, and its size depends on distance type. The Centre‐Loss method was validated on the Self‐Checkout products and Fruits 360 image datasets. Centre‐Loss comparable accuracy and lesser complexity make it a preferred approach over sample‐to‐sample for the class verification task, when the number of reference image per class is high and inference speed is a factor, such as in self‐checkouts.

Abstract Despite the impressive progress in visual localisation, 6‐DoF cross‐view localisation is still a challenging task in the computer vision community due to the huge appearance changes. To address this issue, the authors propose the CLaSP, a coarse‐to‐fine framework, which leverages a synthetic panorama to facilitate cross‐view 6‐DoF localisation in a large‐scale scene. The authors first leverage a segmentation map to correct the prior pose, followed by a synthetic panorama on the ground to enable coarse pose estimation combined with a template matching method. The authors finally formulate the refine localisation process as feature matching and pose refinement to obtain the final result. The authors evaluate the performance of the CLaSP and several state‐of‐the‐art baselines on the Airloc dataset, which demonstrates the effectiveness of our proposed framework.

Abstract Recently, the application of finger vein recognition has become popular. Studies have shown finger vein presentation attacks increasingly threaten these recognition devices. As a result, research on finger vein presentation attack detection (fvPAD) methods has received much attention. However, the current fvPAD methods have two limitations. (1) Most terminal devices cannot train fvPAD models independently due to a lack of data. (2) Several research institutes can train fvPAD models; however, these models perform poorly when applied to terminal devices due to inadequate generalisation. Consequently, it is difficult for threatened terminal devices to obtain an effective fvPAD model. To address this problem, the method of federated finger vein presentation attack detection for various clients is proposed, which is the first study that introduces federated learning (FL) to fvPAD. In the proposed method, the differences in data volume and computing power between clients are considered. Traditional FL clients are expanded into two categories: institutional and terminal clients. For institutional clients, an improved triplet training mode with FL is designed to enhance model generalisation. For terminal clients, their inability is solved to obtain effective fvPAD models. Finally, extensive experiments are conducted on three datasets, which demonstrate the superiority of our method.

Abstract Human action recognition based on graph convolutional networks (GCNs) is one of the hotspots in computer vision. However, previous methods generally rely on handcrafted graph, which limits the effectiveness of the model in characterising the connections between indirectly connected joints. The limitation leads to weakened connections when joints are separated by long distances. To address the above issue, the authors propose a skeleton simplification method which aims to reduce the number of joints and the distance between joints by merging adjacent joints into simplified joints. Group convolutional block is devised to extract the internal features of the simplified joints. Additionally, the authors enhance the method by introducing multi‐scale modelling, which maps inputs into sequences across various levels of simplification. Combining with spatial temporal graph convolution, a multi‐scale skeleton simplification GCN for skeleton‐based action recognition (M3S‐GCN) is proposed for fusing multi‐scale skeleton sequences and modelling the connections between joints. Finally, M3S‐GCN is evaluated on five benchmarks of NTU RGB+D 60 (C‐Sub, C‐View), NTU RGB+D 120 (X‐Sub, X‐Set) and NW‐UCLA datasets. Experimental results show that the authors’ M3S‐GCN achieves state‐of‐the‐art performance with the accuracies of 93.0%, 97.0% and 91.2% on C‐Sub, C‐View and X‐Set benchmarks, which validates the effectiveness of the method.

Abstract Deep learning‐based face recognition models have demonstrated remarkable performance in benchmark tests, and knowledge distillation technology has been frequently accustomed to obtain high‐precision real‐time face recognition models specifically designed for mobile and embedded devices. However, in recent years, the knowledge distillation methods for face recognition, which mainly focus on feature or logit knowledge distillation techniques, neglect the attention mechanism that play an important role in the domain of neural networks. An innovation cross‐stage connection review path of the attention cosine similarity knowledge distillation method that unites the attention mechanism with review knowledge distillation method is proposed. This method transfers the attention map obtained from the teacher network to the student through a cross‐stage connection path. The efficacy and excellence of the proposed algorithm are demonstrated in popular benchmark tests.

Abstract As the proliferation of video content continues, and many video archives lack suitable metadata, therefore, video retrieval, particularly through example‐based search, has become increasingly crucial. Existing metadata often fails to meet the needs of specific types of searches, especially when videos contain elements from different modalities, such as visual and audio. Consequently, developing video retrieval methods that can handle multi‐modal content is essential. An innovative Multi‐modal Video Search by Examples (MVSE) framework is introduced, employing state‐of‐the‐art techniques in its various components. In designing MVSE, the authors focused on accuracy, efficiency, interactivity, and extensibility, with key components including advanced data processing and a user‐friendly interface aimed at enhancing search effectiveness and user experience. Furthermore, the framework was comprehensively evaluated, assessing individual components, data quality issues, and overall retrieval performance using high‐quality and low‐quality BBC archive videos. The evaluation reveals that: (1) multi‐modal search yields better results than single‐modal search; (2) the quality of video, both visual and audio, has an impact on the query precision. Compared with image query results, audio quality has a greater impact on the query precision (3) a two‐stage search process (i.e. searching by Hamming distance based on hashing, followed by searching by Cosine similarity based on embedding); is effective but increases time overhead; (4) large‐scale video retrieval is not only feasible but also expected to emerge shortly.

Abstract With the development of automatic driving and path planning technology, predicting the moving trajectory of pedestrians in dynamic scenes has become one of key and urgent technical problems. However, most of the existing techniques regard all pedestrians in the scene as equally important influence on the predicted pedestrian's trajectory, and the existing methods which use sequence‐based time‐series generative models to obtain the predicted trajectories, do not allow for parallel computation, it will introduce a significant computational overhead. A new social trajectory prediction network, Social‐ATPGNN which integrates both temporal information and spatial one based on ATPGNN is proposed. In space domain, the pedestrians in the predicted scene are formed into an undirected and non fully connected graph, which solves the problem of homogenisation of pedestrian relationships, then, the spatial interaction between pedestrians is encoded to improve the accuracy of modelling pedestrian social consciousness. After acquiring high‐level spatial data, the method uses Temporal Convolutional Network which could perform parallel calculations to capture the correlation of time series of pedestrian trajectories. Through a large number of experiments, the proposed model shows the superiority over the latest models on various pedestrian trajectory datasets.

Abstract The authors study the problem of reconstructing detailed 3D human surfaces in various poses and clothing from images. The parametric human body allows accurate 3D clothed human reconstruction. However, the offset of large and loose clothing from the inferred parametric body mesh confines the generalisation of the existing parametric body‐based methods. A distinctive method that simultaneously generalises well to unseen poses and unseen clothing is proposed. The authors first discover the unbalanced nature of existing implicit function‐based methods. To address this issue, the authors propose to synthesise the balanced training samples with a new dependency coefficient in training. The dependency coefficient can tell the network whether the prior from the parametric body model is reliable. The authors then design a novel positional embedding‐based attenuation strategy to incorporate the dependency coefficient into the implicit function (IF) network. Comprehensive experiments are conducted on the CAPE dataset to study the effectiveness of the authors’ approach. The proposed method significantly surpasses state‐of‐the‐art approaches and generalises well on unseen poses and clothing. As an illustrative example, the proposed method improves the Chamfer Distance Error and Normal Error by 38.2% and 57.6%.

Abstract Although many new methods have emerged in text‐driven images, the large computational power required for model training causes these methods to have a slow training process. Additionally, these methods consume a considerable amount of video random access memory (VRAM) resources during training. When generating high‐resolution images, the VRAM resources are often insufficient, which results in the inability to generate high‐resolution images. Nevertheless, recent Vision Transformers (ViTs) advancements have demonstrated their image classification and recognition capabilities. Unlike the traditional Convolutional Neural Networks based methods, ViTs have a Transformer‐based architecture, leverage attention mechanisms to capture comprehensive global information, moreover enabling enhanced global understanding of images through inherent long‐range dependencies, thus extracting more robust features and achieving comparable results with reduced computational load. The adaptability of ViTs to text‐driven image manipulation was investigated. Specifically, existing image generation methods were refined and the FastFaceCLIP method was proposed by combining the image‐text semantic alignment function of the pre‐trained CLIP model with the high‐resolution image generation function of the proposed FastFace. Additionally, the Multi‐Axis Nested Transformer module was incorporated for advanced feature extraction from the latent space, generating higher‐resolution images that are further enhanced using the Real‐ESRGAN algorithm. Eventually, extensive face manipulation‐related tests on the CelebA‐HQ dataset challenge the proposed method and other related schemes, demonstrating that FastFaceCLIP effectively generates semantically accurate, visually realistic, and clear images using fewer parameters and less time.

Abstract In in‐vehicle driving scenarios, composite action recognition is crucial for improving safety and understanding the driver's intention. Due to spatial constraints and occlusion factors, the driver's range of motion is limited, thus resulting in similar action patterns that are difficult to differentiate. Additionally, collecting skeleton data that characterise the full human posture is difficult, posing additional challenges for action recognition. To address the problems, a novel Graph‐Reinforcement Transformer (GR‐Former) model is proposed. Using limited skeleton data as inputs, by introducing graph structure information to directionally reinforce the effect of the self‐attention mechanism, dynamically learning and aggregating features between joints at multiple levels, the authors’ model constructs a richer feature vector space, enhancing its expressiveness and recognition accuracy. Based on the Drive & Act dataset for composite action recognition, the authors’ work only applies human upper‐body skeleton data to achieve state‐of‐the‐art performance compared to existing methods. Using complete human skeleton data also has excellent recognition accuracy on the NTU RGB + D‐ and NTU RGB + D 120 dataset, demonstrating the great generalisability of the GR‐Former. Generally, the authors’ work provides a new and effective solution for driver action recognition in in‐vehicle scenarios.

Abstract Human action understanding (HAU) is a broad topic that involves specific tasks, such as action localisation, recognition, and assessment. However, most popular HAU datasets are bound to one task based on particular actions. Combining different but relevant HAU tasks to establish a unified action understanding system is challenging due to the disparate actions across datasets. A large‐scale and comprehensive benchmark, namely SkatingVerse is constructed for action recognition, segmentation, proposal, and assessment. SkatingVerse focus on fine‐grained sport action, hence figure skating is chosen as the task object, which eliminates the biases of the object, scene, and space that exist in most previous datasets. In addition, skating actions have inherent complexity and similarity, which is an enormous challenge for current algorithms. A total of 1687 official figure skating competition videos was collected with a total of 184.4 h, exceeding four times over other datasets with a similar topic. SkatingVerse enables to formulate a unified task to output fine‐grained human action classification and assessment results from a raw figure skating competition video. In addition, SkatingVerse can facilitate the study of HAU foundation model due to its large scale and abundant categories. Moreover, image modality is incorporated for human pose estimation task into SkatingVerse. Extensive experimental results show that (1) SkatingVerse significantly helps the training and evaluation of HAU methods, (2) the performance of existing HAU methods has much room to improve, and SkatingVerse helps to reduce such gaps, and (3) unifying relevant tasks in HAU through a uniform dataset can facilitate more practical applications. SkatingVerse will be publicly available to facilitate further studies on relevant problems.

Abstract Detecting small targets in infrared images is crucial for ground surveillance and air traffic control. However, distinguishing small infrared targets from similar backgrounds is challenging due to their lack of structural and textural characteristics. To address these challenges, this study proposes a novel global information‐aware network with global interaction graph attention (GIGA) for infrared small target detection. The GIGA consists of a global interaction layer (GILayer), graph attention weights (GAW), and a global relational learning (GRL) module. Specifically, the GILayer dynamically learns global inter‐pixel relationships of small target images by enhancing the dependencies between feature dimensions. The GAW component calculates pixel‐by‐pixel similarity across the entire feature map using graph attention mechanisms, while the GRL module retains critical similarity features in the feature extraction network, thereby facilitating small target detection. Additionally, the multi‐scale context fusion module utilises self‐attention and dilation convolution to complement richer feature details at different scales. Experimental results on both natural and synthetic datasets demonstrate the proposed method's superiority over other state‐of‐the‐art conventional and deep learning approaches in infrared small target detection.

Abstract Semantic segmentation is a fundamental computer vision task, and deep learning methods have been successfully applied to this field. However, target morphology continues to exhibit the incomplete prediction problem, which is attributable to the low feature utilisation and the insufficiency of spatial location information. This paper proposes a novel cross fusion network with unit attention mechanism (CF‐Net) for semantic segmentation. The two hallmarks of the framework are the usage of a multi‐scale fusion module and the unit attention mechanism. Multi‐scale fusion module can integrate multi‐branch outputs with different receptive fields, which obtain fine‐grained target details and visual contextual information. The cross fusion network is optimised with a unit attention mechanism to fuse intermediate features, which enables the acquisition of more accurate and effective spatial location information while maintaining consistency in feature space. The experimental results demonstrate that the proposed CF‐Net outperforms favourably comparable with other existing methods on the CamVid, Cityscapes, and PASCAL VOC 2012 databases, which also verifies the Effectiveness and reliability of our method.

Abstract Reconstructing geometric shapes from sparse multiview has always been a challenging task. With the development of neural implicit surfaces, geometry‐based volume rendering surface reconstruction methods have been proven to be able to reconstruct high‐quality surfaces. However, existing geometry‐based reconstruction methods completely associate volume density with signed distance function or unsigned distance function, resulting in the same volume density peak that can only be reconstructed near the object surface. When there are transparent surfaces in the scene, existing methods prioritize the reconstruction of opaque surfaces, neglecting the reconstruction of transparent surfaces, which is disadvantageous when reconstructing real scenes. To solve this problem, we introduce local probability standard variance, which calculates volume density together with signed distance function. In this way, it can reconstruct the volume density that matches the transparency characteristics of the object surface. The method can reconstruct the surface of transparent objects, and experiments on two transparent surface datasets show that the method performs better.

Abstract Edge devices are increasingly utilized for defect detection in power line inspection, necessitating algorithms that optimize model size and accuracy. This study introduces a lightweight detection method using the Optimized Feature Network with MobileOne‐FPN‐NASHead (OFN network) OFN network and distillation technique. The OFN incorporates a lightweight backbone, neural network search, re‐parametrization, and a feature pyramid to create a compact yet effective detection network. To enhance feature learning, a heterogeneous distillation approach is applied, leveraging a modified YOLOv8 as a teacher network. This modification includes an explicit visual centre for improved global and local information extraction, crucial for dense target detection in power line inspections. Additionally, the Minimize the points distance IoU (MPDloU) loss function is used to improve localization accuracy over the the Complete‐Intersection Over Union (CIoU) loss. Experimental results show a 1.1% mean Average Precision (mAP) increase for the enhanced YOLOv8 and a 70.2% mAP for the OFN network with 18.95 GFLOPs and 343 FPS, achieving a commendable balance between model efficiency and detection performance. The research underscores the viability of the OFN for edge computing in power line defect detection, highlighting the potential of innovative algorithmic structures in this application.

Abstract Underwater object detection is an important application of underwater vehicles. However, traditional underwater object detection algorithms have several shortcomings in underwater settings. These include imprecise feature extraction, slow detection speeds, and a lack of robustness. To address these shortcomings, a novel approach based on YOLOv5‐MH (Multi‐Head) is proposed in this paper. Firstly, an image enhancement technique is utilised. This technique uses adaptive linear mapping to adjust contrast and improve the quality of underwater images. Secondly, the C2f module for feature extraction is employed to enable more effective capture of object characteristics. Subsequently, the multi‐head self‐attention and coordinate attention are integrated into the network's backbone. This integration increases the attention given to input data, thereby enhancing the network's performance in handling complex tasks. Furthermore, a bidirectional feature pyramid is implemented to adeptly handle objects of varying scales and sizes, and elevate model performance. Finally, through comprehensive testing on the 2018 URPC dataset and deep plastic dataset, this method demonstrates superior performance. This performance is compared to the original YOLOv5 and other similar networks. It holds immense promise for practical applications across a wide spectrum of underwater tasks.

Abstract Emotion recognition in user‐generated videos plays an essential role in affective computing. In general, visual information directly affects human emotions, so the visual modality is significant for emotion recognition. Most classic approaches mainly focus on local temporal information of videos, which potentially restricts their capacity to encode the correlation of long‐range context. To address this issue, a novel network is proposed to recognize emotions in videos. To be specific, a spatio‐temporal correlation‐aware block is designed to depict the long‐range correlations between input tokens, where the convolutional layers are used to learn the local correlations and the inter‐image cross‐attention is designed to learn the long‐range and spatio‐temporal correlations between input tokens. To generate diverse and challenging samples, a dual‐augmentation fusion layer is devised, which fuses each frame with its corresponding frame in the temporal domain. To produce rich video clips, a long‐range sampling layer is designed, which generates clips in a wide range of spatial and temporal domains. Extensive experiments are conducted on two challenging video emotion datasets, namely VideoEmotion‐8 and Ekman‐6. The experimental results demonstrate that the proposed method obtains better performance than baseline methods. Moreover, the proposed method achieves state‐of‐the‐art results on the two datasets. The source code of the proposed network is available at: https://github.com/JinChow/LRCANet.

Abstract Semantic segmentation of underwater fish images is vital for monitoring fish stocks, assessing marine resources, and sustaining fisheries. To tackle challenges such as low segmentation accuracy, inadequate real‐time performance, and imprecise location segmentation in current methods, a novel lightweight U‐Net model is proposed. The proposed model acquires more segmentation details by applying a multiple‐input approach at the first four encoder levels. To achieve both lightweight and high accuracy, a multi‐scale residual structure (MRS) module is proposed to reduce parameters and compensate for the accuracy loss caused by the reduction of channels. To improve segmentation accuracy, a multi‐scale skip connection (MSC) structure is further proposed, and the convolution block attention mechanism (CBAM) is introduced at the end of each decoder level for weight adjustment. Experimental results demonstrate a notable reduction in model volume, parameters, and floating‐point operations by 94.20%, 94.39%, and 51.52% respectively, compared to the original model. The proposed model achieves a high mean intersection over union (mIOU) of 94.44%, mean pixel accuracy (mPA) of 97.03%, and a frame rate of 43.62 frames per second (FPS). With its high precision and minimal parameters, the model strikes a balance between accuracy and speed, making it particularly suitable for underwater image segmentation.

Abstract Presently, State Space Models (SSMs), including frameworks like Mamba, have been incorporated into the realm of computer vision. These models not only sustain remote interactions and encapsulate global semantic information effectively, but also preserve linear computational complexity, offering a balance between performance and computational efficiency. Given that Mamba inherently adheres to the principle of selectivity when constructing sequence models, the goal is to further unleash the potential of Mamba through this innovative combination of convolution and self‐attention, improve accuracy and minimize the number of parameters while achieving linear complexity. Mamba is employed as an encoder to distill semantic information from the image, and it is supplemented with convolutional blocks, thereby conserving the details of the image. Concurrently, embedding prompts at a deeper level enhances its adaptability to cater to diverse requirements. Lastly, a bidirectional attention mechanism is incorporated for inference, striving to retain both global connections and local details to the maximum extent. This culminates in a novel, lightweight medical segmentation model. Exhaustive experiments were executed on six public datasets. The empirical results show that the proposed model exhibits competitive performance in medical image segmentation tasks.

Abstract Image super‐resolution (SR) has been widely applied in remote sensing to generate high‐resolution (HR) images without increasing hardware costs. However, SR is a severe ill‐posed problem. As deep learning advances, existing methods have solved this problem to a certain extent. However, the complex spatial distribution of remote sensing images still poses a challenge in effectively extracting abundant high‐frequency details from the images. Here, a single‐image super‐resolution (SISR) network based on the generative adversarial network (GAN) for remote sensing is presented, called JOA‐GAN. Firstly, a joint‐attention module (JOA) is proposed to focus the network on high‐frequency regions in remote sensing images to enhance the quality of image reconstruction. In the generator network, a multi‐scale densely connected feature extraction block (ERRDB) is proposed, which acquires features at different scales using MSconv blocks containing multi‐scale convolutions and automatically adjusts the features by JOA. In the discriminator network, the relative discriminator is used to compute the relative probability instead of the absolute probability, which helps the network learn clearer and more realistic texture details. JOA‐GAN is compared with other advanced methods, and the results demonstrate that JOA‐GAN has improved objective evaluation metrics and achieved superior visual effects.

Abstract The authors propose Point'n Move, a method that achieves interactive scene object manipulation with exposed region inpainting. Interactivity here further comes from intuitive object selection and real‐time editing. To achieve this, Gaussian Splatting Radiance Field is adopted as the scene representation and its explicit nature and speed advantage are fully leveraged. Its explicit representation formulation allows to devise a 2D prompt points to 3D masks dual‐stage self‐prompting segmentation algorithm, perform mask refinement and merging, minimize changes, and provide good initialization for scene inpainting and perform editing in real‐time without per‐editing training; all lead to superior quality and performance. The method was tested by editing both forward‐facing and 360 scenes. The method is also compared against existing methods, showing superior quality despite being more capable and having a speed advantage.

Abstract Intracranial hematoma, a severe brain injury caused by trauma or cerebrovascular disease, can result in blood accumulation and compression of brain tissue. Untreated cases can cause headaches, impaired consciousness, and even brain tissue damage or death. Therefore, early and accurate diagnosis is crucial. Traditional segmentation methods require physicians with extensive clinical experience and expertise to manually mark out the hematoma region, but for hematoma cases with irregular shapes and uneven grey levels, this process is cumbersome, and the segmentation results are not good. Existing deep learning‐based methods are more likely to perform binary segmentation, considering all hematomas as a class and segmenting them, but this segmentation cannot capture more detailed information and lacks the analysis of different types of hematomas. To address these problems, an ICH segmentation network combining CNN and Transformer Encoder is proposed for accurate segmentation of different types of hematomas. The network incorporated edge information and long‐range context into the segmentation process. Experimental results using the CQ500 dataset demonstrate comparable performance to existing methods, with mIoU (0.8705), TPR (0.9273), mAP (0.9300), and DSC (0.9286) as the best metrics achieved by this paper's method.

Abstract High‐precision relative position and attitude measurement technology has a wide range of applications in aerospace and industrial production. Currently, the commonly used method for measurement is based on visual cooperative signs. However, its accuracy significantly decreases as the distance increases. Therefore, a relative positioning system is designed based on light field spatial coding and visual recognition. The projector emits spatially encoded structured light within the coverage of the light field, while the receiving end captures, identifies, measures the code, and calculates its pose in the light field coordinate system. Compared with the traditional measurement method, the measurement accuracy of this system does not decrease greatly with the increase in distance, the measurement distance can be adjusted in real‐time and does not depend on an external light source. By changing the projection pattern with different resolutions without changing hardware systems, it can adjust effective measurement distance accordingly. Theoretical and experimental results show that the proposed method can maintain measurement accuracy with the increase in distance.

Abstract The high‐precision visual perception results are easily affected by the lens flare issue when the image sensor is facing to strong light. The existing flare removal methods have poor robustness when confronted with flare interference caused by complex nighttime lighting, which has to preserve natural light source information. A simulated dataset for the removal of night flares is created to solve the problem of collecting complete paired training data, and night flare removal network (NFR‐Net) is proposed to remove the interference caused by various light disturbances at night. The light source extraction module is introduced to retain light source information realistically and effectively in night vision scenes. Extensive experimental results demonstrate that the proposed method is superior to the existing related methods in the various complex night vision scenes. The proposed NFR‐Net can enhance visual perception of nighttime images significantly and improve the performance of night vision tasks.

Abstract The complex nature of human brain tissues is important in ensuring accurate diagnosis to save human lives. Research on early detection of brain diseases has gained significant prominence within medical intelligence using highly complex model architectures with only a single label that cannot be verified. This paper introduces an innovative approach to Siamese Network Genetic Algorithm (SN‐GA) leveraging Siamese contrastive learning for classifying brain images across diverse diseases. Our core architecture is a Bi‐Convolutional Neural Network (Bi‐CNN) optimized by a genetic algorithm to enhance brain image classification. Specifically, five widely recognized transfer learning‐based architectures, namely AlexNet, Efficient‐B0, VGG‐16, ResNet‐50, and Inception‐v3, have been incorporated to evaluate the effectiveness of the proposed SN‐GA system. The performance of these models has been rigorously analyzed and compared using two distinct datasets: Brain tumors and Alzheimer's datasets. The experimental results robustly affirm the efficacy of the proposed Siamese model, yielding exceptional levels of accuracy, precision, and recall, all peaking at 97%. These findings underscore the potential and resilience of the optimized Siamese network in the context of brain disease classification, emphasizing its significance in advancing the field of medical imaging and diagnosis, with implications for early intervention and patient care.

Abstract Aiming at the problems of low detection accuracy, high computational complexity and long‐time consumption of visual perception model in a complex mining environment, this research designs a visual information perception system of coal mine comprehensive excavation working face for an edge computing terminal. Firstly, the C3‐Fast feature extraction module, spatial pyramid pooling with cross‐stage partial connection (SPPCSPC) pooling module, bi‐directional feature pyramid network and lightweight decoupled detection head are used to optimize the YOLOv5s model, so as to construct the FSBD‐YOLOv5s multi‐object detection model. Secondly, the pruning and distillation algorithm is used to lighten the FSBD‐YOLOv5s model, and the model complexity is greatly reduced while maintaining the model detection accuracy. Further, the lightweight FSBD‐YOLOv5s model is migrated and deployed to the edge computing terminal platform and the TensorRT engine is used to accelerate model inference. Finally, experiments are carried out based on the data set of the coal mine comprehensive excavation working face. The experimental results show that on the edge computing terminal platform, the parameters and computational volume of the lightweight FSBD‐YOLOv5s model are reduced by 50.8% and 34.0%, while its detection accuracy and speed reach 94.0% and 43.7 fps, which can fully satisfy the requirements of the accuracy and real‐time for the coal mine engineering applications.

Abstract In recent years, multi‐human parsing has become a focal point in research, yet prevailing methods often rely on intermediate stages and lacking pixel‐level analysis. Moreover, their high computational demands limit real‐world efficiency. To address these challenges and enable real‐time performance, low‐latency end‐to‐end network is proposed. This approach leverages vision transformer and convolutional neural network in a dual‐encoded network, featuring a lightweight Transformer‐based vision encoder) and a convolution encoder based on Darknet. This combination adeptly captures long‐range dependencies and spatial relationships. Incorporating a fuse block enables the seamless merging of features from the encoders. Residual connections in the decoder design amplify information flow. Experimental validation on crowd instance‐level human parsing and look into person datasets showcases the WNet's effectiveness, achieving high‐speed multi‐human parsing at 26.7 frames per second. Ablation studies further underscore WNet's capabilities, emphasizing its efficiency and accuracy in complex multi‐human parsing tasks.

Abstract Driving while inattentive or fatigued significantly contributes to traffic accidents and puts road users at a significantly higher risk of collision. The rise in road accidents due to driver inattention resulting from distractive objects, for example, mobile phones, drinking, or tiredness, requires intelligent traffic monitoring systems to promote road safety. However, outdated detection technologies cannot handle the poor accuracy and the lack of real‐time processing possibility especially when combined with the variations of driving environment. This paper introduces “ME‐YOLOv8” which operates driver`s distraction and fatigue through a modified version of YOLOv8, which includes modules multi‐head self‐attention (MHSA) and efficient channel attention (ECA) modules applied, where the goal of MHSA is to improve the sensitivity of global features and the ECA attentions focus on critical features. Additionally, a dataset was created containing 3660 images covering multiple distracted and drowsy driver scenarios. The results reflect the enhanced detection capabilities of ME‐YOLOv8 and demonstrate its effectiveness in real‐time scenarios. This study demonstrates a significant advancement in the application of AI to public safety and highlights the critical role that state‐of‐the‐art deep learning algorithms play in lowering the risks associated with distracted and tired driving.