Search

The network structure of digital watermarking algorithm based on deep learning is usually encoder‐noise layer‐decoder. Most of the existing encoders suffer from the problem of insufficient feature extraction, and the introduction of simulated differentiable joint photographic experts group (JPEG) compression in the noise layer cannot ensure the robustness under real JPEG. In this paper, a watermarking algorithm based on multi‐scale auto‐encoder is proposed, which can effectively extract the image feature information by combining with the channel attention mechanism. At the same time, some parameters of decoder and encoder are shared to reduce redundant feature embedding and improve extraction accuracy. This paper also proposes a robust training scheme against JPEG compression, which can guide the model to store the watermark in the low‐frequency region needed for decoding. Experimental results show that the peak signal‐to‐noise ratio (PSNR) of the proposed algorithm is above 48 and the decoding rate is above 99% under JPEG compression with quality factor Q = 50. Moreover, this scheme can effectively promote the combination of noise layer in training. In addition, the proposed algorithm is also robust to other common network noises. [ABSTRACT FROM AUTHOR]

This paper introduces a new descriptor called orientation‐magnitude response maps as a single 2D image to effectively explore motion patterns. Moreover, boosted multi‐stream CNN‐based model with various attention modules is designed for human action recognition. The model incorporates a convolutional self‐attention autoencoder to represent compressed and high‐level motion features. Sequential convolutional self‐attention modules are used to exploit the implicit relationships within motion patterns. Furthermore, 2D discrete wavelet transform is employed to decompose RGB frames into discriminative coefficients, providing supplementary spatial information related to the actors actions. A spatial attention block, implemented through the weighted inception module in a CNN‐based structure, is designed to weigh the multi‐scale neighbours of various image patches. Moreover, local and global body pose features are combined by extracting informative joints based on geometry features and joint trajectories in 3D space. To provide the importance of specific channels in pose descriptors, a multi‐scale channel attention module is proposed. For each data modality, a boosted CNN‐based model is designed, and the action predictions from different streams are seamlessly integrated. The effectiveness of the proposed model is evaluated across multiple datasets, including HMDB51, UTD‐MHAD, and MSR‐daily activity, showcasing its potential in the field of action recognition. [ABSTRACT FROM AUTHOR]

Various types of foreign fibres may be mixed in the planting, transportation, and production processes of cotton, which not only cause equipment to be out of control, but also leads to a decrease in the quality of cotton textile products and economic losses. The machine vision based detection method for cotton foreign fibres is widely used. Based on existing related research, we construct a classification dataset for cotton foreign fibres in practical scenarios, named the CF2113‐10 dataset. The authors design a basic foreign fibre classification network called CottonNet that balances performance and efficiency. The classification accuracy on the validation set reached 94.2%. In order to enhance the high‐level feature extraction ability, this paper improves the feature fusion method of residual networks and proposes CottonNet‐Res, which improves the classification accuracy to 95.1%. Finally, a classification model based on feature difference fitting, CottonNet‐Fusion, is proposed to address the classification problem of foreign fibre images sampled in complex environments. The classification accuracy of foreign fibre images sampled in ordinary scenes has improved to 97.4%, while the images sampled in complex environments maintain an accuracy of 90.3%. [ABSTRACT FROM AUTHOR]

Accurate classification of substation equipment images remains challenging due to various factors such as unexpected illumination, viewing angles, scale variations, shadows, surface contaminants, and different elements sharing similar appearances. This paper presents a novel two‐stage substation equipment classification method based on dual‐scale attention. Leveraging the region proposal technique from Faster‐regions with CNN features (RCNN), the input images are initially decomposed into multiple scales to capture latent features. A dual‐scale attention module is introduced to enhance the precision of feature extraction. Furthermore, a two‐stage network is proposed to address the challenge of classifying closely similar substation equipment. A multi‐layer perceptron performs a coarse classification to categorize the equipment into broad categories. Then, a lightweight classifier is employed for fine‐grained subclassification, further distinguishing equipment within the same broad category. To mitigate the issue of limited training data, a specialized dataset is collected and annotated for the substation equipment classification. Experimental results demonstrate that the proposed method achieves remarkable accuracy, recall, and F1‐score surpassing 0.91, outperforming mainstream approaches in terms of recall and F1 scores. Ablation experiments further validate the significant contributions of both the dual‐scale attention and the two‐stage classification module in improving the overall performance of the classification network. [ABSTRACT FROM AUTHOR]

In recent years, research on infrared and visible image fusion has mainly focused on deep learning‐based approaches, particularly deep neural networks with auto‐encoder architectures. However, these approaches suffer from problems such as insufficient feature extraction capability and inefficient fusion strategies. Therefore, this paper introduces a novel image fusion network to address the limitations of infrared and visible image fusion networks with auto‐encoder architectures. In the designed network, the encoder employs a multi‐branch cascade structure, and these convolution branches with different kernel sizes provide the encoder with an adaptive receptive field to extract multi‐scale features. In addition, the fusion layer incorporates a non‐local attention module that is inspired by the self‐attention mechanism. With its global receptive field, this module is used to build a non‐local attention fusion network, which works together with the l1${l}_1$‐norm spatial fusion strategy to extract, split, filter, and fuse global and local features. Comparative experiments on the TNO and MSRS datasets demonstrate that the proposed method outperforms other state‐of‐the‐art fusion approaches. [ABSTRACT FROM AUTHOR]

Slug flow processes draw much attention in chemical and pharmaceutical manufacturing thanks to their ability to eliminate downtime costs and batch‐to‐batch variation. Using in‐line video can monitor the current process status and improve the stability of the slug flow. However, there are no effective image processing methods aiming at such videos from chemical experiments due to limited training sample size and the variety of experimental settings. The paper proposes a training‐free method to automatically detect and measure the slugs from in‐line videos. Multiple image features are fused to identify the slug shapes under various lighting conditions, a six‐point model is fitted to achieve better volume estimation, and a consistency score is estimated to quantify the detection uncertainty. The proposed algorithm achieves similar results to manual labeling for various types of slug flow processes, improving the accuracy of slug column estimation by a large margin compared to the existing automatic methods. Finally, using it to monitor the slug flow process's stability and the change of the slug volume in an injection point are demonstrated. The method not only shows that handcrafted image features have the capability for detection and segmentation from chemical experiment images but also paves the road for a training‐based algorithm for the task. [ABSTRACT FROM AUTHOR]

In response to the low matching accuracy of stereo matching algorithms in image regions with specular reflection, this paper proposes a multidimensional fusion stereo matching algorithm named MFANet. The algorithm embeds a multispectral attention module into the residual feature extraction network, utilizing two‐dimensional discrete cosine transforms to extract frequency features. In the pyramid pooling module, a coordinated attention mechanism is introduced to capture relevant positional information. In the cost aggregation part, the MFANet algorithm incorporates a three‐dimensional attention mechanism, focusing on the more important semantic information in high‐level features. By combining detailed information from low‐level features, semantic information from high‐level features, and contextual information, the algorithm generates features that are more conducive to disparity prediction. The MFANet algorithm is evaluated on three standard datasets (SceneFlow, KITTI2015, and KITTI2012). Experimental results demonstrate its robustness against specular reflection interference, accurate prediction of disparities in specular reflection pathological regions, and promising application prospects. [ABSTRACT FROM AUTHOR]

In order to balance accuracy and real‐time performance in semantic segmentation, this paper proposes a real‐time semantic segmentation algorithm model based on attention mechanism and multi‐branch feature fusion using Fast convolutional neural network model (Fast‐SCNN). In this method, the spatial detail feature enhancement branch is introduced to enhance spatial detail features firstly. Then, through rational design of fusion module, the feature information of each branch is optimized to achieve better fusion of deep and shallow features. At the end of the feature fusion module, an adaptive feature enhancement focus module is introduced to capture the interdependence between remote pixels. The experimental results show that the proposed algorithm achieves 71.55% segmentation accuracy on Cityscapes dataset, the reasoning speed FPS is 97.6 frames/s, and the number of parameters is 1.39 M, which verifies the effectiveness of the network model constructed by the algorithm. Code is available at https://github.com/ccchhheeennn/model. [ABSTRACT FROM AUTHOR]

Most existing multi‐view stereo (MVS) methods fail to consider global context information in the stage of feature extraction and cost aggregation. As transformers have shown remarkable performance on various vision tasks due to their ability to perceive global contextual information, this paper proposes a transformer‐based feature enhancement network (TF‐MVSNet) to facilitate feature representation learning by combining local features (both 2D and 3D) with long‐range contextual information. To reduce memory consumption of feature matching, the cross‐attention mechanism is leveraged to efficiently construct 3D cost volumes under the epipolar constraint. Additionally, a colour‐guided network is designed to refine depth maps at a coarse stage, hence reducing incorrect depth predictions at a fine stage. Extensive experiments were performed on the DTU dataset and Tanks and Temples (T&T) benchmark and results are reported. [ABSTRACT FROM AUTHOR]

Pixel‐level labeling of regions of interest in an image is a key step in building a labeled training dataset for supervised deep learning networks of images. However, traditional manual labeling of cancerous regions in digital pathological images by doctors is time‐consuming and inefficient. To address this issue, this paper proposes an automatic labeling method for whole slide images, which combines clustering and edge detection techniques. The proposed method utilizes the multi‐level feature fusion model and the Long‐Short Term Memory network to discriminate the cancerous nature of the whole slide images, thereby improving the classification accuracy of the whole slide images. Subsequently, the automatic labeling of cancerous regions is achieved by integrating a density‐based clustering algorithm and an edge point extraction algorithm, both based on the discriminated results of the cancerous properties of whole slide images. The experimental results demonstrate the effectiveness of the proposed method, which offers an efficient and accurate solution to the challenging task of cancerous region labeling in digital pathological images. [ABSTRACT FROM AUTHOR]

Scene text in natural images carries additional semantic information to aid in image classification. Existing methods lack full consideration of the deep understanding of the text and the visual text relationship, which results in the difficult to judge the semantic accuracy and the relevance of the visual text. This paper proposes image classification based on Cross modal Knowledge Learning of Scene Text (CKLST) method. CKLST consists of three stages: cross‐modal scene text recognition, text semantic enhancement, and visual‐text feature alignment. In the first stage, multi‐attention is used to extract features layer by layer, and a self‐mask‐based iterative correction strategy is utilized to improve the scene text recognition accuracy. In the second stage, knowledge features are extracted using external knowledge and are fused with text features to enhance text semantic information. In the third stage, CKLST realizes visual‐text feature alignment across attention mechanisms with a similarity matrix, thus the correlation between images and text can be captured to improve the accuracy of the image classification tasks. On Con‐Text dataset, Crowd Activity dataset, Drink Bottle dataset, and Synth Text dataset, CKLST can perform significantly better than other baselines on fine‐grained image classification, with improvements of 3.54%, 5.37%, 3.28%, and 2.81% over the best baseline in mAP, respectively. [ABSTRACT FROM AUTHOR]

Video steganography has the potential to be used to convey illegal information, and video steganalysis is a vital tool to detect the presence of this illicit act. Currently, all the motion vector (MV)‐based video steganalysis algorithms extract feature sets directly from the MVs, but ignoring the embedding operation may perturb the statistical distribution of other video encoding elements, such as the skipped macroblocks (no direct MVs). This paper proposes a novel 11‐dimensional feature set to detect MV‐based video steganography based on the above observation. The proposed feature is extracted based on the skipped macroblocks by recompression calibration. Specifically, the feature consists of two components. The first is the probability distribution of motion vector prediction (MVP) difference, and the second is the probability distribution of partition state transfer. Extensive experiments on different conditions demonstrate that the proposed feature set achieves good detection accuracy, especially in lower embedding capacities. In addition, the loss of detection performance caused by recompression calibration using mismatched quantization parameters (QP) is within the acceptable range, so the proposed method can be used in practical scenarios. [ABSTRACT FROM AUTHOR]

Inverse halftoning is a technology that converts a binary image into a continuous tone image. Due to the wide application of inverse halftoning, many scholars have proposed several deep convolutional neural networks (DCNN) to optimize their performance. According to the observation, there is still room for improvement in content generation and detail recovery of the inverse halftone images generated by using the existing methods. Therefore, an inverse halftoning method based on supervised DCNN is proposed in this paper. The method consists of two parts: the multi‐level feature extraction model uses the down‐sampling to extract the features from the halftone image and remove the halftone noise dots on flat areas, which is implemented by four convolutional layers; the image reconstruction model uses up‐sampling to reconstruct image information, which is realized by four convolutional layers and two dense residual blocks. At the same time, in order to further recover the details, the down‐sampling feature maps and up‐sampling feature maps of the same size are concatenated by addition layers. Experimental results show that compared with other methods, the inverse halftone images obtained by the proposed network have better results in both subjective and objective evaluations. [ABSTRACT FROM AUTHOR]

Video‐based emotion recognition has been a long‐standing research topic for computer scientists and psychiatrists. In contrast to traditional discrete emotional models, emotion recognition based on continuous emotional models can better describe the progression of emotions. Quantitative analysis of emotions will have crucial impacts on promoting the development of intelligent products. The current solutions to continuous emotion recognition still have many issues. The original continuous emotion dataset contains incomplete data annotations, and the existing methods often ignore temporal information between frames. The following measures are taken in response to the above problems. Initially, aiming at the problem of incomplete video labels, the correlation between discrete and continuous video emotion labels is used to complete the dataset labels. This correlation is used to propose a mathematical model to fill the missing labels of the original dataset without adding data. Moreover, this paper proposes a continuous emotion recognition network based on an optimized temporal convolutional network, which adds a feature extraction submodule and a residual module to retain shallow features while improving the feature extraction ability. Finally, validation experiments on the Aff‐wild2 dataset achieved accuracies of 0.5159 and 0.65611 on the valence and arousal dimensions, respectively, by adopting the above measures. [ABSTRACT FROM AUTHOR]

The bridge bottom crack detection provides important state information for bridge disease control and safety assessment. This paper proposes a detection method based on deep learning Faster R‐CNN and BIM (Building Information Modeling). The UAV (Unmanned Aerial Vehicle) was used for close aerial photography to obtain high‐resolution crack images of the concrete surface at the bottom of a bridge. Through deep learning algorithms, a Faster R‐CNN model is trained and established for crack identifications. The crack identification accuracy rate and recall rate reach 92.03% and 96.54%, respectively. Crack images are mapped to a BIM model developed for the chosen bridge, and the box girder family with cracks and the three crack families of transverse cracks, longitudinal cracks and turtle cracks are established. The cracks are located and the visualization of the beam bridge with cracks was completed. In order to better assess the health condition of the bridge. The results show that the combination of UAV bridge crack detection and modelling solves the remote, visual and automated detection of cracks on the surface of bridge structures, which are difficult to reach manually, and has important scientific research and engineering application value. [ABSTRACT FROM AUTHOR]

The accurate segmentation of breast tumours is important for the diagnosis and treatment of breast cancer. When using the classic U‐Net, Attention‐UNet, and UNet++ to segment the tumour, there are problems of oversegmentation, incorrect segmentation, and poor edge continuity. In this paper, an effective tumour segmentation method, EfficientUNet, is proposed. EfficientUNet adopts a step‐by‐step enhancement method, combining ResNet18, a channel attention mechanism and deep supervision. ResNet18, as the encoder of the whole network, solves the problem of gradient disappearanceand improves the feature extraction ability of the model. The channel attention module makes the model more accurate in tumour edge processing. The deep supervision technology accelerates the model training and provides the convergence direction for the model. In addition, it is found that when adjusting the size of the image, the method of image filling before clipping (or zooming) is more conducive to model learning than the direct interpolation method. And a comparative experiment wasperformed on dataset B. Compared to U‐Net, Attention‐UNet and UNet++, EfficientUNet has the highest performance. Finally, the ablation experiment also indicated the effectiveness of each module in EfficientUNet. [ABSTRACT FROM AUTHOR]