Search

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Detecting student behavior in smart classrooms is a critical area of research in educational technology that significantly enhances teaching quality and student engagement. This paper introduces an innovative approach using advanced computer vision and artificial intelligence technologies to monitor and analyze student behavior in real time. Such monitoring assists educators in adjusting their teaching strategies effectively, thereby optimizing classroom instruction. However, the application of this technology faces substantial challenges, including the variability in student sizes, the diversity of behaviors, and occlusions among students in complex classroom settings. Additionally, the uneven distribution of student behaviors presents a significant hurdle. To overcome these challenges, we propose Student Behavior Detection Network (SBD-Net), a lightweight target detection model enhanced by the Focal Modulation module for robust multi-level feature fusion, which augments feature extraction capabilities. Furthermore, the model incorporates the ESLoss function to address the imbalance in behavior sample detection effectively. The innovation continues with the Dyhead detection head, which integrates three-dimensional attention mechanisms, enhancing behavioral representation without escalating computational demands. This balance achieves both a high detection accuracy and manageable computational complexity. Empirical results from our bespoke student behavior dataset, Student Classroom Behavior (SCBehavior), demonstrate that SBD-Net achieves a mean Average Precision (mAP) of 0.824 with a low computational complexity of just 9.8 G. These figures represent a 4.3% improvement in accuracy and a 3.8% increase in recall compared to the baseline model. These advancements underscore the capability of SBD-Net to handle the skewed distribution of student behaviors and to perform high-precision detection in dynamically challenging classroom environments. [ABSTRACT FROM AUTHOR]

Aiming at the problems of traditional image super-resolution reconstruction algorithms in the image reconstruction process, such as small receptive field, insufficient multi-scale feature extraction, and easy loss of image feature information, a super-resolution reconstruction algorithm of multi-scale dilated convolution network based on dilated convolution is proposed in this paper. First, the algorithm extracts features from the same input image through the dilated convolution kernels of different receptive fields to obtain feature maps with different scales; then, through the residual attention dense block, further obtain the features of the original low resolution images, local residual connections are added to fuse multi-scale feature information between multiple channels, and residual nested networks and jump connections are used at the same time to speed up deep network convergence and avoid network degradation problems. Finally, deep network extraction features, and it is fused with input features to increase the nonlinear expression ability of the network to enhance the superresolution reconstruction effect. Experimental results show that compared with Bicubic, SRCNN, ESPCN, VDSR, DRCN, LapSRN, MemNet, and DSRNet algorithms on the Set5, Set14, BSDS100, and Urban100 test sets, the proposed algorithm has improved peak signal-to-noise ratio and structural similarity, and reconstructed images. The visual effect is better. [ABSTRACT FROM AUTHOR]

Facial recognition technology has emerged as the most important element in many interactive AI systems due to its ease and accuracy on par with that of humans. Nevertheless, its dependable deployment is constrained by vulnerability to presentation attacks. Therefore, for facial recognition technology to be used safely in unsupervised situations, automatic detection of presentation attacks is crucial. In this context, we propose a novel deep face spoof detection framework, which employs multi-level Elliptical Local Binary Pattern (ELBP) and stacked LSTMs. The ELBP, a variant of Local Binary Patterns (LBPs), is utilized in three levels for three color spaces—RGB, HSV, and RGB + HSV—to acquire discriminating features. We evaluate our framework through extensive experiments on two publicly available and challenging datasets—CASIA-FASD, CASIA-SURF, and OULU-NPU. The experimental results demonstrate that our framework achieves better performance in terms of APCER, NPCER, ACER, EER, ROCs, and confusion matrix. [ABSTRACT FROM AUTHOR]

Retinal disease recognition using Optical Coherence Tomography (OCT) images plays a pivotal role in the early diagnosis and treatment of conditions. However, the previous attempts relied on extracting single-scale features often refined by stacked layered attentions. This paper presents a novel deep learning-based Multiscale Feature Enhancement via a Dual Attention Network specifically designed for retinal disease recognition in OCT images. Our approach leverages the EfficientNetB7 backbone to extract multiscale features from OCT images, ensuring a comprehensive representation of global and local retinal structures. To further refine feature extraction, we propose a Pyramidal Attention mechanism that integrates Multi-Head Self-Attention (MHSA) with Dense Atrous Spatial Pyramid Pooling (DASPP), effectively capturing long-range dependencies and contextual information at multiple scales. Additionally, Efficient Channel Attention (ECA) and Spatial Refinement modules are introduced to enhance channel-wise and spatial feature representations, enabling precise localization of retinal abnormalities. A comprehensive ablation study confirms the progressive impact of integrated blocks and attention mechanisms that enhance overall performance. Our findings underscore the potential of advanced attention mechanisms and multiscale processing, highlighting the effectiveness of the network. Extensive experiments on two benchmark datasets demonstrate the superiority of the proposed network over existing state-of-the-art methods.

In person re-identification (re-id), the key to retrieving the correct person image is to extract discriminative features. The features at different levels are considered complementary. In this work, we design a person re-id learning network that can extract mutually multi-level features called ASCLNet. ASCLNet contains three feature branches, and each branch can extract mutually different levels of features. Furthermore, we propose two novel modules and apply them to learning local and attribute features in ASCLNet. One is the contextual local module, which can learn the local feature with context information from the local body part; the other is the attribute soft-sharing module, which enables shared feature representation among attributes. With the support of these two modules, ASCLNet can extract multi-level features that are more discriminative. Moreover, experimental results show that ASCLNet achieves excellent performances on Market-1501 and DukeMTMC-reID datasets with mAP of 88.85% and 80.18%, respectively. [ABSTRACT FROM AUTHOR]

The presence of image shadows significantly impacts the practical applications of computer vision tasks in real-world environments. For example, image recognition, and image segmentation. Previous methodologies have disregarded the relationship between shadow and non-shadow regions in images, and inadequately utilizing neighboring pixels for shadow removal. In this paper, we introduce CTU-Net, a network model that leverages the relationship between shadows and non-shadows to detect image shadows and subsequently remove them using surrounding pixels. Within CTU-Net, a convolutional transformer fusion module is proposed to merge local and global features within the image. Subsequently, a module termed the shadow and non-shadow complementary mechanism is proposed, which predicts shadow and non-shadow regions simultaneously based on image features, allowing for better shadow prediction through the complementary nature of these regions. Moreover, the image is partitioned into multiple sub-areas, and shadow removal is achieved by combining texture-similar sub-areas with illumination restoration operators, followed by the reconstruction of shadow boundaries based on surrounding pixels. Experimental evaluations conducted on five publicly available datasets demonstrate the efficacy of our proposed shadow detection architecture, with an average performance improvement of 4.62%, and shadow removal algorithm, with an average improvement of 2.42%, measured in terms of balanced error rate and root mean square error.

The lesion area of cancer presents complex physiological structures, posing significant challenges for accurate segmentation, which is required in radiotherapy or chemo-radiotherapy. The existing general-purpose segmentation models have made significant progress, but their segmentation performance remains unsatisfactory in specific cancers. To enhance the segmentation performance of nasopharyngeal carcinoma (NPC) MRI images, we develop a segmentation model by incorporating skip connections and receptive blocks within the deep convolutional neural network. We curate a collection of NPC images and evaluate the segmentation performance of the proposed model in comparison with the ground truth segmented images. The similarity between the two types of images is quantified by the Dice Similarity Coefficient (DSC) score, intersection over Union (IoU) score, recall value, and precision value. The mean DSC score was increased to 0.91, and the mean IoU score was improved to 0.84 in the segmentation of our curated NPC images. Our proposed method not only shows excellent performance in segmenting NPC, but also displays its generalization capacity on three public medical image datasets including skin cancer, lung, and chest image databases. The proposed model can leverage multi-layer feature extraction to improve the accuracy of tumor region segmentation. The improved generalization ability suggests that harnessing multi-level features in deep neural networks can improve segmentation performance.

Recently, point clouds have emerged as a promising research direction for representing 3D visual data in various immersive applications, including augmented reality, self-driving cars and so on. The research on point cloud quality assessment (PCQA) has received significant attention. Although there are various objective PCQA models, their generalization can not satisfy the requirement of practical applications. Obviously, various factors should be considered comprehensively in the PCQA model. The dependence of data and the pertinence of features limit the improvement of the generalization of the PCQA models. Extracting features from different types of point clouds and research directions is also a challenge. To overcome this limitation, we propose a multi-level features model (MFPCQA) that segregates the basic features from the point cloud data into a pool. Then, the extracted basic features are sorted and combined to generate the advanced features, which are more closely related to quality, obtaining a multi-level feature structure. Finally, an efficient quality regressor maps these advanced features to point cloud quality. It is worth noting that, according to different application scenarios, our proposed MFPCQA has carried out detailed experiments on two conditions of whether it can take advantage of the distortion type. Extensive experiments on several publicly available subjective point cloud quality datasets and different distortion type datas validate that our proposed MFPCQA can compete with state-of-the-art full-reference, reduced-reference quality assessment models. The proposed MFPCQA significantly improves the generalization of quality assessment algorithms.

Detecting student behavior in smart classrooms is a critical area of research in educational technology that significantly enhances teaching quality and student engagement. This paper introduces an innovative approach using advanced computer vision and artificial intelligence technologies to monitor and analyze student behavior in real time. Such monitoring assists educators in adjusting their teaching strategies effectively, thereby optimizing classroom instruction. However, the application of this technology faces substantial challenges, including the variability in student sizes, the diversity of behaviors, and occlusions among students in complex classroom settings. Additionally, the uneven distribution of student behaviors presents a significant hurdle. To overcome these challenges, we propose Student Behavior Detection Network (SBD-Net), a lightweight target detection model enhanced by the Focal Modulation module for robust multi-level feature fusion, which augments feature extraction capabilities. Furthermore, the model incorporates the ESLoss function to address the imbalance in behavior sample detection effectively. The innovation continues with the Dyhead detection head, which integrates three-dimensional attention mechanisms, enhancing behavioral representation without escalating computational demands. This balance achieves both a high detection accuracy and manageable computational complexity. Empirical results from our bespoke student behavior dataset, Student Classroom Behavior (SCBehavior), demonstrate that SBD-Net achieves a mean Average Precision (mAP) of 0.824 with a low computational complexity of just 9.8 G. These figures represent a 4.3% improvement in accuracy and a 3.8% increase in recall compared to the baseline model. These advancements underscore the capability of SBD-Net to handle the skewed distribution of student behaviors and to perform high-precision detection in dynamically challenging classroom environments.

Deep convolutional neural networks can effectively improve the performance of single-image super-resolution reconstruction. Deep networks tend to achieve better performance than others. However, the deep CNNs will lead to a dramatic increase in the size of parameters, limiting its application on embedding and resource-constrained devices, such as smart phone. To address the common problems of blurred image edges, inflexible convolution kernel size selection and slow convergence during training procedure due to redundant network structure in image super-resolution algorithms, this paper proposes a lightweight single-image super-resolution network that fuses multi-level features. The components are mainly two-level nested residual blocks. To better extract features and reduce the number of parameters, each residual block adopts an asymmetric structure. Firstly, it expands twice and then compresses the number of channels twice. Secondly, in the residual block, the feature information of different channels is weighted and fused by adding an autocorrelation weight unit. The quality of the reconstructed image of the proposed method is superior to the existing image super-resolution reconstruction methods in both subjective perception and objective evaluation indicators, and the reconstruction performance is better when the factor is large. [ABSTRACT FROM AUTHOR]

Skeleton‐based action recognition has been continuously and intensively studied. However, dynamic 3D skeleton data are difficult to be popularized in practical applications due to the restricted data acquisition conditions. Although the action recognition method based on 2D pose information extracted from RGB video can effectively avoid the influence of complex background, it is susceptible to factors such as video jitter and joint overlap. To reduce the interference of the aforementioned factors, we use two‐dimensional skeletal joint coordinate modal information to represent the changes in human body posture. First, we use a target detector and pose estimation algorithm to obtain the joint coordinates of each frame sample from RGB video. Then the feature extraction network is combined to perform multi‐level feature learning to establish correspondence between actions and corresponding multi‐level features. Finally, the hierarchical attention mechanism is introduced to design the model named CHAN. By calculating the association between elements, the weight of the action classification is redistributed. Extensive experiments on three datasets demonstrate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Visual saliency refers to the human's ability to quickly focus on important parts of their visual field, which is a crucial aspect of image processing, particularly in fields like medical imaging and robotics. Understanding and simulating this mechanism is crucial for solving complex visual problems. In this paper, we propose a salient object detection method based on boundary enhancement, which is applicable to both 2D and 3D sensors data. To address the problem of large-scale variation of salient objects, our method introduces a multi-level feature aggregation module that enhances the expressive ability of fixed-resolution features by utilizing adjacent features to complement each other. Additionally, we propose a multi-scale information extraction module to capture local contextual information at different scales for back-propagated level-by-level features, which allows for better measurement of the composition of the feature map after back-fusion. To tackle the low confidence issue of boundary pixels, we also introduce a boundary extraction module to extract the boundary information of salient regions. This information is then fused with salient target information to further refine the saliency prediction results. During the training process, our method uses a mixed loss function to constrain the model training from two levels: pixels and images. The experimental results demonstrate that our salient target detection method based on boundary enhancement shows good detection effects on targets of different scales, multi-targets, linear targets, and targets in complex scenes. We compare our method with the best method in four conventional datasets and achieve an average improvement of 6.2% on the mean absolute error (MAE) indicators. Overall, our approach shows promise for improving the accuracy and efficiency of salient object detection in a variety of settings, including those involving 2D/3D semantic analysis and reconstruction/inpainting of image/video/point cloud data. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Single image super-resolution (SISR) has achieved great success in recent years due to the representation ability of large and deep models. However, these models usually have a large number of network parameters, which hinders their application to real-world scenarios. To reduce the number of parameters in the SISR models, we propose a super-lightweight model termed s-LMPNet with a multi-stage architecture. Specifically, s-LMPNet includes three sub-networks, which are organized in a cascaded way. Each sub-network is constructed with multiple lightweight cross-group skip-connecting blocks (CGSCBs). To enhance the model performance, a residual feature fusion attention module is adopted to integrate intermediate features from different CGSCBs in a self-adaptive weighted way. A cross-stage feature propagation module is used to propagate the information from the low stage to the high stage, thereby making the network optimization procedure more stable. Extensive experiments are performed on commonly-used super-resolution benchmarks. Experiment results have shown that s-LMPNet achieves promising performance compared to other state-of-the-art lightweight super-resolution methods. [ABSTRACT FROM AUTHOR]

When image quality is evaluated, the human visual system (HVS) infers the details in the image through its internal generative mechanism. In this process, the HVS integrates both local and global information about the image, utilizes contextual information to restore the original image information, and compares it with the distorted image information for image quality evaluation. Inspired by this mechanism, a no-reference image quality assessment method is proposed based on a multitask image restoration network. The multitask image restoration network generates a pseudo-reference image as the main task and produces a structural similarity index measure map as an auxiliary task. By mutually promoting the two tasks, a higher-quality pseudo-reference image is generated. In addition, when predicting the image quality score, both the quality restoration features and the difference features between the distorted and reference images are used, thereby fully utilizing the information from the pseudo-reference image. In order to facilitate the model's ability to extract both global and local features, we introduce a multi-scale feature fusion module. Experimental results demonstrate that the proposed method achieves excellent performance on both synthetically and authentically distorted databases. [ABSTRACT FROM AUTHOR]

No-reference image quality assessment (NR-IQA) has a wide range of application scenarios and occupies an important position in the field of digital image processing. In this paper, we propose a multi-level feature augmentation method for no-reference image quality assessment. This method can effectively extract local features to solve the problem of distortion diversity in authentic distortion images. In this algorithm, an attention enhancement module (AEM) is proposed to strengthen the area of interest, so that obtained features conform to the human visual system (HVS). Then, we propose a residual feature augmentation (RFA) module to supplement the information loss of dimensionality reduction before fusion. At the same time, according to the extracted semantic features of image, the perceptual rule of the quality prediction network is established, and multi-scale features are mapped into the quality prediction network. Extensive experiments show that the model achieves better performance in synthetic distortion image datasets and authentic distortion image datasets. [ABSTRACT FROM AUTHOR]

At present, tobacco leaf grading relies mainly on manual classification, which is highly intensive with respect to labor, materials, and cost; in addition, the performance of manual grading is poor. The realization of automatic grading is an urgent requirement in the tobacco industry. To address this need, we developed assembly line equipment and an RGB image classification method for tobacco grading. There is little difference in appearance between different grades of tobacco leaves, but there are some differences in high-level semantic features; therefore, tobacco grading is fundamentally a fine-grained visual categorization task. It is difficult to classify tobacco using hand-crafted image features. Therefore, in our method, we use a pyramid structure, attention mechanism, and angle decision loss function to improve the bilinear convolutional neural network (a fine-grained visual categorization framework) for tobacco grading. The feature extraction and classification model was trained using 66,966 images, to effectively extract high-level semantic features from a global tobacco image and multi-scale features from a local tobacco image. In an online test on assembly line equipment, the proposed model was able to classify six main grades of tobacco with an accuracy of 80.65%, which is higher than that of the state-of-the-art model. Additionally, the time required to classify each tobacco leaf was 42.1 ms. This work is of great significance for the industrial application of tobacco grading models and grading equipment, and it provides a theoretical reference for the quality grading of other agricultural products.

In the traditional person re-identification model, the CNN network is usually used for feature extraction. When converting the feature map into a feature vector, a large number of convolution operations are used to reduce the size of the feature map. In CNN, since the receptive field of the latter layer is obtained by convolution operation on the feature map of the previous layer, the size of this local receptive field is limited, and the computational cost is large. For these problems, combined with the self-attention characteristics of Transformer, an end-to-end person re-identification model (twinsReID) is designed that integrates feature information between levels in this article. For Transformer, the output of each layer is the correlation between its previous layer and other elements. This operation is equivalent to the global receptive field because each element needs to calculate the correlation with other elements, and the calculation is simple, so its cost is small. From these perspectives, Transformer has certain advantages over CNN's convolution operation. This paper uses Twins-SVT Transformer to replace the CNN network, combines the features extracted from the two different stages and divides them into two branches. First, convolve the feature map to obtain a fine-grained feature map, perform global adaptive average pooling on the second branch to obtain the feature vector. Then divide the feature map level into two sections, perform global adaptive average pooling on each. These three feature vectors are obtained and sent to the Triplet Loss respectively. After sending the feature vectors to the fully connected layer, the output is input to the Cross-Entropy Loss and Center-Loss. The model is verified On the Market-1501 dataset in the experiments. The mAP/rank1 index reaches 85.4%/93.7%, and reaches 93.6%/94.9% after reranking. The statistics of the parameters show that the parameters of the model are less than those of the traditional CNN model.

This manuscript proposed an effective hybrid method based on multi-level convolutional neural network (ML-CNN) and iterative local search (ILS) to solve job shop scheduling problems (JSSP) with small scale training samples and less time. In the proposed method, ML-CNN is proposed to find the global path of JSSP instance from the genetic algorithm (GA); ML-CNN learned global path is fed into ILS to search for the best local path, it is the key to attach excellent adaptivity. In order to find the global path from optimal solutions, the proposed ML-CNN treated the JSSP as some sub-classification tasks first. Each subclass corresponding to one suboperation prioritizes a particular machine according to the production environment. Significantly, the proposed ML-CNN designed two level inputs to represent production statutes. The detailed-level input channel records fourteen detailed statutes, while the system-level input channel records four systematic statutes. Two level inputs are fed into one dimensional (1-D) CNN to extract rich hidden features to predict the priority of each suboperation using a support vector machine (SVM) classifier. At last, the global path (priority sequences) could be obtained, which is encoded as the input of ILS to find the best local path. The authors trained and tested the proposed method on 82 public JSSP instances. The results indicated that the proposed method could obtain optimal solutions for small scale instances and outperform others regarding makespan and computation time for large scale JSSP instances.

The accuracy with which a neural network interprets a point cloud depends on the quality of the features expressed by the network. Addressing this issue, we propose a multi-level feature extraction layer (MFEL) which collects local contextual feature and global information by modeling point clouds at different levels. The MFEL is mainly composed of three independent modules, including the aggregated GAPLayer, the spatial position perceptron, and the RBFLayer, which learn point cloud features from three different scales. The aggregated GAPLayer aggregates the geometry features of neighboring points in a local coordinate system to centroid by graph convolution. Then, the spatial position perceptron independently learns the position features of each point in the world coordinate system. Finally, the RBFLayer aggregates points into pointsets according to the correlation between features, and extracts features from the pointset scale through the quantization layer. Based on the MFEL, an end-to-end classification and segmentation network, namely the MFNet and MFNet-S, is proposed. In the proposed network, the channel-attention mechanism is employed to better aggregate multi-level features. We conduct classification and semantic segmentation experiments on four standard datasets. The results show that the proposed method outperforms the compared methods on the multiple datasets, resulting in 93.1% classification accuracy in ModelNet40. Furthermore, the mIoU of part semantic segmentation in ShapeNet is 85.4%, and the mIoU for semantic segmentation in S3DIS and Semantic3D is 62.9% and 71.9%, respectively. [ABSTRACT FROM AUTHOR]

Visual saliency refers to the human’s ability to quickly focus on important parts of their visual field, which is a crucial aspect of image processing, particularly in fields like medical imaging and robotics. Understanding and simulating this mechanism is crucial for solving complex visual problems. In this paper, we propose a salient object detection method based on boundary enhancement, which is applicable to both 2D and 3D sensors data. To address the problem of large-scale variation of salient objects, our method introduces a multi-level feature aggregation module that enhances the expressive ability of fixed-resolution features by utilizing adjacent features to complement each other. Additionally, we propose a multi-scale information extraction module to capture local contextual information at different scales for back-propagated level-by-level features, which allows for better measurement of the composition of the feature map after back-fusion. To tackle the low confidence issue of boundary pixels, we also introduce a boundary extraction module to extract the boundary information of salient regions. This information is then fused with salient target information to further refine the saliency prediction results. During the training process, our method uses a mixed loss function to constrain the model training from two levels: pixels and images. The experimental results demonstrate that our salient target detection method based on boundary enhancement shows good detection effects on targets of different scales, multi-targets, linear targets, and targets in complex scenes. We compare our method with the best method in four conventional datasets and achieve an average improvement of 6.2% on the mean absolute error (MAE) indicators. Overall, our approach shows promise for improving the accuracy and efficiency of salient object detection in a variety of settings, including those involving 2D/3D semantic analysis and reconstruction/inpainting of image/video/point cloud data.

When image quality is evaluated, the human visual system (HVS) infers the details in the image through its internal generative mechanism. In this process, the HVS integrates both local and global information about the image, utilizes contextual information to restore the original image information, and compares it with the distorted image information for image quality evaluation. Inspired by this mechanism, a no-reference image quality assessment method is proposed based on a multitask image restoration network. The multitask image restoration network generates a pseudo-reference image as the main task and produces a structural similarity index measure map as an auxiliary task. By mutually promoting the two tasks, a higher-quality pseudo-reference image is generated. In addition, when predicting the image quality score, both the quality restoration features and the difference features between the distorted and reference images are used, thereby fully utilizing the information from the pseudo-reference image. In order to facilitate the model’s ability to extract both global and local features, we introduce a multi-scale feature fusion module. Experimental results demonstrate that the proposed method achieves excellent performance on both synthetically and authentically distorted databases.

Guided by the free-energy principle, generative adversarial networks (GAN)-based no-reference image quality assessment (NR-IQA) methods have improved the image quality prediction accuracy. However, the GAN cannot well handle the restoration task for the free-energy principle-guided NR-IQA methods, especially for the severely destroyed images, which results in that the quality reconstruction relationship between the distorted image and its restored image cannot be accurately built. To address this problem, a visual compensation restoration network (VCRNet)-based NR-IQA method is proposed, which uses a non-adversarial model to efficiently handle the distorted image restoration task. The proposed VCRNet consists of a visual restoration network and a quality estimation network. To accurately build the quality reconstruction relationship between the distorted image and its restored image, a visual compensation module, an optimized asymmetric residual block, and an error map-based mixed loss function, are proposed for increasing the restoration capability of the visual restoration network. For further addressing the NR-IQA problem of severely destroyed images, the multi-level restoration features which are obtained from the visual restoration network are used for the image quality estimation. To prove the effectiveness of the proposed VCRNet, seven representative IQA databases are used, and experimental results show that the proposed VCRNet achieves the state-of-the-art image quality prediction accuracy. The implementation of the proposed VCRNet has been released at https://github.com/NUIST-Videocoding/VCRNet. [ABSTRACT FROM AUTHOR]

The variable magnification levels in histopathology images make it difficult to accurately categorize tumor regions in breast cancer histology. In this study, a novel architecture for accurate image interpretation MagNet is presented. With specific modules like Separable Dilation Convolution (SDC), Separable Dilation Skip Block (SDSB), and Point-wise Reformation Block (PRB), MagNet uses a Parallel U-Net (PU-Net) infrastructure. SDC in the PU-Net encoder ensures downsampled generalized feature representations by capturing characteristic attributes at varying magnifications. Using feature upsampling, attribute mapping merging, and PRB for precise feature capture, the decoder improves reconstruction. While PRB combines data from several decoder levels, SDSB creates vital links between the encoder and decoder layers. MagNet requires less processing of histopathology images and improves multi-magnification feature maps. MagNet performs exceptionally well, constantly outperforming rivals in terms of accuracy (0.98), F1 score (0.97), sensitivity (0.96), and specificity (0.97). The effectiveness of MagNet and its revolutionary potential in cancer diagnostics are shown by these quantitative data. [ABSTRACT FROM AUTHOR]

Image fusion operation is beneficial to many applications and is also one of the most common and critical computer vision challenges. The perfect infrared and visible image fusion results should include the important infrared targets while preserving visible textural detail information as much as possible. A novel infrared and visible image fusion framework is proposed for this purpose. In this paper, the proposed fusion network (MIFFuse) is an end-to-end, multi-level-based fusion network for infrared and visible images. The presented approach makes effective use of the intermediate convolution layer’s output features to preserve the primary image fusion information. We also build a cat_block to swap information between two paths to gain more sufficient information during the convolution steps. To reduce the model’s running time even further, the proposed method that reduces the number of feature channels while maintaining the accuracy of the fusion performance. Extensive experiments on the TNO and CVC-14 image fusion datasets show that our MIFFuse outperforms the other methods in terms of both subjective visual effects and quantitative metrics. Furthermore, MIFFuse is approximately twice as fast as the most recent state-of-the-art methods. Our code and models can be found at https://github.com/depeng6/MIFFuse.

Privacy image classification can help people detect privacy images when people share images. In this paper, we propose a novel method using multi-level and multi-scale features for privacy image classification. We first use CNN (Convolutional Neural Network) to extract multi-levels features. Then, max-pooling layers are employed to obtain multi-scale features at each level. Finally, we propose two feature aggregation models, called Privacy-MSML and Privacy-MLMS to fuse those features for image privacy classification. In Privacy-MSML, multi-scale features of the same level are first integrated and then the integrated features are fused. In Privacy-MLMS, multi-level features of the same scale are first integrated and then the integrated features are fused. Our experiments on a real-world dataset demonstrate the proposed method can achieve better performance compared with the state-of-the-art solutions. [ABSTRACT FROM AUTHOR]

Over the years, several methods have been proposed for the computational PPI prediction with different performance evaluation strategies. While attempting to benchmark performance scores, most of these methods often suffer with ill-treated cross-validation strategies, adhoc selection of positive/negative samples etc. To address these issues, in our proposed multi-level feature based PPI prediction approach (JUPPI), using sequence, domain and GO information as features, a refined evaluation strategy has been introduced. During the evaluation process, we first extract high quality negative data using three-stage filtering, and then introduce a pair-input based cross validation strategy with three difficulty levels for test-set predictions. Our proposed evaluation strategy reduces the component-level overlapping issue in test sets. Performance of JUPPI is compared with those of the state-of-the-art approaches in this domain and tested on six independent PPI datasets. In almost all the datasets, JUPPI outperforms the state-of-the-art not only at human proteome level for PPI prediction, but also for prediction of interactors for intrinsic disordered human proteins. https://figshare.com/projects/JUPPI%5fA%5fMulti-level%5fFeature%5fBased%5fMethod%5ffor%5fPPI%5fPrediction%5fand%5fa%5fRefined%5fStrategy%5ffor%5fPerformance%5fAssessment/81656 JUPPI tool and the developed datasets (JUPPId) are available in public domain for academic use along with supplementary materials, which can be found on the Computer Society Digital Library at http://doi.ieeecomputersociety.org/10.1109/TCBB.2020.3004970. [ABSTRACT FROM AUTHOR]

Aiming at the problems of traditional image super-resolution reconstruction algorithms in the image reconstruction process, such as small receptive field, insufficient multi-scale feature extraction, and easy loss of image feature information, a super-resolution reconstruction algorithm of multi-scale dilated convolution network based on dilated convolution is proposed in this paper. First, the algorithm extracts features from the same input image through the dilated convolution kernels of different receptive fields to obtain feature maps with different scales; then, through the residual attention dense block, further obtain the features of the original low resolution images, local residual connections are added to fuse multi-scale feature information between multiple channels, and residual nested networks and jump connections are used at the same time to speed up deep network convergence and avoid network degradation problems. Finally, deep network extraction features, and it is fused with input features to increase the nonlinear expression ability of the network to enhance the super-resolution reconstruction effect. Experimental results show that compared with Bicubic, SRCNN, ESPCN, VDSR, DRCN, LapSRN, MemNet, and DSRNet algorithms on the Set5, Set14, BSDS100, and Urban100 test sets, the proposed algorithm has improved peak signal-to-noise ratio and structural similarity, and reconstructed images. The visual effect is better., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Wang, Zhang and Miao.)

In an attempt to solve the problems of scale change and multi-level feature fusion in population counting method, inspired by U-Net encoder decoder structure network, an adaptive feature fusion network is proposed to carry out accurate population counting. The Adaptive Feature Fusion Module (AFFM) is proposed to efficiently aggregate the high-level semantic information and low- level spatial detail extracted by the encoder branch according to the needs of decoder branch. The Adaptive Context Extractor (ACE) is proposed to extract multi-scale context information on multiple effective field-of-views, then these features are adaptively fused to improve the robustness of the network to scale changes. By conducting exhaustive experiments on Shanghai Tech, UCF-CC-50 and UCF-QNRF, the results show that the network has high accuracy and robustness. [ABSTRACT FROM AUTHOR]

The accuracy with which a neural network interprets a point cloud depends on the quality of the features expressed by the network. Addressing this issue, we propose a multi-level feature extraction layer (MFEL) which collects local contextual feature and global information by modeling point clouds at different levels. The MFEL is mainly composed of three independent modules, including the aggregated GAPLayer, the spatial position perceptron, and the RBFLayer, which learn point cloud features from three different scales. The aggregated GAPLayer aggregates the geometry features of neighboring points in a local coordinate system to centroid by graph convolution. Then, the spatial position perceptron independently learns the position features of each point in the world coordinate system. Finally, the RBFLayer aggregates points into pointsets according to the correlation between features, and extracts features from the pointset scale through the quantization layer. Based on the MFEL, an end-to-end classification and segmentation network, namely the MFNet and MFNet-S, is proposed. In the proposed network, the channel-attention mechanism is employed to better aggregate multi-level features. We conduct classification and semantic segmentation experiments on four standard datasets. The results show that the proposed method outperforms the compared methods on the multiple datasets, resulting in 93.1% classification accuracy in ModelNet40. Furthermore, the mIoU of part semantic segmentation in ShapeNet is 85.4%, and the mIoU for semantic segmentation in S3DIS and Semantic3D is 62.9% and 71.9%, respectively.

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

With the advent of the Internet of Things (IoT) era, many devices have surfaced that capture and generate various visual data. To recognize and extract a meaningful pattern from these visual data, powerful methods are required for different IoT applications. Fortunately, deep convolutional neural networks (CNNs) significantly improve the performance of almost all tasks in computer vision, including semantic image segmentation. However, the feature extraction of CNNs may cause the loss of contextual and spatial information. Moreover, the standard convolutional and pooling layers adopted by most CNN architectures lead to a fixed receptive field, which makes it challenging to deal with multi-scale objects in the image. To remedy these issues of CNNs for semantic image segmentation, this paper proposes a multi-level graph convolutional recurrent neural network (MGCRNN) to combine CNNs and graph neural networks (GNNs) for fusing multi-level features. By applying graph convolutional recurrent neural network (GCRNN), the proposed model acquires a global view of the image and aggregates multi-level contextual and structural information. The experiments verify the ability of GCRNN to obtain a flexible receptive field and learn structure features without losing spatial information. Results of these experiments conducted on the Pascal VOC 2012 and Cityscapes datasets show that the proposed model outperforms baseline approaches and can be competitive with state-of-the-art methods [ABSTRACT FROM AUTHOR]

Due to the remarkable progress in face manipulation technology, malicious applications of these technologies may pose a great threat to the social stability. Therefore, it is essential to carry out the research of deepfake detection. In this paper, we assumed that the illumination on frames that skip a certain space is basically consistent in real videos, but tends to be inconsistent in fake videos. From this point, a network which contains a learnable Image Decomposition Module (IDM) and multi-level feature enhancement is proposed. IDM decomposes frames into illumination and reflection, and frame recomposition is followed to highlight the frame-level illumination inconsistency. Multi-level feature enhancement is proposed to enhance the illumination inconsistency at feature level. In addition, considering the computational complexity and human vision perception mechanism, we train the network in logarithm domain. Experimental results show that the proposed method is effective and superior compared with other state-of-the-art deepfake detection methods on mainstream deepfake datasets. • A learnable Image Decomposition Module (IDM) is proposed. • The recomposition operation is used for highlighting the illumination inconsistency. • Multi-level feature enhancement is proposed to creatively recompose features. • The proposed method is trained in logarithm domain and the IDM is extensible. [ABSTRACT FROM AUTHOR]

Featured Application: The proposed method has a potential application value in assisting the normal driving of unmanned delivery vehicles and unmanned cleaning vehicles in urban street scenes. It can aid unmanned vehicles to detect and segment surrounding objects and plan safe driving routes to avoid obstacles according to the results of instance segmentation. Recently, multi-level feature networks have been extensively used in instance segmentation. However, because not all features are beneficial to instance segmentation tasks, the performance of networks cannot be adequately improved by synthesizing multi-level convolutional features indiscriminately. In order to solve the problem, an attention-based feature pyramid module (AFPM) is proposed, which integrates the attention mechanism on the basis of a multi-level feature pyramid network to efficiently and pertinently extract the high-level semantic features and low-level spatial structure features; for instance, segmentation. Firstly, we adopt a convolutional block attention module (CBAM) into feature extraction, and sequentially generate attention maps which focus on instance-related features along the channel and spatial dimensions. Secondly, we build inter-dimensional dependencies through a convolutional triplet attention module (CTAM) in lateral attention connections, which is used to propagate a helpful semantic feature map and filter redundant informative features irrelevant to instance objects. Finally, we construct branches for feature enhancement to strengthen detailed information to boost the entire feature hierarchy of the network. The experimental results on the Cityscapes dataset manifest that the proposed module outperforms other excellent methods under different evaluation metrics and effectively upgrades the performance of the instance segmentation method. [ABSTRACT FROM AUTHOR]

The Pan-Sharpening (PS) techniques provide a better visualization of a multi-band image using the high-resolution single-band image. To support their development and evaluation, in this paper, a novel, accurate, and automatic No-Reference (NR) PS Image Quality Assessment (IQA) method is proposed. In the method, responses of two complementary network architectures in a form of extracted multi-level representations of PS images are employed as quality-aware information. Specifically, high-dimensional data are separately extracted from the layers of the networks and further processed with the Kernel Principal Component Analysis (KPCA) to obtain features used to create a PS quality model. Extensive experimental comparison of the method on the large database of PS images against the state-of-the-art techniques, including popular NR methods adapted in this study to the PS IQA, indicates its superiority in terms of typical criteria.

Up to now, most of the forensics methods have attached more attention to natural content images. To expand the application of image forensics technology, forgery detection for certificate images that can directly represent people’s rights and interests is investigated in this paper. Variable tampered region scales and diverse manipulation types are two typical characteristics in fake certificate images. To tackle this task, a novel method called Multi-level Feature Attention Network (MFAN) is proposed. MFAN is built following the encoder–decoder network structure. In order to extract features with rich scale information in the encoder, on the one hand, we employ Atrous Spatial Pyramid Pooling (ASPP) on the final layer of a pre-trained residual network to capture the contextual information at different scales; on the other hand, low-level features are concatenated to ensure the sensibility to small targets. Furthermore, the resulting multi-level features are recalibrated on channels for irrelevant information suppression and enhancing the tampered regions, guiding the MFAN to adapt to diverse manipulation traces. In the decoder module, the attentive feature maps are convoluted and unsampled to effectively generate the prediction mask. Experimental results indicate that the proposed method outperforms some state-of-the-art forensics methods.

Abstract Histopathology image analysis is a gold standard for cancer recognition and diagnosis. But typical problems with histopathology images that hamper automatic analysis include complex clinical features, insufficient training data, and large size of a single image (always up to gigapixels). In this paper, an image semantic segmentation algorithm based on feature Pyramid (ResNet50-GICN-GPP) is proposed. Firstly, the patch sampling method is adopted to resample the image, reduce the size of a single sample, and expand the number of training samples. Secondly, design the whole convolution network based on ResNet50 learning feature location information, then use GICN structure and deconvolution network to integrate multi-level features. Finally, in order to solve the problem that the GICN structure may lose the small object, the GPP structure should be joined to explore the multi-scale semantic information. The proposed method achieves 63% of the average segmentation accuracy (Dice coefficient) on Camelyon16 and Gastric WSIs Data, compared with U-Net, FCN and SegNet which has 10~20% improvement, and fully demonstrates the effectiveness of this method in different types of cancer. By experimentally comparing the segmentation accuracy of various scales of cancerous tissues, the performance of ResNet50-GICN-GPP is balanced and the multi-scale information localization is more accurate.

Stance detection is the task of attitude identification toward a standpoint. Previous work of stance detection has focused on feature extraction but ignored the fact that irrelevant features exist as noise during higher-level abstracting. Moreover, because the target is not always mentioned in the text, most methods have ignored target information. In order to solve these problems, we propose a neural network ensemble method that combines the timing dependence bases on long short-term memory (LSTM) and the excellent extracting performance of convolutional neural networks (CNNs). The method can obtain multi-level features that consider both local and global features. We also introduce attention mechanisms to magnify target information-related features. Furthermore, we employ sparse coding to remove noise to obtain characteristic features. Performance was improved by using sparse coding on the basis of attention employment and feature extraction. We evaluate our approach on the SemEval-2016Task 6-A public dataset, achieving a performance that exceeds the benchmark and those of participating teams. [ABSTRACT FROM AUTHOR]

Accurate and automatic vegetation extraction from digital plant images in the field is a widely studied topic in precision agriculture. Many techniques focus on pixels or regions to be segmented as plants or back-grounds, such as colour index-based and learning-based methods. Different from a traditional two-class classification problem, the proposed method regarded vegetation extraction as a multi-class task. In consideration of manually annotated errors at the edge of a plant image, the original marked mask was re-labelled using a Gaussian probability function. To capture more adequate information in the process of feature extraction, 9 pixel-level colour features and 18 region-level statistical characteristics of neighbourhood pixels were computed from three colour spaces. The extracted 27-dimensional features were inputs of a classification model, which output multi-class labels. A suitable threshold was finally selected to obtain the segmented image. Experimental results showed that the proposed multi-class and multi-level features (MCMLF) method achieved better performance than the other approaches. Through the quantitative and qualitative analysis of segmentation results, it was also found that the suggested method had high computation efficiency as well as strong adaptation ability to solve the outdoor challenges, including various lighting conditions, shadow regions, and complex backgrounds. • Vegetation extraction is regarded as a multi-class recognition task. • Pixel-level and region-level features are extracted from three colour spaces. • The method has high computation efficiency as well as strong adaptation ability. [ABSTRACT FROM AUTHOR]

Person re-identification (Re-ID) aims to match the photos of suspects in the gallery database by a query photo. Under Re-ID, the input is usually a query photo of the target person, however, in a realistic setting, such a photo is not always available. In this paper, we study the problem of Sketch Re-ID, which is not based on a person's photo but the professional sketch of the target person. We use a full-body sketch of a target person drawn by a professional to find the photo with the same ID in the gallery. This problem is very challenging because photos and sketches belong to two completely different domains. Specifically, a sketch is a highly abstract description of a person, which only contains some rough outline information. We address the Sketch Re-ID problem by proposing a novel framework to jointly model photos and sketches into a common embedding space. Our framework uses a triplet classification network as a base network. We propose to use a spatial attention module and combine high-level and mid-level output features of CNN to represent the input images. Moreover, we design a novel domain-invariant feature by using a gradient reverse layer (GRL) to solve the domain gap problem. We validated our approach on the Sketch Re-ID dataset, which contains 200 persons, each of whom has a sketch and two photos from different cameras associated. To evaluate the generalization of our method, we also performed experiments on some other public Sketch-based Image Retrieval (SBIR) datasets. The extensive experimental results show that our method can get higher performance than state-of-the-art models. [ABSTRACT FROM AUTHOR]