Search

Image reconstruction-based methods with autoencoder have been widely used for unsupervised anomaly detection. By training the reconstruction on normal samples, autoencoder is supposed to produce higher reconstruction error for anomalous samples, which is used as an indicator for detecting anomalies. However, since autoencoder adopts the bottleneck layer to reconstruct data, it is hard to control its generalization capability. When the generalization capability is high, anomalous features can be confused with normal features, resulting in accurate reconstruction of anomalous regions as well. In this article, we propose a dual-constraint autoencoder to alleviate the problem of feature confusion through the dual constraint of adversarial learning and global memory bank. Given a sample pair consisting of a normal sample and a synthetic anomaly sample generated from it as input, the proposed autoencoder first forces the encoding of the synthetic anomaly image to be close to the normal encoding by adversarial learning. Then, the synthetic anomaly encoding is rerepresented with items in memory bank, which records normal features that facilitate image restoration, and the new feature is fed to the decoder for image inpainting. Under such constraints, the generalization capability of the autoencoder is suppressed and will not reconstruct the anomaly region well. After that, we feed the synthetic anomaly image together with the repaired image into the proposed adaptive weighted similarity spatial attention-based U-Net and produce an accurate anomaly map with the help of our proposed adaptive weighted similarity spatial attention. Results on the a comprehensive real-world dataset for unsupervised anomaly detection (MVTec AD) dataset and the beanTech anomaly detection dataset dataset show that our framework achieves state-of-the-art performance.

With the development of deep learning, significant progress has been made in change detection (CD) methods for remote sensing (RS) images. However, many convolutional neural network (CNN)-based methods are constrained in capturing long-range dependencies due to the limitations of the receptive field. Transformers rely on self-attention mechanisms to effectively achieve global information modeling and are widely used in CD tasks. Nevertheless, transformer-based CD methods still suffer from issues such as pseudochanges and incomplete edges due to the lack of position and temporal correlations in bitemporal RS images. To deal with this issue, we propose a position-temporal awareness transformer (PT-Former), which models position and temporal relations in bitemporal images. Specifically, a Siamese network attached to a position-aware embedding module (PEM) serves as a feature encoder to extract the features of changed areas. Then, a temporal difference perception module (TDPM) is designed to capture the cross-temporal shift and enhance the difference perception ability during cross-temporal interaction. Meanwhile, the contextual information of the ground object is aggregated by the fusion block, and the spatial relation is reconstructed under the guidance of bitemporal features. The experimental results validate the superiority of PT-Former on three benchmark datasets, including the season-varying CD (SVCD) dataset, the learning vision and RS laboratory building CD (LEVIR-CD) dataset, and the WHU-CD dataset confirming the potential of PT-Former for CD tasks in RS images. The code will be available at https://github.com/liuyk29/PT-Former.

In random sample consensus (RANSAC), the problem of ellipsoid fitting can be formulated as a problem of minimization of point-to-model distance, which is realized by maximizing model score. Hence, the performance of ellipsoid fitting is affected by distance metric. In this article, we proposed a novel distance metric called the axial distance, which is converted from the algebraic distance by introducing a scaling factor to solve nongeometric problems of the algebraic distance. There is complementarity between the axial distance and Sampson distance because their combination is a stricter metric when calculating the model score of sample consensus and the weight of the weighted least-squares (WLS) fitting. Subsequently, a novel sample-consensus-based ellipsoid fitting method is proposed using the combination between the axial distance and Sampson distance (CAS). We compare the proposed method with several representative fitting methods through experiments on synthetic and real datasets. The results show that the proposed method has a higher robustness against outliers, consistently high accuracy, and a speed close to that of the method based on sample consensus.

Surface defect detection is a significant step in industrial production, which is also essential for ensuring the quality of industrial products. At the moment, although the defect detection methods based on computer vision have made great progress, they are still tough to automate the detection due to the challenges of large changes in size, low-contrast, strong background interference, large intraclass difference as well as small interclass difference. To address the above issues, we design a deep learning model that mainly consists of a feature augmentation module (FAM), and a multiscale bidirectional feature fusion module (MBFFM), while introducing deformable convolutions into the backbone. The FAM enhances the defect features and magnifies the feature difference between defects and background by combining global context-based self-similarity and spatial attention, which can improve the detection performance for weak defects. The MBFFM is applied to complete the bidirectional fusion of high-level and low-level features for the overall perception of defects. In addition, we design an edge weight loss (EWL) to emphasize defect regions by increasing their weights in the loss. At the same time, by combining EWL and deformable convolution, the backbone can accurately extract comprehensive and complete defect features. Experiments verify that the mean intersection over union (mIoU) of our method on the four datasets is superior to other state-of-the-art methods (MT, miou of 83.75%; NEU_SEG, miou of 88.84%; DAGM_defect, miou of 86.74%; ROAD, miou of 76.71%).

For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral–spatial relationships has attracted widespread attention in the HSI classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch-based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to the Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask-based intrapatch sampling (AMIPS) module is first formulated for intrapatch sampling in an adaptive mask manner based on central spectral vector-oriented spatial relationships. Subsequently, based on the distance covariance descriptor, a dual-channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral–spatial feature representations and exploring spectral–spatial relationships, especially linear and nonlinear interdependence in the spectral domain. Furthermore, considering that the distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement an M2ESL module respecting the Riemannian geometry of the SPD manifold for high-level spectral–spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI datasets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.

To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/CVSSN

The goal of unsupervised surface anomaly detection is to detect areas of the image that are different from the normal pattern, which can be considered as a semantic segmentation problem oriented to anomalous patterns. However, this problem is challenging due to the lack of actual available anomaly sam- ples. In this article, we transform unsupervised anomaly detection into a self-supervised problem by the proposed anomaly simulation strategy. Using only normal samples for training, real anomalies appearing in the inference phase can be detected. Thus, we propose a segmentation network with joint representation and contrast clustering (JRCC-Net). The proposed method learns a joint representation between simulated samples and the magnitude of the differences in distance from their nearest memory samples, as well as a decision boundary between normal and anomalous samples. Moreover, we propose a novel contrast clustering-based representation learning strategy, which allows the model to better learn general patterns from normal samples and mine the latent differences between simulated anomalous and normal samples. JRCC-Net is able to locate anomalies directly in an end-to-end manner and can be trained well with our elaborate anomaly simulation strategy. On the challenging MVTec AD dataset, JRCC-Net outperforms the state-of-the-art unsupervised methods, achieving 98.2% image-level area under the receiver operating characteristics (AUROC) and 97.7% pixel-level AUROC, respectively, and even on the extensively used DAGM dataset, its localization accuracy greatly exceeds that of fully supervised methods.

Semantic segmentation is an extremely challenging task in high-resolution remote sensing (HRRS) images as objects have complex spatial layouts and enormous variations in appearance. Convolutional neural networks (CNNs) have an excellent ability to extract local features and have been widely applied as a feature extractor for various vision tasks. However, due to the inherent inductive bias of convolution operation, CNNs inevitably have limitations in modeling long-range dependencies. Transformer can capture global representations well but unfortunately ignores the details of local features and has high computational and spatial complexity in processing high-resolution feature maps. In this article, we propose a novel hybrid architecture for HRRS image segmentation, termed Enhancing Multiscale Representations with Transformer (EMRT), to exploit the advantages of convolution operations and Transformer to enhance multiscale representation learning. We incorporate the deformable self-attention mechanism in the Transformer to automatically adjust the receptive field and design an encoder–decoder architecture accordingly to achieve efficient context modeling. Specifically, CNN is constructed to extract feature representations. In the encoder, local features and global representations at different resolutions are extracted by the CNN and Transformer, respectively, and fused in an interactive manner. Moreover, a separate spatial branch is designed to extract multiscale contextual information as queries, and global dependencies between features at different scales are efficiently established by the decoder. Extensive experiments on three public remote sensing datasets demonstrate the superiority of EMRT and indicate that the overall performance of our method outperforms state-of-the-art methods. Code is available at https://github.com/peach-xiao/EMRT.

ABSTRACTDeep convolutional neural networks have made significant progress in the field of intelligent analysis of remote-sensing images. However, the semantic segmentation task in high-resolution remote-sensing (HRRS) images always faces the problem of large-scale variation and complex background samples, which causes difficulties in distinguishing confusable ground objects. In this letter, we propose a novel multipath feature refinement network (MFRNet) to alleviate the above problems. We design the feature refinement module (FRM) to fuse features at various scales, which helps to capture different levels of spatial information. It also alleviates the boundary ambiguity problem by enhancing the learning of features with boundary information. The multiscale feature attention module (MFAM) combines atrous convolution and non-local block to obtain larger receptive fields and long-range contextual information, while the feature fusion module (FFM) balances semantic and spatial information, further improving the embedding of locally discriminative features. Experimental results on ISPRS Potsdam and LoveDA datasets indicate that the proposed MFRNet outperforms other semantic segmentation methods and excels in the accuracy and consistency of object boundary segmentation.

This paper proposes a novel learning method of binary local features for recognition of the finger vein. The learning methods existing in local features for image recognition intend to maximize the data variance, reduce quantitative errors, exploit the contextual information within each binary code, or utilize the label information, which all ignore the local manifold structure of the original data. The manifold structure actually plays a very important role in binary code learning, but constructing a similarity matrix for large-scale datasets involves a lot of computational and storage cost. The study attempts to learn a map, which can preserve the manifold structure between the original data and the learned binary codes for large-scale situations. To achieve this goal, we present a learning method using an anchor-based manifold binary pattern (AMBP) for finger vein recognition. Specifically, we first extract the pixel difference vectors (PDVs) in the local patches by calculating the differences between each pixel and its neighbors. Second, we construct an asymmetric graph, on which each data point can be a linear combination of its K-nearest neighbor anchors, and the anchors are randomly selected from the training samples. Third, a feature map is learned to project these PDVs into low-dimensional binary codes in an unsupervised manner, where (i) the quantization loss between the original real-valued vectors and learned binary codes is minimized and (ii) the manifold structure of the training data is maintained in the binary space. Additionally, the study fuses the discriminative binary descriptor and AMBP methods at the image representation level to further boost the performance of the recognition system. Finally, experiments using the MLA and PolyU databases show the effectiveness of our proposed methods.

Finger vein image retrieval is a biometric identification technology that has recently attracted a lot of attention. It has the potential to reduce the search space and has attracted a considerable amount of research effort recently. It is a challenging problem owing to the large number of images in biometric databases and the lack of efficient retrieval schemes. We apply a hierarchical vocabulary tree model-based image retrieval approach because of its good scalability and high efficiency.However, there is a large accumulative quantization error in the vocabulary tree (VT)model that may degrade the retrieval precision. To solve this problem, we improve the vector quantization coding in the VT model by introducing a non-negative locality-constrained constraint: the non-negative locality-constrained vocabulary tree-based image retrieval model. The proposed method can effectively improve coding performance and the discriminative power of local features. Extensive experiments on a large fused finger vein database demonstrate the superiority of our encoding method. Experimental results also show that our retrieval strategy achieves better performance than other state-of-the-art methods, while maintaining low time complexity.

The measurement of the vessel pattern in fingers is a superior method for identifying individuals owing to its convenience and the security it offers. We introduce in this paper a new perspective to accomplish finger vein recognition. This method, which regards deformations as discriminative information, is distinct from existing methods that attempt to prevent the influence of deformations. The proposed technique is based on the observation that regular deformation, which corresponds to a posture change, can only exist in genuine vein patterns. In terms of methodology, we incorporate optimized matching to generate pixelbased 2D displacements that correspond to deformations. The texture of uniformity extracted from the displacement fields is taken as the final matching score. Evaluated on two publicly available databases, PolyU and SDU-MLA, extensive experiments demonstrated that the discriminability of the new feature derived from deformations is preferable. The equal error rate (EER) achieved is the lowest compared to that of state-of-the-art techniques.

Finger vein recognition is a promising biometric technology that has received significant research attention. However, most of the existing works often relied on a single feature, which failed to fully exploit the discriminative information in finger vein images, and therefore led to a limited recognition performance. To overcome this limitation, this paper proposes an encoding coefficient similarity-based multifeature sparse representation method for finger vein recognition. The proposed method not only uses multiple features to extract comprehensive information from finger vein images, but also obtains more discriminative information through constraints in the objective function. The sparsity constraint retains the key information of each feature, and the similarity constraint explores the shared information among the features. Furthermore, the proposed method is capable of fusing all kinds of features, not limited to specific ones. The optimization problem of the proposed method is efficiently solved using the alternating direction multiplier method algorithm. Experimental results on two public finger vein databases HKPU-FV and SDU-FV show that the proposed method achieves good recognition performance.

Research on biometrics for high security applications has not attracted as much attention as civilian or forensic applications. Limited research and deficient analysis so far has led to a lack of general solutions and leaves this as a challenging issue. This work provides a systematic analysis and identification of the problems to be solved in order to meet the performance requirements for high security applications, a double low problem. A hybrid ensemble framework is proposed to solve this problem. Setting an adequately high threshold for each matcher can guarantee a zero false acceptance rate (FAR) and then use the hybrid ensemble framework makes the false reject rate (FRR) as low as possible. Three experiments are performed to verify the effectiveness and generalization of the framework. First, two fingerprint verification algorithms are fused. In this test only 10.55% of fingerprints are falsely rejected with zero false acceptance rate, this is significantly lower than other state of the art methods. Second, in face verification, the framework also results in a large reduction in incorrect classification. Finally, assessing the performance of the framework on a combination of face and gait verification using a heterogeneous database show this framework can achieve both 0% false rejection and 0% false acceptance simultaneously.

In higher education, the initial studying period of each course plays a crucial role for students, and seriously influences the subsequent learning activities. However, given the large size of a course’s students at universities, it has become impossible for teachers to keep track of the performance of individual students. In this circumstance, an academic early warning system is desirable, which automatically detects students with difficulties in learning (i.e., at-risk students) prior to a course starting. However, previous studies are not well suited to this purpose for two reasons: 1) they have mainly concentrated on e-learning platforms, e.g., massive open online courses (MOOCs), and relied on the data about students’ online activities, which is hardly accessed in traditional teaching scenarios; and 2) they have only made performance prediction when a course is in progress or even close to the end. In this paper, for traditional classroom-teaching scenarios, we investigate the task of pre-course student performance prediction, which refers to detecting at-risk students for each course before its commencement. To better represent a student sample and utilize the correlations among courses, we cast the problem as a multi-instance multi-label (MIML) problem. Besides, given the problem of data scarcity, we propose a novel multi-task learning method, i.e., MIML-Circle, to predict the performance of students from different specialties in a unified framework. Extensive experiments are conducted on five real-world datasets, and the results demonstrate the superiority of our approach over the state-of-the-art methods.

Finger vein recognition is a promising biometric recognition technology, which verifies identities via the vein patterns in the fingers. In this paper, (2D)2 PCA is applied to extract features of finger veins, based on which a new recognition method is proposed in conjunction with metric learning. It learns a KNN classifier for each individual, which is different from the traditional methods where a fixed threshold is employed for all individuals. Besides, the SMOTE technology is adopted to solve the class-imbalance problem. Our experiments show that the proposed method is effective by achieving a recognition rate of 99.17%.