Search

Introduction: Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels. Methods: We investigated if the SSL models can be applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network, trained with a contrastive loss, serves as the feature extractor that learns latent representations. The classification head is a single-layer perceptron that is trained to perform diagnostic group separation. We used N = 2,694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its phenotypes. Results: Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV. Conclusion: Our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable. [ABSTRACT FROM AUTHOR]

Self-supervised learning (SSL) has gained attention in the medical field as a deep learning approach utilizing unlabeled data. The Jigsaw puzzle task in SSL enables models to learn both features of images and the positional relationships within images. In breast cancer diagnosis, radiologists evaluate not only lesion-specific features but also the surrounding breast structures. However, deep learning models that adopt a diagnostic approach similar to human radiologists are still limited. This study aims to evaluate the effectiveness of the Jigsaw puzzle task in characterizing breast tissue structures for breast cancer classification on mammographic images. Using the Chinese Mammography Database (CMMD), we compared four pre-training pipelines: (1) IN-Jig, pre-trained with both the ImageNet classification task and the Jigsaw puzzle task, (2) Scratch-Jig, pre-trained only with the Jigsaw puzzle task, (3) IN, pre-trained only with the ImageNet classification task, and (4) Scratch, that is trained from random initialization without any pre-training tasks. All pipelines were fine-tuned using binary classification to distinguish between the presence or absence of breast cancer. Performance was evaluated based on the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Additionally, detailed analysis was conducted for performance across different radiological findings, breast density, and regions of interest were visualized using gradient-weighted class activation mapping (Grad-CAM). The AUC for the four models were 0.925, 0.921, 0.918, 0.909, respectively. Our results suggest the Jigsaw puzzle task is an effective pre-training method for breast cancer classification, with the potential to enhance diagnostic accuracy with limited data. [ABSTRACT FROM AUTHOR]

Spatiotemporal predictive learning is a deep learning method that generates future frames from historical frames in a self-supervised manner. Existing studies face the challenges in capturing long-term dependencies and producing accurate predictions over extended time horizons. To address these limitations, this paper introduces a nested attention module as a special attention mechanism to capture spatiotemporal correlations of input historical frames. Nested attention module decomposes temporal attention into inter-frame channel attention and spatiotemporal attention and uses a nested attention mechanism to capture long-term temporal dependencies, which improves the model's performance and generalization ability. Furthermore, to prevent overfitting in models, a new regularization method is proposed which considers both the intra-frame spatial error and the inter-frame temporal evolution error of sequence frames, and enhances the robustness of the reinforcement learning model to dropout operations. The proposed model achieves state-of-the-art performance on four baseline datasets, including moving MNIST handwritten digit dataset, human 3.6 million dataset, sea surface temperature dataset, and karlsruhe institute of technology and Toyota technological institute dataset. Extended experiments demonstrate the generalization and extensibility of nested attention module on real-world datasets. A dramatic 31.7% mean squared error/26.9% mean absolute error reduction is achieved when predicting 10 frames on moving MNIST. Our proposed model provides a new baseline for future research in spatiotemporal predictive learning tasks. [ABSTRACT FROM AUTHOR]

Hippocampal representations of space and time seem to share a common coding scheme characterized by neurons with bell-shaped tuning curves called place and time cells. The properties of the tuning curves are consistent with Weber's law, such that, in the absence of visual inputs, width scales with the peak time for time cells and with distance for place cells. Building on earlier computational work, we examined how neurons with such properties can emerge through self-supervised learning. We found that a network based on autoencoders can, given a particular inputs and connectivity constraints, produce scale-invariant time cells. When the animal's velocity modulates the decay rate of the leaky integrators, the same network gives rise to scale-invariant place cells. Importantly, this is not the case when velocity is fed as a direct input to the leaky integrators, implying that weight modulation by velocity might be critical for developing scale-invariant spatial receptive fields. Finally, we demonstrated that after training, scale-invariant place cells emerge in environments larger than those used during training. Taken together, these findings bring us closer to understanding the emergence of neurons with bell-shaped tuning curves in the hippocampus and highlight the critical role of velocity modulation in the formation of scale-invariant place cells. [ABSTRACT FROM AUTHOR]

Visual aesthetics has always been an important area of computational vision, and researchers have continued exploring it. To further improve the performance of the image aesthetic evaluation task, we introduce a Transformer into the image aesthetic evaluation task. This paper pioneers a novel self-supervised image aesthetic evaluation model founded upon Transformers. Meanwhile, we expand the pretext task to capture rich visual representations, adding a branch for inpainting the masked images in parallel with the tasks related to aesthetic quality degradation operations. Our model's refinement employs the innovative uncertainty weighting method, seamlessly amalgamating three distinct losses into a unified objective. On the AVA dataset, our approach surpasses the efficacy of prevailing self-supervised image aesthetic assessment methods. Remarkably, we attain results approaching those of supervised methods, even while operating with a limited dataset. On the AADB dataset, our approach improves the aesthetic binary classification accuracy by roughly 16% compared to other self-supervised image aesthetic assessment methods and improves the prediction of aesthetic attributes. [ABSTRACT FROM AUTHOR]

Person Re-Identification (ReID) aims to retrieve relevant individuals in non-overlapping camera images and has a wide range of applications in the field of public safety. In recent years, with the development of Vision Transformer (ViT) and self-supervised learning techniques, the performance of person ReID based on self-supervised pre-training has been greatly improved. Person ReID requires extracting highly discriminative local fine-grained features of the human body, while traditional ViT is good at extracting context-related global features, making it difficult to focus on local human body features. To this end, this article introduces the recently emerged Masked Image Modeling (MIM) self-supervised learning method into person ReID, and effectively extracts high-quality global and local features through large-scale unsupervised pre-training by combining masked image modeling and discriminative contrastive learning, and then conducts supervised fine-tuning training in the person ReID task. This person feature extraction method based on ViT with masked image modeling (PersonViT) has the good characteristics of unsupervised, scalable, and strong generalization capabilities, overcoming the problem of difficult annotation in supervised person ReID, and achieves state-of-the-art results on publicly available benchmark datasets, including MSMT17, Market1501, DukeMTMC-reID, and Occluded-Duke. The code and pre-trained models of the PersonViT method are released at to promote further research in the person ReID field. [ABSTRACT FROM AUTHOR]

Multi-behavior recommendation (MBR) aims to enhance the accuracy of predicting target behavior by considering multiple behaviors simultaneously. Recent researches have attempted to capture the dependencies within behavioral sequences to improve recommendation outcomes, exemplified by the sequential pattern "click → cart → buy". However, their performances are still limited due to the following two problems. Firstly, potential leapfrogging relations among behaviors are underexplored, notably in cases where users purchase directly post-click, bypassing the cart stage. Skipping intermediate behavior allows for better modeling of real-world realities. Secondly, the uneven distribution of user behaviors and item popularity presents a challenge for model training, resulting in prevalence bias and over-reliance issues. To this end, we propose a self-supervised progressive graph neural network model, namely SSPGNN. The model can capture a broader range of behavioral dependencies by using a dual-behavior chain. In addition, we design a self-supervised learning mechanism, including intra- and inter-behavioral self-supervised learning, the former within a single behavior and the latter across multiple behaviors, to address the problems of prevalence bias and overdependence. Extensive experiments on real-world datasets and comparative analyses with state-of-the-art algorithms demonstrate the effectiveness of the proposed SSPGNN. The source codes of this work are available at https://github.com/ZZY-GraphMiningLab/SSPGNN. [ABSTRACT FROM AUTHOR]

The article focuses on the practice of secondary recommendation with collaborative confusion in the context of academic information retrieval. Topics include the risks associated with gathering and transferring user data, the role of collaborative filtering in recommendation systems, and the challenges of preserving study integrity while incorporating new methodologies.

Human skeleton-based action recognition is an important task in the field of computer vision. In recent years, masked autoencoder (MAE) has been used in various fields due to its powerful self-supervised learning ability and has achieved good results in masked data reconstruction tasks. However, in visual classification tasks such as action recognition, the limited ability of the encoder to learn features in the autoencoder structure results in poor classification performance. We propose to enhance the encoder's feature extraction ability in classification tasks by leveraging the latent space of variational autoencoder (VAE) and further replace it with the latent space of vector quantized variational autoencoder (VQVAE). The constructed models are called SkeletonMVAE and SkeletonMVQVAE, respectively. In SkeletonMVAE, we constrain the latent variables to represent features in the form of distributions. In SkeletonMVQVAE, we discretize the latent variables. These help the encoder learn deeper data structures and more discriminative and generalized feature representations. The experiment results on the NTU-60 and NTU-120 datasets demonstrate that our proposed method can effectively improve the classification accuracy of the encoder in classification tasks and its generalization ability in the case of few labeled data. SkeletonMVAE exhibits stronger classification ability, while SkeletonMVQVAE exhibits stronger generalization in situations with fewer labeled data. [ABSTRACT FROM AUTHOR]

Malware detection remains a critical challenge in cybersecurity due to the rapid evolution of sophisticated malware and adversarial threats. Traditional detection systems struggle to adapt to dynamic malware behavior and are particularly vulnerable to adversarial attacks, where subtle modifications evade detection mechanisms. To address these limitations, MIDALF—a novel multimodal image and audio late fusion approach for malware detection is proposed. Unlike previous works that rely on single-modal or static features, MIDALF transforms malware binaries into image and audio representations. Self-supervised learning is employed to extract robust features from image representations, while convolutional neural networks analyze audio spectrograms. To integrate these diverse modalities, five late fusion techniques were explored, identifying logistic regression as the most effective. The proposed approach achieves an accuracy of 99.7% in classifying malware and benign samples on the BODMAS dataset, significantly outperforming baseline methods (p < 0.0001, Wilcoxon signed-rank test). Furthermore, MIDALF demonstrates strong robustness against adversarial malware generated using generative adversarial networks (GANs), maintaining a detection accuracy of 95.1% under adversarial conditions. These findings highlight MIDALF's ability to address limitations in existing works by leveraging multimodal representations, integrating complementary data features, and enhancing resilience against adversarial attacks. This study offers a novel pathway towards more effective and robust malware detection systems. [ABSTRACT FROM AUTHOR]

Data sparsity caused by limited interactions makes it challenging for recommendation to accurately capture user preferences. Contrastive learning effectively alleviates this issue by enriching embedding information through the learning of diverse contrastive views. The effectiveness of contrastive learning in uncovering users' and items' latent preferences largely depends on the construction of data augmentation strategies. Structure and feature perturbations are commonly used augmentation strategies in graph-based contrastive learning. Since graph structure augmentation is time consuming and can disrupt interaction information, this paper proposes a novel feature augmentation contrastive learning method. This approach leverages preference prototypes to guide user and item embeddings in acquiring augmented information. By generating refined prototypes for each user and item based on existing prototypes to better approximate true preferences, it effectively alleviates the over-smoothing issue within similar preferences. To balance feature augmentation, a prototype filtering network is employed to control the flow of prototype information, ensuring consistency among different embeddings. Compared with existing prototype-based methods, ProtoRec achieves maximum gains of up to 16.8% and 20.0% in recall@k and NDCG@k on the Yelp2018, Douban-Book, and Amazon-Book datasets. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Early fault detection technologies play a decisive role in preventing equipment failures in industrial production. The primary challenges in early fault detection for industrial applications include the severe imbalance of time-series data, where normal operating data vastly outnumber anomalous data, and in some cases, anomalies may be virtually absent. Additionally, the frequent changes in operational modes during machinery operation further complicate the detection process, making it difficult to effectively identify faults across varying conditions. This study proposes a bearing early anomaly detection method based on contrastive learning and reconstruction approaches to address the aforementioned issues. The raw time-domain vibration data, which were collected from sensors mounted on the bearings of the machinery, are first preprocessed using the Ricker wavelet transform to effectively remove noise and extract useful signal components. These processed signals are then fed into a BYOL-based contrastive learning network to learn more discriminative global feature representations. In addition, we design the reconstruction loss to complement contrastive learning. By reconstructing the masked original data, the reconstruction loss forces the model to learn detailed information, thereby emphasizing the preservation and restoration of local details. Our model not only eliminates the reliance on negative samples found in mainstream unsupervised methods but also captures data features more comprehensively, achieving superior fault detection accuracy under different operating conditions compared to related methods. Experiments on the widely used CWRU multi-condition-bearing fault dataset demonstrate that our method achieves an average fault detection accuracy of 96.97%. Moreover, the experimental results show that on the full-cycle IMS dataset, our method detects early faults at least 2.3 h earlier than the other unsupervised methods. Furthermore, the validation results for the full-cycle XJTU-SY dataset further demonstrate its excellent generalization ability. [ABSTRACT FROM AUTHOR]

Traditional approaches for human monitoring and motion recognition often rely on wearable sensors, which, while effective, are obtrusive and cause significant discomfort to workers. More recent approaches have employed unobtrusive, real-time sensing using cameras mounted in the manufacturing environment. While these methods generate large volumes of rich data, they require extensive labeling and analysis for machine learning applications. Additionally, these cameras frequently capture irrelevant environmental information, which can hinder the performance of deep learning algorithms. To address these limitations, this paper introduces a novel framework that leverages a contrastive learning approach to learn rich representations from raw images without the need for manual labeling. This framework mitigates the effect of environmental complexity by focusing on critical joint coordinates relevant to manufacturing tasks. This approach ensures that the model learns directly from human-specific data, effectively reducing the impact of the surrounding environment. A custom dataset of human subjects simulating various tasks in a workplace setting is used for training and evaluation. By fine-tuning the learned model for a downstream motion classification task, we achieve up to 90% accuracy, demonstrating the effectiveness of our proposed solution in real-time human motion monitoring. [ABSTRACT FROM AUTHOR]

Multiplexed positron emission tomography (mPET) imaging allows simultaneous observation of physiological and pathological information from multiple tracers in a single PET scan. Although supervised deep learning has demonstrated superior performance in mPET image separation compared to purely model-based methods, acquiring large amounts of paired single-tracer data and multi-tracer data for training poses a practical challenge and needs extended scan durations for patients. In addition, the generalisation ability of the supervised learning framework is a concern, as the patient being scanned and their tracer kinetics may potentially fall outside the training distribution. In this work, we propose a self-supervised learning framework based on the deep image prior (DIP) for mPET image separation using just one dataset. In particular, we integrate the multi-tracer compartmental model into the DIP framework to estimate the parametric maps of each tracer from the measured dynamic dual-tracer activity images. Consequently, the separated dynamic single-tracer activity images can be recovered from the estimated tracer-specific parametric maps. In the proposed method, dynamic dual-tracer activity images are used as the training label, and the static dual-tracer image (reconstructed from the same patient data from the start to the end of acquisition) is used as the network input. The performance of the proposed method was evaluated on a simulated brain phantom for dynamic dual-tracer [18F]FDG+[11C]MET activity image separation and parametric map estimation. The results demonstrate that the proposed method outperforms the conventional voxel-wise multi-tracer compartmental modeling method (vMTCM) and the two-step method DIP-Dn+vMTCM (where dynamic dual-tracer activity images are first denoised using a U-net within the DIP framework, followed by vMTCM separation) in terms of lower bias and standard deviation in the separated single-tracer images and also for the estimated parametric maps for each tracer, at both voxel and ROI levels. [ABSTRACT FROM AUTHOR]

Motion forecasting presents significant challenges critical for ensuring the safety of autonomous driving systems. The accuracy of these forecasts relies heavily on factors such as map topology and the behaviors of vehicles and pedestrians. However, within vast datasets, certain features with unique properties, capable of enhancing representation generalization often remain hidden and overlooked. While self-supervised learning (SSL) has shown promise in uncovering such hidden features through pretext tasks, its application to motion forecasting remains underexplored. In this paper, we propose a novel self-supervised motion forecasting method that exploits the interaction of map topology and actors’ maneuvers within localized focal points to generate more informative and generalizable representations for forecasting task. Since intersections, characterized by intricate structures and frequent motion state changes among actors, serve as pivotal locations where the topology of the intersection map profoundly influences actors’ intentions to change course, we leverage this interplay by calculating map structure-based actors’ attributes, and actors’ maneuver-based map attributes. These attributes yield significant advantages for motion forecasting tasks. Experimentally, our proposed method outperforms the baseline on both the challenging large-scale Argoverse benchmark (Chang et al. 2019) and local test, which demonstrates the effectiveness of the fusion of cross-domain information in a local neighborhood. [ABSTRACT FROM AUTHOR]

In this paper, we propose a self-supervised learning method of point cloud completion for indoor scenes. Considering the limited view of single-view image and the time-consuming and labor-intensive acquisition of multi-view images, we take panoramas as input, which makes the acquisition easier and the scope of the scene wider. As it is difficult to obtain complete scene point cloud, we design an auxiliary task to simulate scene missing area by shifting viewpoint of panorama and extract the supervision information of the scene itself. Given the difficulty to complete large-scale scene point cloud, we design a neighborhood integration and feature spreading module for feature extraction and reservation before substantial point cloud downsampling, enabling the completion network to handle large-scale point cloud. Then we propose a transformer-based scene point cloud completion network and show competitive completion results compared to relevant supervised learning methods. [ABSTRACT FROM AUTHOR]

Introduction: With an enormous number of hand images generated over time, leveraging unlabeled images for pose estimation is an emerging yet challenging topic. While some semi-supervised and self-supervised methods have emerged, they are constrained by their reliance on high-quality keypoint detection models or complicated network architectures. Methods: We propose a novel selfsupervised pretraining strategy for 3D hand mesh regression. Our approach integrates a multi-granularity strategy with pseudo-keypoint alignment in a teacher–student framework, employing self-distillation and masked image modeling for comprehensive representation learning. We pair this with a robust pose estimation baseline, combining a standard vision transformer backbone with a pyramidal mesh alignment feedback head. Results: Extensive experiments demonstrate HandMIM's competitive performance across diverse datasets, notably achieving an 8.00 mm Procrustes alignment vertex-point-error on the challenging HO3Dv2 test set, which features severe hand occlusions, surpassing many specially optimized architectures. [ABSTRACT FROM AUTHOR]

Background: Malocclusion, characterized by dental misalignment and improper occlusal relationships, significantly impacts oral health and daily functioning, with a global prevalence of 56%. Lateral cephalogram is a crucial diagnostic tool in orthodontic treatment, providing insights into various structural characteristics. Methods: This study introduces a pre-training approach using multi-center lateral cephalograms for self-supervised learning, aimed at improving model generalization across diverse clinical data domains. Additionally, a multi-attribute classification network is proposed, leveraging attribute correlations to optimize parameters and enhance classification performance. Results: Comprehensive evaluation on both public and clinical datasets showcases the superiority of the proposed framework, achieving an impressive average accuracy of 90.02%. The developed Self-supervised Pre-training and Multi-Attribute (SPMA) network achieves a best match ratio (MR) score of 71.38% and a low Hamming loss (HL) of 0.0425%, demonstrating its efficacy in orthodontic diagnosis from lateral cephalograms. Conclusions: This work contributes significantly to advancing automated diagnostic tools in orthodontics, addressing the critical need for accurate and efficient malocclusion diagnosis. The outcomes not only improve the efficiency and accuracy of diagnosis, but also have the potential to reduce healthcare costs associated with orthodontic treatments. [ABSTRACT FROM AUTHOR]

Introduction: Accurate recognition of martial arts leg poses is essential for applications in sports analytics, rehabilitation, and human-computer interaction. Traditional pose recognition models, relying on sequential or convolutional approaches, often struggle to capture the complex spatial-temporal dependencies inherent in martial arts movements. These methods lack the ability to effectively model the nuanced dynamics of joint interactions and temporal progression, leading to limited generalization in recognizing complex actions. Methods: To address these challenges, we propose PoseGCN, a Graph Convolutional Network (GCN)-based model that integrates spatial, temporal, and contextual features through a novel framework. PoseGCN leverages spatial-temporal graph encoding to capture joint motion dynamics, an action-specific attention mechanism to assign importance to relevant joints depending on the action context, and a self-supervised pretext task to enhance temporal robustness and continuity. Experimental results on four benchmark datasets—Kinetics-700, Human3.6M, NTU RGB+D, and UTD-MHAD—demonstrate that PoseGCN outperforms existing models, achieving state-of-the-art accuracy and F1 scores. Results and discussion: These findings highlight the model's capacity to generalize across diverse datasets and capture fine-grained pose details, showcasing its potential in advancing complex pose recognition tasks. The proposed framework offers a robust solution for precise action recognition and paves the way for future developments in multi-modal pose analysis. [ABSTRACT FROM AUTHOR]

Background: Diabetic retinopathy (DR) is the leading cause of blindness among working-age adults. Early detection is crucial to reducing DR-related vision loss risk but is fraught with challenges. Manual detection is labor-intensive and often misses tiny DR lesions, necessitating automated detection. Objective: We aimed to develop and validate an annotation-free deep learning strategy for the automatic detection of exudates and bleeding spots on color fundus photography (CFP) images and ultrawide field (UWF) retinal images. Materials and Methods: Three cohorts were created: two CFP cohorts (Kaggle-CFP and E-Ophtha) and one UWF cohort. Kaggle-CFP was used for algorithm development, while E-Ophtha, with manually annotated DR-related lesions, served as the independent test set. For additional independent testing, 50 DR-positive cases from both the Kaggle-CFP and UWF cohorts were manually outlined for bleeding and exudate spots. The remaining cases were used for algorithm training. A multiscale contrast-based shape descriptor transformed DR-verified retinal images into contrast fields. High-contrast regions were identified, and local image patches from abnormal and normal areas were extracted to train a U-Net model. Model performance was evaluated using sensitivity and false positive rates based on manual annotations in the independent test sets. Results: Our trained model on the independent CFP cohort achieved high sensitivities for detecting and segmenting DR lesions: microaneurysms (91.5%, 9.04 false positives per image), hemorrhages (92.6%, 2.26 false positives per image), hard exudates (92.3%, 7.72 false positives per image), and soft exudates (90.7%, 0.18 false positives per image). For UWF images, the model's performance varied by lesion size. Bleeding detection sensitivity increased with lesion size, from 41.9% (6.48 false positives per image) for the smallest spots to 93.4% (5.80 false positives per image) for the largest. Exudate detection showed high sensitivity across all sizes, ranging from 86.9% (24.94 false positives per image) to 96.2% (6.40 false positives per image), though false positive rates were higher for smaller lesions. Conclusions: Our experiments demonstrate the feasibility of training a deep learning neural network for detecting and segmenting DR-related lesions without relying on their manual annotations. [ABSTRACT FROM AUTHOR]

Finding a template location in a query image is a fundamental problem in many computer vision applications, such as localization of known objects, image registration, image matching, and object tracking. Currently available methods fail when insufficient training data are available or big variations in the textures, different modalities, and weak visual features exist in the images, leading to limited applications on real-world tasks. We introduce Self-Supervised Foundation Model for Template Matching (Self-TM), a novel end-to-end approach to self-supervised learning template matching. The idea behind Self-TM is to learn hierarchical features incorporating localization properties from images without any annotations. As going deeper in the convolutional neural network (CNN) layers, their filters begin to react to more complex structures and their receptive fields increase. This leads to loss of localization information in contrast to the early layers. The hierarchical propagation of the last layers back to the first layer results in precise template localization. Due to its zero-shot generalization capabilities on tasks such as image retrieval, dense template matching, and sparse image matching, our pre-trained model can be classified as a foundation one. [ABSTRACT FROM AUTHOR]

Automatically labeling trajectories of multiple agents is key to behavioral analyses but usually requires a large amount of manual annotations. This also applies to the domain of team sport analyses. In this paper, we specifically show how pretraining transformer models improves the classification performance on tracking data from professional soccer. For this purpose, we propose a novel self-supervised masked autoencoder for multiagent trajectories to effectively learn from only a few labeled sequences. Our approach builds upon a factorized transformer architecture for multiagent trajectory data and employs a masking scheme on the level of individual agent trajectories. As a result, our model allows for a reconstruction of masked trajectory segments while being permutation equivariant with respect to the agent trajectories. In addition to experiments on soccer, we demonstrate the usefulness of the proposed pretraining approach on multiagent pose data from entomology. In contrast to related work, our approach is conceptually much simpler, does not require handcrafted features and naturally allows for permutation invariance in downstream tasks. [ABSTRACT FROM AUTHOR]

Although vision transformer has demonstrated superior performance over convolutional networks in computer vision, its potential in iris recognition remains underexplored, primarily due to the neglect of domain-specific iris structure. To bridge this gap, we propose a second-order transformer called RAformer that consists of R&A transformer blocks and affinity pooling stacked in a pyramid structure, which can effectively capture radial and angular structure of iris texture in an efficient manner. In realistic scenarios, however, the appearance changing due to various uncertain acquisition factors inevitably deteriorates model performance of conventional methods, which heavily rely on less diverse data to handle real-world appearance shifts in iris patterns. To address this challenge, we propose a joint style transfer based normalization and variational prototype learning for appearance-shift robust iris recognition. Concretely, by being the first to treat the appearance-changing as fine-grained style shift, we introduce the identity-aware permuted adaptive instance normalization (pAdaIN) module from the perspective of style transfer, which efficiently swaps the intermediate instance-level feature statistics of samples within a batch to construct diverse appearances. Additionally, to allow the model to bear such great style variations, we introduce stochastic neural network (SNN) classifier based on variational prototype learning, in which each class is represented by a distribution rather than a point in the traditional softmax classifier. They are integrated into a joint learning framework and benefit each other. To further adapt the model for data-scarce scenarios, we are the first to introduce self-supervision as an auxiliary task in a joint learning manner, enabling the model to learn richer and more transferable visual representations even with limited labeled samples. Consequently, with self-supervision, our RAformer can be naturally extended to a more practical scenario called semi-supervised iris recognition, where only a few source data are labeled while the majority of them are unlabeled. We report consistent improvements across four challenging iris benchmarks, with more significant advantages over prior arts in semi-supervised learning settings, establishing new state-of-the-art (SOTA) results. Additionally, even without any fine-tuning, the well-trained model from our framework is promising to work directly in deployment environments covering iris acquisitions with different forms of distribution shifts, which strongly demonstrates the superiority and generalization ability of our proposed method. [ABSTRACT FROM AUTHOR]

In heterogeneous graph analysis, existing self-supervised learning (SSL) methods face several key challenges. Primarily, these approaches are tailored for node-level tasks and fail to effectively capture global graph-level features, a crucial aspect for comprehensive graph understanding. Furthermore, they predominantly rely on meta-path-based techniques to unravel graph structures, a process that can be computationally intensive and often intractable for complex networks. Another significant limitation is their inability to account for nonpairwise relationships, a common characteristic in real-world networks like protein-protein interaction and collaboration networks, limiting their effectiveness in graph-level learning where high-order connectivity is essential. To address these issues, we propose an innovative SSL framework for heterogeneous hypergraph embedding, expressly designed to enhance graph-level classification. Our framework introduces multi-hop attention in hypergraph convolution, a significant leap from existing attention mechanisms specifically for hypergraphs that primarily focus on immediate neighborhoods. This multi-hop approach allows for an expansive capture of relational structures, both near and far, uncovering intricate patterns integral to accurate graph-level classification. Complementing this, we implement an informative graph-level attentive pooling mechanism that surpasses traditional aggregation methods. It intelligently synthesizes features, taking into account their structural and semantic importance within the hypergraph, thereby preserving critical contextual information. Furthermore, we refine our contrastive learning approach and introduce targeted negative sampling strategies, creating a more robust learning environment that excels at discerning nuanced graph-level features. Rigorous evaluation against established graph kernels, graph neural networks, and graph pooling methods on real-world datasets demonstrates our model's superior performance, validating its effectiveness in addressing the complexities inherent in heterogeneous graph-level classification. [ABSTRACT FROM AUTHOR]

Airborne magnetic anomaly detection is an important passive remote sensing technique. However, since the magnetic field caused by the aircraft interferes with the detection accuracy, this part of interference should be eliminated by an aeromagnetic compensation method. Most existing compensation methods assume that the ambient magnetic field is uniform when calculating the compensation model parameters. However, as the ambient magnetic field is actually not uniform and varies with the aircraft location, the solved parameters ignore the part of aircraft interference related to the varied ambient magnetic field. Although some of the latest deep learning-based aeromagnetic compensation methods avoid the assumption of uniformity of the ambient magnetic field, the insufficient supervision leads to a poor model generalization. To address these limitations, we propose a self-supervised compensation method. The proposed method utilizes a network to separate the total measured magnetic field into the ambient magnetic field part and the aircraft magnetic field part. By doing so, the method avoids the influence of the uniform ambient magnetic field assumption and enhances the model generalization. In addition, we introduce an improvement ratio loss function to distinguish the aircraft magnetic field from the ambient magnetic field when updating the model parameters. The proposed method is verified using measurement data from real flights. The experimental results indicate that the proposed method significantly outperforms state-of-the-art methods in real flights compensation. [ABSTRACT FROM AUTHOR]

Feature point detection in inverse synthetic aperture radar (ISAR) images of space targets is the foundation for tasks such as analyzing space target motion intent and predicting on-orbit status. Traditional feature point detection methods perform poorly when confronted with the low texture and uneven brightness characteristics of ISAR images. Due to the nonlinear mapping capabilities, neural networks can effectively learn features from ISAR images of space targets, providing new ideas for feature point detection. However, the scarcity of labeled ISAR image data for space targets presents a challenge for research. To address the issue, this paper introduces a self-supervised feature point detection method (SFPD), which can accurately detect the positions of feature points in ISAR images of space targets without true feature point positions during the training process. Firstly, this paper simulates an ISAR primitive dataset and uses it to train the proposed basic feature point detection model. Subsequently, the basic feature point detection model and affine transformation are utilized to label pseudo-ground truth for ISAR images of space targets. Eventually, the labeled ISAR image dataset is used to train SFPD. Therefore, SFPD can be trained without requiring ground truth for the ISAR image dataset. The experiments demonstrate that SFPD has better performance in feature point detection and feature point matching than usual algorithms. [ABSTRACT FROM AUTHOR]

As the domain of artificial intelligence has advanced, the integration of deep learning techniques into terrain and landform analysis has become more prevalent. Nevertheless, many existing methods are fully supervised and designed for specific tasks; thus, their transferability is limited and massive annotated samples are required. This study introduces a geomorphic pretrained model (GeomorPM) capable of performing multiple tasks. First, an architecture was designed that combined a convolution-based Vector Quantised-Variational Autoencoder (VQVAE) with a Transformer-based masked autoencoder (MAE) framework, allowing it to autonomously learn local details and global patterns from large-scale digital elevation model (DEM) data. Subsequently, GeomorPM, based on the VQMAE architecture, was pretrained on massive DEM data and fine-tuned for three specific tasks: DEM void filling, DEM superresolution, and landform classification. GeomorPM outperformed the traditional and other deep learning methods in all three tasks, demonstrating the superior learning ability and transferability of the model. This study provides a practical framework for developing pretrained models based on DEMs that can be expanded to other continuous geoscientific data. [ABSTRACT FROM AUTHOR]

Purpose: To introduce a new method for generalized RF pulse design using physics‐guided self‐supervised learning (GPS), which uses the Bloch equations as the guiding physics model. Theory and Methods: The GPS framework consists of a neural network module and a physics module, where the physics module is a Bloch simulator for MRI applications. For RF pulse design, the neural network module maps an input target profile to an RF pulse, which is subsequently loaded into the physics module. Through the supervision of the physics module, the neural network module designs an RF pulse corresponding to the target profile. GPS was applied to design 1D selective, B1$$ {B}_1 $$‐insensitive, saturation, and multidimensional RF pulses, each conventionally requiring dedicated design algorithms. We further demonstrate our method's flexibility and versatility by compensating for experimental and scanner imperfections through online adaptation. Results: Both simulations and experiments show that GPS can design a variety of RF pulses with corresponding profiles that agree well with the target input. Despite these verifications, GPS‐designed pulses have unique differences compared to conventional designs, such as achieving B1$$ {B}_1 $$‐insensitivity using different mechanisms and using non‐sampled regions of the conventional design to lower its peak power. Experiments, both ex vivo and in vivo, further verify that it can also be used for online adaptation to correct system imperfections, such as B0$$ {B}_0 $$/B1+$$ {B}_1^{+} $$ inhomogeneity. Conclusion: This work demonstrates the generalizability, versatility, and flexibility of the GPS method for designing RF pulses and showcases its utility in several applications. [ABSTRACT FROM AUTHOR]

The ubiquitous availability of smartphones and smartwatches with integrated inertial measurement units (IMUs) enables straightforward capturing of human activities through collecting movement data. For specific applications of sensor-based human activity recognition (HAR), however, logistical challenges and burgeoning costs render especially the ground-truth annotation of such data a difficult endeavor, resulting in limited scale and diversity of datasets available for deriving effective HAR systems and less than ideal recognition capabilities. Transfer learning, i.e., leveraging publicly available labeled datasets to first learn useful representations that can then be fine-tuned using limited amounts of labeled data from a target domain, can alleviate some of the performance issues of contemporary HAR systems. Yet they can fail when the differences between source and target conditions are too large and/or only few samples from a target application domain are available—each of which are typical challenges in real-world human activity recognition scenarios. In this article, we present an approach for economic use of publicly available labeled HAR datasets for effective transfer learning. We introduce a novel transfer learning framework—Cross-Domain HAR—which follows the teacher-student self-training paradigm to more effectively recognize activities with very limited label information. It bridges conceptual gaps between source and target domains, including sensor locations and type of activities. Cross-Domain HAR enables substantial performance improvements over the state-of-the-art in sensor-based HAR scenarios. Through our extensive experimental evaluation on a range of benchmark datasets we specifically demonstrate the effectiveness of our approach for practically relevant few-shot activity recognition scenarios. We also present a detailed analysis into how the individual components of our framework affect downstream performance and provide practical suggestions for using the framework in real-world applications. [ABSTRACT FROM AUTHOR]

Self-supervised learning has gained popularity for reducing the cost of large-scale dataset labeling while improving model generalization and representation. Among self-supervised learning techniques, masked learning is a prominent approach. However, current masking methods typically use regular small block masking after data augmentation, which is not truly random and can degrade the local correlation between image chunks. This paper proposes a novel self-supervised learning method based on spatially selected shifts and irregular image masks (SSIM). The method generates irregular images by threshold binarization, randomly masks the input image, and then performs spatially selective shifting and aggregated input position information operations. This approach not only avoids fixed mask shapes but also preserves and enhances the local correlation between image chunks. We benchmark our method using the DINO model, applying irregular random masking and spatial selective shifting. Experiments on the Imagenet10 dataset show improvements in linear and k-NN accuracy by 7.6% and 5.7%, respectively. The results demonstrate that SSIM outperforms existing self-supervised learning methods using masks. The code of this paper has been open source. [ABSTRACT FROM AUTHOR]

This paper proposes an enhanced contrastive ensemble learning method for anomaly sound detection. The proposed method achieves approximately 6% in the AUC metric in some categories and achieves state-of-the-art performance among self-supervised models on multiple benchmark datasets. The proposed method is effective in automatically monitoring the operating conditions of the production equipment by detecting the sounds emitted by the machine, to provide an early warning of potential production accidents. This method can significantly reduce industrial monitoring costs and increase monitoring efficiency to improve manufacturing facility productivity effectively. Existing detection methods face challenges with data imbalance caused by the scarcity of anomalous samples, leading to performance degradation. This paper proposes an enhanced data augmentation method that improves model robustness by allowing the data to retain the original features while adding noise close to the real environment through a simple operation. Secondly, model feature extraction is enhanced by using channel attention to fuse time-frequency features. Thirdly, this paper proposes a simple anomaly sample generation method, which can automatically generate real pseudo anomaly samples to help the model gain anomaly detection capability and reduce the impact of data imbalance. Finally, this paper proposes a statistical-based bias compensation that further mitigates the impact of data imbalance by distributing samples through statistical induction. Experimental verification confirms that these changes enhance anomalous sound detection capability. [ABSTRACT FROM AUTHOR]

While traditional self-supervised learning methods improve performance and robustness across various medical tasks, they rely on single-vector embeddings that may not capture fine-grained concepts such as anatomical structures or organs. The ability to identify such concepts and their characteristics without supervision has the potential to improve pre-training methods, and enable novel applications such as fine-grained image retrieval and concept-based outlier detection. In this paper, we introduce ConceptVAE, a novel pre-training framework that detects and disentangles fine-grained concepts from their style characteristics in a self-supervised manner. We present a suite of loss terms and model architecture primitives designed to discretise input data into a preset number of concepts along with their local style. We validate ConceptVAE both qualitatively and quantitatively, demonstrating its ability to detect fine-grained anatomical structures such as blood pools and septum walls from 2D cardiac echocardiographies. Quantitatively, ConceptVAE outperforms traditional self-supervised methods in tasks such as region-based instance retrieval, semantic segmentation, out-of-distribution detection, and object detection. Additionally, we explore the generation of in-distribution synthetic data that maintains the same concepts as the training data but with distinct styles, highlighting its potential for more calibrated data generation. Overall, our study introduces and validates a promising new pre-training technique based on concept-style disentanglement, opening multiple avenues for developing models for medical image analysis that are more interpretable and explainable than black-box approaches. [ABSTRACT FROM AUTHOR]

Accurate and efficient localization of root cause instances in large-scale microservice systems is of paramount importance. Unfortunately, prevailing methods face several limitations. Notably, some recent methods rely on supervised learning which necessitates a substantial amount of labeled data. However, labeling root cause instances is time-consuming and laborious, especially with multiple modalities of data including logs, traces, metrics, and so on. Moreover, some approaches favor deep learning for localization but lack interpretability and continuous improvement mechanisms. To address the above challenges, we propose DeepHunt, a novel root cause localization method based on multimodal data analysis. Firstly, DeepHunt introduces root cause score (RCS) by integrating reconstruction errors and failure propagation patterns (upstream–downstream relationships), imparting interpretability to the localization of root causes. Then, it embraces graph autoencoder (GAE) to address the limitation imposed by scarce labeled data. It employs data augmentation to mitigate the adverse effects of insufficient historical training samples. We evaluate DeepHunt on two open source datasets, and it outperforms existing methods when facing a zero-label cold start. DeepHunt can be further improved by continuously fine-tuning through a feedback mechanism. [ABSTRACT FROM AUTHOR]

Deep-learning-based MR image reconstruction in settings where large fully sampled dataset collection is infeasible requires methods that effectively use both under-sampled and fully sampled datasets. This paper evaluates a weakly supervised, multi-coil, physics-guided approach to MR image reconstruction, leveraging both dataset types, to improve both the quality and robustness of reconstruction. A physics-guided end-to-end variational network (VarNet) is pretrained in a self-supervised manner using a 4 × under-sampled dataset following the self-supervised learning via data undersampling (SSDU) methodology. The pre-trained weights are transferred to another VarNet, which is fine-tuned using a smaller, fully sampled dataset by optimizing multi-scale structural similarity (MS-SSIM) loss in image space. The proposed methodology is compared with fully self-supervised and fully supervised training. Reconstruction quality improvements in SSIM, PSNR, and NRMSE when abundant training data is available (the high-data regime), and enhanced robustness when training data is scarce (the low-data regime) are demonstrated using weak supervision for knee and brain MR image reconstructions at 8 × and 10 × acceleration, respectively. Multi-coil physics-guided MR image reconstruction using both under-sampled and fully sampled datasets is achievable with transfer learning and fine-tuning. This methodology can provide improved reconstruction quality in the high-data regime and improved robustness in the low-data regime at high acceleration rates. [ABSTRACT FROM AUTHOR]

Artificial Intelligence (AI) has emerged as a promising solution for real-time monitoring of the quality of additively manufactured (AM) metallic parts. This study focuses on the Laser-based Directed Energy Deposition (L-DED) process and utilizes embedded vision systems to capture critical melt pool characteristics for continuous monitoring. Two self-learning frameworks based on Convolutional Neural Networks and Transformer architecture are applied to process zone images from different DED process regimes, enabling in-situ monitoring without ground truth information. The evaluation is based on a dataset of process zone images obtained during the deposition of titanium powder (Cp-Ti, grade 1), forming a cube geometry using four laser regimes. By training and evaluating the Deep Learning (DL) algorithms using a co-axially mounted Charged Couple Device (CCD) camera within the process zone, the down-sampled representations of process zone images are effectively used with conventional classifiers for L-DED process monitoring. The high classification accuracies achieved validate the feasibility and efficacy of self-learning strategies in real-time quality assessment of AM. This study highlights the potential of AI-based monitoring systems and self-learning algorithms in quantifying the quality of AM metallic parts during fabrication. The integration of embedded vision systems and self-learning algorithms presents a novel contribution, particularly in the context of the L-DED process. The findings open avenues for further research and development in AM process monitoring, emphasizing the importance of self-supervised in situ monitoring techniques in ensuring part quality during fabrication. [ABSTRACT FROM AUTHOR]

Compressed sensing (CS) is a promising tool for reducing sampling costs. Current deep neural network (NN)-based CS approaches face the challenges of collecting labeled measurement-ground truth (GT) data and generalizing to real applications. This paper proposes a novel Self-supervised sCalable deep CS method, comprising a deep Learning scheme called SCL and a family of Networks named SCNet, which does not require GT and can handle arbitrary sampling ratios and matrices once trained on a partial measurement set. Our SCL contains a dual-domain loss and a four-stage recovery strategy. The former encourages a cross-consistency on two measurement parts and a sampling-reconstruction cycle-consistency regarding arbitrary ratios and matrices to maximize data utilization. The latter can progressively leverage the common signal prior in external measurements and internal characteristics of test samples and learned NNs to improve accuracy. SCNet combines both the explicit guidance from optimization algorithms and the implicit regularization from advanced NN blocks to learn a collaborative signal representation. Our theoretical analyses and experiments on simulated and real captured data, covering 1-/2-/3-D natural and scientific signals, demonstrate the effectiveness, superior performance, flexibility, and generalization ability of our method over existing self-supervised methods and its significant potential in competing against many state-of-the-art supervised methods. Code is available at https://github.com/Guaishou74851/SCNet. [ABSTRACT FROM AUTHOR]

Research on reconstructing imperfect faces is a challenging task. In this study, we explore a data-driven approach using a pre-trained MICA (MetrIC fAce) model combined with 3D printing to address this challenge. We propose a training strategy that utilizes the pre-trained MICA model and self-supervised learning techniques to improve accuracy and reduce the time needed for 3D facial structure reconstruction. Our results demonstrate high accuracy, evaluated by the geometric loss function and various statistical measures. To showcase the effectiveness of the approach, we used 3D printing to create a model that covers facial wounds. The findings indicate that our method produces a model that fits well and achieves comprehensive 3D facial reconstruction. This technique has the potential to aid doctors in treating patients with facial injuries. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Shanghai Jiaotong University (Science) is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Smart cities deploy various sensors such as microphones and RGB cameras to collect data to improve the safety and comfort of the citizens. As data annotation is expensive, self-supervised methods such as contrastive learning are used to learn audio-visual representations for downstream tasks. Focusing on surveillance data, we investigate two common limitations of audio-visual contrastive learning: false negatives and the minimal sufficient information bottleneck. Irregular, yet frequently recurring events can lead to a considerable number of false-negative pairs and disrupt the model's training. To tackle this challenge, we propose a novel method for generating contrastive pairs based on the distance between embeddings of different modalities, rather than relying solely on temporal cues. The semantically synchronized pairs can then be used to ease the minimal sufficient information bottleneck along with the new loss function for multiple positives. We experimentally validate our approach on real-world data and show how the learnt representations can be used for different downstream tasks, including audio-visual event localization, anomaly detection, and event search. Our approach reaches similar performance as state-of-the-art modality- and task-specific approaches. [ABSTRACT FROM AUTHOR]

Similar to traditional machine learning, federated learning is susceptible to adversarial attacks. Existing defense methods against federated attacks often rely on extensive labeling during the local training process to enhance model robustness. However, labeling typically requires significant resources. To address the challenges posed by expensive labeling and the robustness issues in federated learning, we propose the Unsupervised Perturbation based Self-Supervised Federated Adversarial Training (UPFAT) framework. Within local clients, we introduce an innovative unsupervised adversarial sample generation method, which adapts the classical self-supervised framework BYOL (Bootstrap Your Own Latent). This method maximizes the distances between embeddings of various transformations of the same input, generating unsupervised adversarial samples aimed at confusing the model. For model communication, we present the Robustness-Enhanced Moving Average (REMA) module, which adaptively utilizes global model updates based on the local model’s robustness.Extensive experiments demonstrate that UPFAT outperforms existing methods by 3 ∼ 4 % . [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Hyperspectral images are typically composed of hundreds of narrow and contiguous spectral bands, each containing information regarding the material composition of the imaged scene. However, these images can be affected by various sources of noise, distortions, or data loss, which can significantly degrade their quality and usefulness. This paper introduces a convergent guaranteed algorithm, LRS-PnP-DIP(1-Lip), which successfully addresses the instability issue of DHP that has been reported before. The proposed algorithm extends the successful joint low-rank and sparse model to further exploit the underlying data structures beyond the conventional and sometimes restrictive unions of subspace models. A stability analysis guarantees the convergence of the proposed algorithm under mild assumptions, which is crucial for its application in real-world scenarios. Extensive experiments demonstrate that the proposed solution consistently delivers visually and quantitatively superior inpainting results, establishing state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Monocular depth estimation (MDE) is a critical task in computer vision with applications in autonomous driving, robotics, and augmented reality. However, predicting depth from a single image poses significant challenges, especially in dynamic scenes where moving objects introduce scale ambiguity and inaccuracies. In this paper, we propose the Dynamic Iterative Monocular Depth Estimation (DI-MDE) framework, which integrates an iterative refinement process with a novel scale-alignment module to address these issues. Our approach combines elastic depth bins that adjust dynamically based on uncertainty estimates with a scale-alignment mechanism to ensure consistency between static and dynamic regions. Leveraging self-supervised learning, DI-MDE does not require ground truth depth labels, making it scalable and applicable to real-world environments. Experimental results on standard datasets such as SUN RGB-D and KITTI demonstrate that our method achieves state-of-the-art performance, significantly improving depth prediction accuracy in dynamic scenes. This work contributes a robust and efficient solution to the challenges of monocular depth estimation, offering advancements in both depth refinement and scale consistency. [ABSTRACT FROM AUTHOR]

In recent years, spatio-temporal graph neural networks (GNNs) have successfully been used to improve traffic prediction by modeling intricate spatio-temporal dependencies in irregular traffic networks. However, these approaches may not capture the intrinsic properties of traffic data and can suffer from overfitting due to their local nature. This paper introduces the Implicit Sensing Self-Supervised learning model (ISSS), which leverages a multi-pretext task framework for traffic flow prediction. By transforming data into an alternative feature space, ISSS effectively captures both specific and general representations through self-supervised tasks, including contrastive learning and spatial jigsaw puzzles. This enhancement promotes a deeper understanding of traffic features, improved regularization, and more accurate representations. Comparative experiments on six datasets demonstrate the effectiveness of ISSS in learning general and discriminative features in both supervised and unsupervised modes. ISSS outperforms existing models, demonstrating its capabilities in improving traffic flow predictions while addressing challenges associated with local operations and overfitting. Comprehensive evaluations across various traffic prediction datasets, have established the validity of the proposed approach. Unsupervised learning scenarios have shown the improvements in RMSE for the METR-LA and PEMSBAY datasets of 0.39 and 0.35 for location-dependent and location-independent tasks, respectively. In supervised learning scenarios, for the same datasets, the improvements were 1.16 for location-dependent tasks and 0.55 for location-independent tasks. [ABSTRACT FROM AUTHOR]

Graph neural networks (GNN) offer an alternative approach to boost the screening effectiveness in drug discovery. However, their efficacy is often hindered by limited datasets. To address this limitation, we introduced a robust GNN training framework, applied to various chemical databases to identify potent non‐nucleoside reverse transcriptase inhibitors (NNRTIs) against the challenging K103N‐mutated HIV‐1 RT. Leveraging self‐supervised learning (SSL) pre‐training to tackle data scarcity, we screened 1,824,367 compounds, using multi‐step approach that incorporated machine learning (ML)‐based screening, analysis of absorption, distribution, metabolism, and excretion (ADME) prediction, drug‐likeness properties, and molecular docking. Ultimately, 45 compounds were left as potential candidates with 17 of the compounds were previously identified as NNRTIs, exemplifying the model's efficacy. The remaining 28 compounds are anticipated to be repurposed for new uses. Molecular dynamics (MD) simulations on repurposed candidates unveiled two promising preclinical drugs: one designed against Plasmodium falciparum and the other serving as an antibacterial agent. Both have superior binding affinity compared to anti‐HIV drugs. This conceptual framework could be adapted for other disease‐specific therapeutics, facilitating the identification of potent compounds effective against both WT and mutants while revealing novel scaffolds for drug design and discovery. [ABSTRACT FROM AUTHOR]

Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing various deep learning architectures and training them in a fully supervised manner. Although significant progress has been achieved by these supervised methods, the longstanding lack of large-scale high-quality labeled data severely hinders their further improvements. Motivated by the recent success of self-supervised learning in computer vision, this paper introduces a self-supervised approach, termed Self-supervised Video Facial Affect Perceiver (SVFAP), to address the dilemma faced by supervised methods. Specifically, SVFAP leverages masked facial video autoencoding to perform self-supervised pre-training on massive unlabeled facial videos. Considering that large spatiotemporal redundancy exists in facial videos, we propose a novel temporal pyramid and spatial bottleneck Transformer as the encoder of SVFAP, which not only largely reduces computational costs but also achieves excellent performance. To verify the effectiveness of our method, we conduct experiments on nine datasets spanning three downstream tasks, including dynamic facial expression recognition, dimensional emotion recognition, and personality recognition. Comprehensive results demonstrate that SVFAP can learn powerful affect-related representations via large-scale self-supervised pre-training and it significantly outperforms previous state-of-the-art methods on all datasets. [ABSTRACT FROM AUTHOR]

In recent years, significant progress has been made in the research of attribute graph clustering. However, existing methods are mostly based on the homogeneity assumption, thereby neglecting the application scenarios of heterogeneous graphs, leading to the loss of high-frequency information and poor clustering results during the clustering process. To address this issue, a novel dual-channel attribute graph clustering (DCAGC) method was proposed. A mixture of Gaussian models was used to predict the homogeneity of node connections and two views of homogeneous and heterogeneous were built, based on this prediction to capture low-frequency and high-frequency information in the graph from different perspectives. Simultaneously, by integrating contrastive learning and clustering, more precise node embeddings were achieved. Compared to other methods, DCAGC demonstrates significant clustering performance when handling heterogeneous graph datasets and exhibits strong resilience to anomalous connections. [ABSTRACT FROM AUTHOR]

Self-supervised learning (SSL) has been incorporated into many state-of-the-art models in various domains, where SSL defines pretext tasks based on unlabeled datasets to learn contextualized and robust representations. Recently, SSL has become a new trend in exploring the representation learning capability in the realm of tabular data, which is more challenging due to not having explicit relations for learning descriptive representations. This survey aims to systematically review and summarize the recent progress and challenges of SSL for non-sequential tabular data (SSL4NS-TD). We first present a formal definition of NS-TD and clarify its correlation to related studies. Then, these approaches are categorized into three groups—predictive learning, contrastive learning, and hybrid learning, with their motivations and strengths of representative methods in each direction. Moreover, application issues of SSL4NS-TD are presented, including automatic data engineering, cross-table transferability, and domain knowledge integration. In addition, we elaborate on existing benchmarks and datasets for NS-TD applications to analyze the performance of existing tabular models. Finally, we discuss the challenges of SSL4NS-TD and provide potential directions for future research. We expect our work to be useful in terms of encouraging more research on lowering the barrier to entry SSL for the tabular domain, and of improving the foundations for implicit tabular data. [ABSTRACT FROM AUTHOR]

Self-supervised contrastive learning (CL) has achieved state-of-the-art performance in representation learning by minimizing the distance between positive pairs while maximizing that of negative ones. Recently, it has been verified that the model learns better representation with diversely augmented positive pairs because they enable the model to be more view-invariant. However, only a few studies on CL have considered the difference between augmented views, and have not gone beyond the hand-crafted findings. In this paper, we first observe that the score-matching function can measure how much data has changed from the original through augmentation. With the observed property, every pair in CL can be weighted adaptively by the difference of score values, resulting in boosting the performance. We show the generality of our method, referred to as ScoreCL, by consistently improving various CL methods, SimCLR, SimSiam, W-MSE, and VICReg, up to 3%p in image classifcation on CIFAR and ImageNet datasets. Moreover, we have conducted exhaustive experiments and ablations, including results on diverse downstream tasks, comparison with possible baselines, and further applications when used with other augmentation methods. We hope our exploration will inspire more research in exploiting the score matching for CL. [ABSTRACT FROM AUTHOR]