Search

Deep learning methods are leveraged in various computer science and artificial intelligence areas, including image classification. Convolutional neural network (CNN) is one of the most widely used deep neural networks for which, several highly effective architectures for image classification have been presented. In this paper, an improved version of the recently introduced CondenseNet is provided as a new network architecture. On the other hand, due to the necessity of reducing the dependence on labeled data in the training process of neural networks, a self-supervised learning method is also proposed for labeling unlabeled images. The results of the experiments show the proper performance of the proposed self-supervised CondenseNet method compared to the basic version of CondenseNet. The experiments are conducted on CIFAR_10 and CIFAR-100 datasets and show better accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Recent single-view reconstruction methods have sought to reconstruct 3D point clouds from images and corresponding silhouette collections alone. However, merely utilizing input images as supervision without any auxiliary methods amplifies the matching ambiguity. To address this issue, we propose a self-supervised 3D point cloud reconstruction method based on generative adversarial network (GAN) inversion. Three novel components are introduced to solve the intrinsic challenges of cross-dimensional inversion. First, we develop a uniform loss to enhance the uniformity of the point clouds generated by the GAN. Second, we devise a coarse-to-fine differentiable point cloud renderer to facilitate accurate projections. Third, we design a pseudo ground-truth pose predictor that can estimate the precise viewpoints of the input images. Experimental results on both synthetic datasets and real-world datasets demonstrate that our approach outperforms existing state-of-the-art 2D supervised reconstruction methods and is comparable to 3D supervised approaches. [ABSTRACT FROM AUTHOR]

Seismic deconvolution is a useful tool in seismic data processing. Classical non-machine learning deconvolution methods usually apply quite a few constraints to both wavelet inversion and reflectivity inversion. Supervised machine learning deconvolution methods often require appropriate training labels. The existing self-supervised machine learning deconvolution methods need a given wavelet, which is a non-blind process. To overcome these issues, we propose a blind deconvolution method based on self-supervised machine learning. This method first estimates an initial zero-phase wavelet by smoothing the amplitude spectrum of averaged seismic data. Then, the loss function of self-supervised machine learning is taken as the error between the observed seismic data and the reconstructed seismic data that come from the convolution of phase-rotated wavelet and reflectivity generated by the network. We utilize a residual neural network with long skip connections as the reflectivity inversion network and a fully connected convolutional neural network as the wavelet phase inversion network. Numerical experiments on synthetic data and field data show that the proposed method can obtain reflectivity inversion results with higher resolution than the existing self-supervised machine learning method without given wavelet. [ABSTRACT FROM AUTHOR]

Enhancing image quality provides more interpretability for both human beings and machines. Traditional image enhancement techniques work well for specific uses, but they struggle with images taken in extreme conditions, such as varied distortions, noise, and contrast deformations. Deep-learning-based methods produce superior quality in enhancing images since they are capable of learning the spatial characteristics within the images. However, deeper models increase the computational costs and require additional modules for particular problems. In this paper, we propose self-supervised and supervised image enhancement models based on the time-shift image enhancement method (TS-IEM). We embedded the TS-IEM into a four-layer CNN model and reconstructed the reference images for the self-supervised model. The reconstructed images are also used in the supervised model as an additional layer to improve the learning process and obtain better-quality images. Comprehensive experiments and qualitative and quantitative analysis are performed using three benchmark datasets of different application domains. The results showed that the self-supervised model could provide reasonable results for the datasets without reference images. On the other hand, the supervised model outperformed the state-of-the-art methods in quantitative analysis by producing well-enhanced images for different tasks. [ABSTRACT FROM AUTHOR]

Navigation of mobile agents in unknown, unmapped environments is a critical task for achieving general autonomy. Recent advancements in combining Reinforcement Learning with Deep Neural Networks have shown promising results in addressing this challenge. However, the inherent complexity of these approaches, characterized by multi-layer networks and intricate reward objectives, limits their autonomy, increases memory footprint, and complicates adaptation to energy-efficient edge hardware. To overcome these challenges, we propose a brain-inspired method that employs a shallow architecture trained by a local learning rule for self-supervised navigation in uncharted environments. Our approach achieves performance comparable to a stateof-the-art Deep Q Network (DQN) method with respect to goal-reaching accuracy and path length, with a similar (slightly lower) number of parameters, operations, and training iterations. Notably, our self-supervised approach combines novelty-based and random walks to alleviate the need for objective reward definition and enhance agent autonomy. At the same time, the shallow architecture and local learning rule do not call for error backpropagation, decreasing the memory overhead and enabling implementation on edge neuromorphic processors. These results contribute to the potential of embodied neuromorphic agents utilizing minimal resources while effectively handling variability. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background. To make the question text represent more information and construct an end-to-end text clustering model, we propose a double-target self-supervised clustering with multi-feature fusion (MF-DSC) for texts which describe questions related to the medical field. Since medical question-and-answer data are unstructured texts and characterized by short characters and irregular language use, the features extracted by a single model cannot fully characterize the text content. Methods. Firstly, word weights were obtained based on term frequency, and word vectors were generated according to lexical semantic information. Then we fused term frequency and lexical semantics to obtain weighted word vectors, which were used as input to the model for deep learning. Meanwhile, a self-attention mechanism was introduced to calculate the weight of each word in the question text, i.e., the interactions between words. To learn fusing cross-document topic features and build an end-to-end text clustering model, two target functions, L cluster and L topic, were constructed and integrated to a unified clustering framework, which also helped to learn a friendly representation that facilitates text clustering. After that, we conducted comparison experiments with five other models to verify the effectiveness of MF-DSC. Results. The MF-DSC outperformed other models in normalized mutual information (NMI), adjusted Rand indicator (ARI) average clustering accuracy (ACC) and F1 with 0.4346, 0.4934, 0.8649 and 0.5737, respectively. [ABSTRACT FROM AUTHOR]

The traditional supervised anomaly detection methods have developed rapidly. In order to reduce the dependence on labels, self-supervised pre-training methods are widely studied, and the studies show that additional intrinsic semantic knowledge embedding is crucial for table learning. In order to mine the rich knowledge information in tabular data, the self-supervised tabular data anomaly detection method based on knowledge enhancement (STKE) is proposed with the following improvements. The proposed data processing module integrates domain knowledge (semantics) and statistical mathematics knowledge into feature construction. At the same time, self-supervised pre-training (parameter learning) provides contextual knowledge priors to achieve the rich information transfer of tabular data. The mask mechanism is used on the original data to learn the masked features by learning the relevant non-masked features, and predict the original value of the additive Gaussian noise in the hidden layer space of the data. This strategy promotes the model even in the presence of noisy inputs. The original feature information can also be recovered. A hybrid attention mechanism is used to effectively extract association information between data features. The experimental results of the proposed method on six datasets show superior performance. [ABSTRACT FROM AUTHOR]

In recent years, self-supervised learning-based models have been widely used for electrocardiogram (ECG) representation learning. However, most of the models utilize contrastive learning that strongly depend on data augmentation. In this paper, we propose a masked self-supervised learning model based on multiview information bottleneck principle. Our method masks the ECG signal instances in the time and frequency domains at a high ratio and then uses the autoencoder to reconstruct the original input. Not only the intra-view relations within each view but also the inter-view relations between two views are exploited in ECG representation learning. Furthermore, we use the multiview information bottleneck principle to remove redundant information in the time and frequency domains, so that the representations of both views contain more task-relevant information. Our model is pre-trained on three larger ECG datasets at once and fine-tuned on each classification task. Experimental results show that our model not only outperforms state-of-the-art models with self-supervised learning, but also outperforms models with supervised learning. [ABSTRACT FROM AUTHOR]

Probe-based confocal laser endoscopy (pCLE) has emerged as a powerful tool for disease diagnosis, yet it faces challenges such as the formation of hexagonal patterns in images due to the inherent characteristics of fiber bundles. Recent advancements in deep learning offer promise in image denoising, but the acquisition of clean-noisy image pairs for training networks across all potential scenarios can be prohibitively costly. Few studies have explored training denoising networks on such pairs. Here, we propose an innovative self-supervised denoising method. Our approach integrates noise prediction networks, image quality assessment networks, and denoising networks in a collaborative, jointly trained manner. Compared to prior self-supervised denoising methods, our approach yields superior results on pCLE images and fluorescence microscopy images. In summary, our novel self-supervised denoising technique enhances image quality in pCLE diagnosis by leveraging the synergy of noise prediction, image quality assessment, and denoising networks, surpassing previous methods on both pCLE and fluorescence microscopy images. [ABSTRACT FROM AUTHOR]

Fluorescence microscopy data describe protein localization patterns at single-cell resolution and have the potential to reveal whole-proteome functional information with remarkable precision. Yet, extracting biologically meaningful representations from cell micrographs remains a major challenge. Existing approaches often fail to learn robust and noise-invariant features or rely on supervised labels for accurate annotations. We developed PIFiA (Protein Image-based Functional Annotation), a self-supervised approach for protein functional annotation from single-cell imaging data. We imaged the global yeast ORF-GFP collection and applied PIFiA to generate protein feature profiles from single-cell images of fluorescently tagged proteins. We show that PIFiA outperforms existing approaches for molecular representation learning and describe a range of downstream analysis tasks to explore the information content of the feature profiles. Specifically, we cluster extracted features into a hierarchy of functional organization, study cell population heterogeneity, and develop techniques to distinguish multi-localizing proteins and identify functional modules. Finally, we confirm new PIFiA predictions using a colocalization assay, suggesting previously unappreciated biological roles for several proteins. Paired with a fully interactive website (https://thecellvision.org/pifia/), PIFiA is a resource for the quantitative analysis of protein organization within the cell. Synopsis: PIFiA is a self-supervised deep-learning approach for protein functional annotation from single-cell images. It generates feature profiles from images of the yeast ORF-GFP collection that can be used in downstream analyses. PIFiA features identify new functional groups of proteins within organelles and proteins with heterogeneous localizations. PIFiA features successfully predict protein–protein interactions and members of protein complexes. PIFiA outperforms previous methods on four different standards of protein function. Images and analysis are available at thecellvision.org/pifia. PIFiA is a self-supervised deep-learning approach for protein functional annotation from single-cell images. It generates feature profiles from images of the yeast ORF-GFP collection that can be used in downstream analyses. [ABSTRACT FROM AUTHOR]

Addressing the issues of low negative sampling efficiency and tendency towards overfitting in existing dense passage retrieval (DPR) models, this paper proposed a DPR model based on query semantic characteristics (Q-DPR). Firstly, it introduced a negative sampling method based on neighbor queries for the negative sampling process. This method constructed high-quality negative samples rapidly by retrieving neighboring queries, thereby reducing the training costs. Secondly, to mitigate overfitting, it proposed a query self-supervised method based on contrastive learning. This method alleviated overfitting to training labels by establishing a self-supervised contrastive loss among queries, thereby enhancing retrieval accuracy. Q-DPR performed exceptionally well on the large-scale MSMARCO dataset for open-domain question answering, achieving a mean reciprocal rank of 0.348 and a recall rate of 0.975. Experimental results demonstrate that this model successfully reduces trai-ning overhead while also improving retrieval performance. [ABSTRACT FROM AUTHOR]

Next-generation wireless network aims to support low-latency, high-speed data transmission services by incorporating artificial intelligence (AI) technologies. To fulfill this promise, AI-based network traffic prediction is essential for pre-allocating resources, such as bandwidth and computing power. This can help reduce network congestion and improve the quality of service (QoS) for users. Most studies achieve future traffic prediction by exploiting deep learning and reinforcement learning, to mine spatio-temporal correlated variables. Nevertheless, the prediction results obtained only by the spatio-temporal correlated variables cannot reflect real traffic changes. This phenomenon prevents the true prediction variables from being inferred, making the prediction algorithm perform poorly. Inspired by causal science, we propose a novel network traffic prediction method based on self-supervised spatio-temporal causal discovery (SSTCD). We first introduce the Granger causal discovery algorithm to build a causal graph among prediction variables and obtain spatio-temporal causality in the observed data, which reflects the real reasons affecting traffic changes. Next, a graph neural network (GNN) is adopted to incorporate causality for traffic prediction. Furthermore, we propose a self-supervised method to implement causal discovery to to address the challenge of lacking ground-truth causal graphs in the observed data. Experimental results demonstrate the effectiveness of the SSTCD method. [ABSTRACT FROM AUTHOR]

Semantic segmentation plays an important role in many fields because of its powerful ability to classify each pixel efficiently and accurately, but it relies on a large amount of manual annotations. In many cases, the annotations are very scarce and expensive, such as in medical image segmentation. To address this problem, researchers have been increasingly concerned about building efficient deep learning algorithms using rough label information in the past few years, with weakly supervised semantic segmentation method being one of them. Currently, most weakly supervised semantic segmentation methods rely on prototype learning to obtain the correlation between pixels; when the images of different categories are similar or indistinguishable, the extracted prototype has no representativeness to guide the training of model. Inspired by metric learning, we construct the pixel-level pairwise samples and propose a new self-supervised contrastive loss based on them, which makes full use of the class activation maps to reduce the intra-class difference and increase the inter-class difference; we also propose a novel prototype loss by a superpixel-guided clustering method to mine the valuable information in the image, which gathers the similar feature vectors to obtain the prototypes more accurately. The comparative experiments are carried out on PASCAL VOC 2012 and MS COCO 2014, the segmentation mIoU on the test set of PASCAL VOC 2012 has reached 69.5%, and the mIoU on the test set of MS COCO 2014 has reached 40.6%. The experimental results demonstrate our method achieves new state-of-the-art performance, which verifies the effectiveness and feasibility of the proposed method. [ABSTRACT FROM AUTHOR]

Cancer, a significant global public health issue, resulted in about 10 million deaths in 2022. Anticancer peptides (ACPs), as a category of bioactive peptides, have emerged as a focal point in clinical cancer research due to their potential to inhibit tumor cell proliferation with minimal side effects. However, the recognition of ACPs through wet-lab experiments still faces challenges of low efficiency and high cost. Our work proposes a recognition method for ACPs named ACP-DRL based on deep representation learning, to address the challenges associated with the recognition of ACPs in wet-lab experiments. ACP-DRL marks initial exploration of integrating protein language models into ACPs recognition, employing in-domain further pre-training to enhance the development of deep representation learning. Simultaneously, it employs bidirectional long short-term memory networks to extract amino acid features from sequences. Consequently, ACP-DRL eliminates constraints on sequence length and the dependence on manual features, showcasing remarkable competitiveness in comparison with existing methods. [ABSTRACT FROM AUTHOR]

Ocean Bottom Node (OBN) acquisition is a technique for marine seismic survey that has gained increased attention in recent years. The removal of shear wave noise from the vertical component of receivers plays a crucial role in the subsequent processing and interpretation of OBN data. Previous solutions suffer from noise residue or signal impairment for complex noise and signal overlap scenarios. In this work, we present and explore a self-supervised deep learning approach to attenuate shear wave noise in OBN data. It applies a deep neural network (DNN) to perform adaptive subtraction and comprises two steps to remove the noise associated with the two horizontal components of receivers, respectively. The two horizontal components are considered as noise reference and are sequentially fed into the DNN, and the DNN predicts the actual leaked noise from the contaminated vertical components data. The self-supervised method achieves improvements in the signal-to-noise ratio (SNR) on a set of synthetic data. The implementation of our method on field data demonstrates that it effectively attenuates the shear wave noise and preserves the valid signal. [ABSTRACT FROM AUTHOR]

The current network for the dual-task grinding wheel defect semantic segmentation lacks high-precision lightweight designs, making it challenging to balance lightweighting and segmentation accuracy, thus severely limiting its practical application in grinding wheel production lines. Additionally, recent approaches for addressing the natural class imbalance in defect segmentation fail to leverage the inexhaustible unannotated raw data on the production line, posing huge data wastage. Targeting these two issues, firstly, by discovering the similarity between Coordinate Attention (CA) and ASPP, this study has introduced a novel lightweight CA-ASP module to the DeeplabV3+, which is 45.3% smaller in parameter size and 53.2% lower in FLOPs compared to the ASPP, while achieving better segmentation precision. Secondly, we have innovatively leveraged the Masked Autoencoder (MAE) to address imbalance. By developing a new Hybrid MAE and applying it to self-supervised pretraining on tremendous unannotated data, we have significantly uplifted the network's semantic understanding on the minority classes, which leads to further rises in both the overall accuracy and accuracy of the minorities without additional computational growth. Lastly, transfer learning has been deployed to fully utilize the highly related dual tasks. Experimental results demonstrate that the proposed methods with a real-time latency of 9.512 ms obtain a superior segmentation accuracy on the mIoU score over the compared real-time state-of-the-art methods, excelling in managing the imbalance and ensuring stability on the complicated scenes across the dual tasks. [ABSTRACT FROM AUTHOR]

Deep learning (DL) techniques have recently attracted considerable attention in the field of seismic data denoising. However, most DL-based seismic denoising models require a considerable number of paired noisy–clean samples for training, which limits their application in practice. In this paper, a novel self-supervised DL scheme for noise attenuation in reflection seismic data is proposed based on the Neighbor2Neighbor strategy. The proposed method adopts a U-shaped convolutional network as the main network and incorporates regularization loss during training to improve the stability of network training, thereby constructing an end-to-end self-learning process for seismic data denoising. Specifically, a neighborhood subsampling workflow is built on single noisy seismic data to generate paired noisy images for training. Next, an unsupervised evaluation metric, grounded solely in noisy data, is adopted to quantitatively evaluate the performance of the denoising algorithm without labeled data. The application to synthetic datasets suggests that the proposed method achieves better denoising results than traditional (e.g., f−x deconvolution and wavelet analysis) and self-supervised methods, and rivals a widely used supervised DL model, the denoising convolution neural network. In real application to seismic data denoising without noise-free labels, the proposed method more effectively suppresses migration artifacts and random noise while better preserving the signal integrity compared to other methods. These improvements demonstrate that the proposed method is an independent, novel, and powerful data-driven DL scheme suitable for real seismic data denoising. [ABSTRACT FROM AUTHOR]

Metagenome binning is a key step, downstream of metagenome assembly, to group scaffolds by their genome of origin. Although accurate binning has been achieved on datasets containing multiple samples from the same community, the completeness of binning is often low in datasets with a small number of samples due to a lack of robust species co-abundance information. In this study, we exploited the chromatin conformation information obtained from Hi-C sequencing and developed a new reference-independent algorithm, Metagenome Binning with Abundance and Tetra-nucleotide frequencies-Long Range (metaBAT-LR), to improve the binning completeness of these datasets. This self-supervised algorithm builds a model from a set of high-quality genome bins to predict scaffold pairs that are likely to be derived from the same genome. Then, it applies these predictions to merge incomplete genome bins, as well as recruit unbinned scaffolds. We validated metaBAT-LR's ability to bin-merge and recruit scaffolds on both synthetic and real-world metagenome datasets of varying complexity. Benchmarking against similar software tools suggests that metaBAT-LR uncovers unique bins that were missed by all other methods. MetaBAT-LR is open-source and is available at https://bitbucket.org/project-metabat/metabat-lr.

Abstract Fluorescence microscopy data describe protein localization patterns at single-cell resolution and have the potential to reveal whole-proteome functional information with remarkable precision. Yet, extracting biologically meaningful representations from cell micrographs remains a major challenge. Existing approaches often fail to learn robust and noise-invariant features or rely on supervised labels for accurate annotations. We developed PIFiA (Protein Image-based Functional Annotation), a self-supervised approach for protein functional annotation from single-cell imaging data. We imaged the global yeast ORF-GFP collection and applied PIFiA to generate protein feature profiles from single-cell images of fluorescently tagged proteins. We show that PIFiA outperforms existing approaches for molecular representation learning and describe a range of downstream analysis tasks to explore the information content of the feature profiles. Specifically, we cluster extracted features into a hierarchy of functional organization, study cell population heterogeneity, and develop techniques to distinguish multi-localizing proteins and identify functional modules. Finally, we confirm new PIFiA predictions using a colocalization assay, suggesting previously unappreciated biological roles for several proteins. Paired with a fully interactive website ( https://thecellvision.org/pifia/ ), PIFiA is a resource for the quantitative analysis of protein organization within the cell.

The growth of the number of connected devices and network densification is driving an increasing demand for radio network resources, particularly Radio Frequency (RF) spectrum. Given the dynamic and complex nature of contemporary wireless environments, characterized by a wide variety of devices and multiple RATs, spectrum sensing is envisioned to become a building component of future 6G, including as a components within O-RAN or digital twins. However, the current SotA research for RAT classification predominantly revolves around supervised Convolutional Neural Network (CNN)- based approach that require extensive labeled dataset. Due to this, it is unclear how existing models behave in environments for which training data is unavailable thus leaving open questions regarding their generalization capabilities. In this paper, we propose a new spectrum sensing workflow in which the model training does not require any prior knowledge of the RATs transmitting in that area (i.e., no labelled data) and the class assignment can be easily done through manual mapping. Furthermore, we adaptat a SSL deep clustering architecture capable of autonomously extracting spectrum features from raw 1D Fast Fourier Transform (FFT) data. We evaluate the proposed architecture on three real-world datasets from three European cities, in the 868 MHz, 2.4 GHz and 5.9 GHz bands containing over 10 RATs and show that the developed model achieves superior performance by up to 35 percentage points with 22% fewer trainable parameters and 50% less floating-point operations per second (FLOPS) compared to an SotA AE-based reference architecture.

In recent years, speech denoising has greatly benefited from the rapid development of neural networks. However, these models require substantial noisy-clean speech pairs for supervised training, which limits their widespread use. Although there have been attempts to train denoising networks with only noisy speech data, existing self-supervised methods often suffer from a lack of continuity, low noise reduction performance, or heavy dependence on noise modeling. In this work, we introduce an efficient self-supervised Dynamic Multi-Focusing Network (DMFNet), a noise-only trained speech denoising network that utilizes a multi-scale connected encoder-decoder architecture as its backbone. Specifically, we have designed an efficient Spectral Dynamic Focusing Unit (SDFU) that enables the network to dynamically adapt the shape of its convolutional kernels while learning features, thus effectively focusing on the spectral structure of the human voice. Additionally, we introduce a Complex Attention Module (CAM), designed with a cross-space structure specialized for feature interaction and extraction. Finally, to further enhance the recovery of fine spectral details, we propose the Complex Multi-Scale Feature Fusion Unit (CMFFU) and Complex Scope Fusion Unit (CSFU) to adaptively fuse the features from different stages in the encoding process. Extensive evaluations across multiple datasets demonstrate that the proposed DMFNet significantly outperforms other state-of-the-art methods.

Spectral clustering, as an algorithm based on graph theory and spectral theory, has shown excellent performance in classification tasks of hyperspectral images in recent years. Although better results have been achieved, some challenges still exist. The inclusion of a priori information can increase the performance of spectral clustering algorithms; however, in practice, it is often unable to meet the demand for a priori information; at the same time, the two-stage structure of spectral clustering leads to bias and information loss during its clustering process. To address the above problems, we propose a one-step joint learning of self-supervised spectral clustering with anchor graph and fuzzy clustering. In this algorithm, we first use the BKHK method to generate anchor points that vary with the size of the dataset to enhance the quality of the similarity graph. After that, we use the idea of fuzzy clustering to optimize the membership and simultaneously learn the local and global structures of the data. Moreover, clustering information is integrated with pairwise constraint information in a unified update framework, which achieves mutual learning of clustering results and constraints under soft partitioning. After comparing with the existing algorithms, the proposed algorithm has better performance in classifying numerical datasets, remote sensing images, and hyperspectral datasets.

Image denoising is a critical task in image processing aimed at removing noise artifacts. Typically, supervised deep learning often necessitates a large number of pairs of noisy and noise-free images for training. Noise2Noise techniques have demonstrated efficiency in noise removal without relying on a noise-free ground truth. This is achieved through a learning process that approaches input to target points, balancing results across all training inputs. While Noise2Noise can be adapted for single image denoising, it still faces challenges in single image and blind noise scenarios. To address this issue, our research introduces the concept of self-augmented noisy images for self-supervised Noise2Noise single image denoising. The proposed method leverages the behavior of the training process, which strives to balance the loss values appropriately for each training set. By utilizing the same noisy image for both input and validation to learn self-identification, it produces another set of noisy images that mimic the input noisy images. From the experimental results, measured using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) metrics, it is evident that the proposed self-augmented strategy enables Noise2Noise to remove noise in single image scenarios. Additionally, it achieves performance comparable to other unsupervised denoising methods without requiring additional augmentation manipulations.

We introduce Pyramidal Recursive learning (PyRv), a novel method for text representation learning. This approach constructs a pyramidal hierarchy by recursively building representations of phrases, starting from tokens (characters, subwords, or words). At each level, N representations are recursively combined, resulting in N-1 representations on the level above, abstracting the input text from characters or subwords to words, phrases, and potentially sentences. The proposed method employs two learning approaches: autoencoding and autoregression. The autoencoding head decodes encoded representation pairs, while the autoregressive head predicts neighboring representations on both the left and right. This method exhibits four key properties: hierarchical representation, representation compositionality, representation decodability, and self-supervised learning. To implement and validate the proposed method, we train the Pyramidal Recursive Neural Network (PyRvNN) model. Evaluation metrics include autoencoder decodability, plagiarism detection, memorization, and readability. The accuracy of autoencoder decodability serves as an indicator of the validity of the four key properties. Preliminary assessments demonstrate promising results, particularly in machine-paraphrased plagiarism, text readability, and a memorization experiment.

Recently, deep learning-based methods have emerged as the preferred approach for ultrasound data analysis. However, these methods often require large-scale annotated datasets for training deep models, which are not readily available in practical scenarios. Additionally, the presence of speckle noise and other imaging artifacts can introduce numerous hard examples for ultrasound data classification. In this paper, drawing inspiration from self-supervised learning techniques, we present a pre-training method based on mask modeling specifically designed for ultrasound data. Our study investigates three different mask modeling strategies: random masking, vertical masking, and horizontal masking. By employing these strategies, our pre-training approach aims to predict the masked portion of the ultrasound images. Notably, our method does not rely on externally labeled data, allowing us to extract representative features without the need for human annotation. Consequently, we can leverage unlabeled datasets for pre-training. Furthermore, to address the challenges posed by hard samples in ultrasound data, we propose a novel hard sample mining strategy. To evaluate the effectiveness of our proposed method, we conduct experiments on two datasets. The experimental results demonstrate that our approach outperforms other state-of-the-art methods in ultrasound image classification. This indicates the superiority of our pre-training method and its ability to extract discriminative features from ultrasound data, even in the presence of hard examples.

Nowadays, an increasing number of hyperspectral images (HSIs) are becoming available. However, the utilization of unlabeled HSIs is extremely low due to high annotation costs. Thus, it is crucial to figure out how to use these unlabeled HSIs and enhance the classification performance. Fortunately, self-supervised training enables us to acquire latent features from unlabeled HSIs, thereby enhancing network performance via transfer learning. Whereas, most current networks for HSIs are inflexible, it is challenging for them to perform learning and accommodate multimodal HSIs. Therefore, we devise a scalable self-supervised network called instructional mask autoencoder, which can extract general patterns of HSIs by these unannotated data. It primarily consists of a spatial–spectral embedding block and a transformer-based masked autoencoder, which are utilized for projecting input samples into the same latent space and learning higher level semantic information, respectively. Moreover, we utilize a random token called $ins\_{t}oken$ to instruct the model learn components of global information, which are highly correlated with the target pixel in HSI samples. In the fine-tuning stage, we design a learnable aggregation mechanism to put all tokens into full play. The obtained results illustrate that our method exhibits robust generalization performance and accelerates convergence across diverse datasets. In cases of limited samples, we conducted experiments on three structurally distinct HSIs, all of which achieved competitive performance. Compared to state-of-the-art methods, our approach demonstrated respective improvements of 1.97%, 0.44%, and 3.35% on these three datasets.

An increasing availability of remote sensing data in the era of geo big-data makes producing well-represented, reliable training data to be more challenging and requires an excessive amount of human labor. In addition, the rapid increase in graphics processing unit processing power has enabled the development of advanced deep learning algorithms, which achieve impressive results in the field of satellite image processing. However, they require a huge and comprehensive training dataset to avoid overfitting problems and to represent a generalizable model. Thus, moving toward the development of nonsupervised deep learning (NSDL) models in different remote sensing applications is an inevitable need. To provide an initial response to that need, this article performs a comprehensive review and systematic meta-analysis of recently published research articles focusing on the applications of NSDL for remote sensing data processing. In order to identify future research directions and formulate recommendations, we extract trends and highlight interesting approaches from this large body of literature. Consequently, current challenges, prospects, and recommendations are also discussed to uncover the trend. According to the results, there is a sharp increasing trend in the applicability of NSDL methods during these few years particularly, with the advent of new deep architectures, such as adversarial, graph, and transformer models. As a result, this review article discusses different remote sensing data processing applications and challenges that can be addressed using NSDL approaches.

Monocular depth estimation is a crucial task in computer vision, and self-supervised algorithms are gaining popularity due to their independence from expensive ground truth supervision. However, current self-supervised algorithms may not provide accurate estimation and may suffer from distorted boundaries when applied to indoor scenes. Combining multi-scale features is an important research direction in image segmentation to achieve accurate estimation and resolve boundary distortion. However, there are few studies on indoor self-supervised algorithms in this regard. To solve this issue, we propose a novel full-scale feature information fusion approach that includes a full-scale skip-connection and a full-scale feature fusion block. This approach can aggregate the high-level and low-level information of all scale feature maps during the network's encoding and decoding process to compensate for the network's loss of cross-layer feature information. The proposed full-scale feature fusion improves accuracy and reduces the decoder parameters. To fully exploit the superiority of the full-scale feature fusion module, we replace the encoder backbone from ResNet with the more advanced ResNeSt. Combining these two methods results in a significant improvement in prediction accuracy. We have extensively evaluated our approach on the indoor benchmark datasets NYU Depth V2 and ScanNet. Our experimental results demonstrate that our method outperforms existing algorithms, particularly on NYU Depth V2, where our precision is raised to 83.8%. [ABSTRACT FROM AUTHOR]

Seismic fault interpretation holds great significance in the fields of geophysics and geology. However, conventional methods of seismic fault recognition encounter various issues. For example, models trained on synthetic data often exhibit inadequate generalization when applied to field seismic data, and supervised learning is heavily dependent on the quantity and quality of annotated data, being susceptible to the subjectivity of interpreters. To address these challenges, we propose applying self-supervised pre-training methods to seismic fault recognition, exploring the transfer of 3D Transformer-based backbone networks and different pre-training methods on fault recognition tasks, thereby enabling the model to learn more powerful feature representations from extensive unlabeled datasets. Additionally, we propose an innovative pre-training strategy for the entire segmentation network based on the characteristics of seismic data and introduce a multi-scale decoding and fusion module that significantly improves recognition accuracy. Specifically, during the pre-training stage, we compare various self-supervision methods, like MAE, SimMIM, SimCLR, and a joint self-supervised learning approach. We adopt multi-scale decoding step-by-step fitting expansion targets during the fine-tuning stage. Ultimately merging features to refine fault edges, the model displays superior adaptability when handling narrow, elongated, and unevenly distributed fault annotations. Experiments demonstrate that our proposed method achieves state-of-the-art performance on Thebe, the currently largest publicly annotated dataset in this field. [ABSTRACT FROM AUTHOR]

Change detection (CD) in remote sensing (RS) imagery is a pivotal method for detecting changes in the Earth's surface, finding wide applications in urban planning, disaster management, and national security. Recently, deep learning (DL) has experienced explosive growth and, with its superior capabilities in feature learning and pattern recognition, it has introduced innovative approaches to CD. This review explores the latest techniques, applications, and challenges in DL-based CD, examining them through the lens of various learning paradigms, including fully supervised, semi-supervised, weakly supervised, and unsupervised. Initially, the review introduces the basic network architectures for CD methods using DL. Then, it provides a comprehensive analysis of CD methods under different learning paradigms, summarizing commonly used frameworks. Additionally, an overview of publicly available datasets for CD is offered. Finally, the review addresses the opportunities and challenges in the field, including: (a) incomplete supervised CD, encompassing semi-supervised and weakly supervised methods, which is still in its infancy and requires further in-depth investigation; (b) the potential of self-supervised learning, offering significant opportunities for Few-shot and One-shot Learning of CD; (c) the development of Foundation Models, with their multi-task adaptability, providing new perspectives and tools for CD; and (d) the expansion of data sources, presenting both opportunities and challenges for multimodal CD. These areas suggest promising directions for future research in CD. In conclusion, this review aims to assist researchers in gaining a comprehensive understanding of the CD field. [ABSTRACT FROM AUTHOR]

The electrocardiogram (ECG) measurements with clinical diagnostic labels are intrinsically limited. We propose a generative learning based self-supervised method for general ECG representations applicable to various downstream tasks, thus achieving the goal of reducing the dependence on labeled data. However, existing self-supervised methods either fail to provide satisfactory ECG representations or require too much effort to curate a large amount of expert-annotated datasets. We propose a spatio-temporal joint detection based self-supervised method with little or no human supervision to label massive datasets. Considering the spatio-temporal characteristics of ECG signals, we dynamically randomly mask the original signal (temporal detection) and disrupt the order of leads (spatial detection) to complete the learning through reconstructing the original signal and predicting the lead numbers. To validate the effectiveness of the proposed method, we use several publicly available ECG databases as well as a private ECG data of ventricular tachycardia to pre-train our model. We use diagnostic classification of 27 arrhythmia types and localization of ventricular tachycardia origin sites as two downstream tasks, respectively. The results show that learning ECG representations with this method is effective. This effort demonstrates the feasibility of learning representations from ECG data by self-supervised learning. Our self-supervised method uses only 60% of the labeled data used by the supervised method to achieve the same performance. Using the same amount of data, our self-supervised approach shows 1.3% and 8.6% improvement in classification and localization accuracy compared to the model with random initialization on two types of downstream tasks, respectively. [ABSTRACT FROM AUTHOR]

Purpose: In twin-to-twin transfusion syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the two fetuses. In the current practice, TTTS is treated surgically by closing abnormal anastomoses using laser ablation. This surgery is minimally invasive and relies on fetoscopy. Limited field of view makes anastomosis identification a challenging task for the surgeon. Methods: To tackle this challenge, we propose a learning-based framework for in vivo fetoscopy frame registration for field-of-view expansion. The novelties of this framework rely on a learning-based keypoint proposal network and an encoding strategy to filter (i) irrelevant keypoints based on fetoscopic semantic image segmentation and (ii) inconsistent homographies. Results: We validate our framework on a dataset of six intraoperative sequences from six TTTS surgeries from six different women against the most recent state-of-the-art algorithm, which relies on the segmentation of placenta vessels. Conclusion: The proposed framework achieves higher performance compared to the state of the art, paving the way for robust mosaicking to provide surgeons with context awareness during TTTS surgery. [ABSTRACT FROM AUTHOR]