Search

Tumor mutation burden (TMB) is a potential biomarker for evaluating the prognosis and response to immune checkpoint inhibitors, but its costly and time-consuming method of measurement limits its widespread application. This study aimed to identify the TMB-related histopathologic features from hematoxylin and eosin slides and explore their prognostic value in gliomas. TMB-related features were detected using a graph convolutional neural network from whole-slide images of patients from The Cancer Genome Atlas data set (619 patients), and the correlation between features and TMB was evaluated in an external validation set (237 patients). TMB-related features were used for predicting overall survival (OS) of patients to investigate whether these features have potential for prognostic prediction. Moreover, biological pathways underlying the prognostic value of the features were further explored. Histopathologic features derived from whole-slide images were significantly associated with patient TMB (P = 0.007 in the external validation set). TMB-related features showed excellent performance for OS prediction, and patients with lower-grade gliomas could be further stratified into different risk groups according to the features (P = 0.00013; hazard ratio, 4.004). Pathways involved in the cell cycle and execution of immune response were enriched in patients with higher OS risk. The TMB-related features could be used to estimate TMB and aid in prognostic risk stratification of patients with glioma with dysregulated biological pathways., Competing Interests: Disclosure Statement None declared., (Copyright © 2023 American Society for Investigative Pathology. Published by Elsevier Inc. All rights reserved.)

Background and Objective: Adjuvant chemotherapy is recommended as standard treatment for colorectal cancer (CRC) with stage III according to TNM stage. However, outcomes are varied even among patients receiving similar treatments. We aimed to develop a prognostic signature to stratify outcomes and benefit from different chemotherapy regimens by analyzing whole slide images (WSI) using deep learning., Methods: We proposed an unsupervised deep learning network (variational autoencoder and generative adversarial network) in 180,819 image tiles from the training set (147 patients) to develop a WSI signature for predicting the disease-free survival (DFS) and overall survival (OS) of patients, and tested in validation set of 63 patients. An integrated nomogram was constructed to investigate the incremental value of deep learning signature (DLS) to TNM stage for individualized outcomes prediction., Results: The DLS was associated with DFS and OS in both training and validation sets and proved to be an independent prognostic factor. Integrating the DLS and clinicopathologic factors showed better performance (C-index: DFS, 0.748; OS, 0.794; in the validation set) than TNM stage. In patients whose DLS and clinical risk levels were inconsistent, their risk of relapse was reclassified. In the subgroup of patients treated with 3 months, high-DLS was associated with worse DFS (hazard ratio: 3.622-7.728)., Conclusions: The proposed based-WSI DLS improved risk stratification and could help identify patients with stage III CRC who may benefit from the prolonged duration of chemotherapy., Competing Interests: Declaration of Competing Interest The authors declare no potential conflicts of interest., (Copyright © 2022. Published by Elsevier B.V.)

Deep learning is the fastest growing field in artificial intelligence and has led to many transformative innovations in various domains. However, lack of interpretability sometimes hinders its application in hypothesis-driven domains such as biology and healthcare. In this paper, we propose a novel deep learning model with individual feature ranking. Several simulated datasets with the scenarios that contributing features are correlated and buried among non-contributing features were used to characterize the novel analysis approach. A publicly available clinical dataset was also applied. The performance of the individual level dropout feature ranking model was compared with commonly used artificial neural network model, random forest model, and population level dropout feature ranking model. The individual level dropout feature ranking model provides a reasonable prediction of the outcomes. Unlike the random forest model and population level dropout feature ranking model, which can only identify global-wise contributing features (i.e., at population level), the individual level dropout feature ranking model allows further identification of impactful features on response at individual level. Therefore, it provides a basis for clustering patients into subgroups. This may provide a new tool for enriching patients in clinical drug development and developing personalized or individualized medicine.

3D object detection is a new research direction in recent years, and its main task is the location and recognization of targets in space. The existing methods for 3D object detection using monocular or binocular stereo vision are easily affected by object occlusion, viewpoint changing and scale changing in 3D scene, there will be problems such as poor detection accuracy and robustness. LiDAR point cloud can provide 3D scene information, so using deep learning method to complete 3D object detection based on LiDAR point cloud has become a research hotspot in the field of 3D vision. Aiming at the 3D object detection based on LiDAR point cloud, the relevant research in recent years is reviewed. Firstly, the 3D object detection methods based on LiDAR point cloud are divided into point cloud based, point cloud projection based, point cloud voxelization based and multi-modal fusion based 3D object detection methods according to the data form of network input, and the most representative methods in each category are described in detail. Then common datasets are introduced, and the performance of representative methods is evaluated, and the advantages and limitations of each method are discussed from several aspects. Finally, the shortcomings and difficulties are given, and the future development directions are also discussed and put forward.

The integration of machine/deep learning and sensing technologies is transforming healthcare and medical practice. However, inherent limitations in healthcare data, namely scarcity, quality, and heterogeneity, hinder the effectiveness of supervised learning techniques which are mainly based on pure statistical fitting between data and labels. In this article, we first identify the challenges present in machine learning for pervasive healthcare and we then review the current trends beyond fully supervised learning that are developed to address these three issues. Rooted in the inherent drawbacks of empirical risk minimization that underpins pure fully supervised learning, this survey summarizes seven key lines of learning strategies, to promote the generalization performance for real-world deployment. In addition, we point out several directions that are emerging and promising in this area, to develop data-efficient, scalable, and trustworthy computational models, and to leverage multi-modality and multi-source sensing informatics, for pervasive healthcare. [ABSTRACT FROM AUTHOR]

As an important agricultural facility that supports efficient crop production and precise water management, center-pivot irrigated cropland (CPIC) plays a critical role in increasing crop yield and ensuring food security globally. Since the 1980s, CPIC has expanded significantly and has become one of the most valuable irrigated croplands in Brazil. However, national-scale and long-term CPIC products are currently unavailable, making spatiotemporal analysis for related ecological and economic factors challenging. In this study, we proposed a novel framework combining cloud platforms and deep learning algorithms to produce annual CPIC maps (ACMs) with 30 m spatial resolution in Brazil from 1985 to 2021. Leveraging all available Landsat-5, 7, and 8 images in Brazil, we generated three-band annual composite images to obtain stable image features for CPIC and alleviate the heterogeneity of CPIC. Then we constructed a large-scale training dataset covering different climatic zones of Brazil and spanning multiple years. Based on this dataset a CNN model was trained and applied to the whole of Brazil and generalized to other years. We deployed the CNN model to the cloud platform and combined it with Google Earth Engine to produce Brazil's annual CPIC Maps (ACMs). Moreover, we analyzed the spatiotemporal dynamics of CPIC and explored the impact of CPIC expansion on forests. The accuracy assessment demonstrated that our ACMs reached an average mIoU of 0.947 ± 0.021 for all study years, and the estimated CPIC area was highly consistent with official statistics. Our results indicated that Brazilian CPICs reached 1.8 × 106 ha in 2021, which was 45 times more than that in 1985. CPIC expansion occurred mainly in the states of Minas Gerais, Goiás, and Bahia over the past few decades, with an average annual increase in area of over 8,000 ha. We also found that CPIC expansion had encroached on 42,000 ha of shrubs and 25,000 ha of forests since 1985, and the impact was continuing and intensifying. To the best of our knowledge, the produced ACMs are the longest and most spatially detailed CPIC products in Brazil that reflect the CPIC spatiotemporal changes in the last four decades and can provide essential information services for Brazil's water resource management and agricultural decision-making. [ABSTRACT FROM AUTHOR]

With the large-scale application of the Internet of Things (IoT), security issues have become increasingly prominent. Device identification is an effective way to secure IoT environment by quickly identifying the category or model of devices in the network. Currently, the passive fingerprinting method used for IoT device identification based on network traffic flow mostly focuses on protocol features in packet headers but does not consider the direction and length of packet sequences. This paper proposes a device identification method for the IoT based on directional packet length sequences in network flows and a deep convolutional neural network. Each value in a packet length sequence represents the size and transmission direction of the corresponding packet. This method constructs device fingerprints from packet length sequences and uses convolutional layers to extract deep features from the device fingerprints. Experimental results show that this method can effectively recognize device identity with accuracy, recall, precision, and f1-score over 99%. Compared with methods using traditional machine learning and feature extraction techniques, our feature representation is more intuitive, and the classification model is effective. [ABSTRACT FROM AUTHOR]

Conventional biometric modalities, such as the face, fingerprint, and iris, are vulnerable against imitation and circumvention. Accordingly, secure biometric modalities with cancelable properties are needed for personal identification, especially in smart healthcare applications. Here we developed a person identification model using high-density surface electromyography (HD-sEMG) as biometric traits. In this model, the HD-sEMG biometric templates are cancelable and could be customized by the users through finger isometric contractions. A deep feature learning approach, implemented by convolutional neural networks (CNNs) is used to capture user-specific patterns from HD-sEMG signals and make identification decisions. This model has been validated on twenty-two subjects, with training and testing data acquired from two different days. The rank-1 identification accuracy and equal error rate for 44 identities (22 subjects × 2 accounts) can reach 87.23% and 4.66%, respectively. The cross-day identification accuracy of the proposed model is higher than the results of previous methods reported in the literature. The usability and efficiency of the proposed model are also investigated, indicating its potentials for practical applications. [ABSTRACT FROM AUTHOR]

This article addresses the problem of remote sensing image pan-sharpening from the perspective of generative adversarial learning. We propose a novel deep neural network-based method named pansharpening GAN (PSGAN). To the best of our knowledge, this is one of the first attempts at producing high-quality pan-sharpened images with generative adversarial networks (GANs). The PSGAN consists of two components: a generative network (i.e., generator) and a discriminative network (i.e., discriminator). The generator is designed to accept panchromatic (PAN) and multispectral (MS) images as inputs and maps them to the desired high-resolution (HR) MS images, and the discriminator implements the adversarial training strategy for generating higher fidelity pan-sharpened images. In this article, we evaluate several architectures and designs, namely, two-stream input, stacking input, batch normalization layer, and attention mechanism to find the optimal solution for pan-sharpening. Extensive experiments on QuickBird, GaoFen-2, and WorldView-2 satellite images demonstrate that the proposed PSGANs not only are effective in generating high-quality HR MS images and superior to state-of-the-art methods but also generalize well to full-scale images. [ABSTRACT FROM AUTHOR]

Hand gesture recognition via high-density surface electromyogram (HDsEMG) has received increasing attentions in human–machine interactions. However, with an extremely large number of electrodes in the electrode arrays, the computational burden substantially increased in real world mobile computing applications. Additionally, with diverse features extracted from all electrodes, large information redundancy reduced the learning efficiency of machine learning models. Furthermore, most machine learning models were employed as black-box modules, increasing concerns on both ethics and the user trust on machine learning techniques, especially in human-centered applications. In this work, we applied deep forest in HDsEMG-based hand gesture recognition. Deep forest is a new deep learning architecture where information could be processed layer-by-layer via cascaded random forest modules. Built on highly interpretable decision trees, deep forest models retain the interpretability of decision trees. With the inherent interpretability of deep forest, the most important features and electrodes that deep forest focused on in the decision making process can also be used to optimize the feature set and electrode configuration. We also compared the model interpretation results with the anatomy structure of forearm muscles, providing relations between the interpretations of deep forest and the physiological basis of neuromuscular systems. HDsEMG (256 electrodes) from 20 subjects were used in our analyses. To simulate realistic application scenarios, all analyses and evaluations were performed in inter-subject validations. Results showed that with optimized feature set and electrode configuration, the number of features/electrodes was substantially reduced, with the overall classification accuracy improved. Data and codes are available online. [ABSTRACT FROM AUTHOR]

• Two-stream fusion CNN is proposed for pan-sharpening. • Using skip connection to compensate detail lost in down-sampling. • Demonstrate ℓ 1 is more suitable for pan-sharpening than ℓ 2. • Residual learning is utilized to further boost the performance. Remote sensing image fusion (also known as pan-sharpening) aims at generating a high resolution multi-spectral (MS) image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral image. Inspired by the astounding achievements of convolutional neural networks (CNNs) in a variety of computer vision tasks, in this paper we propose a Two-stream Fusion Network (TFNet) to address the problem of pan-sharpening. Unlike many previous CNN based methods that consider pan-sharpening as a super-resolution problem and perform pan-sharpening through mapping the stacked PAN and MS to the target high resolution MS image, the proposed TFNet aims to fuse PAN and MS images in feature domain and reconstruct the pan-sharpened image from the fused features. The TFNet mainly consists of three parts. The first part is comprised of two networks extracting features from PAN and MS images, respectively. The subsequent network fuses them together to form compact features that represent both spatial and spectral information of PAN and MS images, simultaneously. Finally, the desired high spatial resolution MS image is recovered from the fused features through an image reconstruction network. Experiments on Quickbird and GaoFen-1 images demonstrate that the proposed TFNet can fuse PAN and MS images effectively, and produce pan-sharpened images competitive with even superior to state of the arts. [ABSTRACT FROM AUTHOR]

Traditional methods of classification and retrieval for non-rigid 3D models, such as topological structure-based methods and spectral analysis-based methods, high rely on manual interventions, which leads to the fail of generalization. Recently, methods based on deep learning have been proposed for rigid 3D models, but they are usually not suitable for non-rigid 3D models. Due to excessive redundant calculations, point clouds-based methods and multi-view-based methods are also inefficient. In this paper, a novel deep learning framework for non-rigid 3D model classifications is presented. Different from other point clouds-based methods and multi-view-based method, this framework utilize a 3D sphere view and a new convolution way. First, an encoded feature sphere (EFS) is constructed to express the non-rigid 3D model through feature encoding, spherical projection, and crucial points extraction. Then, loop convolution and longitudinal–latitudinal loop convolution are combined into EFSCNN to extract the discriminative feature descriptor of the non-rigid 3D model. EFSCNN is a hierarchical network based on multi-direction convolution, which expands the receptive field and enhances the ability to capture local features. With the single view input EFS, EFSCNN achieves competitive performance on major non-rigid model classification and retrieval benchmarks with significantly faster runtime than previous studies. • A new 3D model representation method with spherical projection is proposed termed Encoded Feature Sphere (EFS), which can represent the 3D model with only a single view. • The loop convolution is presented, which can learn deep features of projected points on the sphere. Based on the loop convolution, EFSCNN, a network that extends CNN on the sphere, is designed to obtain the deep features of 3D models. • The proposed framework does not require the construction of local neighborhoods (or graph) or a high-computational network to extract features from multi-view. Thus, EFSCNN achieves state-of-the-art and satisfactory efficiency on the classification and retrieval of non-rigid 3D models. [ABSTRACT FROM AUTHOR]

Influence maximization (IM) has drawn significant attention in recent years. Most existing IM methods primarily focus on homogeneous networks, and do not take into account the heterogeneity and the attributes of different types of nodes in heterogeneous networks. However, heterogeneous networks are ubiquitous in real world, encompassing rich semantics and complex structural information. Additionally, the clustering characteristics inherent in a network have a critical and substantial impact on the process of information diffusion, which is often overlooked in IM models designed for heterogeneous networks. To address the challenges posed by the heterogeneity and clustering structure in heterogeneous networks, we propose a novel deep learning framework based on a self-supervised clustered heterogeneous graph transformer for IM in heterogeneous networks, which we have named SCHGT-IM. SCHGT-IM aggregates the heterogeneity and clustering information in heterogeneous networks and incorporates a clustered cascade (CC) model as an information diffusion model to enhance the realism of simulations. We evaluate the performance of SCHGT-IM in comparison with that of state-of-the-art IM models using three academic heterogeneous networks extracted from the DBLP dataset. The experimental results on influence spread demonstrate that SCHGT-IM is superior to fourteen state-of-the-art algorithms and is highly effective in selecting influential seed nodes of different types from heterogeneous networks. • A self-supervised clustered HGT method SCHGT-IM for IM on heterogeneous network is proposed. • SCHGT-IM integrates cluster information from heterogeneous networks. • SCHGT-IM includes information diffusion model, named the clustered cascade model. • SCHGT-IM outperforms fourteen state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Opium poppies are a major source of traditional drugs, which are not only harmful to physical and mental health, but also threaten the economy and society. Monitoring poppy cultivation in key regions through remote sensing is therefore a crucial task; the location coordinates of poppy parcels represent particularly important information for their eradication by local governments. We propose a new methodology based on deep learning target detection to identify the location of poppy parcels and map their spatial distribution. We first make six training datasets with different band combinations and slide window sizes using two ZiYuan3 (ZY3) remote sensing images and separately train the single shot multibox detector (SSD) model. Then, we choose the best model and test its performance using 225 km2 verification images from Lao People's Democratic Republic (Lao PDR), which exhibits a precision of 95% for a recall of 85%. The speed of our method is 4.5 km2/s on 1080TI Graphics Processing Unit (GPU). This study is the first attempt to monitor opium poppies with the deep learning method and achieve a high recognition rate. Our method does not require manual feature extraction and provides an alternative way to rapidly obtain the exact location coordinates of opium poppy cultivation patches. [ABSTRACT FROM AUTHOR]

Existing methods of 3D model recognition mainly depend on deep learning algorithms. However, the extracted deep feature lacks the geometric information of the models. Thus, these methods fail to identify rigid and non-rigid 3D models simultaneously, i.e., universal 3D models recognition. In this work, we propose a novel method, G eometric- a ware F eature L earning (GaFL), to further investigate the combination mechanism of geometric feature and deep learning in universal 3D models recognition. In GaFL, we design the Layer-Projected and Ray-Projected feature extraction policies to obtain depth values, which contain rich geometric information. Furthermore, the sphere convolution is proposed to guarantee the continuity and integrity of the ray feature when feeding into a deep network and the feature inactivation fusion module is designed to achieve the complementarity between layer and ray feature. Finally, the final merged feature vector contains enough geometric information as well as high-level semantic information, which are critical to universal 3D models recognition. In the experiments, GaFL achieves 95.0% and 95.2% classification accuracy in the rigid 3D models dataset ModelNet40 and the non-rigid 3D models dataset SHREC16, respectively, indicating that GaFL is powerful in universal 3D models recognition. Moreover, its significant advantage over state-of-the-art methods has also been validated on three other datasets, i.e., ShapeNet Core55, ScanObjectNN and SHREC15. • We propose a novel geometric feature input source, which enriches the data source. • Introduce geometric aware feature learning for universal 3D models recognition. • Extensive experimental results demonstrate the competitiveness of our method. [ABSTRACT FROM AUTHOR]

• Proposed model improved prognostic prediction in locally advanced rectal cancer. • Non-invasive imaging signatures were independent prognostic factors. • An individualized tool linking chemotherapy decisions and benefits was provided. • More patients benefiting from adjuvant chemotherapy would be identified. • The balance between benefit and toxicity should be considered in specific subgroups. Adjuvant therapy is recommended to minimize the risk of distant metastasis (DM) and local recurrence (LR) in patients with locally advanced rectal cancer (LARC). However, its role is controversial. We aimed to develop a pretreatment MRI-based deep learning model to predict LR, DM, and overall survival (OS) over 5 years after surgery and to identify patients benefitting from adjuvant chemotherapy (AC). The multi-survival tasks network (MuST) model was developed in a primary cohort (n = 308) and validated using two external cohorts (n = 247, 245). An AC decision tree integrating the MuST-DM score, perineural invasion (PNI), and preoperative carbohydrate antigen 19–9 (CA19-9) was constructed to assess chemotherapy benefits and aid personalized treatment of patients. We also quantified the prognostic improvement of the decision tree. The MuST network demonstrated high prognostic accuracy in the primary and two external cohorts for the prediction of three different survival tasks. Within the stratified analysis and decision tree, patients with CA19-9 levels > 37 U/mL and high MuST-DM scores exhibited favorable chemotherapy efficacy. Similar results were observed in PNI-positive patients with low MuST-DM scores. PNI-negative patients with low MuST-DM scores exhibited poor chemotherapy efficacy. Based on the decision tree, 14 additional patients benefiting from AC and 391 patients who received over-treatment were identified in this retrospective study. The MuST model accurately and non-invasively predicted OS, DM, and LR. A specific and direct tool linking chemotherapy decisions and benefit quantification has also been provided. [ABSTRACT FROM AUTHOR]

Ocean wave energy is attracting more and more attention from researchers on observation of its clean and sustainable properties. Significant wave height (SWH) is one of the key wave parameters, making accurately forecasting the SWH important for coastal/ocean engineers. In this paper, we propose a new framework for forecasting the SWH. The data were decomposed and reconstructed, and the lag features length of the input data were adaptively optimized using the Bayesian optimization (BO) algorithm. A new paradigm for end-to-end dynamic modeling (EEDM) forecast is then proposed, where data are modeled and forecasted separately for buoys at various geographical locations, with automated machine learning (AutoML) as the back-end modeling support for the paradigm. It was also trained and tested on nine buoys from NOAA National Data Buoy Center, which are located at sites with different water depths. The results show that the forecast framework has provided reliable forecasts. We also discussed the reasons for the buoy with the worst forecast in terms of model interpretability and data quality. Finally, we compared three deep learning models (simple recurrent network, long short-term memory and gate recurrent unit) and three machine learning models (principal component regression, support vector machine and K-nearest neighbor). The comparisons indicate that the AutoML turns out to be the best. • The feasibility of automated machine learning for forecasting significant wave height is verified for the first time. • The input lag features length (rolling window size) is optimized as a black-box problem. • Significant wave height end-to-end dynamic modeling (EEDM) framework is proposed to provide a new paradigm. • The proposed forecast framework can be adopted in coastal/ocean engineering. [ABSTRACT FROM AUTHOR]