Search

Over the past few years, the COVID‐19 virus has had a significant impact on the physical and mental health of people around the world. Therefore, in order to effectively distinguish COVID‐19 patients, many deep learning efforts have used chest medical images to detect COVID‐19. As with model accuracy, interpretability is also important in the work related to human health. This work introduces an interpretable vision transformer that uses the prototype method for the detection of positive patients with COVID‐19. The model can learn the prototype features of each category based on the structural characteristics of ViT. The predictions of the model are obtained by comparing all the features of the prototype in the designed prototype block. The proposed model was applied to two chest X‐ray datasets and one chest CT dataset, achieving classification performance of 99.3%, 96.8%, and 98.5% respectively. Moreover, the prototype method can significantly improve the interpretability of the model. The decisions of the model can be interpreted based on prototype parts. In the prototype block, the entire inference process of the model can be shown and the predictions of the model can be demonstrated to be meaningful through the visualization of the prototype features. [ABSTRACT FROM AUTHOR]

Contemporary multiscale construction object detection algorithms rely predominantly on fully-supervised deep learning, requiring arduous and time-consuming labeling process. This paper presents a novel semisupervised multiscale construction objects detection (SS-MCOD) by harnessing nearly infinite unlabeled images along with limited labels, achieving more accurate and robust detection results. SS-MCOD uses a deformable convolutional network (DCN)-based teacher-student joint learning framework. DCN uses deformable advantages to extract and fuse multiscale construction object features. The teacher module generates pseudolabels for construction objects in unlabeled images, while the student module learns the location and classification of construction objects in both labeled images and unlabeled images with pseudolabels. Experimental validation using commonly used construction datasets demonstrates the accuracy and generalization performance of SS-MCOD. This research can provide insights for other detection tasks with limited labels in the construction domain. [ABSTRACT FROM AUTHOR]

Objectives: To develop a deep learning (DL) method that can determine the Liver Imaging Reporting and Data System (LI-RADS) grading of high-risk liver lesions and distinguish hepatocellular carcinoma (HCC) from non-HCC based on multiphase CT. Methods: This retrospective study included 1049 patients with 1082 lesions from two independent hospitals that were pathologically confirmed as HCC or non-HCC. All patients underwent a four-phase CT imaging protocol. All lesions were graded (LR 4/5/M) by radiologists and divided into an internal (n = 886) and external cohort (n = 196) based on the examination date. In the internal cohort, Swin-Transformer based on different CT protocols were trained and tested for their ability to LI-RADS grading and distinguish HCC from non-HCC, and then validated in the external cohort. We further developed a combined model with the optimal protocol and clinical information for distinguishing HCC from non-HCC. Results: In the test and external validation cohorts, the three-phase protocol without pre-contrast showed κ values of 0.6094 and 0.4845 for LI-RADS grading, and its accuracy was 0.8371 and 0.8061, while the accuracy of the radiologist was 0.8596 and 0.8622, respectively. The AUCs in distinguishing HCC from non-HCC were 0.865 and 0.715 in the test and external validation cohorts, while those of the combined model were 0.887 and 0.808. Conclusion: The Swin-Transformer based on three-phase CT protocol without pre-contrast could feasibly simplify LI-RADS grading and distinguish HCC from non-HCC. Furthermore, the DL model have the potential in accurately distinguishing HCC from non-HCC using imaging and highly characteristic clinical data as inputs. Clinical relevance statement: The application of deep learning model for multiphase CT has proven to improve the clinical applicability of the Liver Imaging Reporting and Data System and provide support to optimize the management of patients with liver diseases. Key Points: • Deep learning (DL) simplifies LI-RADS grading and helps distinguish hepatocellular carcinoma (HCC) from non-HCC. • The Swin-Transformer based on the three-phase CT protocol without pre-contrast outperformed other CT protocols. • The Swin-Transformer provide help in distinguishing HCC from non-HCC by using CT and characteristic clinical information as inputs. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have achieved promising performance in the field of deep learning, but the manual design turns out to be very difficult due to the increasingly complex topologies of CNNs. Recently, neural architecture search (NAS) methods have been proposed to automatically design network architectures, which are superior to handcrafted counterparts. Unfortunately, most current NAS methods suffer from either highly computational complexity of generated architectures or limitations in the flexibility of architecture design. To address above issues, this article proposes an evolutionary neural architecture search (ENAS) method based on improved Transformer and multi-branch ConvNet. The multi-branch block enriches the feature space and enhances the representational capacity of a network by combining paths with different complexities. Since convolution is inherently a local operation, a simple yet powerful "batch-free normalization Transformer Block" (BFNTBlock) is proposed to leverage both local information and long-range feature dependencies. In particular, the design of batch-free normalization (BFN) and batch normalization (BN) mixed in the BFNTBlock blocks the accumulation of estimation shift ascribe to the stack of BN, which has favorable effects for performance improvement. The proposed method achieves remarkable accuracies, 97.24 % and 80.06 % on CIFAR10 and CIFAR100, respectively, with high computational efficiency, i.e. only 1.46 and 1.53 GPU days. To validate the universality of our method in application scenarios, the proposed algorithm is verified on two real-world applications, including the GTSRB and NEU-CLS dataset, and achieves a better performance than common methods. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have achieved promising performance in the field of deep learning, but the manual design turns out to be very difficult due to the increasingly complex topologies of CNNs. Recently, neural architecture search (NAS) methods have been proposed to automatically design network architectures, which are superior to handcrafted counterparts. Unfortunately, most current NAS methods suffer from either highly computational complexity of generated architectures or limitations in the flexibility of architecture design. To address above issues, this article proposes an evolutionary neural architecture search (ENAS) method based on improved Transformer and multi-branch ConvNet. The multi-branch block enriches the feature space and enhances the representational capacity of a network by combining paths with different complexities. Since convolution is inherently a local operation, a simple yet powerful "batch-free normalization Transformer Block" (BFNTBlock) is proposed to leverage both local information and long-range feature dependencies. In particular, the design of batch-free normalization (BFN) and batch normalization (BN) mixed in the BFNTBlock blocks the accumulation of estimation shift ascribe to the stack of BN, which has favorable effects for performance improvement. The proposed method achieves remarkable accuracies, 97.24 % and 80.06 % on CIFAR10 and CIFAR100, respectively, with high computational efficiency, i.e. only 1.46 and 1.53 GPU days. To validate the universality of our method in application scenarios, the proposed algorithm is verified on two real-world applications, including the GTSRB and NEU-CLS dataset, and achieves a better performance than common methods. [ABSTRACT FROM AUTHOR]

Background: The common tea tree disease known as "tea coal disease" (Neocapnodium theae Hara) can have a negative impact on tea yield and quality. The majority of conventional approaches for identifying tea coal disease rely on observation with the human naked eye, which is labor- and time-intensive and frequently influenced by subjective factors. The present study developed a deep learning model based on RGB and hyperspectral images for tea coal disease rapid classification. Results: Both RGB and hyperspectral could be used for classifying tea coal disease. The accuracy of the classification models established by RGB imaging using ResNet18, VGG16, AlexNet, WT-ResNet18, WT-VGG16, and WT-AlexNet was 60%, 58%, 52%, 70%, 64%, and 57%, respectively, and the optimal classification model for RGB was the WT-ResNet18. The accuracy of the classification models established by hyperspectral imaging using UVE-LSTM, CARS-LSTM, NONE-LSTM, UVE-SVM, CARS-SVM, and NONE-SVM was 80%, 95%, 90%, 61%, 77%, and 65%, respectively, and the optimal classification model for hyperspectral was the CARS-LSTM, which was superior to the model based on RGB imaging. Conclusions: This study revealed the classification potential of tea coal disease based on RGB and hyperspectral imaging, which can provide an accurate, non-destructive, and efficient classification method for monitoring tea coal disease. [ABSTRACT FROM AUTHOR]

In recent years, segmentation details and computing efficiency have become more important in medical image segmentation for clinical applications. In deep learning, UNet based on a convolutional neural network is one of the most commonly used models. UNet 3+ was designed as a modified UNet by adopting the architecture of full-scale skip connections. However, full-scale feature fusion can result in excessively redundant computations. This study aimed to reduce the network parameters of UNet 3+ while further improving the feature extraction capability. First, to eliminate redundancy and improve computational efficiency, we prune the full-scale skip connections of UNet 3+. In addition, we use the attention module called Convolutional Block Attention Module (CBAM) to capture more essential features and thus improve the feature expression capabilities. The performance of the proposed model was validated by three different types of datasets: skin cancer segmentation, breast cancer segmentation, and lung segmentation. The parameters are reduced by about 36% and 18% compared to UNet and UNet 3+, respectively. The results show that the proposed method not only outperformed the comparison models in a variety of evaluation metrics but also achieved more accurate segmentation results. The proposed models have lower network parameters that enhance feature extraction and improve segmentation performance efficiently. Furthermore, the models have great potential for application in medical imaging computer-aided diagnosis. [ABSTRACT FROM AUTHOR]

Fine-grained visual classification (FGVC) has small inter-class variations and large intra-class variations, therefore, recognizing sub-classes belonging to the same meta-class is a difficult task. Recent studies have primarily addressed this problem by locating the most discriminative image regions, and the extracted image regions have been used to improve the ability to capture subtle differences. Most of these studies used regular anchors to extract local features. However, the local features of the target are mostly irregular geometric shapes. These methods cannot fully extract the features and inevitably include a large amount of irrelevant information, resulting in reduced credibility of the evaluation results. However, the spatial relationship between the features is easily overlooked. This study proposes a novel local feature extraction anchor generator (LFEAG) to simulate the shapes of irregular features. Thus, discriminative features can be fully included in the extracted features. In addition, an effective symmetrized local feature extraction module (SLFEM) based on an attention mechanism is proposed to fully use the spatial relationship between the extracted local features and highlight discriminative features. Experiments on six popular fine-grained benchmark datasets: CUB-200-2011, Stanford Dogs, Food-101, Oxford-IIIT Pets, Aircraft and NA-Birds, are conducted to demonstrate the advantages of our proposed method. [ABSTRACT FROM AUTHOR]

• A segmentation model based on 3D U-Net can help extract depth features from images. • A classification model based on Swin transformer can improve predictive ability. • The multi-model and multi-task modeling method based on deep learning is feasible. To develop a deep learning (DL) model based on preoperative contrast-enhanced computed tomography (CECT) images to predict microvascular invasion (MVI) and pathological differentiation of hepatocellular carcinoma (HCC). This retrospective study included 640 consecutive patients who underwent surgical resection and were pathologically diagnosed with HCC at two medical institutions from April 2017 to May 2022. CECT images and relevant clinical parameters were collected. All the data were divided into 368 training sets, 138 test sets and 134 validation sets. Through DL, a segmentation model was used to obtain a region of interest (ROI) of the liver, and a classification model was established to predict the pathological status of HCC. The liver segmentation model based on the 3D U-Network had a mean intersection over union (mIoU) score of 0.9120 and a Dice score of 0.9473. Among all the classification prediction models based on the Swin transformer, the fusion models combining image information and clinical parameters exhibited the best performance. The area under the curve (AUC) of the fusion model for predicting the MVI status was 0.941, its accuracy was 0.917, and its specificity was 0.908. The AUC values of the fusion model for predicting poorly differentiated, moderately differentiated and highly differentiated HCC based on the test set were 0.962, 0.957 and 0.996, respectively. The established DL models established can be used to noninvasively and effectively predict the MVI status and the degree of pathological differentiation of HCC, and aid in clinical diagnosis and treatment. [ABSTRACT FROM AUTHOR]

Motivation Deep learning approaches have empowered single-cell omics data analysis in many ways and generated new insights from complex cellular systems. As there is an increasing need for single-cell omics data to be integrated across sources, types and features of data, the challenges of integrating single-cell omics data are rising. Here, we present an unsupervised deep learning algorithm that learns discriminative representations for single-cell data via maximizing mutual information, SMILE (Single-cell Mutual Information Learning). Results Using a unique cell-pairing design, SMILE successfully integrates multisource single-cell transcriptome data, removing batch effects and projecting similar cell types, even from different tissues, into the shared space. SMILE can also integrate data from two or more modalities, such as joint-profiling technologies using single-cell ATAC-seq, RNA-seq, DNA methylation, Hi-C and ChIP data. When paired cells are known, SMILE can integrate data with unmatched feature, such as genes for RNA-seq and genome-wide peaks for ATAC-seq. Integrated representations learned from joint-profiling technologies can then be used as a framework for comparing independent single source data. Availability and implementation The source code of SMILE including analyses of key results in the study can be found at: https://github.com/rpmccordlab/SMILE , implemented in Python. Supplementary information Supplementary data are available at Bioinformatics online. [ABSTRACT FROM AUTHOR]

Background: The rise of spatial transcriptomics technologies is leading to new insights about how gene regulation happens in a spatial context. Determining which genes are expressed in similar spatial patterns can reveal gene regulatory relationships across cell types in a tissue. However, many current analysis methods do not take full advantage of the spatial organization of the data, instead treating pixels as independent features. Here, we present CoSTA: a novel approach to learn spatial similarities between gene expression matrices via convolutional neural network (ConvNet) clustering. Results: By analyzing simulated and previously published spatial transcriptomics data, we demonstrate that CoSTA learns spatial relationships between genes in a way that emphasizes broader spatial patterns rather than pixel-level correlation. CoSTA provides a quantitative measure of expression pattern similarity between each pair of genes rather than only classifying genes into categories. We find that CoSTA identifies narrower, but biologically relevant, sets of significantly related genes as compared to other approaches. Conclusions: The deep learning CoSTA approach provides a different angle to spatial transcriptomics analysis by focusing on the shape of expression patterns, using more information about the positions of neighboring pixels than would an overlap or pixel correlation approach. CoSTA can be applied to any spatial transcriptomics data represented in matrix form and may have future applications to datasets such as histology in which images of different genes are from similar but not identical biological sections. [ABSTRACT FROM AUTHOR]

This study takes advantage of recent advances in machine learning to establish a physics-based data analytic platform for distributed reconstruction of mechanical properties in layered components from full waveform data. In this vein, two logics, namely the direct inversion and physics-informed neural networks (PINNs), are explored. The direct inversion entails three steps: (i) spectral denoising and differentiation of the full-field data, (ii) building appropriate neural maps to approximate the profile of unknown physical and regularization parameters on their respective domains, and (iii) simultaneous training of the neural networks by minimizing the Tikhonov-regularized PDE loss using data from (i). PINNs furnish efficient surrogate models of complex systems with predictive capabilities via multitask learning where the field variables are modeled by neural maps endowed with (scaler or distributed) auxiliary parameters such as physical unknowns and loss function weights. PINNs are then trained by minimizing a measure of data misfit subject to the underlying physical laws as constraints. In this study, to facilitate learning from ultrasonic data, the PINNs loss adopts (a) wavenumber-dependent Sobolev norms to compute the data misfit, and (b) non-adaptive weights in a specific scaling framework to naturally balance the loss objectives by leveraging the form of PDEs germane to elastic-wave propagation. Both paradigms are examined via synthetic and laboratory test data. In the latter case, the reconstructions are performed at multiple frequencies and the results are verified by a set of complementary experiments highlighting the importance of verification and validation in data-driven modeling. [ABSTRACT FROM AUTHOR]

• A task-significance-aware meta-learning paradigm is proposed for multi-type structural damage segmentation. • An interpretable task generation strategy is designed by high-dimensional feature density clustering using a refined Jaccard distance. • A task-significance-aware mechanism is introduced into meta-learning using a standard query pool. • The proposed method outperforms directly training a recognition model and conventional MAML using only a few images. • Ablation experiments and comparative studies are performed to further demonstrate the effectiveness, necessity, generalization ability, and robustness. Recently, structural damage recognition has gained significant progress using deep learning and computer vision techniques. However, the recognition accuracy highly relies on massive training images, the inter-class balance, and the completeness of different damage categories. In addition, the generalization ability for new damage categories and robustness under real-world scenarios are challenging. This study proposes a task-aware meta-learning paradigm using limited images for universal structural damage segmentation. First, an interpretable task generation strategy instead of random sampling is designed based on feature density clustering, and a synthetical metric of Jaccard distance and Euclidean distance is established to measure the feature similarity and discover the class separability in the high-level feature space. Second, a dual-stage optimization framework is built based on Model-Agnostic Meta-Learning (MAML), comprising an internal optimization of the inner semantic segmentation model and an external optimization of the meta-learning machine. Third, core samples around the cluster center are selected to form a query pool and evaluate the task-significance scores of different tasks within a meta-batch, which are utilized in the external optimization to control the orientation of gradient updates towards more significant tasks. Finally, a multi-type structural damage dataset, including concrete crack, steel fatigue crack, concrete spalling, cable corrosion, and cable clamp slipping, is utilized to verify the effectiveness and necessity. The results show that the segmentation accuracy better outperforms directly training the inner semantic segmentation model and the conventional MAML algorithm using fewer training images. The generalization ability for new structural damage is further verified. [ABSTRACT FROM AUTHOR]

Based on the current concentration levels of gas components in the air, indoor air quality may be assessed. Gas information of mixed gas with various concentrations can be acquired and assessed using an electronic nose (E-nose). For mixed gas of varied concentrations, the E-nose's prediction accuracy using the conventional regression prediction algorithm is unsatisfactory. We propose a two-channel temporal convolutional network based on a model temporal convolutional network (TCN) suitable for time series processing that was utilized to increase the precision of the regression prediction of mixed gas concentration on the E-nose. In addition to improve the structure of the model, we also analyse its activation function. In the experiment, we use two types of gas mixture data with different concentrations, methane-ethylene and carbon monoxide(CO)-ethylene, in our experiments. It is compared with some baseline models including long short-term memory (LSTM), gated recurrent unit (GRU) and generic temporal convolutional network (TCN). In our experiments, we show that the two-channel TCN architecture is superior to both the models mentioned above and the generic TCN architecture. • An electronic nose is used to make regression prediction of mixed gas concentration. • Successfully determined CO-ethylene and methane-ethylene concentrations. • A new deep learning model called two-channel TCN is proposed which is based on TCN. • The proposed two-channel TCN achieves outstanding regression prediction performance. [ABSTRACT FROM AUTHOR]

Summary: This paper proposes an identification framework based on a restricted Boltzmann machine (RBM) for crack identification and extraction from images containing cracks and complicated background inside steel box girders of bridges. The original images that include fatigue crack and other background information are obtained by a consumer‐grade camera inside the steel box girder. The original images are cut into a number of elements with small size as the input dataset, and a state representation vector is artificially labeled to every image element used for the crack identification. A deep learning model or network consisting of multiple processing RBM layers to learn the abstract features is constructed to match the input image elements with corresponding state representation vectors. Next, a three‐layer RBM with 500; 500; and 2,000 hidden units is trained as the hidden layers in the deep learning network. A contrastive divergence learning algorithm is employed for training the deep network to update and obtain the optimal parameters (i.e., the biases and weights). The new input image elements labeled as crack are sorted out and assembled to form an output image. A deep network is modeled through the consumer‐grade camera images containing cracks and complicated background information using the proposed approach. The accuracy and ability to identify cracks from new images with different resolutions using the trained deep network are validated. Furthermore, effects of element size on reconstruction error and identification accuracy are investigated. The results show that there exists optimal element size; that is, too small and too large element sizes both increase the reconstruction error and decrease the identification accuracy. [ABSTRACT FROM AUTHOR]

As an important task for the management of bike sharing systems, accurate forecast of travel demand could facilitate dispatch and relocation of bicycles to improve user satisfaction. In recent years, many deep learning algorithms have been introduced to improve bicycle usage forecast. A typical practice is to integrate convolutional (CNN) and recurrent neural network (RNN) to capture spatial–temporal dependency in historical travel demand. For typical CNN, the convolution operation is conducted through a kernel that moves across a "matrix-format" city to extract features over spatially adjacent urban areas. This practice assumes that areas close to each other could provide useful information that improves prediction accuracy. However, bicycle usage in neighboring areas might not always be similar, given spatial variations in built environment characteristics and travel behavior that affect cycling activities. Yet, areas that are far apart can be relatively more similar in temporal usage patterns. To utilize the hidden linkage among these distant urban areas, the study proposes an irregular convolutional Long-Short Term Memory model (IrConv+LSTM) to improve short-term bike sharing demand forecast. The model modifies traditional CNN with irregular convolutional architecture to leverage the hidden linkage among "semantic neighbors". The proposed model is evaluated with a set of benchmark models in five study sites, which include one dockless bike sharing system in Singapore, and four station-based systems in Chicago, Washington, D.C., New York, and London. We find that IrConv+LSTM outperforms other benchmark models in the five cities. The model also achieves superior performance in areas with varying levels of bicycle usage and during peak periods. The findings suggest that "thinking beyond spatial neighbors" can further improve short-term travel demand prediction of urban bike sharing systems. • Propose an Irregular Convolutional Long Short-Term Memory Network (IrConv+LSTM) model. • Introduce the novel concept of semantic neighbors. • IrConv+LSTM with semantic neighbors outperforms benchmark models in the five cities. • Leveraging hidden linkage over areas with similar usage patterns facilitates the demand forecast. [ABSTRACT FROM AUTHOR]

Hepatocellular carcinoma (HCC) is a common malignancy with poor survival and requires long-term follow-up. Hence, we collected information on patients with Primary Hepatocellular Carcinoma in the United States from the Surveillance, Epidemiology, and EndResults (SEER) database. We used this information to establish a deep learning with a multilayer neural network (the NMTLR model) for predicting the survival rate of patients with Primary Hepatocellular Carcinoma. HCC patients pathologically diagnosed between January 2011 and December 2015 in the SEER (Surveillance, Epidemiology, and End Results) database of the National Cancer Institute of the United States were selected as study subjects. We utilized two deep learning-based algorithms (DeepSurv and Neural Multi-Task Logistic Regression [NMTLR]) and a machine learning-based algorithm (Random Survival Forest [RSF]) for model training. A multivariable Cox Proportional Hazards (CoxPH) model was also constructed for comparison. The dataset was randomly divided into a training set and a test set in a 7:3 ratio. The training dataset underwent hyperparameter tuning through 1000 iterations of random search and fivefold cross-validation. Model performance was assessed using the concordance index (C-index), Brier score, and Integrated Brier Score (IBS). The accuracy of predicting 1-year, 3-year, and 5-year survival rates was evaluated using Receiver Operating Characteristic (ROC) curves, calibration plots, and Area Under the Curve (AUC). The primary outcomes were the 1-year, 3-year, and 5-year overall survival rates. Models were developed using DeepSurv, NMTLR, RSF, and Cox Proportional Hazards regression. Model differentiation was evaluated using the C-index, calibration with concordance plots, and risk stratification capability with the log-rank test. The study included 2197 HCC patients, randomly divided into a training cohort (70%, n = 1537) and a testing cohort (30%, n = 660). Clinical characteristics between the two cohorts showed no significant statistical difference (p > 0.05). The deep learning models outperformed both RSF and CoxPH models, with C-indices of 0.735 (NMTLR) and 0.731 (DeepSurv) in the test dataset. The NMTLR model demonstrated enhanced accuracy and well-calibrated survival estimates, achieving an Area Under the Curve (AUC) of 0.824 for 1-year survival predictions, 0.813 for 3-year, and 0.803 for 5-year survival rates. This model's superior calibration and discriminative ability enhance its utility for clinical prognostication in Primary Hepatocellular Carcinoma. We deployed the NMTLR model as a web application for clinical practice. The NMTLR model have potential advantages over traditional linear models in prognostic assessment and treatment recommendations. This novel analytical approach may provide reliable information on individual survival and treatment recommendations for patients with primary liver cancer. [ABSTRACT FROM AUTHOR]

The precision monitoring of film-mulched winter wheat growth facilitates field management optimization and further improves yield. Unmanned aerial vehicle (UAV) is an effective tool for crop monitoring at the field scale. However, due to the interference of background effects caused by soil and mulch, achieving accurate monitoring of crop growth in complex backgrounds for UAV remains a challenge. Additionally, the simultaneous inversion of multiple growth parameters helped us to comprehensively monitor the overall crop growth status. This study conducted field experiments including three winter wheat mulching treatments: ridge mulching, ridge–furrow full-mulching, and flat cropping full-mulching. Three machine learning algorithms (partial least squares, ridge regression, and support vector machines) and deep neural network were employed to process the vegetation indices (VIs) feature data, and the residual neural network 50 (ResNet 50) was used to process the image data. Then the two modalities (VI feature data and image data) were fused to obtain a multi-modal fusion (MMF) model. Meanwhile, a film-mulched winter wheat growth monitoring model that simultaneously predicted leaf area index (LAI), aboveground biomass (AGB), plant height (PH), and leaf chlorophyll content (LCC) was constructed by coupling multi-task learning techniques. The results showed that the image-based ResNet 50 outperformed the VI feature-based model. The MMF improved prediction accuracy for LAI, AGB, PH, and LCC with coefficients of determination of 0.73–0.92, mean absolute errors of 0.29–3.89 and relative root mean square errors of 9.48–12.99%. A multi-task MMF model with the same loss weight distribution ([1/4, 1/4, 1/4, 1/4]) achieved comparable accuracy to the single-task MMF model, improving training efficiency and providing excellent generalization to different film-mulched sample areas. The novel technique of the multi-task MMF model proposed in this study provides an accurate and comprehensive method for monitoring the growth status of film-mulched winter wheat. [ABSTRACT FROM AUTHOR]

In the literature pertaining to futures price prediction using decomposition ensemble algorithms, little has been discussed in regard to determining the optimal number of decompositions and the extent to which information should be integrated. To investigate the effectiveness of multiple price decompositions in improving the accuracy of deep learning models for futures prices, we conducted an empirical study on the daily closing prices of China's main contracts for gold, zinc, soybean meal, crude oil, iron ore, and PTA futures. We utilized a variety of prevalent deep learning models and two groups of benchmark models: shallow machine learning models and regression models. To split the prices into basic components, we proposed a VMD-EEMD technique and fed them to the models. By controlling the decomposition times and model types, we further studied the predictive performance of the deep learning models against benchmark models without decomposition or with only VMD. Our empirical evidence showed that multiple price decompositions, as an input engineering technique, proved to be an effective approach for enhancing the prediction accuracy of deep learning models applied to financial equities. The average percentages of error reduction of double decomposition against single decomposition are 6.13%, 9.45%, 7.09%, and 4.91% for the four common performance metrics: MAE, MSE, MAPE, and RMSE, respectively. Moreover, we explored the necessity of secondary decomposition and optimal decomposition times to balance prediction accuracy and model complexity. These findings provide important theoretical implications and practical guidance for researchers. [ABSTRACT FROM AUTHOR]

In recent years, neural architecture search (NAS) algorithms based on Evolutionary Computation (EC) have demonstrated immense potential in the automated design of deep neural network architectures, garnering widespread attention in the field of deep learning. Most EC-based NAS algorithms select the best individual based on overall fitness score. However, some eliminated suboptimal individuals may only perform weakly in overall classification performance, but perform well on certain classes. To search valuable suboptimal individuals and prevent them from being eliminated, we propose a multi-population evolutionary NAS algorithm with stacked generalization (MPE-NAS). Each population evolves based on the classification accuracy of different classes. After completing the evolution process, the stacked generalization approach is utilized to fuse the searched architectures. Moreover, an integrated performance predictor based on k-nearest neighbor (KNN) regression, random forest (RF) and support vector machine (SVM) is proposed to alleviate computational cost during architecture performance evaluation. On the CIFAR benchmark dataset, the proposed algorithm is examined and compared with the most advanced algorithms, and its effectiveness is confirmed based on experiments. In addition, the proposed multi-population evolutionary (MPE) search strategy is applied to others EC-based NAS algorithms, and achieves the performance improvement without increasing computational resources. • A novel multi-population evolutionary search strategy is proposed. • A performance predictor was designed using stacked generalization. • The multi-head attention mechanism was employed. [ABSTRACT FROM AUTHOR]

• A quantitative analysis of the movement predictability of urban tourists and visitors using a nationwide mobile phone dataset. • A Markov chain model and a recurrent neural network are used to assess travelers' movement predictability. • Travelers' movement predictability is impacted by length of stay and activeness in travel patterns. • A notable geographic variation is observed: travelers' movements are more predictable in some cities, but less in others. The abilities to predict tourist movements are critical to many urban applications, such as travel recommendations, targeted advertising, and infrastructure planning. Despite its importance, our understanding on the movement predictability of urban tourists and visitors is still limited, partially due to difficulties in accessing large scale mobility observations. In this study, we aim to bridge this gap by analyzing a nationwide mobile phone dataset. The dataset captures movement traces of a large number of international travelers who visited South Korea in 2018. By introducing two prediction models, one being Markov chain and the other with a recurrent neural network architecture, we assess how well travelers' movements can be predicted under different model settings, and examine how predictability relates to travelers' length of stay and activeness in travel patterns. Since travelers' destination choices are quite diverse in South Korea, this enables us to further investigate the geographic variation of the models' performance. Results show that the Markov chain model achieves an overall accuracy between 33.4% (@Acc1 metric) and 64.2% (@Acc5 metric), compared to 41.9% (@Acc1) and 67.7% (@Acc5) for the recurrent neural network model. The prediction capabilities of both models are largely unequal across individuals, with active travelers being more predictable in general. There is a notable geographic variation in the models' performance, meaning that travelers' movements are more predictable in some cities, but less in others. We believe this study represents a new effort in portraying the movement predictability of urban tourists and visitors. The analytical framework can be applied to assist tourism planning and service deployment in cities. [ABSTRACT FROM AUTHOR]

In aerospace, marine, and other heavy industries, bearing fault diagnosis has been an essential part of improving machine life, reducing economic losses, and avoiding safety problems caused by machine bearing failures. Most existing bearing fault diagnosis methods face challenges in extracting the fault features from raw bearing fault data. Compared with traditional methods for bearing fault characteristics extraction, deep neural networks can automatically extract intrinsic features without expert knowledge. The convolutional neural network (CNN) was utilized most widely in extracting representative features of bearing faults. Fundamental to this, the hybrid models based on the CNN and individual classifiers were proposed to diagnose bearing faults. However, CNN may not be suitable for all bearing fault classifiers. It is crucial to identify the classifiers which can maximize the CNN feature extraction ability. In this paper, four hybrid models based on CNN were built, and their fault detection accuracy and efficiency were compared. The comparative analysis showed that the random forest (RF) and support vector machine (SVM) could make full use of the CNN feature extraction ability. [ABSTRACT FROM AUTHOR]

• SMOTE-REG is proposed for generating samples for small dataset to address regression tasks. • SMOTE-REG discovers new samples that match the original data distribution. • Deep neural network is applied for regression based on synthetic samples. • Experimental results demonstrate significant performance on prediction and inference. • The method enables deep learning for solving problems with a small dataset. Precise control over dimension of nanocrystals is critical to tune the properties for various applications. However, the traditional control through experimental optimization is slow, tedious and time consuming. Herein a robust deep neural network-based regression algorithm has been developed for precise prediction of length, width, and aspect ratios of semiconductor nanorods (NRs). Given there is limited experimental data available (28 samples), a Synthetic Minority Oversampling Technique for regression (SMOTE-REG) is employed first for data generation. Deep neural network is further applied to develop regression model which demonstrated the well performed prediction on both the original and generated data with a similar distribution. The prediction model is further validated with additional experimental data, showing accurate prediction results. Additionally, Local Interpretable Model-Agnostic Explanations (LIME) is used to interpret the weight for each sample, corresponding to its importance towards the target dimension, which is well validated by experimental observations. [ABSTRACT FROM AUTHOR]

• A hybrid deep learning method is proposed for bearing fault detection. • Convolutional neural network and deep forest are appropriately integrated using continuous wavelet transform. • Experimental result demonstrate good prediction accuracy of the new method. • The proposed method is able to deal with different sizes of datasets. Detection accuracy of bearing faults is crucial in saving economic loss for industrial applications. Deep learning is capable of producing high accuracy for bearing fault diagnosis; however, in most of existing deep-learning models such as a convolutional neural network (CNN) model or a deep forest (gcForest) model, the fault feature extraction process is ignored. In order to address this issue, this study develops a hybrid deep-learning model based on CNN and gcForest. In this new method, bearing vibration signals were converted into time-frequency images using the continuous wavelet transform (CWT). Then, CNN was used to extract intrinsic fault features from the images and feed them into a gcForest classifier. Experimental bearing data provided by Case Western Reserve University (CWRU) and Xi'an Jiaotong University (XJTU-SY) were used to evaluate the performance of the proposed method. The analysis results demonstrated that the proposed hybrid deep learning model can achieve higher detection accuracy than CNN and gcForest, which may be favorable to practical applications. [ABSTRACT FROM AUTHOR]

Recent years have witnessed the exponential growth of time series data as the popularity of sensing devices and development of IoT techniques; time series classification has been considered as one of the most challenging studies in time series data mining, attracting great interest over the last two decades. According to the empirical evidences, temporal representation learning-based time series classification has more superiority of accuracy, efficiency and interpretability as compared to hundreds of existing time series classification methods. However, due to the high time complexity of feature process, the performance of these methods has been severely restricted. In this paper, we first presented an efficient shapelet transformation method to improve the overall efficiency of time series classification, and then, we further developed a novel enhanced recurrent neural network model for deep representation learning to further improve the classification accuracy. Experimental results on typical real-world datasets have justified the superiority of our models over several shallow and deep representation learning competitors. [ABSTRACT FROM AUTHOR]

[Display omitted] • H-CRNN is proposed for early prediction of low concentration CO. • A novel activation function Hardswish is successfully applied to H-CRNN. • A gated attention mechanism is proposed to enhance the reliability of H-CRNN. • H-CRNN achieves outstanding CO concentration prediction performance. Among harmful gases, carbon monoxide (CO) has posed a significant threat to human safety due to its concealment and harmfulness. Therefore, it's imperative for the preservation of human safety to implement early detection for CO concentration. However, traditional methods for gas detection are either plagued by the limitations of insufficient accuracy like electronic nose (E-nose) equipped with relatively simple machine learning algorithms such as vanilla SVM or RNN, or limited by the incapacity to make early prediction such as sensory analysis, which requires a large amount of manpower resources and time. To address this issue, a novel technique, namely H-CRNN, is presented for E-nose to perform early prediction of low concentration CO, leveraging the strengths of convolutional and recurrent neural network to efficiently capture long-term dependencies within data. Also, shortcut connection and a novel gated attention mechanism is applied to enhance the capacity to capture crucial features. Furthermore, a linear process unit has been incorporated to handle the scale insensitivity issue. The experimental results show that the proposed H-CRNN outperforms RNN with a notable average reduction of 50.51% in the root relative squared error and 53.8% in the relative absolute error, and exhibits varying degrees of improvement when compared to state-of-the-art algorithms (TCN, TPA-LSTM, STCN, LSTNet, LSTM). Additionally, the H-CRNN achieved an average accuracy rate of 96.42%, surpassing all other algorithms and demonstrating the highest level of performance. Therefore, our work substantiates the proposed H-CRNN as an applicable method for early prediction of CO concentration. [ABSTRACT FROM AUTHOR]

During the construction process, many construction vehicles gather in a small area in a short period, thus the accurate identification of dense multiple vehicles is of great significance for ensuring the safety of construction sites. In this study, a novel end-to-end deep learning network, namely orientation-aware feature fusion single-stage detection (OAFF-SSD), is proposed for the precise detection of dense multiple construction vehicles using images from Unmanned Aerial Vehicle (UAV). The proposed OAFF-SSD consists of three main modules: (1) multi-level feature extraction, (2) novel feature fusion, and (3) new orientation-aware bounding box (OABB) proposal and regression. Meanwhile, specific strategies are designated for the fast convergence of training losses. The application of OAFF-SSD to real construction sites vehicle detection and comparison with the well-known SSD (a benchmark using traditional bounding box) and orientation-aware SSD (OA-SSD) demonstrate the efficiency and accuracy of the proposed method. • An end-to-end deep learning approach is proposed to detect vehicles from UAV. • An OABB proposal module is presented for dense multiple detection. • A feature fusion module is added to the network for more precise detection. [ABSTRACT FROM AUTHOR]

Accurate and effective monitoring of environmental parameters in tea seedling greenhouses is an important basis for regulating the seedling environment, which is crucial for improving the seedling growth quality. This study proposes a tea seedling growth simulation (TSGS) model based on deep learning. The Internet of Things system was used to measure environmental change during the whole seedling process. The correlation between the environmental parameters and the biomass growth of tea seedlings in various varieties was analyzed. A CNN-LSTM network was proposed to build the TSGS model of light, temperature, water, gas, mineral nutrition, and growth biomass. The results showed that: (1) the average correlation coefficients of air temperature, soil temperature, and soil moisture with the biomass growth of tea seedlings were 0.78, 0.84, and −0.63, respectively, which were three important parameters for establishing the TSGS model. (2) For evaluating the TSGS model of a single variety, the accuracy of ZM's TSGS based on the CNN-LSTM network was the highest (Rp2 = 0.98, RMSEP = 0.14). (3) For evaluating the TSGS model of multiple varieties, the accuracy of TSGS based on the CNN-LSTM network was the highest (Rp2 = 0.96, RMSEP = 0.17). This study provided effective technical parameters for intelligent control of tea-cutting growth and a new method for rapid breeding. [ABSTRACT FROM AUTHOR]

Accurate prediction of gas concentrations can help people to know if there is hazardous gas in the environment, thus protecting people's health. The electronic nose (E-nose) performance in predicting gas concentrations mostly depends on its algorithm. In recent years, recurrent neural networks (RNNs) have demonstrated their suitability for processing time series data. Typical models are long short-term memory (LSTM) neural networks and gated recursive units (GRUs). However, traditional recurrent neural networks use a gate structure to drop some information randomly, which can cause important features to be discarded. The temporal convolutional neural network (TCN) based on convolutional structures is more suitable for time series prediction. To enhance the accurate model forecasts, we first optimized the critical parameters in the model. Then, we adapted the residual structure of TCN and replaced the activation function of TCN with Gaussian error linear units (GELUs), And the modified model is named Gaussian-TCN. Finally, the experimental results show that the proposed Gaussian-TCN outperforms traditional recurrent networks regarding prediction accuracy. • An electronic nose with sensors is used to predict CO concentrations under different conditions. • Large-scale measurements with more than ten million data points are used to train a deep learning model for the electronic nose. • The paper proposes a new deep learning model called Gaussian-TCN, which is based on the temporal convolutional neural network and Gaussian error linear units. • The proposed Gaussian-TCN achieves significant predictive performance and outperforms TCN, GRU, and LSTM. [ABSTRACT FROM AUTHOR]

Accurate prediction of gas concentrations can help people to know if there is hazardous gas in the environment, thus protecting people's health. The electronic nose (E-nose) performance in predicting gas concentrations mostly depends on its algorithm. In recent years, recurrent neural networks (RNNs) have demonstrated their suitability for processing time series data. Typical models are long short-term memory (LSTM) neural networks and gated recursive units (GRUs). However, traditional recurrent neural networks use a gate structure to drop some information randomly, which can cause important features to be discarded. The temporal convolutional neural network (TCN) based on convolutional structures is more suitable for time series prediction. To enhance the accurate model forecasts, we first optimized the critical parameters in the model. Then, we adapted the residual structure of TCN and replaced the activation function of TCN with Gaussian error linear units (GELUs), And the modified model is named Gaussian-TCN. Finally, the experimental results show that the proposed Gaussian-TCN outperforms traditional recurrent networks regarding prediction accuracy. • An electronic nose with sensors is used to predict CO concentrations under different conditions. • Large-scale measurements with more than ten million data points are used to train a deep learning model for the electronic nose. • The paper proposes a new deep learning model called Gaussian-TCN, which is based on the temporal convolutional neural network and Gaussian error linear units. • The proposed Gaussian-TCN achieves significant predictive performance and outperforms TCN, GRU, and LSTM. [ABSTRACT FROM AUTHOR]

• Tea seedling growth is monitored by hyperspectral imaging. • Spectrum is extracted by Mask R-CNN network with high precision. • UVE is the best among the feature band selection methods. • The CNN-GRU regression model is proposed, and the effect is better than single-layer deep learning and machine learning models. The shoot and root growth not only serve as important indicators for measuring the quality of tea cutting seedlings, but also can provide scientific basis for agricultural production and fine seedling management. Currently, manual weighing is the main method used to measure shoot and root growth, which is both destructive and inefficient. The study proposes a method for monitoring the shoot and root growth in tea cutting seedlings using hyperspectral imaging. First, mature leaves and shoots spectra were extracted from tea cutting seedlings using Mask R-CNN (Epoch=20 and Learning-rate=0.001). The spectra were then preprocessed by MSC, S-G, 1-D filtering techniques, with feature bands screened by UVE (Optimal factor number=5), CARS (Monte Carlo sampling times=300), and SPA (Epoch=25). Finally, a CNN-GRU (Epoch=100 and Learning-rate=0.01) network was employed to predict shoot and root biomass, and compared with machine learning methods of SVR (Kernel function=Polynomial), RFR (Ntrees=200), and PLSR (Latent variable=16) and deep learning methods (Epoch=100 and Learning-rate=0.01) of CNN and LSTM. The results show that (i) Mask R-CNN can accurately extract the spectral of mature leaves (precision=97.8 %) and shoots (precision=91.5 %), (ii) The spectral feature bands of shoot (number=212) and root (number=105) biomass screened by UVE were more abundant than CARS and SPA, (iii) The UVE+CNN-GRU model (Rp2=0.90, RMSEP = 0.12, RPD=2.43) based on shoots spectrum provided the optimal estimation of shoot biomass. The SPA+LSTM model (Rp2 = 0.65, RMSEP = 0.05, RPD=1.67) based on the mature leaves spectrum provided the optimal estimation of root biomass. These findings demonstrate that hyperspectral imaging technology combined with deep learning algorithms can accurately monitor the growth of cutting seedlings quickly without causing damage. This not only provides a new data sources along with technical means for efficient screening excellent tea varieties, but also improves agricultural production efficiency and resource utilization. [ABSTRACT FROM AUTHOR]

• An intelligent segmentation method for tunnel water leakage, named TR-Unet, is proposed based on super-resolution reconstruction and triple attention. • Real-ERSGAN super-resolution reconstruction algorithm is used to perform resolution enhancement on water leakage images to strengthen the edge details of the defect regions. • An triple attention mechanism is established by fusing channel attention, spatial attention, and self-attention to enhance the model's resistance to environmental interference. • The water leakage is classified into three types: dark water leakage, green water leakage area and white dry water stained by analyzing the characteristics of the defect area. To effectively solve the issues of poor anti-environmental interference and inaccurate defect boundary segmentation in tunnel water leakage intelligent identification methods, an intelligent segmentation method based on super-resolution reconstruction and triple attention is proposed in this paper, named TR-Unet. In TR-Unet, the Real-ERSGAN super-resolution reconstruction algorithm is used to perform resolution enhancement on water leakage images to strengthen the edge details of the defect regions. Then triple attention mechanism is established by fusing channel attention, spatial attention, and self-attention, and is introduced into the encoder of the Unet network, thus enhancing the model's resistance to environmental interference. The results show that the mIoU and mPA of the TR-Unet are 84.31% and 88.95%, respectively, which are higher than the other compared models. In addition, TR-Unet has better segmentation effect of water leakage, which mainly reflected in less false recognition and better edge details. [ABSTRACT FROM AUTHOR]

Recently, deep learning has drawn significant attention in hyperspectral image (HSI) classification. With the growth of network depth and feature integration, deep learning demands abundant labeled samples to optimize many parameters. Unfortunately, most hyperspectral data are unlabeled and the available labeled samples are extremely limited. How to obtain richer features under limited training samples is a challenge for HSI classification. To tackle this issue, a new supervised multiscale alternately updated clique network (MSCN) is proposed for HSI classification to fully employ HSI features in different scales. Based on the Clique Block, we design the multiscale alternately updated clique block (MSCB) that applies convolution kernels of various sizes to adaptively exploit the multiscale HSI information and merge them within the block. Meanwhile, the recurrent feedback architecture is introduced to reuse high-level visual information and network parameters. The proposed MSCN includes two MSCBs to capture the multiscale spectral and spatial information in turn. The MSCN improves the information flow and the efficiency of parameter tuning through the feedback mechanism and the cross-utilization of multiscale feature. It not only obtains more abstract HSI information, but also reduces the network depth and the number of parameters, thereby improving the classification accuracy under limited samples. To certify the validity of the proposed MSCN, experiments are conducted on three real HSI datasets and compared with multiple state-of-the-art deep learning-based approaches. The experimental results demonstrate that the presented multiscale network achieves superior performance, especially in the case of a small number of training samples. [ABSTRACT FROM AUTHOR]

• Instance segmentation model is proposed for complex rock fragment recognition. • Multilevel feature fusion, prior anchors, and self-attention modules are designed. • Dark illumination is solved by homomorphic filtering and histogram equalisation. • Extremely large and small rock fragments can be recognised with good accuracy. • Effectiveness is verified by comparing different models in ablation experiments. Timely recognition of rock fragments and their morphological sizes can allow adjustment of excavation parameters during tunnel boring machine (TBM) tunnelling. Traditional manual inspection strongly relies on the experience and subjective determination of the human operators, and sieving tests cannot be conducted in real time and are energy-consuming. Rock fragments in real-world images are frequently observed against a dark background, distributed with a high size diversity, and blocked by each other. A novel deep learning-based method is proposed in this paper to achieve real-time on-site rock fragment recognition. The proposed instance segmentation model comprises two subnetworks: object detection and semantic segmentation. The object detection subnetwork is designed based on a modified single-shot detector architecture, and multilevel feature fusion, prior anchors, and self-attention modules are utilised to localise rock fragment regions. The semantic segmentation subnetwork is designed based on U-net. Down-sampling stages use the structures of the object detection subnetwork to share the extracted features of the rock fragments, and up-sampling stages employ skip connection and self-attention modules to accomplish binary segmentation in each detected bounding box. A total of 50 original images with a resolution of 4096 × 3072 were collected: 35 for training and 15 for testing. The results showed that 88.8% of the rock fragments can be recognised and that the average recall and average intersection-over-union values reach 0.87 and 0.76, respectively. Small rock fragments inevitably missed in the labelling process and extremely large ones can also be recognised. The predicted size distributions of the rock fragments fit well with the ground truth ones. Ablation experiments were conducted to further demonstrate the effectiveness of the proposed method. This study presents both visual recognition and statistical results of the size distribution of rock fragments during TBM tunnelling, which can assist in the prediction of rock properties and the adjustment of excavation parameters. [ABSTRACT FROM AUTHOR]

Heterologous expression is the main approach for recombinant protein production ingenetic synthesis, for which codon optimization is necessary. The existing optimization methods are based on biological indexes. In this paper, we propose a novel codon optimization method based on deep learning. First, we introduce the concept of codon boxes, via which DNA sequences can be recoded into codon box sequences while ignoring the order of bases. Then, the problem of codon optimization can be converted to sequence annotation of corresponding amino acids with codon boxes. The codon optimization models for Escherichia Coli were trained by the Bidirectional Long-Short-Term Memory Conditional Random Field. Theoretically, deep learning is a good method to obtain the distribution characteristics of DNA. In addition to the comparison of the codon adaptation index, protein expression experiments for plasmodium falciparum candidate vaccine and polymerase acidic protein were implemented for comparison with the original sequences and the optimized sequences from Genewiz and ThermoFisher. The results show that our method for enhancing protein expression is efficient and competitive. [ABSTRACT FROM AUTHOR]

• Construction of tea cold injury index for better frost damage assessment. • Multimodal data fusion performed better than unimodal data. • CNN-GRU model performed better than baseline models (GRU, PLSR, SVR, and RFR). • Soil information provided important information for the dense tea canopy. In the context of climate change, extreme weather events, represented by frost injury, are increasingly having a negative impact on the growth of tea plants. This has brought huge losses to the tea industry. Traditionally, the freezing injury of tea plants in the field was assessed by vision. This is labor-intensive and subjective. In this research, multimodal remote sensing data from different periods of natural overwintering tea plantations were collected by using unmanned aerial vehicles (UAV) equipped with multispectral (MS), thermal infrared (TIR) and RGB sensors. And the physiological data of tea leaves on the same day were obtained to construct a tea cold injury score (TCIS). Then, a convolutional neural networks-gate recurrent unit (CNN-GRU) model was improved for estimating TCIS. To better compare the performance of CNN-GRU, a single GRU model and three classical machine learning models were also used for comparison. The study found that: (1) The multimodal data fusion was superior to the unimodal data. The best prediction results were achieved for the combined bimodal MS + RGB data (Rp2 = 0.862, RMSEP = 0.138, RPD = 2.220); (2) The CNN-GRU hybrid model was superior to the other four baseline models. The best effect was achieved based on the multivariate input of MS + RGB (Rp2 = 0.862) or MS + RGB + TIR (Rp2 = 0.850); (3) The accuracy of the model after removing soil features was lower than that of the model without background removal. Therefore, the TCIS-CNN-GRU model combined with multi-source remote sensing data can objectively and accurately evaluate the cold injury phenotype of tea plants, making the CNN-GRU model more scientific and promising. [ABSTRACT FROM AUTHOR]

Thermal discharge from nuclear power plants poses a threat to the received natural water bodies, but the long-term extent and intensity of their surface thermal plumes remain unclear. In this study, we proposed a method to determine the background area for each drainage outlet and delineate the mixed surface thermal plumes based on 7,172 Landsat thermal infrared images. We further used a deep convolutional neural network integrated with prior location knowledge to extract core surface thermal plumes for 74 drainage outlets of 66 nuclear power plants worldwide. Our final model achieved a mean Intersection over Union (mIoU) of 0.8998 and an F1 score of 0.8886. We found that the mean maximal water surface temperature (WST) increment of the studied plants globally was 4.80 K. The Tianwan plant in China experienced the highest WST increase (8.51 K), followed by the Gravelines plant in France and the Ohi plant in Japan (7.91 K and 7.71 K, respectively). The Bruce plant in Canada had the largest thermal-polluted surface area (7.22 km2). We also provided the dataset, Global Coastal Nuclear power plant Thermal Plume (GCNT-Plume), to describe the long-term occurrence of water surface thermal plumes. Three influencing factors of the water surface thermal plume were further analyzed in this study, including total capacity, drainage type, and location type, which were associated with operating power, drainage method, and geographical features, respectively. Total capacity was more statistically related to the maximum of WST increment under shallow drainage condition. The mean WST increment of shallow drainage was 1.22 K higher than that of deep drainage. Surface plumes larger than 4 km2 frequently occurred in the Great Lakes, while small surface thermal plumes (< 1 km2) were primarily found in estuaries. The proposed method provides an important framework for future operational water surface thermal plume detection using remotely sensed observations and deep learning. • A practical method was developed to capture mixed thermal plumes from nuclear power plants using Landsat images. • A deep learning model integrated with prior location knowledge was applied to extract core areas of mixed thermal plumes. • The first map of the thermal pollution footprint of 66 global nuclear power plants was provided. • The intensity and extent of thermal pollution detected are related to total capacity, drainage type, and location type. [ABSTRACT FROM AUTHOR]

• We propose a model which can detect the lesion areas, diagnose pathological information and output diagnostic reports for medical images simultaneously. • Our model can learn to fuse pathological information into the generation procedure of diagnostic reports, so that our model can generate reports with higher metric scores and more accurate pathological information. • To the best of our knowledge, this is the first approach attempt to improve accuracy of pathological information in generated diagnostic reports, and we have achieved a competitive result. Background and objectives: Writing diagnostic reports for medical images is a heavy and tedious work. The automatic generation of medical image diagnostic reports can assist doctors to reduce their workload and improve diagnosis efficiency. It is of great significance to introduce image caption algorithm into medical image processing. Existing approaches attempt to generate medical image diagnostic reports using image caption algorithms but without taking the accuracy of pathological information in generated diagnostic reports into account. Methods: To solve the mentioned problem, we propose a Semantic Fusion Network (SFNet) including a lesion area detection model and a diagnostic generation model. The lesion area detection model can extract visual and pathological information from medical image, and the diagnostic report generation model can learn to fuse the two kinds of information to generate reports. Thus, the pathological information in the generated diagnostic reports can be more accurate. Results: Experimental results have verified the performance of our model (Accuracy increases 1.2% on the Ultrasound Image Dataset and 2.4% on the Open-i X-ray Image Dataset), compared with the model only using visual feature to generate diagnostic reports. Conclusions: This work utilizes computer algorithms to generate the more accurate diagnostic reports for medical images automatically, which expands the application of computer-aided diagnosis and promotes the implementation of deep learning in the medical image analysis field. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification. [ABSTRACT FROM AUTHOR]

• A spatiotemporal SDLRR model is developed for large basins and sub-basins of UYR. • Spatial features of regional SPPs are extracted based on stacked ConvLSTM. • Use static features to distinguish catchment-specific rainfall–runoff behaviors. • SDLRR improves the accuracy of runoff forecasting compared with benchmark models. • SPPs can be incidentally assessed based on the model performance of forecasting. Rainfall-runoff modeling is a complex nonlinear spatiotemporal prediction problem. However, few studies have considered the spatial characteristics of rainfall-runoff relationship in runoff forecasts based on machine learning. With the emergence of high-resolution Satellite-based Precipitation Products (SPPs) and the continuous improvement of rainfall estimation accuracy, the shortcoming of sparse spatial information for in-situ rainfall monitoring has been made up. Therefore, this study developed a large scale spatiotemporal deep learning rainfall-runoff (SDLRR) forecasting model for hydrological stations in the upper Yangtze River, and evaluated the positive impact of utilizing spatial information of three SPPs on reducing errors of runoff forecasts. The adopted remote sensing precipitation products are bias-corrected Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS), Integrated Multi-satellite Retrievals for Global Precipitation Measurement data (IMERG) and Tropical Rainfall Measuring Mission Multi-satellite Precipitation Analysis data (TMPA). For runoff forecasting at the Luoduxi (LDX) hydrological station, compared to regular Long Short Term Memory Network (LSTM) model, the proposed SDLRR model that utilizing IMERG data as precipitation input (IMERG_SDLRR) improved 15% in terms of Coefficient of Determination (R2) and improved 25% in terms of Root Mean Squared Error (RMSE). Compared to the best performance model among models using area-averaged precipitation as input, IMERG_SDLRR improved 5% in terms of R2 and 11% in terms of RMSE. Good performance was also acquired in the other hydrological stations. For extreme flood forecasts, IMERG_SDLRR decreased Mean Relative Error (MRE) by 0.29 and increased Qualified Rate (QR) by 53% compared to LSTM, and decreased MRE by 0.08 and increased QR by 6% compared to the best performance model using area-averaged precipitation as input. The utilization of IMERG or TMPA spatial information improved the accuracy of runoff forecasting. The accuracy evaluation of SPPs based on the results of spatiotemporal rainfall-runoff forecasts method was also demonstrated. The research is of great significance for developing runoff forecasting methods and optimizing water resources management. [ABSTRACT FROM AUTHOR]

Current mainstream networks for hyperspectral image (HSI) classification employ image patches as inputs for feature extraction. Spatial information extraction is limited by the size of inputs, which makes networks unable to perform effective learning and reasoning from the global perspective. As a common component for capturing long-range dependencies, non-local networks with pixel-by-pixel information interaction bring unaffordable computational costs and information redundancy. To address the above issues, we propose a class feature fused fully convolutional network (CFF-FCN) with a local feature extraction block (LFEB) and a class feature fusion block (CFFB) to jointly utilize local and global information. LFEB based on dilated convolutions and reverse loop mechanism can acquire the local spectral–spatial features at multiple levels and deliver shallower layer features for coarse classification. CFFB calculates global class representation to enhance pixel features. Robust global information is propagated to every pixel with low computational cost. CFF-FCN considers a fully global class context and obtains more discriminative representation by concatenating high-level local features and re-integrated global features. Experimental results conducted on three real HSI data sets demonstrate that the proposed fully convolutional network is superior to multiple state-of-the-art deep learning-based approaches, especially in the case of a small number of training samples. [ABSTRACT FROM AUTHOR]