Search

Cardiac magnetic resonance imaging (CMR) has emerged as a valuable diagnostic tool for cardiac diseases. However, a significant drawback of CMR is its slow imaging speed, resulting in low patient throughput and compromised clinical diagnostic quality. The limited temporal resolution also causes patient discomfort and introduces artifacts in the images, further diminishing their overall quality and diagnostic value. There has been growing interest in deep learning-based CMR imaging algorithms that can reconstruct high-quality images from highly under-sampled k-space data. However, the development of deep learning methods requires large training datasets, which have so far not been made publicly available for CMR. To address this gap, we released a dataset that includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects. Imaging studies include cardiac cine and mapping sequences. The 'CMRxRecon' dataset contains raw k-space data and auto-calibration lines. Our aim is to facilitate the advancement of state-of-the-art CMR image reconstruction by introducing standardized evaluation criteria and making the dataset freely accessible to the research community., (© 2024. The Author(s).)

Background: Early identification of the malignant propensity of pulmonary ground-glass nodules (GGNs) can relieve the pressure from tracking lesions and personalized treatment adaptation. The purpose of this study was to develop a deep learning-based method using sequential computed tomography (CT) imaging for diagnosing pulmonary GGNs., Methods: This diagnostic study retrospectively enrolled 762 patients with GGNs from West China Hospital of Sichuan University between July 2009 and March 2019. All patients underwent surgical resection and at least two consecutive time-point CT scans. We developed a deep learning-based method to identify GGNs using sequential CT imaging on a training set consisting of 1524 CT sections from 508 patients and then evaluated 256 patients in the testing set. Afterwards, an observer study was conducted to compare the diagnostic performance between the deep learning model and two trained radiologists in the testing set. We further performed stratified analysis to further relieve the impact of histological types, nodule size, time interval between two CTs, and the component of GGNs. Receiver operating characteristic (ROC) analysis was used to assess the performance of all models., Results: The deep learning model that used integrated DL-features from initial and follow-up CT images yielded the best diagnostic performance, with an area under the curve of 0.841. The observer study showed that the accuracies for the deep learning model, junior radiologist, and senior radiologist were 77.17%, 66.89%, and 77.03%, respectively. Stratified analyses showed that the deep learning model and radiologists exhibited higher performance in the subgroup of nodule sizes larger than 10 mm. With a longer time interval between two CTs, the deep learning model yielded higher diagnostic accuracy, but no general rules were yielded for radiologists. Different densities of components did not affect the performance of the deep learning model. In contrast, the radiologists were affected by the nodule component., Conclusions: Deep learning can achieve diagnostic performance on par with or better than radiologists in identifying pulmonary GGNs., (© 2022 The Authors. Thoracic Cancer published by China Lung Oncology Group and John Wiley & Sons Australia, Ltd.)

Lung cancer is one of the malignant tumors with the highest fatality rate and nearest to our lives. It poses a great threat to human health and it mainly occurs in smokers. In our country, with the acceleration of industrialization, environmental pollution, and population aging, the cancer burden of lung cancer is increasing day by day. In the diagnosis of lung cancer, Computed Tomography (CT) images are a fairly common visualization tool. CT images visualize all tissues based on the absorption of X-rays. The diseased parts of the lung are collectively referred to as pulmonary nodules, the shape of nodules is different, and the risk of cancer will vary with the shape of nodules. Computer-aided diagnosis (CAD) is a very suitable method to solve this problem because the computer vision model can quickly scan every part of the CT image of the same quality for analysis and will not be affected by fatigue and emotion. The latest advances in deep learning enable computer vision models to help doctors diagnose various diseases, and in some cases, models have shown greater competitiveness than doctors. Based on the opportunity of technological development, the application of computer vision in medical imaging diagnosis of diseases has important research significance and value. In this paper, we have used a deep learning-based model on CT images of lung cancer and verified its effectiveness in the timely and accurate prediction of lungs disease. The proposed model has three parts: (i) detection of lung nodules, (ii) False Positive Reduction of the detected nodules to filter out "false nodules," and (iii) classification of benign and malignant lung nodules. Furthermore, different network structures and loss functions were designed and realized at different stages. Additionally, to fine-tune the proposed deep learning-based mode and improve its accuracy in the detection Lung Nodule Detection, Noudule-Net, which is a detection network structure that combines U-Net and RPN, is proposed. Experimental observations have verified that the proposed scheme has exceptionally improved the expected accuracy and precision ratio of the underlined disease., Competing Interests: The authors declare that they have no conflicts of interest., (Copyright © 2022 Wenfa Jiang et al.)

Prognosis of primary pontine hemorrhage (PPH) is important for treatment planning and patient management. However, only few clinical factors were reported to have prognostic value to PPH. Here, we propose a deep learning (DL) model that mines high-dimensional prognostic information from computed tomography (CT) images and combines clinical factors for predicting individualized prognosis of PPH. We proposed a multi-task DL model to learn high-dimensional CT features of hematoma and perihematomal areas for predicting the risk of 30-day mortality, 90-day mortality and 90-day functional outcome of PPH simultaneously. We further explored the combination of the DL model and clinical factors by building a combined model. All the models were trained in a training cohort (n = 219) and tested in an independent testing cohort (n = 35). The DL model achieved area under the curve (AUC) of 0.886, 0.886, and 0.759 in predicting 30-day mortality, 90-day mortality and 90-day functional outcome of PPH in the independent testing cohort, which improved over the previously reported new PPH score and the clinical model. When combining the DL model and clinical factors, the combined model achieved improved performance (AUC = 0.920, 0.941, and 0.894), indicating that DL model mines CT information that complements clinical factors. Through DL visualization technique, we found that the internal structure of hematoma and its expansion to perihematomal regions are important for predicting the prognosis of PPH. This DL model provides an easy-to-use way for predicting individualized prognosis of PPH by mining high-dimensional information from CT images, and showed improvement over clinical factors and present methods., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 The Authors. Published by Elsevier Inc. All rights reserved.)

Objective: In a few patients with mild COVID-19, there is a possibility of the infection becoming severe or critical in the future. This work aims to identify high-risk patients who have a high probability of changing from mild to critical COVID-19 (only account for 5% of cases)., Methods: Using traditional convolutional neural networks for classification may not be suitable to identify this 5% of high risk patients from an entire dataset due to the highly imbalanced label distribution. To address this problem, we propose a Mix Contrast model, which matches original features with mixed features for contrastive learning. Three modules are proposed for training the model: 1) a cumulative learning strategy for synthesizing the mixed feature; 2) a commutative feature combination module for learning the commutative law of feature concatenation; 3) a united pairwise loss assigning adaptive weights for sample pairs with different class anchors based on their current optimization status., Results: We collect a multi-center computed tomography dataset including 918 confirmed COVID-19 patients from four hospitals and evaluate the proposed method on both the COVID-19 mild-to-critical prediction and COVID-19 diagnosis tasks. For mild-to-critical prediction, the experimental results show a recall of 0.80 and a specificity of 0.815. For diagnosis, the model shows comparable results with deep neural networks using a large dataset. Our method demonstrates improvements when the amount of training data is small or imbalanced., Significance: Identifying mild-to-critical COVID-19 patients is important for early prevention and personalized treatment planning.

Coronavirus disease 2019 (COVID-19) has spread globally, and medical resources become insufficient in many regions. Fast diagnosis of COVID-19 and finding high-risk patients with worse prognosis for early prevention and medical resource optimisation is important. Here, we proposed a fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis by routinely used computed tomography.We retrospectively collected 5372 patients with computed tomography images from seven cities or provinces. Firstly, 4106 patients with computed tomography images were used to pre-train the deep learning system, making it learn lung features. Following this, 1266 patients (924 with COVID-19 (471 had follow-up for >5 days) and 342 with other pneumonia) from six cities or provinces were enrolled to train and externally validate the performance of the deep learning system.In the four external validation sets, the deep learning system achieved good performance in identifying COVID-19 from other pneumonia (AUC 0.87 and 0.88, respectively) and viral pneumonia (AUC 0.86). Moreover, the deep learning system succeeded to stratify patients into high- and low-risk groups whose hospital-stay time had significant difference (p=0.013 and p=0.014, respectively). Without human assistance, the deep learning system automatically focused on abnormal areas that showed consistent characteristics with reported radiological findings.Deep learning provides a convenient tool for fast screening of COVID-19 and identifying potential high-risk patients, which may be helpful for medical resource optimisation and early prevention before patients show severe symptoms., Competing Interests: Conflict of interest: S. Wang has nothing to disclose. Conflict of interest: Y. Zha has nothing to disclose. Conflict of interest: W. Li has nothing to disclose. Conflict of interest: Q. Wu has nothing to disclose. Conflict of interest: X. Li has nothing to disclose. Conflict of interest: M. Niu has nothing to disclose. Conflict of interest: M. Wang has nothing to disclose. Conflict of interest: X. Qiu has nothing to disclose. Conflict of interest: H. Li has nothing to disclose. Conflict of interest: H. Yu has nothing to disclose. Conflict of interest: W. Gong has nothing to disclose. Conflict of interest: Y. Bai has nothing to disclose. Conflict of interest: L. Li has nothing to disclose. Conflict of interest: Y. Zhu has nothing to disclose. Conflict of interest: L. Wang has nothing to disclose. Conflict of interest: J. Tian has nothing to disclose., (Copyright ©ERS 2020.)

Importance: Accurate identification of lymph node metastasis preoperatively and noninvasively in patients with cervical cancer can avoid unnecessary surgical intervention and benefit treatment planning., Objective: To develop a deep learning model using preoperative magnetic resonance imaging for prediction of lymph node metastasis in cervical cancer., Design, Setting, and Participants: This diagnostic study developed an end-to-end deep learning model to identify lymph node metastasis in cervical cancer using magnetic resonance imaging (MRI). A total of 894 patients with stage IB to IIB cervical cancer who underwent radical hysterectomy and pelvic lymphadenectomy were reviewed. All patients underwent radical hysterectomy and pelvic lymphadenectomy, received pelvic MRI within 2 weeks before the operations, had no concurrent cancers, and received no preoperative treatment. To achieve the optimal model, the diagnostic value of 3 MRI sequences was compared, and the outcomes in the intratumoral and peritumoral regions were explored. To mine tumor information from both image and clinicopathologic levels, a hybrid model was built and its prognostic value was assessed by Kaplan-Meier analysis. The deep learning model and hybrid model were developed on a primary cohort consisting of 338 patients (218 patients from Sun Yat-sen University Cancer Center, Guangzhou, China, between January 2011 and December 2017 and 120 patients from Henan Provincial People's Hospital, Zhengzhou, China, between December 2016 and June 2018). The models then were evaluated on an independent validation cohort consisting of 141 patients from Yunnan Cancer Hospital, Kunming, China, between January 2011 and December 2017., Main Outcomes and Measures: The primary diagnostic outcome was lymph node metastasis status, with the pathologic characteristics diagnosed by lymphadenectomy. The secondary primary clinical outcome was survival. The primary diagnostic outcome was assessed by receiver operating characteristic (area under the curve [AUC]) analysis; the primary clinical outcome was assessed by Kaplan-Meier survival analysis., Results: A total of 479 patients (mean [SD] age, 49.1 [9.7] years) fulfilled the eligibility criteria and were enrolled in the primary (n = 338) and validation (n = 141) cohorts. A total of 71 patients (21.0%) in the primary cohort and 32 patients (22.7%) in the validation cohort had lymph node metastais confirmed by lymphadenectomy. Among the 3 image sequences, the deep learning model that used both intratumoral and peritumoral regions on contrast-enhanced T1-weighted imaging showed the best performance (AUC, 0.844; 95% CI, 0.780-0.907). These results were further improved in a hybrid model that combined tumor image information mined by deep learning model and MRI-reported lymph node status (AUC, 0.933; 95% CI, 0.887-0.979). Moreover, the hybrid model was significantly associated with disease-free survival from cervical cancer (hazard ratio, 4.59; 95% CI, 2.04-10.31; P < .001)., Conclusions and Relevance: The findings of this study suggest that deep learning can be used as a preoperative noninvasive tool to diagnose lymph node metastasis in cervical cancer.

Epidermal growth factor receptor (EGFR) genotyping is critical for treatment guidelines such as the use of tyrosine kinase inhibitors in lung adenocarcinoma. Conventional identification of EGFR genotype requires biopsy and sequence testing which is invasive and may suffer from the difficulty of accessing tissue samples. Here, we propose a deep learning model to predict EGFR mutation status in lung adenocarcinoma using non-invasive computed tomography (CT).We retrospectively collected data from 844 lung adenocarcinoma patients with pre-operative CT images, EGFR mutation and clinical information from two hospitals. An end-to-end deep learning model was proposed to predict the EGFR mutation status by CT scanning.By training in 14 926 CT images, the deep learning model achieved encouraging predictive performance in both the primary cohort (n=603; AUC 0.85, 95% CI 0.83-0.88) and the independent validation cohort (n=241; AUC 0.81, 95% CI 0.79-0.83), which showed significant improvement over previous studies using hand-crafted CT features or clinical characteristics (p<0.001). The deep learning score demonstrated significant differences in EGFR-mutant and EGFR-wild type tumours (p<0.001).Since CT is routinely used in lung cancer diagnosis, the deep learning model provides a non-invasive and easy-to-use method for EGFR mutation status prediction., Competing Interests: Conflict of interest: S. Wang has nothing to disclose. Conflict of interest: J. Shi has nothing to disclose. Conflict of interest: Z. Ye has nothing to disclose. Conflict of interest: D. Dong has nothing to disclose. Conflict of interest: D. Yu has nothing to disclose. Conflict of interest: M. Zhou has nothing to disclose. Conflict of interest: Y. Liu has nothing to disclose. Conflict of interest: O. Gevaert has nothing to disclose. Conflict of interest: K. Wang has nothing to disclose. Conflict of interest: Y. Zhu has nothing to disclose. Conflict of interest: H. Zhou has nothing to disclose. Conflict of interest: Z. Liu has nothing to disclose. Conflict of interest: J. Tian has nothing to disclose., (Copyright ©ERS 2019.)

In recent years, animal behavior analysis has become one of the most important methods in the fields of neuroscience and artificial intelligence. Taking advantage of the powerful deep-learning-based image analysis technology, researchers have developed state-of-the-art automatic animal behavior analysis methods with complex functions. Compared with traditional methods of animal behavior analysis, special labeling is not required in these methods, animal pose can be efficiently estimated and tracked. These methods like in a natural environment, which hold the potential for complex animal behavior experiments. Therefore, the application of deep learning in animal behavior analysis is reviewed. Firstly, this paper analyzes the tasks and current status of animal behavior analysis. Then, it highlights and compares existing deep learning-based animal behavior analysis tools. According to the dimension of experimental analysis, the deep learning-based animal behavior analysis tools are divided into two-dimensional animal behavior analysis tools and three-dimensional animal behavior analysis tools, and the functions, performance and scope of application of tools are discussed. Furthermore, the existing animal datasets and evaluation metrics are introduced, and the algorithm mechanism used in the existing animal behavior analysis tool is summarized from the advantages, limitations and applicable scenarios. Finally, the deep learning-based animal behavior analysis tools are prospected from the aspects of dataset, experimental paradigm and low latency.

Purpose: While mammography has excellent sensitivity for the detection of breast lesions, its specificity is limited. Adjunct screening with ultrasound may partially alleviate this issue but also increases false positives, resulting in unnecessary biopsies. Our study investigated the use of Google AutoML Vision (Mountain View, California), a commercially available machine learning service, to both identify and characterize indeterminate breast lesions on ultrasound. Approach: B-mode images from 253 independent cases of indeterminate breast lesions scheduled for core biopsy were used for model creation and validation. The performances of two sub-models from AutoML Vision, the image classification model and object detection model, were evaluated, while also investigating training strategies to enhance model performances. Pathology from the patients biopsy was used as a reference standard. Results: The image classification models trained under different conditions demonstrated areas under the precision-recall curve (AUC) ranging from 0.85 to 0.96 during internal validation. Once deployed, the model with highest internal performance demonstrated a sensitivity of 100% [95% confidence interval (CI) of 73.5% to 100%], specificity of 83.3% (CI=51.6% to 97.9%), positive predictive value (PPV) of 85.7% (CI=62.9% to 95.5%), and negative predictive value (NPV) of 100% (CI non-evaluable) in an independent dataset. The object detection model demonstrated lower performance internally during development (AUC=0.67) and during prediction in the independent dataset [sensitivity=75% (CI=42.8 to 94.5), specificity=80% (CI=51.9 to 95.7), PPV=75% (CI=50.8 to 90.0), and NPV=80% (CI=59.3% to 91.7%)], but was able to demonstrate the location of the lesion within the image. Conclusions: Two models appear to be useful tools for identifying and classifying suspicious areas on B-mode images of indeterminate breast lesions.

Optical encryption based on single-pixel imaging (SPI) has made great advances with the introduction of deep learning. However, the use of deep neural networks usually requires a long training time, and the networks need to be retrained once the target scene changes. With this in mind, we propose an SPI encryption scheme based on an attention-inserted physics-driven neural network. Here, an attention module is used to encrypt the single-pixel measurement value sequences of two images, together with a sequence of cryptographic keys, into a one-dimensional ciphertext signal to complete image encryption. Then, the encrypted signal is fed into a physics-driven neural network for high-fidelity decoding (i.e., decryption). This scheme eliminates the need for pre-training the network and gives more freedom to spatial modulation. Both simulation and experimental results have demonstrated the feasibility and eavesdropping resistance of this scheme. Thus, it will lead SPI-based optical encryption closer to intelligent deep encryption. [ABSTRACT FROM AUTHOR]

In recent years, deep learning-based classification methods for hyperspectral images (HSIs) have gained widespread popularity in fields such as agriculture, environmental monitoring, and geological exploration. This is owing to their ability to automatically extract features and deliver outstanding performance. This study provides a new Dilated Spectral–Spatial Gaussian Transformer Net (DSSGT) model. The DSSGT model incorporates dilated convolutions as shallow feature extraction units, which allows for an expanded receptive field while maintaining computational efficiency. We integrated transformer architecture to effectively capture feature relationships and generate deep fusion features, thereby enhancing classification accuracy. We used consecutive dilated convolutional layers to extract joint low-level spectral–spatial features. We then introduced Gaussian Weighted Pixel Embedding blocks, which leverage Gaussian weight matrices to transform the joint features into pixel-level vectors. By combining the features of each pixel with its neighbouring pixels, we obtained pixel-level representations that are more expressive and context-aware. The transformed vector matrix was fed into the transformer encoder module, enabling the capture of global dependencies within the input data and generating higher-level fusion features with improved expressiveness and discriminability. We evaluated the proposed DSSGT model using five hyperspectral image datasets through comparative experiments. Our results demonstrate the superior performance of our approach compared to those of current state-of-the-art methods, providing compelling evidence of the DSSGT model's effectiveness. [ABSTRACT FROM AUTHOR]

Tree species identification is a critical component of forest resource monitoring, and timely and accurate acquisition of tree species information is the basis for sustainable forest management and resource assessment. Airborne hyperspectral images have rich spectral and spatial information and can detect subtle differences among tree species. To fully utilize the advantages of hyperspectral images, we propose a double-branch spatial–spectral joint network based on the SimAM attention mechanism for tree species classification. This method achieved high classification accuracy on three tree species datasets (93.31% OA value obtained in the TEF dataset, 95.7% in the Tiegang Reservoir dataset, and 98.82% in the Xiongan New Area dataset). The network consists of three parts: spectral branch, spatial branch, and feature fusion, and both branches make full use of the spatial–spectral information of pixels to avoid the loss of information. In addition, the SimAM attention mechanism is added to the feature fusion part of the network to refine the features to extract more critical features for high-precision tree species classification. To validate the robustness of the proposed method, we compared this method with other advanced classification methods through a series of experiments. The results show that: (1) Compared with traditional machine learning methods (SVM, RF) and other state-of-the-art deep learning methods, the proposed method achieved the highest classification accuracy in all three tree datasets. (2) Combining spatial and spectral information and incorporating the SimAM attention mechanism into the network can improve the classification accuracy of tree species, and the classification performance of the double-branch network is better than that of the single-branch network. (3) The proposed method obtains the highest accuracy under different training sample proportions, and does not change significantly with different training sample proportions, which are stable. This study demonstrates that high-precision tree species classification can be achieved using airborne hyperspectral images and the methods proposed in this study, which have great potential in investigating and monitoring forest resources. [ABSTRACT FROM AUTHOR]

With the continuous development of surface observation methods and technologies, we can acquire multiple sources of data more effectively in the same geographic area. The quality and availability of these data have also significantly improved. Consequently, how to better utilize multi-source data to represent ground information has become an important research question in the field of geoscience. In this paper, a novel model called multi-modal transformer cascaded fusion net (MMTCFN) is proposed for fusion and classification of multi-modal remote sensing data, Hyperspectral Imagery (HSI) and LiDAR data. Feature fusion and feature extraction are the two stages of the model. First, in the feature extraction stage, a three-branch cascaded Convolutional Neural Network (CNN) framework is employed to fully leverage the advantages of convolutional operators in extracting shallow-level local features. Based on this, we generated multi-modal long-range integrated deep features utilizing the transformer-based vectorized pixel group transformer (VPGT) module during the feature fusion stage. In the VPGT block, we designed a vectorized pixel group embedding that preserves the global features extracted from the three branches in a non-overlapping multi-space manner. Moreover, we introduce the DropKey mechanism into the multi-head self-attention (MHSA) to alleviate overfitting caused by insufficient training samples. Finally, we employ a probabilistic decision fusion strategy to integrate multiple class estimations, assigning a specific category to each pixel. This model was experimented on three HSI-LiDAR datasets with balanced and unbalanced training samples. The proposed model outperforms the other seven SOTA approaches in terms of OA performance, proving the superiority of MMTCFN for the HSI-LiDAR classification task. [ABSTRACT FROM AUTHOR]

Based on the monthly precipitation data of 139 meteorological stations from 1981 to 2015, the trend line method was used to analyze the variation of the drought area and sown area from 1981 to 2015 and the characteristics of drought in Hebei Province. The results showed the following. (1) the frequency of drought was the highest in winter and the lowest in summer, which was 43.2 and 26.2%, respectively. (2) The ratio of drought stations was highest in winter and lowest in summer. The frequency of global drought was 37 and 17% in summer and winter, respectively. Except for the increasing trend of drought in summer, all other seasons showed a decreasing trend which was significant in autumn (α = 0.05). (3) The drought intensity of the four seasons was all less than 1. The drought intensity in spring and summer showed an upward trend, while that in autumn and winter showed a downward trend. The drought intensity in autumn showed a significant downward trend (α = 0.05). (4) Drought disaster rate was positively correlated with summer drought intensity and drought ratio. [ABSTRACT FROM AUTHOR]

Surgical skill assessment can quantify the quality of the surgical operation via the motion state of the surgical instrument tip (SIT), which is considered one of the effective primary means by which to improve the accuracy of surgical operation. Traditional methods have displayed promising results in skill assessment. However, this success is predicated on the SIT sensors, making these approaches impractical when employing the minimally invasive surgical robot with such a tiny end size. To address the assessment issue regarding the operation quality of robot-assisted minimally invasive surgery (RAMIS), this paper proposes a new automatic framework for assessing surgical skills based on visual motion tracking and deep learning. The new method innovatively combines vision and kinematics. The kernel correlation filter (KCF) is introduced in order to obtain the key motion signals of the SIT and classify them by using the residual neural network (ResNet), realizing automated skill assessment in RAMIS. To verify its effectiveness and accuracy, the proposed method is applied to the public minimally invasive surgical robot dataset, the JIGSAWS. The results show that the method based on visual motion tracking technology and a deep neural network model can effectively and accurately assess the skill of robot-assisted surgery in near real-time. In a fairly short computational processing time of 3 to 5 s, the average accuracy of the assessment method is 92.04% and 84.80% in distinguishing two and three skill levels. This study makes an important contribution to the safe and high-quality development of RAMIS. [ABSTRACT FROM AUTHOR]

Imagining recognition of behaviors from video sequences for a machine is full of challenges but meaningful. This work aims to predict students' behavior in an experimental class, which relies on the symmetry idea from reality to annotated reality centered on the feature space. A heteromorphic ensemble algorithm is proposed to make the obtained features more aggregated and reduce the computational burden. Namely, the deep learning models are improved to obtain feature vectors representing gestures from video frames and the classification algorithm is optimized for behavior recognition. So, the symmetric idea is realized by decomposing the task into three schemas including hand detection and cropping, hand joints feature extraction, and gesture classification. Firstly, a new detector method named YOLOv4-specific tiny detection (STD) is proposed by reconstituting the YOLOv4-tiny model, which could produce two outputs with some attention mechanism leveraging context information. Secondly, the efficient pyramid squeeze attention (EPSA) net is integrated into EvoNorm-S0 and the spatial pyramid pool (SPP) layer to obtain the hand joint position information. Lastly, the D–S theory is used to fuse two classifiers, support vector machine (SVM) and random forest (RF), to produce a mixed classifier named S–R. Eventually, the synergetic effects of our algorithm are shown by experiments on self-created datasets with a high average recognition accuracy of 89.6%. [ABSTRACT FROM AUTHOR]

Nowadays, the banks are facing increasing business pressure in loan allocations, because more and more enterprises are applying for it and financial risk is becoming vaguer. To this end, it is expected to investigate effective autonomous loan allocation decision schemes that can provide guidance for banks. However, in many real-world scenarios, the credit status information of enterprises is unknown and needs to be inferred from business status. To handle such an issue, this paper proposes a two-stage loan allocation decision framework for enterprises with unknown credit status. And the proposal is named as TLAD-UC for short. For the first stage, the idea of deep machine learning is introduced to train a prediction model that can generate credit status prediction results for enterprises with unknown credit status. For the second stage, a dynamic planning model with both optimization objective and constraint conditions is established. Through such model, both the profit and risk of banks can be well described. Solving such a dynamic planning model via computer simulation programs, the optimal allocation schemes can be suggested. [ABSTRACT FROM AUTHOR]

Recently, many models based on the combination of graph convolutional networks and deep learning have attracted extensive attention for their superior performance in graph clustering tasks. However, the existing models have the following limitations: (1) Existing models are limited by the calculation method of graph convolution, and their computational cost will increase exponentially as the graph scale grows. (2) Stacking too many convolutional layers causes the over-smoothing issue and neglects the local graph structure. (3) Expanding the range of the neighborhood and the model depth together is difficult due to the orthogonal relationship between them. Inspired by personalized pagerank and auto-encoder, we conduct the node-wise graph clustering task in the undirected simple graph as the research direction and propose a Scalable Deep Network (SDN) for graph clustering via personalized pagerank. Specifically, we utilize the combination of multi-layer perceptrons and linear propagation layer based on personalized pagerank as the backbone network (i.e., the Quasi-GNN module) and employ a DNN module for auto-encoder to learn different dimensions embeddings. After that, SDN combines the two embeddings correspondingly; then, it utilizes a dual self-supervised module to constrain the training of the embedding and clustering process. Our proposed Quasi-GNN module reduces the computational costs of traditional GNN models in a decoupled approach and solves the orthogonal relationship between the model depth and the neighborhood range. Meanwhile, it also alleviates the degraded clustering effect caused by the over-smoothing issue. We conducted experiments on five widely used graph datasets. The experimental results demonstrate that our model achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Lung cancer is the leading cause of the cancer deaths. Therefore, predicting the survival status of lung cancer patients is of great value. However, the existing methods mainly depend on statistical machine learning (ML) algorithms. Moreover, they are not appropriate for high-dimensionality genomics data, and deep learning (DL), with strong high-dimensional data learning capability, can be used to predict lung cancer survival using genomics data. The Cancer Genome Atlas (TCGA) is a great database that contains many kinds of genomics data for 33 cancer types. With this enormous amount of data, researchers can analyze key factors related to cancer therapy. This paper proposes a novel method to predict lung cancer long-term survival using gene expression data from TCGA. Firstly, we select the most relevant genes to the target problem by the supervised feature selection method called mutual information selector. Secondly, we propose a method to convert gene expression data into two kinds of images with KEGG BRITE and KEGG Pathway data incorporated, so that we could make good use of the convolutional neural network (CNN) model to learn high-level features. Afterwards, we design a CNN-based DL model and added two kinds of clinical data to improve the performance, so that we finally got a multimodal DL model. The generalized experiments results indicated that our method performed much better than the ML models and unimodal DL models. Furthermore, we conduct survival analysis and observe that our model could better divide the samples into high-risk and low-risk groups. [ABSTRACT FROM AUTHOR]

The aim of this study was to determine whether left ventricular ejection fraction (LVEF) and right ventricular ejection fraction (RVEF) and left ventricular mass (LVM) measurements made using 3 fully automated deep learning (DL) algorithms are accurate and interchangeable and can be used to classify ventricular function and risk-stratify patients as accurately as an expert. Artificial intelligence is increasingly used to assess cardiac function and LVM from cardiac magnetic resonance images. Two hundred patients were identified from a registry of individuals who underwent vasodilator stress cardiac magnetic resonance. LVEF, LVM, and RVEF were determined using 3 fully automated commercial DL algorithms and by a clinical expert (CLIN) using conventional methodology. Additionally, LVEF values were classified according to clinically important ranges: <35%, 35% to 50%, and ≥50%. Both ejection fraction values and classifications made by the DL ejection fraction approaches were compared against CLIN ejection fraction reference. Receiver-operating characteristic curve analysis was performed to evaluate the ability of CLIN and each of the DL classifications to predict major adverse cardiovascular events. Excellent correlations were seen for each DL-LVEF compared with CLIN-LVEF (r = 0.83-0.93). Good correlations were present between DL-LVM and CLIN-LVM (r = 0.75-0.85). Modest correlations were observed between DL-RVEF and CLIN-RVEF (r = 0.59-0.68). A >10% error between CLIN and DL ejection fraction was present in 5% to 18% of cases for the left ventricle and 23% to 43% for the right ventricle. LVEF classification agreed with CLIN-LVEF classification in 86%, 80%, and 85% cases for the 3 DL-LVEF approaches. There were no differences among the 4 approaches in associations with major adverse cardiovascular events for LVEF, LVM, and RVEF. This study revealed good agreement between automated and expert-derived LVEF and similarly strong associations with outcomes, compared with an expert. However, the ability of these automated measurements to accurately classify left ventricular function for treatment decision remains limited. DL-LVM showed good agreement with CLIN-LVM. DL-RVEF approaches need further refinements. [Display omitted] [ABSTRACT FROM AUTHOR]

The increasing volume and types of malwares bring a great threat to network security. The malware binary detection with deep convolutional neural networks (CNNs) has been proved to be an effective method. However, the existing malware classification methods based on CNNs are unsatisfactory to this day because of their poor extraction ability, insufficient accuracy of malware classification, and high cost of detection time. To solve these problems, a novel approach, namely, multiscale feature fusion convolutional neural networks (MFFCs), was proposed to achieve an effective classification of malware based on malware visualization utilizing deep learning, which can defend against malware variants and confusing malwares. The approach firstly converts malware code binaries into grayscale images, and then, these images will be normalized in size by utilizing the MFFC model to identify malware families. Comparative experiments were carried out to verify the performance of the proposed method. The results indicate that the MFFC stands out among the recent advanced methods with an accuracy of 98.72% and an average cost of 5.34 milliseconds on the Malimg dataset. Our method can effectively identify malware and detect variants of malware families, which has excellent feature extraction capability and higher accuracy with lower detection time. [ABSTRACT FROM AUTHOR]

Previous deep learning methods usually required large‐scale annotated data, which is computationally exhaustive and unrealistic in certain scenarios. Therefore, few‐shot learning, where only a few annotated training images are available for training, has attracted increasing attention these days, showing huge potential in practical applications, such as portable equipment or security inspection, and so on. However, current few‐shot learning methods usually neglect the valuable semantic correlations between samples, thereby failing in extracting discriminating relations to achieve accurate predictive results. In this work, extending on a recent state‐of‐the‐art few‐shot learning method, transductive relation‐propagation network (TRPN), which considers the correlations between training samples, a constrained relation‐propagation network is proposed to further regularise the distilled correlations and thus achieve favourable few‐shot classification performance. The proposed framework contains three main components, namely preprocess module, relational propagation module, and relation constraint module. First, sample features are extracted and a relation graph node is constructed by treating the relation of each support–query pair as a graph node in the preprocess module. After that, in the relation propagation module (RPM), the valuable information of support–query pairs is modelled and propagated to directly generate the relational representations for further prediction. Then, a relation constraint module is introduced to regularise the relational representations and make it consistent with the ground‐truth relations as much as possible. With the guidance of the effective RPM and relation constraint module, the relational representations of the support–query pairs are distinguishable and thus can achieve accurate predictive results. Comprehensive experiments conducted on widely used benchmarks validate the effectiveness of our method compared to state‐of‐the‐art few‐shot classification approaches. [ABSTRACT FROM AUTHOR]

• Proposed a knowledge framework for 3D reconstruction in civil engineering. • Provided a technical framework of image-based 3D reconstruction with deep learning. • Summarized multi-scale state-of-the-practice of 3D reconstruction in civil engineering. • Pointed out eight research directions such as automated modeling, model fusion, etc. As a bridge between physical objects and as-built models, image-based 3D reconstruction performs a vital role by generating point cloud models, mesh models, textured models, and eventually BIMs from images. This study provides a quantitative and qualitative summary of image-based 3D reconstruction for civil engineering projects in the last decade. A bibliometric analysis of 286 journal papers suggested that 3D reconstruction is an interdisciplinary field that integrates photogrammetry, 3D point cloud analysis, semantic segmentation, and deep learning. Based on the analysis, we proposed a 3D reconstruction knowledge framework with three dimensions — essential elements, use phases, and reconstruction scales. The "essential elements" dimension is a technical framework of visual geometry and deep learning methods for 3D model generation. The "use phases" emphasize using 3D reconstruction techniques during the construction, operation, and maintenance phases, which are driven by the demands of visual inspection in various contexts. The "reconstruction scales" dimension synthesizes 3D reconstruction applications from the component level to the city scale with highlights of their opportunities and challenges. This 3D reconstruction knowledge framework sheds light on eight future research directions: automated modeling, model fusion, performance optimization, data fusion, enhanced virtual experience, real-time modeling, standardized reference, and in-depth deep learning research. This review can help scholars understand the present status and highlight research trends of image-based 3D reconstruction in civil engineering associated with the integration of deep learning methods. [ABSTRACT FROM AUTHOR]

Accurate RNA secondary structure information is the cornerstone of gene function research and RNA tertiary structure prediction. However, most traditional RNA secondary structure prediction algorithms are based on the dynamic programming (DP) algorithm, according to the minimum free energy theory, with both hard and soft constraints. The accuracy is particularly dependent on the accuracy of soft constraints (from experimental data like chemical and enzyme detection). With the elongation of the RNA sequence, the time complexity of DP-based algorithms will increase geometrically, as a result, they are not good at coping with relatively long sequences. Furthermore, due to the complexity of the pseudoknots structure, the secondary structure prediction method, based on traditional algorithms, has great defects which cannot predict the secondary structure with pseudoknots well. Therefore, few algorithms have been available for pseudoknots prediction in the past. The ATTfold algorithm proposed in this article is a deep learning algorithm based on an attention mechanism. It analyzes the global information of the RNA sequence via the characteristics of the attention mechanism, focuses on the correlation between paired bases, and solves the problem of long sequence prediction. Moreover, this algorithm also extracts the effective multi-dimensional features from a great number of RNA sequences and structure information, by combining the exclusive hard constraints of RNA secondary structure. Hence, it accurately determines the pairing position of each base, and obtains the real and effective RNA secondary structure, including pseudoknots. Finally, after training the ATTfold algorithm model through tens of thousands of RNA sequences and their real secondary structures, this algorithm was compared with four classic RNA secondary structure prediction algorithms. The results show that our algorithm significantly outperforms others and more accurately showed the secondary structure of RNA. As the data in RNA sequence databases increase, our deep learning-based algorithm will have superior performance. In the future, this kind of algorithm will be more indispensable. [ABSTRACT FROM AUTHOR]

Compared to traditional data-driven learning methods, recently developed deep reinforcement learning (DRL) approaches can be employed to train robot agents to obtain control policies with appealing performance. However, learning control policies for real-world robots through DRL is costly and cumbersome. A promising alternative is to train policies in simulated environments and transfer the learned policies to real-world scenarios. Unfortunately, due to the reality gap between simulated and real-world environments, the policies learned in simulated environments often cannot be generalized well to the real world. Bridging the reality gap is still a challenging problem. In this paper, we propose a novel real–sim–real (RSR) transfer method that includes a real-to-sim training phase and a sim-to-real inference phase. In the real-to-sim training phase, a task-relevant simulated environment is constructed based on semantic information of the real-world scenario and coordinate transformation, and then a policy is trained with the DRL method in the built simulated environment. In the sim-to-real inference phase, the learned policy is directly applied to control the robot in real-world scenarios without any real-world data. Experimental results in two different robot control tasks show that the proposed RSR method can train skill policies with high generalization performance and significantly low training costs. [ABSTRACT FROM AUTHOR]

Locating diseases precisely from medical images, like ultrasonic and CT images, have been one of the most challenging problems in medical image analysis. In recent years, the vigorous development of deep learning models have greatly improved the accuracy in disease location on medical images. However, there are few artificial intelligent methods for identifying cholelithiasis and classifying gallstones on CT images, since no open source CT images dataset of cholelithiasis and gallstones is available for training the models and verifying their performance. In this paper, we build up the first medical image dataset of cholelithiasis by collecting 223846 CT images with gallstone of 1369 patients. With these CT images, a neural network is trained to “pick up” CT images of high quality as training set, and then a novel Yolo neural network, named Yolov3-arch neural network, is proposed to identify cholelithiasis and classify gallstones on CT images. Identification and classification accuracies are obtained by 10-fold cross-validations. It is obtained that our Yolov3-arch model is with average accuracy 92.7% in identifying granular gallstones and average accuracy 80.3% in identifying muddy gallstones. This achieves 3.5% and 8% improvements in identifying granular and muddy gallstones to general Yolo v3 model, respectively. Also, the average cholelithiasis identifying accuracy is improved to 86.50% from 80.75%. Meanwhile, our method can reduce the misdiagnosis rate of negative samples by the object detection model. [ABSTRACT FROM AUTHOR]

• Computing on edge end improves the efficiency and reliability of data analysis. • A lightweight CNN model is efficient for real-time crowd density estimation on edge. • Better crowd density inference speed with a slight increase in estimation accuracy. • Equip the model in an IoT device to monitor the crowd density in a Subway Station. Crowd stampedes and incidents are critical threats to public security that have caused countless deaths during the past few decades. To avoid crowd stampedes, real-time crowd density estimation can help monitor crowd movements, and thus support a timely evacuation strategy development. In previous studies, scholars and engineers developed multiple video-based crowd density estimation algorithms based on deep neural networks. The excessive computational complexity of deep learning algorithms exacerbated the algorithm's efficiency, causing unacceptable real-time performance. In the Internet of Things era, deploying the crowd density estimation task with edge computing is an advanced strategy to maintain the real-time performance of the entire system. Considering the limited computational resources on the edge devices, deep learning-based crowd density estimation algorithms normally cannot be handled. To fulfill the deployment on the edge device, the algorithms need to be optimized with a smaller model size. Therefore, this paper proposes a lightweight Convolutional Neural Networks (CNN) based crowd density estimation model by combining the modified MobileNetv2 and the dilated convolution. Public crowd image data sets are used to conduct experiments for evaluating the performance of the proposed algorithm in terms of accuracy and inference speed. The results show that our model achieves much better inference speed accompanied by a slight increase in accuracy. The proposed method of this study can enhance the performance of the crowd monitoring system, and therefore help avoid crowd stampedes and incidents. [ABSTRACT FROM AUTHOR]

• Combination of deep learning network and clinical criteria ACR TI-RADS. • Large-scale and high-quality thyroid nodule ultrasound image dataset. • Efficient and lightweight network. Thyroid nodule is one of the most common endocrine diseases in the adult population. Early screening and diagnosis of thyroid nodules is of great significance to patients' subsequent treatment. An objective and accurate diagnosis algorithm for thyroid nodules is vital to improve the efficiency of clinical diagnosis and reduce the work pressure of doctors. In this paper, we propose a fused deep learning model for the diagnosis of benign and malignant thyroid nodules. Based on the ACR TI-RADS proposed by the American Academy of Radiology, the model first extracts 33 clinically significant statistical features of composition, echogenicity, shape, margin, and echogenic foci. After PCA dimensionality reduction, the top 4 features are fused with the feature map of EfficientNet B3. The thyroid nodules are classified into benign and malignant based on the fused feature map. This model is trained and tested on 3828 clinically collected ultrasound images of thyroid nodules. It achieves an accuracy of 96.4% and an AUC of 0.965. Compared with traditional machine learning algorithms and state-of-the-art deep learning networks, the fused model we proposed has improved classification performance and amount of parameters, and is also clinically interpretable. [ABSTRACT FROM AUTHOR]

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization workflow, which is time-consuming and requires prior information. Convolutional neural networks (CNNs) can regress the six degrees of freedom (6-DOF) extrinsic parameters from raw LiDAR and image data. However, these CNN-based methods just learn the representations of the projected LiDAR and image and ignore the correspondences at different locations. The performances of these CNN-based methods are unsatisfactory and worse than those of non-CNN methods. In this paper, we propose a novel CNN-based LiDAR-camera extrinsic calibration algorithm named CFNet. We first decided that a correlation layer should be used to provide matching capabilities explicitly. Then, we innovatively defined calibration flow to illustrate the deviation of the initial projection from the ground truth. Instead of directly predicting the extrinsic parameters, we utilize CFNet to predict the calibration flow. The efficient Perspective-n-Point (EPnP) algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D–3D correspondences constructed by the calibration flow. Due to its consideration of the geometric information, our proposed method performed better than the state-of-the-art CNN-based methods on the KITTI datasets. Furthermore, we also tested the flexibility of our approach on the KITTI360 datasets. [ABSTRACT FROM AUTHOR]

The design of a ship detection model that can be adapted to both foggy and clear images faces significant challenges. Existing methods are either not accurate enough, or have a high amount of model parameters, making them difficult to deploy to lightweight front-ends. To address these issues, a lightweight deep learning model based on combined optimization of dehazing and detection is proposed, focusing on self-adaptive ship detection. Firstly, a self-adaptive image dehazing module is designed and placed ahead of the detection network, including a dehazing parameter predictor and an improved dehazing method. Subsequently, a lightweight-improved object detection deep learning model integrated with the dehazing module is devised to detect the ship in the foggy image. Experimental results demonstrate the effectiveness of this approach in enabling efficient and accurate ship detection under foggy conditions. Through the joint optimization of the dehazing module and the detection module, it can be seen from the experiments that our Dehazing + Detection model has the highest detection accuracy and performs well in terms of detection speed, parameter amount, and weight file size. The detection accuracy has reached 97.1%, which is better than that of the other three dehazing + detection models. • A model based on combined optimization of dehazing and detection is designed. • A self-adaptive dehazing module is designed to work alongside the detection module. • We have improved the baseline model (YOLO v5s) to enhance the model's performance. • Extensive experiments are conducted to confirm the superiority of our model. [ABSTRACT FROM AUTHOR]

Effective prediction of PM 2.5 long-term concentration can help reduce exposure risks, but few current studies based on machine learning have been able to credibly predict concentration changing trends after 12 h. To address this issue, this study employed the complementary ensemble empirical mode decomposition (CEEMD) method combined with sample entropy (SE) index, as well as bidirectional long short-term memory (BiLSTM) model stacked denoising auto-encoder (AE) method for better decomposition and prediction of long-term trends in PM 2.5 time series data. The new CEEMD-AE-BiLSTM model has the best long-term prediction results at different sites, and the R2 value is 0.871, and the RMSE is 9.28 μg/m3 at T+12 moment in the 1002A site. The new model effectively captures a significant portion of the long-term concentration trends characteristically and has been proven to be an effective tool for long-term predictions. This is attributed to the robust noise resistance of the AE method and the appropriate parameter setting of the CEEMD method. • The new model can predict the long-term trend in PM 2.5 concentration. • The CEEMD method can better decompose the time series data of PM 2.5 concentration. • The AE method can effectively resist the noise of component and enhance the accuracy of the prediction results. • The SE index can effectively couple decomposition method and time series prediction model. [ABSTRACT FROM AUTHOR]