Search

Magnetic Resonance Imaging (MRI) is increasingly being used in treatment planning due to its superior soft tissue contrast, which is useful for tumor and soft tissue delineation compared to computed tomography (CT). However, MRI cannot directly provide mass density or relative stopping power (RSP) maps, which are required for calculating proton radiotherapy doses. Therefore, the integration of artificial intelligence (AI) into MRI-based treatment planning to estimate mass density and RSP directly from MRI has generated significant interest. A deep learning (DL) based framework was developed to establish a voxel-wise correlation between MR images and mass density as well as RSP. To facilitate the study, five tissue substitute phantoms were created, representing different tissues such as skin, muscle, adipose tissue, 45% hydroxyapatite (HA), and spongiosa bone. The composition of these phantoms was based on information from ICRP reports. Additionally, two animal tissue phantoms, simulating pig brain and liver, were prepared for DL training purposes. The phantom study involved the development of two DL models. The first model utilized clinical T1 and T2 MRI scans as input, while the second model incorporated zero echo time (ZTE) MRI scans. In the patient application study, two more DL models were trained: one using T1 and T2 MRI scans as input, and another model incorporating synthetic dual-energy computed tomography (sDECT) images to provide accurate bone tissue information. The DECT empirical model was used as a reference to evaluate the proposed models in both phantom and patient application studies. The DECT empirical model was selected as the reference for evaluating the proposed models in both phantom and patient application studies. In the phantom study, the DL model based on T1, and T2 MRI scans demonstrated higher accuracy in estimating mass density and RSP for skin, muscle, adipose tissue, brain, and liver. The mean absolute percentage errors (MAPE) were 0.42%, 0.14%, 0.19%, 0.78%, and 0.26% for mass density, and 0.30%, 0.11%, 0.16%, 0.61%, and 0.23% for RSP, respectively. The DL model incorporating ZTE MRI further improved the accuracy of mass density and RSP estimation for 45% HA and spongiosa bone, with MAPE values of 0.23% and 0.09% for mass density, and 0.19% and 0.07% for RSP, respectively. These results demonstrate the feasibility of using an MRI-only approach combined with DL methods for mass density and RSP estimation in proton therapy treatment planning. By employing this approach, it is possible to obtain the necessary information for proton radiotherapy directly from MRI scans, eliminating the need for additional imaging modalities., (© 2024. The Author(s).)

The advantage of proton therapy as compared to photon therapy stems from the Bragg peak effect, which allows protons to deposit most of their energy directly at the tumor while sparing healthy tissue. However, even with such benefits, proton therapy does present certain challenges. The biological effectiveness differences between protons and photons are not fully incorporated into clinical treatment planning processes. In current clinical practice, the relative biological effectiveness (RBE) between protons and photons is set as constant 1.1. Numerous studies have suggested that the RBE of protons can exhibit significant variability. Given these findings, there is a substantial interest in refining proton therapy treatment planning to better account for the variable RBE. Dose-average linear energy transfer (LET d ) is a key physical parameter for evaluating the RBE of proton therapy and aids in optimizing proton treatment plans. Calculating precise LET d distributions necessitates the use of intricate physical models and the execution of specialized Monte-Carlo simulation software, which is a computationally intensive and time-consuming progress. In response to these challenges, we propose a deep learning based framework designed to predict the LET d distribution map using the dose distribution map. This approach aims to simplify the process and increase the speed of LET d map generation in clinical settings. The proposed CycleGAN model has demonstrated superior performance over other GAN-based models. The mean absolute error (MAE), peak signal-to-noise ratio and normalized cross correlation of the LET d maps generated by the proposed method are 0.096 ± 0.019 keV μ m -1 , 24.203 ± 2.683 dB, and 0.997 ± 0.002, respectively. The MAE of the proposed method in the clinical target volume, bladder, and rectum are 0.193 ± 0.103, 0.277 ± 0.112, and 0.211 ± 0.086 keV μ m -1 , respectively. The proposed framework has demonstrated the feasibility of generating synthetic LET d maps from dose maps and has the potential to improve proton therapy planning by providing accurate LET d information., (Creative Commons Attribution license.)

Objective: Mapping CT number to material property dominates the proton range uncertainty. This work aims to develop a physics-constrained deep learning-based multimodal imaging (PDMI) framework to integrate physics, deep learning, MRI, and advanced dual-energy CT (DECT) to derive accurate patient mass density maps., Methods: Seven tissue substitute MRI phantoms were used for validation including adipose, brain, muscle, liver, skin, spongiosa, 45% hydroxyapatite (HA) bone. MRI images were acquired using T 1 weighted Dixon and T 2 weighted short tau inversion recovery sequences. Training inputs are from MRI and twin-beam dual-energy images acquired at 120 kVp with gold/tin filters. The feasibility investigation included an empirical model and four residual networks (ResNet) derived from different training inputs and strategies by PDMI framework. PRN-MR-DE and RN-MR-DE denote ResNet (RN) trained with and without a physics constraint (P) using MRI (MR) and DECT (DE) images. PRN-DE stands for RN trained with a physics constraint using only DE images. A retrospective study using institutional patient data was also conducted to investigate the feasibility of the proposed framework., Results: For the tissue surrogate study, PRN-MR-DE, PRN-DE, and RN-MR-DE result in mean mass density errors: -0.72%/2.62%/-3.58% for adipose; -0.03%/-0.61%/-0.18% for muscle; -0.58%/-1.36%/-4.86% for 45% HA bone. The retrospective patient study indicated that PRN-MR-DE predicted the densities of soft tissue and bone within expected intervals based on the literature survey, while PRN-DE generated large density deviations., Conclusion: The proposed PDMI framework can generate accurate mass density maps using MRI and DECT images. The supervised learning can further enhance model efficacy, making PRN-MR-DE outperform RN-MR-DE. The patient investigation also shows that the framework can potentially improve proton range uncertainty with accurate patient mass density maps., Advances in Knowledge: PDMI framework is proposed for the first time to inform deep learning models by physics insights and leverage the information from MRI to derive accurate mass density maps.

Flexible modeling of the protein-ligand complex structure is a fundamental challenge for in silico drug development. Recent studies have improved commonly used docking tools by incorporating extra-deep learning-based steps. However, such strategies limit their accuracy and efficiency because they retain massive sampling pressure and lack consideration for flexible biomolecular changes. In this study, we propose FlexPose, a geometric graph network capable of direct flexible modeling of complex structures in Euclidean space without the following conventional sampling and scoring strategies. Our model adopts two key designs: scalar-vector dual feature representation and SE(3)-equivariant network, to manage dynamic structural changes, as well as two strategies: conformation-aware pretraining and weakly supervised learning, to boost model generalizability in unseen chemical space. Benefiting from these paradigms, our model dramatically outperforms all tested popular docking tools and recently advanced deep learning methods, especially in tasks involving protein conformation changes. We further investigate the impact of protein and ligand similarity on the model performance with two conformation-aware strategies. Moreover, FlexPose provides an affinity estimation and model confidence for postanalysis.

Proton therapy is a type of radiation therapy that can provide better dose distribution compared to photon therapy by delivering most of the energy at the end of range, which is called the Bragg peak (BP). The protoacoustic technique was developed to determine the BP locations in vivo , but it requires a large dose delivery to the tissue to obtain a high number of signal averaging (NSA) to achieve a sufficient signal-to-noise ratio (SNR), which is not suitable for clinical use. A novel deep learning-based technique has been proposed to denoise acoustic signals and reduce BP range uncertainty with much lower doses. Three accelerometers were placed on the distal surface of a cylindrical polyethylene (PE) phantom to collect protoacoustic signals. In total, 512 raw signals were collected at each device. Device-specific stack autoencoder (SAE) denoising models were trained to denoise the noise-containing input signals, which were generated by averaging only 1, 2, 4, 8, 16, or 24 raw signals (low NSA signals), while the clean signals were obtained by averaging 192 raw signals (high NSA). Both supervised and unsupervised training strategies were employed, and the evaluation of the models was based on mean squared error (MSE), SNR, and BP range uncertainty. Overall, the supervised SAEs outperformed the unsupervised SAEs in BP range verification. For the high accuracy detector, it achieved a BP range uncertainty of 0.20 ± 3.44 mm by averaging over 8 raw signals, while for the other two low accuracy detectors, they achieved the BP uncertainty of 1.44 ± 6.45 mm and -0.23 ± 4.88 mm by averaging 16 raw signals, respectively. This deep learning-based denoising method has shown promising results in enhancing the SNR of protoacoustic measurements and improving the accuracy in BP range verification. It greatly reduces the dose and time for potential clinical applications., (Creative Commons Attribution license.)

Background: Endoscopy artifacts are widespread in real capsule endoscopy (CE) images but not in high-quality standard datasets., Aim: To improve the segmentation performance of polyps from CE images with artifacts based on ensemble learning., Methods: We collected 277 polyp images with CE artifacts from 5760 h of videos from 480 patients at Guangzhou First People's Hospital from January 2016 to December 2019. Two public high-quality standard external datasets were retrieved and used for the comparison experiments. For each dataset, we randomly segmented the data into training, validation, and testing sets for model training, selection, and testing. We compared the performance of the base models and the ensemble model in segmenting polyps from images with artifacts., Results: The performance of the semantic segmentation model was affected by artifacts in the sample images, which also affected the results of polyp detection by CE using a single model. The evaluation based on real datasets with artifacts and standard datasets showed that the ensemble model of all state-of-the-art models performed better than the best corresponding base learner on the real dataset with artifacts. Compared with the corresponding optimal base learners, the intersection over union (IoU) and dice of the ensemble learning model increased to different degrees, ranging from 0.08% to 7.01% and 0.61% to 4.93%, respectively. Moreover, in the standard datasets without artifacts, most of the ensemble models were slightly better than the base learner, as demonstrated by the IoU and dice increases ranging from -0.28% to 1.20% and -0.61% to 0.76%, respectively., Conclusion: Ensemble learning can improve the segmentation accuracy of polyps from CE images with artifacts. Our results demonstrated an improvement in the detection rate of polyps with interference from artifacts., Competing Interests: Conflict-of-interest statement: All the authors report no relevant conflicts of interest for this article., (©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.)

Objective . Computed tomography (CT) to material property conversion dominates proton range uncertainty, impacting the quality of proton treatment planning. Physics-based and machine learning-based methods have been investigated to leverage dual-energy CT (DECT) to predict proton ranges. Recent development includes physics-informed deep learning (DL) for material property inference. This paper aims to develop a framework to validate Monte Carlo dose calculation (MCDC) using CT-based material characterization models. Approach. The proposed framework includes two experiments to validate in vivo dose and water equivalent thickness (WET) distributions using anthropomorphic and porcine phantoms. Phantoms were irradiated using anteroposterior proton beams, and the exit doses and residual ranges were measured by MatriXX PT and a multi-layer strip ionization chamber. Two pre-trained conventional and physics-informed residual networks (RN/PRN) were used for mass density inference from DECT. Additional two heuristic material conversion models using single-energy CT (SECT) and DECT were implemented for comparisons. The gamma index was used for dose comparisons with criteria of 3%/3 mm (10% dose threshold). Main results . The phantom study showed that MCDC with PRN achieved mean gamma passing rates of 95.9% and 97.8% for the anthropomorphic and porcine phantoms. The rates were 86.0% and 79.7% for MCDC with the empirical DECT model. WET analyses indicated that the mean WET variations between measurement and simulation were -1.66 mm, -2.48 mm, and -0.06 mm for MCDC using a Hounsfield look-up table with SECT and empirical and PRN models with DECT. Validation experiments indicated that MCDC with PRN achieved consistent dose and WET distributions with measurement. Significance . The proposed framework can be used to identify the optimal CT-based material characterization model for MCDC to improve proton range uncertainty. The framework can systematically verify the accuracy of proton treatment planning, and it can potentially be implemented in the treatment room to be instrumental in online adaptive treatment planning., (© 2022 Institute of Physics and Engineering in Medicine.)

Proton therapy requires accurate dose calculation for treatment planning to ensure the conformal doses are precisely delivered to the targets. The conversion of CT numbers to material properties is a significant source of uncertainty for dose calculation. The aim of this study is to develop a physics-informed deep learning (PIDL) framework to derive accurate mass density and relative stopping power maps from dual-energy computed tomography (DECT) images. The PIDL framework allows deep learning (DL) models to be trained with a physics loss function, which includes a physics model to constrain DL models. Five DL models were implemented including a fully connected neural network (FCNN), dual-FCNN (DFCNN), and three variants of residual networks (ResNet): ResNet-v1 (RN-v1), ResNet-v2 (RN-v2), and dual-ResNet-v2 (DRN-v2). An artificial neural network (ANN) and the five DL models trained with and without physics loss were explored to evaluate the PIDL framework. Two empirical DECT models were implemented to compare with the PIDL method. DL training data were from CIRS electron density phantom 062M (Computerized Imaging Reference Systems, Inc., Norfolk, VA). The performance of DL models was tested by CIRS adult male, adult female, and 5-year-old child anthropomorphic phantoms. For density map inference, the physics-informed RN-v2 was 3.3%, 2.9% and 1.9% more accurate than ANN for the adult male, adult female, and child phantoms. The physics-informed DRN-v2 was 0.7%, 0.6%, and 0.8% more accurate than DRN-v2 without physics training for the three phantoms, respectfully. The results indicated that physics-informed training could reduce uncertainty when ANN/DL models without physics training were insufficient to capture data structures or derived significant errors. DL models could also achieve better image noise control compared to the empirical DECT parametric mapping methods. The proposed PIDL framework can potentially improve proton range uncertainty by offering accurate material properties conversion from DECT., (© 2022 Institute of Physics and Engineering in Medicine.)

Aiming at the problems that the traditional remote sensing image classification methods cannot effectively integrate a variety of deep learning features and poor classification performance, a land resource use classification method based on a convolutional neural network (CNN) in ecological remote sensing images is proposed. In this study, a seven-layer convolution neural network is constructed, and then the two fully connected layer features of the improved CNN network training output are fused with the fifth layer pooled layer features after dimensionality reduction by principal component analysis (PCA), so as to obtain an effective remote sensing image feature of land resources based on deep learning. Further, the classification of land resources remote sensing images is completed based on a support vector machine classifier. The remote sensing images of Pingshuo mining area in Shanxi Province are used to analyze the proposed method. The results show that the edge of the recognized image is clear, the classification accuracy, misclassification rate, and kappa coefficient are 0.9472, 0.0528, and 0.9435, respectively, and the model has excellent overall performance and good classification effect., Competing Interests: The authors declare that they have no conflicts of interest., (Copyright © 2022 Bin Xia et al.)

The dark-field microscopy (DFM) imaging technology has the advantage of a high signal-to-noise ratio, and it is often used for real-time monitoring of plasmonic resonance scattering and biological imaging at the single-nanoparticle level. Due to the limitation of the optical diffraction limit, it is still a challenging task to accurately distinguish two or more nanoparticles whose distance is less than the diffraction limit. Here, we propose a computational strategy based on a deep learning framework (NanoNet), which will realize the effective segmentation of the scattered light spots in diffraction-limited DFM images and obtain high-resolution plasmonic light scattering imaging. A small data set of DFM and the corresponding scanning electron microscopy (SEM) image pairs are used to learn for obtaining a highly resolved semantic imaging model using NanoNet, and thus highly resolved DFM images matching the resolution of those acquired using SEM can be obtained. Our method has the ability to transform diffraction-limited DFM images to highly resolved ones without adding a complex optical system. As a proof of concept, a highly resolved DFM image of living cells through the NanoNet technique is successfully made, opening up a new avenue for high-resolution optical nanoscopic imaging.

Background: Gastric cancer (GC) is a highly heterogeneous tumor with different responses to immunotherapy. Identifying immune subtypes and landscape of GC could improve immunotherapeutic strategies., Methods: Based on the abundance of tumor-infiltrating immune cells in GC patients from The Cancer Genome Atlas, we used unsupervised consensus clustering algorithm to identify robust clusters of patients, and assessed their reproducibility in an independent cohort from Gene Expression Omnibus. We further confirmed the feasibility of our immune subtypes in five independent pan-cancer cohorts. Finally, functional enrichment analyses were provided, and a deep learning model studying the pathological images was constructed to identify the immune subtypes., Results: We identified and validated three reproducible immune subtypes presented with diverse components of tumor-infiltrating immune cells, molecular features, and clinical characteristics. An immune-inflamed subtype 3, with better prognosis and the highest immune score, had the highest abundance of CD8+ T cells, CD4+ T-activated cells, follicular helper T cells, M1 macrophages, and NK cells among three subtypes. By contrast, an immune-excluded subtype 1, with the worst prognosis and the highest stromal score, demonstrated the highest infiltration of CD4+ T resting cells, regulatory T cells, B cells, and dendritic cells, while an immune-desert subtype 2, with an intermediate prognosis and the lowest immune score, demonstrated the highest infiltration of M2 macrophages and mast cells, and the lowest infiltration of M1 macrophages. Besides, higher proportion of EVB and MSI of TCGA molecular subtyping, over expression of CTLA4, PD1, PDL1, and TP53, and low expression of JAK1 were observed in immune subtype 3, which consisted with the results from Gene Set Enrichment Analysis. These subtypes may suggest different immunotherapy strategies. Finally, deep learning can predict the immune subtypes well., Conclusion: This study offers a conceptual frame to better understand the tumor immune microenvironment of GC. Future work is required to estimate its reference value for the design of immune-related studies and immunotherapy selection., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Chen, Sun, Chen, Liu, Chai, Ding, Liu, Feng, Zhou, Shen, Huang and Xu.)

Purpose: In intensity-modulated proton therapy (IMPT), protons are used to deliver highly conformal dose distributions, targeting tumors, and sparing organs-at-risk. However, due to uncertainties in both patient setup and relative stopping power (RSP) calculation, margins are added to the treatment volume during treatment planning, leading to higher doses to normal tissues. Cone-beam computed tomography (CBCT) images are taken daily before treatment; however, the poor image quality of CBCT limits the use of these images for online dose calculation. In this work, we use a deep-learning-based method to predict RSP maps from daily CBCT images, allowing for online dose calculation in a step toward adaptive radiation therapy., Methods: Twenty-three head-and-neck cancer patients were simulated using a Siemens TwinBeam dual-energy CT (DECT) scanner. Mixed-energy scans (equivalent to a 120 kVp single-energy CT scan) were converted to RSP maps for treatment planning. Cone-beam computed tomography images were taken on the first day of treatment, and the planning RSP maps were registered to these images. A deep learning network based on a cycle-GAN architecture, relying on a compound loss function designed for structural and contrast preservation, was then trained to create an RSP map from a CBCT image. Leave-one-out and holdout cross validations were used for evaluation, and mean absolute error (MAE), mean error (ME), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) were used to quantify the differences between the CT-based and CBCT-based RSP maps. The proposed method was compared to a deformable image registration-based method which was taken as the ground truth and two other deep learning methods. For one patient who underwent resimulation, the new planning RSP maps and CBCT images were used for further evaluation and validation., Results: The CBCT-based RSP generation method was evaluated on 23 head-and-neck cancer patients. From leave-one-out testing, the MAE between CT-based and CBCT-based RSP was 0.06 ± 0.01 and the ME was -0.01 ± 0.01. The proposed method statistically outperformed the comparison DL methods in terms of MAE and ME when compared to the planning CT. In terms of dose comparison, the mean gamma passing rate at 3%/3 mm was 94% when three-dimensional (3D) gamma index was calculated per plan and 96% when gamma index was calculated per field., Conclusions: The proposed method provides sufficiently accurate RSP map generation from CBCT images, allowing for evaluation of daily dose based on CBCT and possibly allowing for CBCT-guided adaptive treatment planning for IMPT., (© 2020 American Association of Physicists in Medicine.)

Deep learning has been used to analyze and diagnose various skin diseases through medical imaging. However, recent researches show that a well-trained deep learning model may not generalize well to data from different cohorts due to domain shift. Simple data fusion techniques such as combining disease samples from different data sources are not effective to solve this problem. In this paper, we present two methods for a novel task of cross-domain skin disease recognition. Starting from a fully supervised deep convolutional neural network classifier pre-trained on ImageNet, we explore a two-step progressive transfer learning technique by fine-tuning the network on two skin disease datasets. We then propose to adopt adversarial learning as a domain adaptation technique to perform invariant attribute translation from source to target domain in order to improve the recognition performance. In order to evaluate these two methods, we analyze generalization capability of the trained model on melanoma detection, cancer detection, and cross-modality learning tasks on two skin image datasets collected from different clinical settings and cohorts with different disease distributions. The experiments prove the effectiveness of our method in solving the domain shift problem.

Purpose; For shoot-through proton treatments, like FLASH radiotherapy, there will be protons exiting the patient which can be used for proton portal imaging (PPI), revealing valuable information for the validation of tumor location in the beam's-eye-view at native gantry angles. However, PPI has poor inherent contrast and spatial resolution. To deal with this issue, we propose a deep-learning-based method to use kV digitally reconstructed radiographs (DRR) to improve PPI image quality. Method; We used a residual generative adversarial network (GAN) framework to learn the nonlinear mapping between PPIs and DRRs. Residual blocks were used to force the model to focus on the structural differences between DRR and PPI. To assess the accuracy of our method, we used 149 images for training and 30 images for testing. PPIs were acquired using a double-scattered proton beam. The DRRs acquired from CT acted as learning targets in the training process and were used to evaluate results from the proposed method using a six-fold cross-validation scheme. Results; Qualitatively, the corrected PPIs showed enhanced spatial resolution and captured fine details present in the DRRs that are missed in the PPIs. The quantitative results for corrected PPIs show average normalized mean error (NME), normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index of -0.1%, 0.3%, 39.14 dB, and 0.987, respectively. Conclusion; The results indicate the proposed method can generate high quality corrected PPIs and this work shows the potential to use a deep-learning model to make PPI available in proton radiotherapy. This will allow for beam's-eye-view (BEV) imaging with the particle used for treatment, leading to a valuable alternative to orthogonal x-rays or cone-beam CT for patient position verification.

The purpose of this work is to validate the application of a deep learning-based method for pelvic synthetic CT (sCT) generation that can be used for prostate proton beam therapy treatment planning. We propose to integrate dense block minimization into 3D cycle-consistent generative adversarial networks (cycleGAN) framework to effectively learn the nonlinear mapping between MRI and CT pairs. A cohort of 17 patients with co-registered CT and MR pairs were used to test the deep learning-based sCT generation method by leave-one-out cross-validation. Image quality between the sCT and CT images, gamma analysis passing rate, dose-volume metrics, distal range displacement, and the individual pencil beam Bragg peak shift between sCT- and CT-based proton plans were evaluated. The average mean absolute error (MAE) was 51.32  ±  16.91 HU. The relative differences of the statistics of the PTV dose-volume histogram (DVH) metrics in between sCT and CT were generally less than 1%. Mean values of dose difference, absolute dose difference (in percent of the prescribed dose) were  -0.07%  ±  0.07% and 0.23%  ±  0.08%. Mean gamma analysis pass rate of 1 mm/1%, 2 mm/2%, 3 mm/3% criteria with 10% dose threshold were 92.39%  ±  5.97%, 97.95%  ±  2.95% and 98.97%  ±  1.62% respectively. The median, mean and standard deviation of absolute maximum range differences were 0.09 cm and 0.23  ±  0.25 cm. The median and mean Bragg peak shifts among the 17 patients were 0.09 cm and 0.18  ±  0.07 cm. The image similarity, dosimetric and distal range agreement between sCT and original CT suggests the feasibility of further development of an MRI-only workflow for prostate proton radiotherapy.

Magnetic resonance imaging (MRI) has been widely used in combination with computed tomography (CT) radiation therapy because MRI improves the accuracy and reliability of target delineation due to its superior soft tissue contrast over CT. The MRI-only treatment process is currently an active field of research since it could eliminate systematic MR-CT co-registration errors, reduce medical cost, avoid diagnostic radiation exposure, and simplify clinical workflow. The purpose of this work is to validate the application of a deep learning-based method for abdominal synthetic CT (sCT) generation by image evaluation and dosimetric assessment in a commercial proton pencil beam treatment planning system (TPS). This study proposes to integrate dense block into a 3D cycle-consistent generative adversarial networks (cycle GAN) framework in an effort to effectively learn the nonlinear mapping between MRI and CT pairs. A cohort of 21 patients with co-registered CT and MR pairs were used to test the deep learning-based sCT image quality by leave-one-out cross validation. The CT image quality, dosimetric accuracy and the distal range fidelity were rigorously checked, using side-by-side comparison against the corresponding original CT images. The average mean absolute error (MAE) was 72.87  ±  18.16 HU. The relative differences of the statistics of the PTV dose volume histogram (DVH) metrics between sCT and CT were generally less than 1%. Mean 3D gamma analysis passing rate of 1 mm/1%, 2 mm/2%, 3 mm/3% criteria with 10% dose threshold were 90.76%  ±  5.94%, 96.98%  ±  2.93% and 99.37%  ±  0.99%, respectively. The median, mean and standard deviation of absolute maximum range differences were 0.170 cm, 0.186 cm and 0.155 cm. The image similarity, dosimetric and distal range agreement between sCT and original CT suggests the feasibility of further development of an MRI-only workflow for liver proton radiotherapy.

Magnetic Resonance Imaging (MRI) is increasingly being used in treatment planning due to its superior soft tissue contrast, which is useful for tumor and soft tissue delineation compared to computed tomography (CT). However, MRI cannot directly provide mass density or relative stopping power (RSP) maps, which are required for calculating proton radiotherapy doses. Therefore, the integration of artificial intelligence (AI) into MRI-based treatment planning to estimate mass density and RSP directly from MRI has generated significant interest. A deep learning (DL) based framework was developed to establish a voxel-wise correlation between MR images and mass density as well as RSP. To facilitate the study, five tissue substitute phantoms were created, representing different tissues such as skin, muscle, adipose tissue, 45% hydroxyapatite (HA), and spongiosa bone. The composition of these phantoms was based on information from ICRP reports. Additionally, two animal tissue phantoms, simulating pig brain and liver, were prepared for DL training purposes. The phantom study involved the development of two DL models. The first model utilized clinical T1 and T2 MRI scans as input, while the second model incorporated zero echo time (ZTE) MRI scans. In the patient application study, two more DL models were trained: one using T1 and T2 MRI scans as input, and another model incorporating synthetic dual-energy computed tomography (sDECT) images to provide accurate bone tissue information. The DECT empirical model was used as a reference to evaluate the proposed models in both phantom and patient application studies. The DECT empirical model was selected as the reference for evaluating the proposed models in both phantom and patient application studies. In the phantom study, the DL model based on T1, and T2 MRI scans demonstrated higher accuracy in estimating mass density and RSP for skin, muscle, adipose tissue, brain, and liver. The mean absolute percentage errors (MAPE) were 0.42%, 0.14%, 0.19%, 0.78%, and 0.26% for mass density, and 0.30%, 0.11%, 0.16%, 0.61%, and 0.23% for RSP, respectively. The DL model incorporating ZTE MRI further improved the accuracy of mass density and RSP estimation for 45% HA and spongiosa bone, with MAPE values of 0.23% and 0.09% for mass density, and 0.19% and 0.07% for RSP, respectively. These results demonstrate the feasibility of using an MRI-only approach combined with DL methods for mass density and RSP estimation in proton therapy treatment planning. By employing this approach, it is possible to obtain the necessary information for proton radiotherapy directly from MRI scans, eliminating the need for additional imaging modalities.

Fueled by substantial advancements in deep learning, the domain of autonomous driving is swiftly advancing towards more robust and effective intelligent systems. One of the critical challenges in this field is achieving accurate 3D object detection, which is often hindered by data sparsity and occlusion. To address these issues, we propose a method centered around a multi-modal fusion strategy that leverages vehicle-road cooperation to enhance perception capabilities. Our approach integrates label information from roadside perception point clouds to harmonize and enrich the representation of image and LiDAR data. This comprehensive integration significantly improves detection accuracy by providing a fuller understanding of the surrounding environment. Rigorous evaluations of our proposed method on two benchmark datasets, KITTI and Waymo Open, demonstrate its superior performance, with our model achieving 87.52% 3D Average Precision (3D AP) and 93.71% Bird's Eye View Average Precision (BEV AP) on the KITTI v a l set. These results highlight the effectiveness of our method in detecting sparse and distant objects, contributing to the development of safer and more efficient autonomous driving solutions. [ABSTRACT FROM AUTHOR]

Industrial defect detection has great significance in product quality improvement, and deep learning methods are now the dominant approach. However, the volume of industrial products is enormous and mainstream detectors are unable to maintain a high accuracy rate during rapid detection. To address the above issues, this paper proposes AMFF-YOLOX, an improved industrial defect detector based on YOLOX. The proposed method can reduce the activation function and normalization operation of the bottleneck in the backbone network, and add an attention mechanism and adaptive spatial feature fusion within the feature extraction network to enable the network to better focus on the object. Ultimately, the accuracy of the prediction is enhanced without excessive loss of speed in network prediction, with competitive performance compared to mainstream detectors. Experiments show that the proposed method in this paper achieves 61.06% (85.00%) mAP@0.5:0.95 (mAP@0.5) in the NRSD-MN dataset, 51.58% (91.09%) is achieved in the PCB dataset, and 49.08% (80.48%) is achieved in the NEU-DET dataset. A large number of comparison and ablation experiments validate the effectiveness and competitiveness of the model in industrial defect detection scenarios. [ABSTRACT FROM AUTHOR]

Research on image classification sparked the latest deep-learning boom. Many downstream tasks, including semantic segmentation, benefit from it. The state-of-the-art semantic segmentation models are all based on deep learning, and they sometimes make some semantic mistakes. In a semantic segmentation dataset with a small number of categories, images are often collected from a single scene, and there is a close semantic connection between any two categories. However, in the semantic segmentation dataset collected from multiple scenes, two categories may be irrelevant. The probability of objects in one category appearing next to objects in other categories is different, which is the basis of the paper. Semantic segmentation methods need to solve two problems of positioning and classification. This paper is dedicated to correcting those clearly wrong classifications that are contrary to reality. Specifically, we first calculate the relevancy between different class pairs. Then, based on this knowledge, we infer the category of a connected component according to the relationships of this connected component with its surrounding connected components and correct the obviously wrong classifications made by a deep learning semantic segmentation model. Several well-performing deep learning models are experimented on two challenging public datasets in the field of semantic image segmentation. Our proposed method improves the performance of UPerNet, OCRNet and SETR from 40.7%, 43% and 48.64% to 42.07%, 44.09% and 49.09% mean IoU on the ADE20K validation set, and the performance of PSPNet, DeepLabV3 and OCRNet from 37.26%, 37.3% and 39.5% to 38.93%, 38.95% and 40.63% mean IoU on the COCO-Stuff dataset, which shows the effectiveness of the method. [ABSTRACT FROM AUTHOR]

Purpose: A quality assurance (QA) CT scans are usually acquired during cancer radiotherapy to assess for any anatomical changes, which may cause an unacceptable dose deviation and therefore warrant a replan. Accurate and rapid deformable image registration (DIR) is needed to support contour propagation from the planning CT (pCT) to the QA CT to facilitate dose volume histogram (DVH) review. Further, the generated deformation maps are used to track the anatomical variations throughout the treatment course and calculate the corresponding accumulated dose from one or more treatment plans. Methods: In this study, we aim to develop a deep learning (DL)‐based method for automatic deformable registration to align the pCT and the QA CT. Our proposed method, named dual‐feasible framework, was implemented by a mutual network that functions as both a forward module and a backward module. The mutual network was trained to predict two deformation vector fields (DVFs) simultaneously, which were then used to register the pCT and QA CT in both directions. A novel dual feasible loss was proposed to train the mutual network. The dual‐feasible framework was able to provide additional DVF regularization during network training, which preserves the topology and reduces folding problems. We conducted experiments on 65 head‐and‐neck cancer patients (228 CTs in total), each with 1 pCT and 2–6 QA CTs. For evaluations, we calculated the mean absolute error (MAE), peak‐signal‐to‐noise ratio (PSNR), structural similarity index (SSIM), target registration error (TRE) between the deformed and target images and the Jacobian determinant of the predicted DVFs. Results: Within the body contour, the mean MAE, PSNR, SSIM, and TRE are 122.7 HU, 21.8 dB, 0.62 and 4.1 mm before registration and are 40.6 HU, 30.8 dB, 0.94, and 2.0 mm after registration using the proposed method. These results demonstrate the feasibility and efficacy of our proposed method for pCT and QA CT DIR. Conclusion: In summary, we proposed a DL‐based method for automatic DIR to match the pCT to the QA CT. Such DIR method would not only benefit current workflow of evaluating DVHs on QA CTs but may also facilitate studies of treatment response assessment and radiomics that depend heavily on the accurate localization of tissues across longitudinal images. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has indeed emerged as a powerful tool for rapidly and accurately predicting materials properties from big data, such as the design of current commercial Li-ion batteries. However, its practical utility for multivalent metal-ion batteries (MIBs), the most promising future solution of large-scale energy storage, is limited due to scarce MIB data availability and poor DL model interpretability. Here, we develop an interpretable DL model as an effective and accurate method for learning electrode voltages of multivalent MIBs (divalent magnesium, calcium, zinc, and trivalent aluminum) at small dataset limits (150–500). Using the experimental results as validation, our model is much more accurate than machine-learning models, which usually are better than DL in the small dataset regime. Besides the high accuracy, our feature-engineering-free DL model is explainable, which automatically extracts the atom covalent radius as the most important feature for the voltage learning by visualizing vectors from the layers of the neural network. The presented model potentially accelerates the design and optimization of multivalent MIB materials with fewer data and less domain-knowledge restriction and is implemented into a publicly available online tool kit in http://batteries.2dmatpedia.org/ for the battery community. [ABSTRACT FROM AUTHOR]

Deep learning (DL) based hyperspectral images (HSIs) denoising approaches directly learn the nonlinear mapping between noisy and clean HSI pairs. They usually do not consider the physical characteristics of HSIs. This drawback makes the models lack interpretability that is key to understanding their denoising mechanism and limits their denoising ability. In this paper, we introduce a novel model-guided interpretable network for HSI denoising to tackle this problem. Fully considering the spatial redundancy, spectral low-rankness, and spectral-spatial correlations of HSIs, we first establish a subspace-based multidimensional sparse (SMDS) model under the umbrella of tensor notation. After that, the model is unfolded into an end-to-end network named SMDS-Net, whose fundamental modules are seamlessly connected with the denoising procedure and optimization of the SMDS model. This makes SMDS-Net convey clear physical meanings, i.e., learning the low-rankness and sparsity of HSIs. Finally, all key variables are obtained by discriminative training. Extensive experiments and comprehensive analysis on synthetic and real-world HSIs confirm the strong denoising ability, strong learning capability, promising generalization ability, and high interpretability of SMDS-Net against the state-of-the-art HSI denoising methods. The source code and data of this article will be made publicly available at https://github.com/bearshng/smds-net for reproducible research. [ABSTRACT FROM AUTHOR]

The equity of CE is an important issue in the whole educational cause. Its level has a significant impact on China's overall quality and economic and social progress. The BDCE is the basic realization of educational equity in China. It also promotes the sustained and healthy growth of economy and society and provides a strong guarantee for building a well-off society in an all-round way. In this paper, the PageRank algorithm of deep learning is used to collect data related to rural education and deeply analyze the problems existing in rural education, providing data support for the development of rural education. Firstly, this paper introduces the BDCE, expounds the problems and reasons in the progress of rural education, and studies the application of data analysis. Then, this paper uses the method of questionnaire to investigate and analyze the difficult problems and current situation of rural education. Finally, the result is that the quality balance of students needs to be further improved. Although the resources of rural education are unbalanced, more than 70% of teachers have bachelor's degree or above, and there is no significant difference among schools. It can be seen that highly educated teachers have become the norm in the stage of CE. Teachers are generally satisfied and recognized with the teaching process. In rural school education, the degree of dissatisfaction with the school curriculum and course selection system is greater than the degree of satisfaction. This shows that there is a lack of diverse and rich teaching courses in rural CE. [ABSTRACT FROM AUTHOR]

Disease and pest are the main factors causing grape yield reduction. Correct and timely identification of these symptoms are necessary for the vineyard. However, the commonly used CNN models limit their performance on leaf images with complex backgrounds, due to the lack of global receptive field. In this article, we propose an effective and accurate approach based on Ghost-convolution and Transformer networks for diagnosing grape leaf in field. First, a grape leaf disease and pest dataset containing 11 classes and 12,615 images, namely GLDP12k is collected. Ghost network is adopted as the convolutional backbone to generate intermediate feature maps with cheap linear operations. Transformer encoders with multi-head self-attention are integrated behind to extract deep semantic features. Then we get the Ghost enlightened Transformer model, namely GeT. After analyzing five hyper-parameters, the optimized GeT is transfer-learnt from ImageNet which provides a 4.3% accuracy bonus. As the results show, with 180 frame-per-second, 1.16 M weights and 98.14% accuracy, GeT surpasses other models, and is 1.7 times faster and 3.6 times lighter than MobilenetV3_large (97.7%). This study shows that the GeT model is effective and provides an optional benchmark for field grape leaf diagnosis. [ABSTRACT FROM AUTHOR]

The noise and high randomness of the stock market are primary obstacles to profitability. These factors cause stock returns to consist of short-term predictable and stock-specific residual parts. Therefore, it is beneficial to separate and model the two items for forecasting returns in the randomness environment. On this basis, we propose a novel Temporal Return Separation and Restore Forecasting (TRSRF) method. TRSRF first extracts price- and relation-based features from historical stock inputs and uses them to decompose and restore predictable and residual elements of historical returns , respectively. Then it forecasts these components based on historical values. We conducted experiments on the Chinese small and mid-cap, blue-chip, and large-cap indexes over ten years to compare with seven proposed baselines. On the three Chinese indexes our method separately exceeds 6%, 16%, and 22% absolute annualized returns than our baseline. The ablation study also proves the necessity of return decomposition and inter-stock relation modeling. We find that modeling inter-stock relations on small-cap stocks works better than on large-cap stocks. Furthermore, we prove our method might be an implicit ensemble model of the predictable and residual parts, which is why it works stability under various market conditions. • We propose to separate stock returns into predictable and residual parts. • We present a novel stock movement separation and prediction method. • Our method significantly outperforms the best baselines. • Our method are verified stability as an implicit ensemble strategy. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) denoising is an ill-posed inverse problem. The underlying physical model is always important to tackle this problem, which is unfortunately ignored by most of the current deep learning (DL)-based methods, producing poor denoising performance. To address this issue, this article introduces an end-to-end model-aided nonlocal neural network (MAC-Net) which simultaneously takes the spectral low-rank model and spatial deep prior into account for HSI noise reduction. Specifically, motivated by the success of the spectral low-rank model in depicting the strong spectral correlations and the nonlocal similarity prior in capturing spatial long-range dependencies, we first build a spectral low-rank model and then integrate a nonlocal U-Net into the model. In this way, we obtain a hybrid model-based and DL-based HSI denoising method where the spatial local and nonlocal multi-scale and spectral low-rank structures are effectively exploited. After that, we cast the optimization and denoising procedure of the hybrid method as a forward process of a neural network and introduce a set of learnable modules to yield our MAC-Net. Compared with traditional model-based methods, our MAC-Net overcomes the difficulties of accurate modeling, thanks to the strong learning and representation ability of DL. Unlike most “black-box” DL-based methods, the spectral low-rank model is beneficial to increase the generalization ability of the network and decrease the requirement of training samples. Experimental results on the natural and remote-sensing HSIs show that MAC-Net achieves state-of-the-art performance over both model-based and DL-based methods. The source code and data of this article will be made publicly available at https://github.com/bearshng/mac-net for reproducible research. [ABSTRACT FROM AUTHOR]

Ultrasound imaging of the lung has played an important role in managing patients with COVID-19–associated pneumonia and acute respiratory distress syndrome (ARDS). During the COVID-19 pandemic, lung ultrasound (LUS) or point-of-care ultrasound (POCUS) has been a popular diagnostic tool due to its unique imaging capability and logistical advantages over chest X-ray and CT. Pneumonia/ARDS is associated with the sonographic appearances of pleural line irregularities and B-line artefacts, which are caused by interstitial thickening and inflammation, and increase in number with severity. Artificial intelligence (AI), particularly machine learning, is increasingly used as a critical tool that assists clinicians in LUS image reading and COVID-19 decision making. We conducted a systematic review from academic databases (PubMed and Google Scholar) and preprints on arXiv or TechRxiv of the state-of-the-art machine learning technologies for LUS images in COVID-19 diagnosis. Openly accessible LUS datasets are listed. Various machine learning architectures have been employed to evaluate LUS and showed high performance. This paper will summarize the current development of AI for COVID-19 management and the outlook for emerging trends of combining AI-based LUS with robotics, telehealth, and other techniques. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) denoising is a crucial preprocessing task to improve the performance of the subsequent HSI interpretation and applications. With recent progress in deep learning, HSI denoising methods based on deep neural networks have attracted increasing interest and achieved the state-of-the-art performance. Nevertheless, most of these methods are based on network structures originally developed for grayscale and color images and require a change of network structure to be applicable to HSIs. The new network architectures often lead to complicated models and limited flexibility, which, in turn, result in difficulty in learning and demand of a large number of training samples. In this article, we propose an innovative two-stage learning method including pretraining and fine-tuning procedures. In the first stage, a denoising convolutional neural network can be pretrained with pairs of corrupted and clean images. In the second stage, the pretrained network is fine-tuned via a self-supervised learning strategy to capture the spectral correlation in HSIs. The training pairs in the second stage are constructed from the neighboring band images in the target noisy HSI, leading to a novel idea of embedding spectral information into denoiser through the target image, rather than the change of the network architecture. This model has strong adaptability such that many image denoising networks can be easily adopted for HSIs, while the external hyperspectral training set is optional but not mandatory. Experimental results show that our method has competitive performance compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Hyperspectral unmixing is recognized as an important tool to learn the constituent materials and corresponding distribution in a scene. The physical spectral mixture model is always important to tackle this problem because of its highly ill-posed nature. In this article, we introduce a linear spectral mixture model (LMM)-based end-to-end deep neural network named SNMF-Net for hyperspectral unmixing. SNMF-Net shares an alternating architecture and benefits from both model-based methods and learning-based methods. On the one hand, SNMF-Net is of high physical interpretability as it is built by unrolling $L_{p}$ sparsity constrained nonnegative matrix factorization ($L_{p}$ -NMF) model belonging to LMM families. On the other hand, all the parameters and submodules of SNMF-Net can be seamlessly linked with the alternating optimization algorithm of $L_{p}$ -NMF and unmixing problem. This enables us to reasonably integrate the prior knowledge on unmixing, the optimization algorithm, and the sparse representation theory into the network for robust learning, so as to improve unmixing. Experimental results on the synthetic and real-world data show the advantages of the proposed SNMF-Net over many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper presents a novel method for content-based color image retrieval that combines color vector quantization and visual primary features into a compact feature representation. Color vector quantization is proposed to describe the image in a compressed stream by preserving the contrast of an image and produce two color quantizers which are processed by vector quantization to preserve the content of a color image. Loosely inspired by human visual system and its mechanism in effectively recognizing objects in an image by its edges and color distribution, we propose extraction of visual primary features based on edge texture orientation and color moments features. The representation of the proposed method utilizes histogram-based features which are populated by color quantization and visual primary features that are used to measure the similarity between the two color images by a specific distance metric computation. The proposed method proves to be efficient and adaptive to the particulars of image retrieval, while it does not require any training information, making it suitable for real time color CBIR applications.The smaller feature size is an additional benefit of the proposed methodology which makes it ideal to use for embedded computing, mobile computing, energy efficient computing and high-throughput systems. An extensive experimental evaluation conducted on four publicly available datasets namely Wang, Vistex-640, Corel-5k and Corel-10k datasets against well-known fusion based, non-fusion based, and deep learning methods highlights the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Purpose: Auto‐segmentation algorithms offer a potential solution to eliminate the labor‐intensive, time‐consuming, and observer‐dependent manual delineation of organs‐at‐risk (OARs) in radiotherapy treatment planning. This study aimed to develop a deep learning‐based automated OAR delineation method to tackle the current challenges remaining in achieving reliable expert performance with the state‐of‐the‐art auto‐delineation algorithms. Methods: The accuracy of OAR delineation is expected to be improved by utilizing the complementary contrasts provided by computed tomography (CT) (bony‐structure contrast) and magnetic resonance imaging (MRI) (soft‐tissue contrast). Given CT images, synthetic MR images were firstly generated by a pre‐trained cycle‐consistent generative adversarial network. The features of CT and synthetic MRI were then extracted and combined for the final delineation of organs using mask scoring regional convolutional neural network. Both in‐house and public datasets containing CT scans from head‐and‐neck (HN) cancer patients were adopted to quantitatively evaluate the performance of the proposed method against current state‐of‐the‐art algorithms in metrics including Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS). Results: Across all of 18 OARs in our in‐house dataset, the proposed method achieved an average DSC, HD95, MSD, and RMS of 0.77 (0.58–0.90), 2.90 mm (1.32–7.63 mm), 0.89 mm (0.42–1.85 mm), and 1.44 mm (0.71–3.15 mm), respectively, outperforming the current state‐of‐the‐art algorithms by 6%, 16%, 25%, and 36%, respectively. On public datasets, for all nine OARs, an average DSC of 0.86 (0.73–0.97) were achieved, 6% better than the competing methods. Conclusion: We demonstrated the feasibility of a synthetic MRI‐aided deep learning framework for automated delineation of OARs in HN radiotherapy treatment planning. The proposed method could be adopted into routine HN cancer radiotherapy treatment planning to rapidly contour OARs with high accuracy. [ABSTRACT FROM AUTHOR]

Against the social background of the gradual transformation of the social pension mode and the arrival of the ageing age, the tool design of fitness walkers for the elderly was studied. In this paper, the physiological and psychological status of the elderly with mild or moderate stroke or unable to exercise due to the decline of physical function was analysed. In addition, the corresponding elderly fitness walker product market research was studied. Aiming at these problems, this paper proposes a set of elderly fitness walker frames based on the ergonomic semiotic approach of the product architecture design (SAPAD) model. First, the development and structure of artificial engineering is introduced in detail. Then the framework of walking aid for the elderly based on the SAPAD model is proposed, and the SAPAS model and walking aid for the elderly are introduced, respectively, along with the framework of simulation design and application. The experimental results show that the framework of elderly fitness walker based on the ergonomic SAPAD model can be implemented. It can be applied to the elderly’s fitness assistance. A fitness walker can be suitable for the elderly who want to continue to exercise; it also hopes to care for the elderly who need to exercise from the perspective of the product.

In the hyperspectral image, each pixel corresponds to a small area on the Earth’s surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial–spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial–spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples. [ABSTRACT FROM AUTHOR]

This document is a correction notice for an article titled "Global and local representation collaborative learning for few-shot learning" published in the Journal of Intelligent Manufacturing. The correction addresses errors in Table 2 of the article. Specifically, two values in column 6 have been corrected. The original article has been updated to reflect these corrections. The correction notice is provided by Jun Zhou and Qingling Cai. Springer Nature, the publisher, remains neutral in terms of jurisdictional claims and institutional affiliations. [Extracted from the article]

Benefiting from the booming of deep learning, the state-of-the-art models achieved great progress. But they are huge in terms of parameters and floating point operations, which makes it hard to apply them to real-time applications. In this paper, we propose a novel deep neural network architecture, named MPDNet, for fast and efficient semantic segmentation under resource constraints. First, we use a light-weight classification model pretrained on ImageNet as the encoder. Second, we use a cost-effective upsampling datapath to restore prediction resolution and convert features for classification into features for segmentation. Finally, we propose to use a multi-path decoder to extract different types of features, which are not ideal to process inside only one convolutional neural network. The experimental results of our model outperform other models aiming at real-time semantic segmentation on Cityscapes. Based on our proposed MPDNet, we achieve 76.7% mean IoU on Cityscapes test set with only 118.84GFLOPs and achieves 37.6 Hz on 768 × 1536 images on a standard GPU. [ABSTRACT FROM AUTHOR]

Although object detection techniques have been widely employed in various practical applications, automatic tree detection is still a difficult challenge, especially for street-view images. In this article, we propose a unified end-to-end trainable network for automatic street tree detection based on a state-of-the-art deep learning-based object detector. We tackle low illumination and heavy occlusion conditions in tree detection, which have not been extensively studied until now, due to clear challenges. Existing generic object detectors cannot be directly applied to this task due to aforementioned challenges. To address these issues, we first present a simple, yet effective image brightness adjustment method to handle low illuminance cases. Moreover, we propose a novel loss and a tree part-attention module to reduce false detections caused by heavy occlusion, inspired by the previously proposed occlusion-aware region-convolutional neural network (R-CNN) work. We train and evaluate several versions of the proposed network and validate the importance of each component. It is demonstrated that the resulting framework, part attention network for tree detection (PANTD), can efficiently detect trees in street-view images. The experimental results show that our approach achieves high accuracy and robustness under various conditions. [ABSTRACT FROM AUTHOR]

map matching has played a crucial role in technologies related to indoor positioning. Conventional map matching algorithms based on particle filter (PF) have some limitations, such as the limited use of map information, poor generalization and low precision. To solve these problems, we propose an adaptable particle filter network (AdaPFnet), a novel map matching technique that integrates particle filter algorithm into a neural network. AdaPFnet uses local views of particles to represent particles so that the map information about location can be learned sufficiently through a neural network. To demonstrate the performance of the model, it has conducted extensive experiments using 1540 real-world data. The results show that AdaPFnet outperforms PF by up to 21% and remains a strong adaptability for different environments. [ABSTRACT FROM AUTHOR]

• Grading modeling and decision-making framework were proposed for yield estimation. • Normalized weight decision-making strategy improved the estimation precision. • Deep learning network ConvNeXt was compared for modeling performance. • Proposed framework performs precisely and robustly using UAV RGB images. Rice yield estimation is of great significance for ensuring food security and breeding new varieties with high yield and good stress resistance. The popular yield estimation method is to combine unmanned aerial vehicle (UAV) images to extract vegetation index (VI) for multi-variable regression, whose application is always limited by expensive equipment and complex data processing. In this study, based on the deep learning network, ConvNeXt, a robust framework developed by grading modeling and normalized weight decision-making strategy was innovatively proposed to estimate the rice yield, only using RGB images collected by UAVs. The main results are: (1) the yield estimation performance of yield grading model was better than that of regression model, and R2, mean absolute error (MAE), and mean absolute percentage error (MAPE) on the test dataset using the grading model were 0.97, 410.94 kg/ha, and 3.96 % respectively; (2) the confidence scores of the grading model were adopted as the weights, which could correct the estimated yield of the misclassified samples and further reduce the estimated error. The normalized weight decision-making after optimization had obvious advantages, the performance of 2-weight strategy was the best, whose R2, MAE, and MAPE reached 0.98, 386.08 kg/ha, and 3.79 %, respectively; (3) the MAPE results of generalization assessment showed that the generalization of the grading (12.98 %) and regression (10.88 %) models was inferior to the proposed framework (8.71 %), which could reduce the MAPE on the generalization evaluation dataset to less than 10 %. Furthermore, the framework exhibited good adaptability when applied to the rice yield estimation of the new data in 2022, with an MAPE of 4.54 %. Considering both application potential and adaptability, this framework constructs a novel strategy and method for rice yield estimation using grading modeling and normalized weight decision-making strategy, which provides a reference for future real-time and precise yield estimation using UAV remote sensing. [ABSTRACT FROM AUTHOR]

Internet companies are facing the need for handling large-scale machine learning applications on a daily basis and distributed implementation of machine learning algorithms which can handle extra-large-scale tasks with great performance is widely needed. Deep forest is a recently proposed deep learning framework which uses tree ensembles as its building blocks and it has achieved highly competitive results on various domains of tasks. However, it has not been tested on extremely large-scale tasks. In this work, based on our parameter server system, we developed the distributed version of deep forest. To meet the need for real-world tasks, many improvements are introduced to the original deep forest model, including MART (Multiple Additive Regression Tree) as base learners for efficiency and effectiveness consideration, the cost-based method for handling prevalent class-imbalanced data, MART based feature selection for high dimension data, and different evaluation metrics for automatically determining the cascade level. We tested the deep forest model on an extra-large-scale task, i.e., automatic detection of cash-out fraud, with more than 100 million training samples. Experimental results showed that the deep forest model has the best performance according to the evaluation metrics from different perspectives even with very little effort for parameter tuning. This model can block fraud transactions in a large amount of money each day. Even compared with the best-deployed model, the deep forest model can additionally bring a significant decrease in economic loss each day. [ABSTRACT FROM AUTHOR]

Purpose: The incorporation of cone‐beam computed tomography (CBCT) has allowed for enhanced image‐guided radiation therapy. While CBCT allows for daily 3D imaging, images suffer from severe artifacts, limiting the clinical potential of CBCT. In this work, a deep learning‐based method for generating high quality corrected CBCT (CCBCT) images is proposed. Methods: The proposed method integrates a residual block concept into a cycle‐consistent adversarial network (cycle‐GAN) framework, called res‐cycle GAN, to learn a mapping between CBCT images and paired planning CT images. Compared with a GAN, a cycle‐GAN includes an inverse transformation from CBCT to CT images, which constrains the model by forcing calculation of both a CCBCT and a synthetic CBCT. A fully convolution neural network with residual blocks is used in the generator to enable end‐to‐end CBCT‐to‐CT transformations. The proposed algorithm was evaluated using 24 sets of patient data in the brain and 20 sets of patient data in the pelvis. The mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR), normalized cross‐correlation (NCC) indices, and spatial non‐uniformity (SNU) were used to quantify the correction accuracy of the proposed algorithm. The proposed method is compared to both a conventional scatter correction and another machine learning‐based CBCT correction method. Results: Overall, the MAE, PSNR, NCC, and SNU were 13.0 HU, 37.5 dB, 0.99, and 0.05 in the brain, 16.1 HU, 30.7 dB, 0.98, and 0.09 in the pelvis for the proposed method, improvements of 45%, 16%, 1%, and 93% in the brain, and 71%, 38%, 2%, and 65% in the pelvis, over the CBCT image. The proposed method showed superior image quality as compared to the scatter correction method, reducing noise and artifact severity. The proposed method produced images with less noise and artifacts than the comparison machine learning‐based method. Conclusions: The authors have developed a novel deep learning‐based method to generate high‐quality corrected CBCT images. The proposed method increases onboard CBCT image quality, making it comparable to that of the planning CT. With further evaluation and clinical implementation, this method could lead to quantitative adaptive radiation therapy. [ABSTRACT FROM AUTHOR]

Purpose: Stereotactic radiosurgery (SRS) is widely used to obliterate arteriovenous malformations (AVMs). Its performance relies on the accuracy of delineating the target AVM. Manual segmentation during a framed SRS procedure is time consuming and subject to inter‐ and intraobserver variation. To address these drawbacks, we proposed a deep learning‐based method to automatically segment AVMs on CT simulation image sets. Methods: We developed a deep learning‐based method using a deeply supervised three‐dimensional (3D) V‐Net with a compound loss function. A 3D supervision mechanism was integrated into a residual network, V‐Net, to deal with the optimization difficulties when training deep networks with limited training data. The proposed compound loss function including logistic and Dice losses encouraged similarity and penalized discrepancy simultaneously between prediction and training dataset; this was utilized to supervise the 3D V‐Net at different stages. To evaluate the accuracy of segmentation, we retrospectively investigated 80 AVM patients who had CT simulation and digital subtraction angiography (DSA) acquired prior to treatment. The AVM target volume was segmented by our proposed method. They were compared with clinical contours approved by physicians with regard to Dice overlapping, difference in volume and centroid, and dose coverage changes on original plan. Results: Contours created by the proposed method demonstrated very good visual agreement to the ground truth contours. The mean Dice similarity coefficient (DSC), sensitivity and specificity of the contours delineated by our method were >0.85 among all patients. The mean centroid distance between our results and ground truth was 0.675 ± 0.401 mm, and was not significantly different in any of the three orthogonal directions. The correlation coefficient between ground truth and AVM volume resulting from the proposed method was 0.992 with statistical significance. The mean volume difference among all patients was 0.076 ± 0.728 cc; there was no statistically significant difference. The average differences in dose metrics were all less than 0.2 Gy, with standard deviation less than 1 Gy. No statistically significant differences were observed in any of the dose metrics. Conclusion: We developed a novel, deeply supervised, deep learning‐based approach to automatically segment the AVM volume on CT images. We demonstrated its clinical feasibility by validating the shape and positional accuracy, and dose coverage of the automatic volume. These results demonstrate the potential of a learning‐based segmentation method for delineating AVMs in the clinical setting. [ABSTRACT FROM AUTHOR]