Search

Multispectral photoacoustic tomography (PAT) is an imaging modality that utilizes the photoacoustic effect to achieve non-invasive and high-contrast imaging of internal tissues but also molecular functional information derived from multi-spectral measurements. However, the hardware cost and computational demand of a multispectral PAT system consisting of up to thousands of detectors are huge. To address this challenge, we propose an ultra-sparse spiral sampling strategy for multispectral PAT, which we named U3S-PAT. Our strategy employs a sparse ring-shaped transducer that, when switching excitation wavelengths, simultaneously rotates and translates. This creates a spiral scanning pattern with multispectral angle-interlaced sampling. To solve the highly ill-conditioned image reconstruction problem, we propose a self-supervised learning method that is able to introduce structural information shared during spiral scanning. We simulate the proposed U3S-PAT method on a commercial PAT system and conduct in vivo animal experiments to verify its performance. The results show that even with a sparse sampling rate as low as 1/30, our U3S-PAT strategy achieves similar reconstruction and spectral unmixing accuracy as non-spiral dense sampling. Given its ability to dramatically reduce the time required for three-dimensional multispectral scanning, our U3S-PAT strategy has the potential to perform volumetric molecular imaging of dynamic biological activities.

Photoacoustic tomography (PAT), as a novel medical imaging technology, provides structural, functional, and metabolism information of biological tissue in vivo. Sparse Sampling PAT, or SS-PAT, generates images with a smaller number of detectors, yet its image reconstruction is inherently ill-posed. Model-based methods are the state-of-the-art method for SS-PAT image reconstruction, but they require design of complex handcrafted prior. Owing to their ability to derive robust prior from labeled datasets, deep-learning-based methods have achieved great success in solving inverse problems, yet their interpretability is poor. Herein, we propose a novel SS-PAT image reconstruction method based on deep algorithm unrolling (DAU), which integrates the advantages of model-based and deep-learning-based methods. We firstly provide a thorough analysis of DAU for PAT reconstruction. Then, in order to incorporate the structural prior constraint, we propose a nested DAU framework based on plug-and-play Alternating Direction Method of Multipliers (PnP-ADMM) to deal with the sparse sampling problem. Experimental results on numerical simulation, in vivo animal imaging, and multispectral un-mixing demonstrate that the proposed DAU image reconstruction framework outperforms state-of-the-art model-based and deep-learning-based methods.

Photoacoustic tomography (PAT) is a promising imaging technique that can visualize the distribution of chromophores within biological tissue. However, the accuracy of PAT imaging is compromised by light fluence (LF), which hinders the quantification of light absorbers. Currently, model-based iterative methods are used for LF correction, but they require extensive computational resources due to repeated LF estimation based on differential light transport models. To improve LF correction efficiency, we propose to use Fourier neural operator (FNO), a neural network specially designed for estimating partial differential equations, to learn the forward projection of light transport in PAT. Trained using paired finite-element-based LF simulation data, our FNO model replaces the traditional computational heavy LF estimator during iterative correction, such that the correction procedure is considerably accelerated. Simulation and experimental results demonstrate that our method achieves comparable LF correction quality to traditional iterative methods while reducing the correction time by over 30 times.

In gradient echo MRI, quantitative susceptibility mapping (QSM) quantifies the magnetic susceptibility distributions of tissues, which has great potential in detecting brain diseases. However, QSM reconstruction is an ill-conditional inversion problem because of the zeros in the frequency domain of the dipole kernel. The intrinsic nature of the ill-posedness would affect the accuracy of quantifying tissue susceptibility. Recently, deep learning-based methods have been proposed to improve accuracy by suppressing the streaking artifacts. In this work, we proposed a hybrid architecture to enforce data consistency by involving numerical optimization blocks within convolutional neural networks (CNN), which aimed to reconstruct high-quality QSM images, referred to as NoQSM-net. The Calculation of Susceptibility through Multiple Orientation Sampling (COSMOS) QSM maps were used as labels for training. The performance of the proposed method was evaluated on two healthy volunteers and brain images of patients with diseases. Our experiments showed that the proposed method achieved good performance in terms of quantitative metrics and could effectively suppress artifacts in reconstructed QSM images, demonstrating its potential for future applications. For experiments on patients with multiple sclerosis (MS), the proposed method could better detect lesion regions in the results of NoQSM-net.

BackgroundAccurate phase unwrapping is a critical prerequisite for successful applications in phase-related MRI, including quantitative susceptibility mapping (QSM) and susceptibility weighted imaging. However, many existing 3D phase unwrapping algorithms face challenges in the presence of severe noise, rapidly changing phase, and open-end cutline.MethodsIn this study, we introduce a novel 3D phase unwrapping approach utilizing region partitioning and a local polynomial model. Initially, the method leverages phase partitioning to create initial regions. Noisy voxels connecting areas within these regions are excluded and grouped into residual voxels. The connected regions within the region of interest are then reidentified and categorized into blocks and residual voxels based on voxel count thresholds. Subsequently, the method sequentially performs inter-block and residual voxel phase unwrapping using the local polynomial model. The proposed method was evaluated on simulation and in vivo abdominal QSM data, and was compared with the classical Region-growing, Laplacian_based, Graph-cut, and PRELUDE methods.ResultsSimulation experiments, conducted under different signal-to-noise ratios and phase change levels, consistently demonstrate that the proposed method achieves accurate unwrapping results, with mean error ratios not exceeding 0.01%. In contrast, the error ratios of Region-growing (N/A, 84.47%), Laplacian_based (20.65%, N/A), Graph-cut (2.26%, 20.71%), and PRELUDE (4.28%, 10.33%) methods are all substantially higher than those of the proposed method. In vivo abdominal QSM experiments further confirm the effectiveness of the proposed method in unwrapping phase data and successfully reconstructing susceptibility maps, even in scenarios with significant noise, rapidly changing phase, and open-end cutline in a large field of view.ConclusionThe proposed method demonstrates robust and accurate phase unwrapping capabilities, positioning it as a promising option for abdominal QSM applications.

Objective: To comprehensively capture intra-tumor heterogeneity in head and neck cancer (HNC) and maximize the use of valid information collected in the clinical field, we propose a novel multi-modal image–text fusion strategy aimed at improving prognosis. Method: We have developed a tailored diagnostic algorithm for HNC, leveraging a deep learning-based model that integrates both image and clinical text information. For the image fusion part, we used the cross-attention mechanism to fuse the image information between PET and CT, and for the fusion of text and image, we used the Q-former architecture to fuse the text and image information. We also improved the traditional prognostic model by introducing time as a variable in the construction of the model, and finally obtained the corresponding prognostic results. Result: We assessed the efficacy of our methodology through the compilation of a multicenter dataset, achieving commendable outcomes in multicenter validations. Notably, our results for metastasis-free survival (MFS), recurrence-free survival (RFS), overall survival (OS), and progression-free survival (PFS) were as follows: 0.796, 0.626, 0.641, and 0.691. Our results demonstrate a notable superiority over the utilization of CT and PET independently, and exceed the result derived without the clinical textual information. Conclusions: Our model not only validates the effectiveness of multi-modal fusion in aiding diagnosis, but also provides insights for optimizing survival analysis. The study underscores the potential of our approach in enhancing prognosis and contributing to the advancement of personalized medicine in HNC.

Introduction: The estimation of myocardial motion abnormalities has great potential for the early diagnosis of myocardial infarction (MI). This study aims to quantitatively analyze the segmental and transmural myocardial motion in MI rats by incorporating two novel strategies of algorithm parameter optimization and transmural motion index (TMI) calculation.Methods: Twenty-one rats were randomly divided into three groups (n = 7 per group): sham, MI, and ischemia–reperfusion (IR) groups. Ultrasound radio-frequency (RF) signals were acquired from each rat heart at 1 day and 28 days after animal model establishment; thus, a total of six datasets were represented as Sham1, Sham28, MI1, MI28, IR1, and IR28. The systolic cumulative displacement was calculated using our previously proposed vectorized normalized cross-correlation (VNCC) method. A semiautomatic regional and layer-specific myocardium segmentation framework was proposed for transmural and segmental myocardial motion estimation. Two novel strategies were proposed: the displacement-compensated cross-correlation coefficient (DCCCC) for algorithm parameter optimization and the transmural motion index (TMI) for quantitative estimation of the cross-wall transmural motion gradient.Results: The results showed that an overlap value of 80% used in VNCC guaranteed a more accurate displacement calculation. Compared to the Sham1 group, the systolic myocardial motion reductions were significantly detected (p < 0.05) in the middle anteroseptal (M-ANT-SEP), basal anteroseptal (B-ANT-SEP), apical lateral (A-LAT), middle inferolateral (M-INF-LAT), and basal inferolateral (B-INF-LAT) walls as well as a significant TMI drop (p < 0.05) in the M-ANT-SEP wall in the MI1 rats; significant motion reductions (p < 0.05) were also detected in the B-ANT-SEP and A-LAT walls in the IR1 group. The motion improvements (p < 0.05) were detected in the M-INF-LAT wall in the MI28 group and the apical septal (A-SEP) wall in the IR28 group compared to the MI1 and IR1 groups, respectively.Discussion: Our results show that the MI-induced reductions and reperfusion-induced recovery in systolic myocardial contractility could be successfully evaluated using our method, and most post-MI myocardial segments could recover systolic function to various extents in the remodeling phase. In conclusion, the ultrasound-based quantitative estimation framework for estimating segmental and transmural motion of the myocardium proposed in our study has great potential for non-invasive, novel, and early MI detection.

Photoacoustic tomography (PAT) images contain inherent distortions due to the imaging system and heterogeneous tissue properties. Improving image quality requires the removal of these system distortions. While model-based approaches and data-driven techniques have been proposed for PAT image restoration, achieving accurate and robust image recovery remains challenging. Recently, deep-learning-based image deconvolution approaches have shown promise for image recovery. However, PAT imaging presents unique challenges, including spatially varying resolution and the absence of ground truth data. Consequently, there is a pressing need for a novel learning strategy specifically tailored for PAT imaging. Herein, we propose a configurable network model named Deep hybrid Image-PSF Prior (DIPP) that builds upon the physical image degradation model of PAT. DIPP is an unsupervised and deeply learned network model that aims to extract the ideal PAT image from complex system degradation. Our DIPP framework captures the degraded information solely from the acquired PAT image, without relying on ground truth or labeled data for network training. Additionally, we can incorporate the experimentally measured Point Spread Functions (PSFs) of the specific PAT system as a reference to further enhance performance. To evaluate the algorithm’s effectiveness in addressing multiple degradations in PAT, we conduct extensive experiments using simulation images, publicly available datasets, phantom images, and in vivo small animal imaging data. Comparative analyses with classical analytical methods and state-of-the-art deep learning models demonstrate that our DIPP approach achieves significantly improved restoration results in terms of image details and contrast.

Magnetic resonance imaging (MRI) and photoacoustic tomography (PAT) offer two distinct image contrasts. To integrate these two modalities, we present a comprehensive hardware-software solution for the successive acquisition and co-registration of PAT and MRI images in in vivo animal studies. Based on commercial PAT and MRI scanners, our solution includes a 3D-printed dual-modality imaging bed, a 3-D spatial image co-registration algorithm with dual-modality markers, and a robust modality switching protocol for in vivo imaging studies. Using the proposed solution, we successfully demonstrated co-registered hybrid-contrast PAT-MRI imaging that simultaneously displays multi-scale anatomical, functional and molecular characteristics on healthy and cancerous living mice. Week-long longitudinal dual-modality imaging of tumor development reveals information on size, border, vascular pattern, blood oxygenation, and molecular probe metabolism of the tumor micro-environment at the same time. The proposed methodology holds promise for a wide range of pre-clinical research applications that benefit from the PAT-MRI dual-modality image contrast.

In Photoacoustic Tomography (PAT), the acquired image represents a light energy deposition map of the imaging object. For quantitative imaging, the PAT image is converted into an absorption coefficient (μa) map by dividing the light fluence (LF). Previous methods usually assume a uniform tissue μa distribution, and consequently degrade the LF correction results. Here, we propose a simple method to reconstruct the pixel-wise μa map. Our method is based on a non-segmentation-based iterative algorithm, which alternately optimizes the LF distribution and the μa map. Using simulation data, as well as phantom and animal data, we implemented our algorithm and compared it to segmentation-based correction methods. The results show that our method can obtain accurate estimation of the LF distribution and therefore improve the image quality and feature visibility of the μa map. Our method may facilitate efficient calculation of the concentration distributions of endogenous and exogenous agents in vivo.

Photoacoustic tomography (PAT) is a promising imaging technique that utilizes the detection of light-induced acoustic waves for both morphological and functional biomedical imaging. However, producing high-quality images using PAT is still challenging and requires further research. Besides improving image reconstruction, which turns the raw photoacoustic signal into a PAT image, an alternative way to address this issue is through image post-processing, which can enhance and optimize the reconstructed PAT image. Image post-processing methods have rapidly emerged in PAT and are proven to be essential in improving image quality in recent research. In this review, we investigate the need for image post-processing in PAT imaging. We conduct a thorough literature review on the latest PAT image post-processing articles, including both general and PAT-specific post-processing techniques. In contrast to previous reviews, our analysis focuses specifically on advanced image post-processing rather than image reconstruction methods. By highlighting their potential applications, we hope to encourage further research and development in PAT image post-processing technology.

Purpose: This study aims to investigate the impact of aggregation methods used for the generation of texture features on their robustness of nasopharyngeal carcinoma (NPC) based on 18F-FDG PET/CT images. Methods: 128 NPC patients were enrolled and 95 texture features were extracted for each patient including six feature families under different aggregation methods. For GLCM and GLRLM features, six aggregation methods were considered. For GLSZM, GLDZM, NGTDM and NGLDM features, three aggregation methods were considered. The robustness of the features affected by aggregation methods was assessed by the pair-wise intra-class correlation coefficient (ICC). Furthermore, the effects of discretization and partial volume correction (PVC) on the percent of ICC categories of all texture features were evaluated by overall ICC instead of the pair-wise ICC. Results: There were 12 features with excellent pair-wise ICCs varying aggregation methods, namely joint average, sum average, autocorrelation, long run emphasis, high grey level run emphasis, short run high grey level emphasis, long run high grey level emphasis, run length variance, SZM high grey level emphasis, DZM high grey level emphasis, high grey level count emphasis and dependence count percentage. For GLCM and GLRLM features, 19/25 and 14/16 features showed excellent pair-wise ICCs varying aggregation methods (averaged and merged) on the same dimensional features (2D, 2.5D or 3D). Different discretization levels and partial volume corrections lead to consistent robustness of textural features affected by aggregation methods. Conclusion: Different dimensional features with the same aggregation methods showed worse robustness compared with the same dimensional features with different aggregation methods. Different discretization levels and PVC algorithms had a negligible effect on the percent of ICC categories of all texture features.

Skull stripping is an initial and critical step in the pipeline of mouse fMRI analysis. Manual labeling of the brain usually suffers from intra- and inter-rater variability and is highly time-consuming. Hence, an automatic and efficient skull-stripping method is in high demand for mouse fMRI studies. In this study, we investigated a 3D U-Net based method for automatic brain extraction in mouse fMRI studies. Two U-Net models were separately trained on T2-weighted anatomical images and T2*-weighted functional images. The trained models were tested on both interior and exterior datasets. The 3D U-Net models yielded a higher accuracy in brain extraction from both T2-weighted images (Dice > 0.984, Jaccard index > 0.968 and Hausdorff distance < 7.7) and T2*-weighted images (Dice > 0.964, Jaccard index > 0.931 and Hausdorff distance < 3.3), compared with the two widely used mouse skull-stripping methods (RATS and SHERM). The resting-state fMRI results using automatic segmentation with the 3D U-Net models are highly consistent with those obtained by manual segmentation for both the seed-based and group independent component analysis. These results demonstrate that the 3D U-Net based method can replace manual brain extraction in mouse fMRI analysis.