Search

This paper addresses digital staining and classification of the unstained white blood cell images obtained with a differential contrast microscope. We have data coming from multiple domains that are partially labeled and partially matching across the domains. Using unstained images removes time-consuming staining procedures and could facilitate and automatize comprehensive diagnostics. To this aim, we propose a method that translates unstained images to realistically looking stained images preserving the inter-cellular structures, crucial for the medical experts to perform classification. We achieve better structure preservation by adding auxiliary tasks of segmentation and direct reconstruction. Segmentation enforces that the network learns to generate correct nucleus and cytoplasm shape, while direct reconstruction enforces reliable translation between the matching images across domains. Besides, we build a robust domain agnostic latent space by injecting the target domain label directly to the generator, i.e., bypassing the encoder. It allows the encoder to extract features independently of the target domain and enables an automated domain invariant classification of the white blood cells. We validated our method on a large dataset composed of leukocytes of 24 patients, achieving state-of-the-art performance on both digital staining and classification tasks. [ABSTRACT FROM AUTHOR]

Machine learning has been widely used in medical image analysis under an assumption that the training and test data are under the same feature distributions. However, medical images from difference devices or the same device with different parameter settings are often contaminated with different amount and types of noises, which violate the above assumption. Therefore, the models trained using data from one device or setting often fail to work for that from another. Moreover, it is very expensive and tedious to label data and re-train models for all different devices or settings. To overcome this noise adaptation issue, it is necessary to leverage on the models trained with data from one device or setting for new data. In this paper, we reformulate this noise adaptation task as an image-to-image translation task such that the noise patterns from the test data are modified to be similar to those from the training data while the contents of the data are unchanged. In this paper, we propose a novel Noise Adaptation Generative Adversarial Network (NAGAN), which contains a generator and two discriminators. The generator aims to map the data from source domain to target domain. Among the two discriminators, one discriminator enforces the generated images to have the same noise patterns as those from the target domain, and the second discriminator enforces the content to be preserved in the generated images. We apply the proposed NAGAN on both optical coherence tomography (OCT) images and ultrasound images. Results show that the method is able to translate the noise style. In addition, we also evaluate our proposed method with segmentation task in OCT and classification task in ultrasound. The experimental results show that the proposed NAGAN improves the analysis outcome. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have recently been used in biomedical imaging applications with great success. In this paper, we investigated the classification performance of CNN models on diffusion weighted imaging (DWI) streamlines defined by functional MRI (fMRI) and electrical stimulation mapping (ESM). To learn a set of discriminative and interpretable features from the extremely unbalanced dataset, we evaluated different CNN architectures with multiple loss functions (e.g., focal loss and center loss) and a soft attention mechanism and compared our models with current state-of-the-art methods. Through extensive experiments on streamlines collected from 70 healthy children and 70 children with focal epilepsy, we demonstrated that our deep CNN model with focal and central losses and soft attention outperforms all existing models in the literature and provides clinically acceptable accuracy (73%–100%) for the objective detection of functionally important white matter pathways, including ESM determined eloquent areas such as primary motors, aphasia, speech arrest, auditory, and visual functions. The findings of this paper encourage further investigations to determine if DWI-CNN analysis can serve as a noninvasive diagnostic tool during pediatric presurgical planning by estimating not only the location of essential cortices at the gyral level but also the underlying fibers connecting these cortical areas to minimize or predict postsurgical functional deficits. This paper translates an advanced CNN model to clinical practice in the pediatric population where currently available approaches (e.g., ESM and fMRI) are suboptimal. The implementation will be released at https://github.com/HaotianMXu/Brain-fiber-classification-using-CNNs. [ABSTRACT FROM AUTHOR]

This paper introduces a new estimation-theoretic framework for experiment design in the context of MR image reconstruction under sparsity constraints. The new framework is called OEDIPUS (Oracle-based Experiment Design for Imaging Parsimoniously Under Sparsity constraints) and is based on combining the constrained Cramér-Rao bound with classical experiment design techniques. Compared to popular random sampling approaches, OEDIPUS is fully deterministic and automatically tailors the sampling pattern to the specific imaging context of interest (i.e., accounting for coil geometry, anatomy, image contrast, etc.). OEDIPUS-based experiment designs are evaluated using retrospectively subsampled in vivo MRI data in several different contexts. The results demonstrate that OEDIPUS-based experiment designs have some desirable characteristics relative to conventional MRI sampling approaches. [ABSTRACT FROM AUTHOR]

Atlas-based segmentation is used in radiotherapy planning to accelerate the delineation of organs at risk (OARs). Atlas selection has been proposed to improve the performance of segmentation, assuming that the more similar the atlas is to the patient, the better the result. It follows that the larger the database of atlases from which to select, the better the results should be. This paper seeks to estimate a clinically achievable expected performance under this assumption. Assuming a perfect atlas selection, an extreme value theory has been applied to estimate the accuracy of single-atlas and multi-atlas segmentation given a large database of atlases. For this purpose, clinical contours of most common OARs on computed tomography of the head and neck (${N}=\textsf {316}$) and thoracic (${N}=\textsf {280}$) cases were used. This paper found that while for most organs, perfect segmentation cannot be reasonably expected, auto-contouring performance of a level corresponding to clinical quality could be consistently expected given a database of 5000 atlases under the assumption of perfect atlas selection. [ABSTRACT FROM AUTHOR]

As a follow-up to the first IEEE Transactions on Medical Imaging (TMI) special issue on the theme of deep tomographic reconstruction, the second special issue is assembled to reflect the status and momentum of this rapidly emerging field. In this editorial, we provide a brief background illustrating the motivation for the development of network-based, data-driven, and learning-oriented reconstruction methods, summarize the included papers, and report our verification of the shared deep learning codes. Finally, we discuss several important research topics to facilitate further investigation and collaboration. [ABSTRACT FROM AUTHOR]

This paper describes a new image reconstruction method for digital breast tomosynthesis (DBT). The new method incorporates detector blur into the forward model. The detector blur in DBT causes correlation in the measurement noise. By making a few approximations that are reasonable for breast imaging, we formulated a regularized quadratic optimization problem with a data-fit term that incorporates models for detector blur and correlated noise (DBCN). We derived a computationally efficient separable quadratic surrogate (SQS) algorithm to solve the optimization problem that has a non-diagonal noise covariance matrix. We evaluated the SQS-DBCN method by reconstructing DBT scans of breast phantoms and human subjects. The contrast-to-noise ratio and sharpness of microcalcifications were analyzed and compared with those by the simultaneous algebraic reconstruction technique. The quality of soft tissue lesions and parenchymal patterns was examined. The results demonstrate the potential to improve the image quality of reconstructed DBT images by incorporating the system physics model. This paper is a first step toward model-based iterative reconstruction for DBT. [ABSTRACT FROM AUTHOR]

Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. In this paper, we propose a novel box-based instance segmentation method. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region. However, existing methods can hardly differentiate the target from its neighboring objects within the same bounding box region due to their similar textures and low-contrast boundaries. To deal with this problem, in this paper, we propose an object-guided instance segmentation method. Our method first detects the center points of the objects, from which the bounding box parameters are then predicted. To perform segmentation, an object-guided coarse-to-fine segmentation branch is built along with the detection branch. The segmentation branch reuses the object features as guidance to separate target object from the neighboring ones within the same bounding box region. To further improve the segmentation quality, we design an auxiliary feature refinement module that densely samples and refines point-wise features in the boundary regions. Experimental results on three biological image datasets demonstrate the advantages of our method. The code will be available at https://github.com/yijingru/ObjGuided-Instance-Segmentation. [ABSTRACT FROM AUTHOR]

Multi-modal imaging data fusion has attracted much attention in medical data analysis because it can provide complementary information for more accurate analysis. Integrating functional and structural multi-modal imaging data has been increasingly used in the diagnosis of brain diseases, such as epilepsy. Most of the existing methods focus on the feature space fusion of different modalities but ignore the valuable high-order relationships among samples and the discriminative fused features for classification. In this paper, we propose a novel framework by fusing data from two modalities of functional MRI (fMRI) and diffusion tensor imaging (DTI) for epilepsy diagnosis, which effectively captures the complementary information and discriminative features from different modalities by high-order feature extraction with the attention mechanism. Specifically, we propose a triple network to explore the discriminative information from the high-order representation feature space learned from multi-modal data. Meanwhile, self-attention is introduced to adaptively estimate the degree of importance between brain regions, and the cross-attention mechanism is utilized to extract complementary information from fMRI and DTI. Finally, we use the triple loss function to adjust the distance between samples in the common representation space. We evaluate the proposed method on the epilepsy dataset collected from Jinling Hospital, and the experiment results demonstrate that our method is significantly superior to several state-of-the-art diagnosis approaches. [ABSTRACT FROM AUTHOR]

Image reconstruction for positron emission tomography (PET) is challenging because of the ill-conditioned tomographic problem and low counting statistics. Kernel methods address this challenge by using kernel representation to incorporate image prior information in the forward model of iterative PET image reconstruction. Existing kernel methods construct the kernels commonly using an empirical process, which may lead to unsatisfactory performance. In this paper, we describe the equivalence between the kernel representation and a trainable neural network model. A deep kernel method is then proposed by exploiting a deep neural network to enable automated learning of an improved kernel model and is directly applicable to single subjects in dynamic PET. The training process utilizes available image prior data to form a set of robust kernels in an optimized way rather than empirically. The results from computer simulations and a real patient dataset demonstrate that the proposed deep kernel method can outperform the existing kernel method and neural network method for dynamic PET image reconstruction. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axis cardiac shape such that the segmented sequences can benefit from temporal and anatomical consistency constraints. Our method is a post-processing that takes as input segmented echocardiographic sequences produced by any state-of-the-art method and processes it in two steps to (i) identify spatio-temporal inconsistencies according to the overall dynamics of the cardiac sequence and (ii) correct the inconsistencies. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes, where we can both detect and fix anomalies. We tested our framework on 98 full-cycle sequences from the CAMUS dataset, which are available alongside this paper. Our temporal regularization method not only improves the accuracy of the segmentation across the whole sequences, but also enforces temporal and anatomical consistency. [ABSTRACT FROM AUTHOR]

The gold standard for diagnosing lymph node metastasis of papillary thyroid carcinoma is to analyze the whole slide histopathological images (WSIs). Due to the large size of WSIs, recent computer-aided diagnosis approaches adopt the multi-instance learning (MIL) strategy and the key part is how to effectively aggregate the information of different instances (patches). In this paper, a novel transformer-guided framework is proposed to predict lymph node metastasis from WSIs, where we incorporate the transformer mechanism to improve the accuracy from three different aspects. First, we propose an effective transformer-based module for discriminative patch feature extraction, including a lightweight feature extractor with a pruned transformer (Tiny-ViT) and a clustering-based instance selection scheme. Next, we propose a new Transformer-MIL module to capture the relationship of different discriminative patches with sparse distribution on WSIs and better nonlinearly aggregate patch-level features into the slide-level prediction. Considering that the slide-level annotation is relatively limited to training a robust Transformer-MIL, we utilize the pathological relationship between the primary tumor and its lymph node metastasis and develop an effective attention-based mutual knowledge distillation (AMKD) paradigm. Experimental results on our collected WSI dataset demonstrate the efficiency of the proposed Transformer-MIL and attention-based knowledge distillation. Our method outperforms the state-of-the-art methods by over 2.72% in AUC (area under the curve). [ABSTRACT FROM AUTHOR]

Macrovascular invasion (MaVI) is a major threat to survival in hepatocellular carcinoma (HCC), which should be treated as early as possible to ensure safety and efficacy. In this aspect, MaVI prediction can be helpful. However, MaVI prediction is difficult because of the inter-class similarity and intra-class variation of HCC in computed tomography (CT) images. Moreover, existing methods fail to include clinical priori knowledge associated with HCC, leading to incomprehensive information extraction. In this paper, we proposed a prior knowledge-aware fusion network (PKAFnet) to accurately achieve MaVI prediction in CT images. First, a perception module was presented to extract features related to tumor marginal heterogeneity in the graph domain, which contributed to rotation invariance and captured intensity variations of tumor margin. Second, a tumor segmentation network was built to obtain global information of a 3D tumor image and information associated with tumor internal heterogeneity in the image domain. Finally, multi-domain features associated with the tumor margin and tumor region were combined by using a multi-domain attentional feature fusion module. Thus, by incorporating MaVI-related prior knowledge, our PKAFnet can alleviate overfitting, which can improve the discriminative ability. The proposed PKAFnet was validated on a multi-center dataset, and remarkable performance was achieved in an independent testing set. Moreover, the interpretability of perception module and segmentation network were presented in our paper, which illustrated the effectiveness and credibility of PKAFnet. Therefore, the proposed method showed great application potential for MaVI prediction. [ABSTRACT FROM AUTHOR]

Histopathological tissue classification is a simpler way to achieve semantic segmentation for the whole slide images, which can alleviate the requirement of pixel-level dense annotations. Existing works mostly leverage the popular CNN classification backbones in computer vision to achieve histopathological tissue classification. In this paper, we propose a super lightweight plug-and-play module, named Pyramidal Deep-Broad Learning (PDBL), for any well-trained classification backbone to improve the classification performance without a re-training burden. For each patch, we construct a multi-resolution image pyramid to obtain the pyramidal contextual information. For each level in the pyramid, we extract the multi-scale deep-broad features by our proposed Deep-Broad block (DB-block). We equip PDBL in three popular classification backbones, ShuffLeNetV2, EfficientNetb0, and ResNet50 to evaluate the effectiveness and efficiency of our proposed module on two datasets (Kather Multiclass Dataset and the LC25000 Dataset). Experimental results demonstrate the proposed PDBL can steadily improve the tissue-level classification performance for any CNN backbones, especially for the lightweight models when given a small among of training samples (less than 10%). It greatly saves the computational resources and annotation efforts. The source code is available at: https://github.com/linjiatai/PDBL. [ABSTRACT FROM AUTHOR]

Most deep learning models for temporal regression directly output the estimation based on single input images, ignoring the relationships between different images. In this paper, we propose deep relation learning for regression, aiming to learn different relations between a pair of input images. Four non-linear relations are considered: “cumulative relation,” “relative relation,” “maximal relation” and “minimal relation.” These four relations are learned simultaneously from one deep neural network which has two parts: feature extraction and relation regression. We use an efficient convolutional neural network to extract deep features from the pair of input images and apply a Transformer for relation learning. The proposed method is evaluated on a merged dataset with 6,049 subjects with ages of 0–97 years using 5-fold cross-validation for the task of brain age estimation. The experimental results have shown that the proposed method achieved a mean absolute error (MAE) of 2.38 years, which is lower than the MAEs of 8 other state-of-the-art algorithms with statistical significance (p<0.05) in paired T-test (two-side). [ABSTRACT FROM AUTHOR]

Spectral computed tomography (CT) reconstructs images from different spectral data through photon counting detectors (PCDs). However, due to the limited number of photons and the counting rate in the corresponding spectral segment, the reconstructed spectral images are usually affected by severe noise. In this paper, we propose a fourth-order nonlocal tensor decomposition model for spectral CT image reconstruction (FONT-SIR). To maintain the original spatial relationships among similar patches and improve the imaging quality, similar patches without vectorization are grouped in both spectral and spatial domains simultaneously to form the fourth-order processing tensor unit. The similarity of different patches is measured with the cosine similarity of latent features extracted using principal component analysis (PCA). By imposing the constraints of the weighted nuclear and total variation (TV) norms, each fourth-order tensor unit is decomposed into a low-rank component and a sparse component, which can efficiently remove noise and artifacts while preserving the structural details. Moreover, the alternating direction method of multipliers (ADMM) is employed to solve the decomposition model. Extensive experimental results on both simulated and real data sets demonstrate that the proposed FONT-SIR achieves superior qualitative and quantitative performance compared with several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Encoding-decoding (ED) CNNs have demonstrated state-of-the-art performance for noise reduction over the past years. This has triggered the pursuit of better understanding the inner workings of such architectures, which has led to the theory of deep convolutional framelets (TDCF), revealing important links between signal processing and CNNs. Specifically, the TDCF demonstrates that ReLU CNNs induce low-rankness, since these models often do not satisfy the necessary redundancy to achieve perfect reconstruction (PR). In contrast, this paper explores CNNs that do meet the PR conditions. We demonstrate that in these type of CNNs soft shrinkage and PR can be assumed. Furthermore, based on our explorations we propose the learned wavelet-frame shrinkage network, or LWFSN and its residual counterpart, the rLWFSN. The ED path of the (r)LWFSN complies with the PR conditions, while the shrinkage stage is based on the linear expansion of thresholds proposed Blu and Luisier. In addition, the LWFSN has only a fraction of the training parameters (<1%) of conventional CNNs, very small inference times, low memory footprint, while still achieving performance close to state-of-the-art alternatives, such as the tight frame (TF) U-Net and FBPConvNet, in low-dose CT denoising. [ABSTRACT FROM AUTHOR]

Recent works show that generative adversarial networks (GANs) can be successfully applied to image synthesis and semi-supervised learning, where, given a small labeled database and a large unlabeled database, the goal is to train a powerful classifier. In this paper, we trained a retinal image synthesizer and a semi-supervised learning method for automatic glaucoma assessment using an adversarial model on a small glaucoma-labeled database and a large unlabeled database. Various studies have shown that glaucoma can be monitored by analyzing the optic disc and its surroundings, and for that reason, the images used in this paper were automatically cropped around the optic disc. The novelty of this paper is to propose a new retinal image synthesizer and a semi-supervised learning method for glaucoma assessment based on the deep convolutional GANs. In addition, and to the best of our knowledge, this system is trained on an unprecedented number of publicly available images (86926 images). This system, hence, is not only able to generate images synthetically but to provide labels automatically. Synthetic images were qualitatively evaluated using t-SNE plots of features associated with the images and their anatomical consistency was estimated by measuring the proportion of pixels corresponding to the anatomical structures around the optic disc. The resulting image synthesizer is able to generate realistic (cropped) retinal images, and subsequently, the glaucoma classifier is able to classify them into glaucomatous and normal with high accuracy (AUC = 0.9017). The obtained retinal image synthesizer and the glaucoma classifier could then be used to generate an unlimited number of cropped retinal images with glaucoma labels. [ABSTRACT FROM AUTHOR]

Current clinical practice relies on clinical history to determine the time since stroke (TSS) onset. Imaging-based determination of acute stroke onset time could provide critical information to clinicians in deciding stroke treatment options, such as thrombolysis. The patients with unknown or unwitnessed TSS are usually excluded from thrombolysis, even if their symptoms began within the therapeutic window. In this paper, we demonstrate a machine learning approach for TSS classification using routinely acquired imaging sequences. We develop imaging features from the magnetic resonance (MR) images and train machine learning models to classify the TSS. We also propose a deep-learning model to extract hidden representations for the MR perfusion-weighted images and demonstrate classification improvement by incorporating these additional deep features. The cross-validation results show that our best classifier achieved an area under the curve of 0.765, with a sensitivity of 0.788 and a negative predictive value of 0.609, outperforming existing methods. We show that the features generated by our deep-learning algorithm correlate with the MR imaging features, and validate the robustness of the model on imaging parameter variations (e.g., year of imaging). This paper advances magnetic resonance imaging analysis one-step-closer to an operational decision support tool for stroke treatment guidance. [ABSTRACT FROM AUTHOR]

In recent years, we have witnessed the explosion of large-scale data in various fields. Classical statistical methodologies, such as linear regression or generalized linear regression, often show inadequate performance on heterogeneous data because the key homogeneity assumption fails. In this paper, we present a flexible framework to handle heterogeneous populations that can be naturally grouped into several ordered subtypes. A local model technique utilizing ordinal class labels during the training stage is proposed. We define a new “progression score” that captures the progression of ordinal classes, and use a truncated Gaussian kernel to construct the weight function in a local regression framework. Furthermore, given the weights, we apply sparse shrinkage on the local fitting to handle high dimensionality. In this way, our local model is able to conduct variable selection on each query point. Numerical studies show the superiority of our proposed method over several existing ones. Our method is also applied to the Alzheimer’s Disease Neuroimaging Initiative data to make predictions on the longitudinal clinical scores based on different modalities of baseline brain image features. [ABSTRACT FROM AUTHOR]

Accurate surgery duration estimation is necessary for optimal OR planning, which plays an important role in patient comfort and safety as well as resource optimization. It is, however, challenging to preoperatively predict surgery duration since it varies significantly depending on the patient condition, surgeon skills, and intraoperative situation. In this paper, we propose a deep learning pipeline, referred to as RSDNet, which automatically estimates the remaining surgery duration (RSD) intraoperatively by using only visual information from laparoscopic videos. The previous state-of-the-art approaches for RSD prediction are dependent on manual annotation, whose generation requires expensive expert knowledge and is time-consuming, especially considering the numerous types of surgeries performed in a hospital and the large number of laparoscopic videos available. A crucial feature of RSDNet is that it does not depend on any manual annotation during training, making it easily scalable to many kinds of surgeries. The generalizability of our approach is demonstrated by testing the pipeline on two large datasets containing different types of surgeries: 120 cholecystectomy and 170 gastric bypass videos. The experimental results also show that the proposed network significantly outperforms a traditional method of estimating RSD without utilizing manual annotation. Further, this paper provides a deeper insight into the deep learning network through visualization and interpretation of the features that are automatically learned. [ABSTRACT FROM AUTHOR]

In this paper, we developed a deep convolutional neural network (CNN) for the classification of malignant and benign masses in digital breast tomosynthesis (DBT) using a multi-stage transfer learning approach that utilized data from similar auxiliary domains for intermediate-stage fine-tuning. Breast imaging data from DBT, digitized screen-film mammography, and digital mammography totaling 4039 unique regions of interest (1797 malignant and 2242 benign) were collected. Using cross validation, we selected the best transfer network from six transfer networks by varying the level up to which the convolutional layers were frozen. In a single-stage transfer learning approach, knowledge from CNN trained on the ImageNet data was fine-tuned directly with the DBT data. In a multi-stage transfer learning approach, knowledge learned from ImageNet was first fine-tuned with the mammography data and then fine-tuned with the DBT data. Two transfer networks were compared for the second-stage transfer learning by freezing most of the CNN structures versus freezing only the first convolutional layer. We studied the dependence of the classification performance on training sample size for various transfer learning and fine-tuning schemes by varying the training data from 1% to 100% of the available sets. The area under the receiver operating characteristic curve (AUC) was used as a performance measure. The view-based AUC on the test set for single-stage transfer learning was 0.85 ± 0.05 and improved significantly (${p} < 0.05$) to 0.91 ± 0.03 for multi-stage learning. This paper demonstrated that, when the training sample size from the target domain is limited, an additional stage of transfer learning using data from a similar auxiliary domain is advantageous. [ABSTRACT FROM AUTHOR]

Many new clinical investigations of microwave breast imaging have been published in recent years. Trials with over one hundred participants have indicated the potential of microwave imaging to detect breast cancer, with particularly encouraging sensitivity results reported from women with dense breasts. The next phase of clinical trials will involve larger and more diverse populations, including women with no breast abnormalities or benign breast diseases. These trials will need to address clinical efficacy in terms of sensitivity and specificity. A number of challenges exist when using microwave imaging with broad populations: 1) addressing the substantial variance in breast composition observed in the population and 2) achieving high specificity given differences between individuals. This paper analyses these challenges using a diverse phantom set which models the variance in breast composition and tumor shape and size seen in the population. The data show that the sensitivity of microwave breast imaging in breasts of differing density can suffer if patient-specific beamforming is not used. Moreover, the results suggest that achieving high specificity in dense breasts may be difficult, but that patient-specific beamforming does not adversely affect the expected specificity. In summary, this paper finds that patient-specific beamforming has a tangible impact on expected sensitivity in experimental cases and that achieving high specificity in dense breasts may be challenging. [ABSTRACT FROM AUTHOR]

This paper proposes a mixed low-rank approximation and second-order tensor-based total variation (LRSOTTV) approach for the super-resolution and denoising of retinal optical coherence tomography (OCT) images through effective utilization of nonlocal spatial correlations and local smoothness properties. OCT imaging relies on interferometry, which explains why OCT images suffer from a high level of noise. In addition, data subsampling is conducted during OCT A-scan and B-scan acquisition. Therefore, using effective super-resolution algorithms is necessary for reconstructing high-resolution clean OCT images. In this paper, a low-rank regularization approach is proposed for exploiting nonlocal self-similarity prior to OCT image reconstruction. To benefit from the advantages of the redundancy of multi-slice OCT data, we construct a third-order tensor by extracting the nonlocal similar three-dimensional blocks and grouping them by applying the k-nearest-neighbor method. Next, the nuclear norm is used as a regularization term to shrink the singular values of the constructed tensor in the non-local correlation direction. Further, the regularization approaches of the first-order tensor-based total variation (FOTTV) and SOTTV are proposed for better preservation of retinal layers and suppression of artifacts in OCT images. The alternative direction method of multipliers (ADMM) technique is then used to solve the resulting optimization problem. Our experiments show that integrating SOTTV instead of FOTTV into a low-rank approximation model can achieve noticeably improved results. Our experimental results on the denoising and super-resolution of OCT images demonstrate that the proposed model can provide images whose numerical and visual qualities are higher than those obtained by using state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Alzheimer’s Disease (AD), one of the main causes of death in elderly people, is characterized by Mild Cognitive Impairment (MCI) at prodromal stage. Nevertheless, only part of MCI subjects could progress to AD. The main objective of this paper is thus to identify those who will develop a dementia of AD type among MCI patients. 18F-FluoroDeoxyGlucose Positron Emission Tomography (18F-FDG PET) serves as a neuroimaging modality for early diagnosis as it can reflect neural activity via measuring glucose uptake at resting-state. In this paper, we design a deep network on 18F-FDG PET modality to address the problem of AD identification at early MCI stage. To this end, a Multi-view Separable Pyramid Network (MiSePyNet) is proposed, in which representations are learned from axial, coronal and sagittal views of PET scans so as to offer complementary information and then combined to make a decision jointly. Different from the widely and naturally used 3D convolution operations for 3D images, the proposed architecture is deployed with separable convolution from slice-wise to spatial-wise successively, which can retain the spatial information and reduce training parameters compared to 2D and 3D networks, respectively. Experiments on ADNI dataset show that the proposed method can yield better performance than both traditional and deep learning-based algorithms for predicting the progression of Mild Cognitive Impairment, with a classification accuracy of 83.05%. [ABSTRACT FROM AUTHOR]

Volume Projection Imaging from ultrasound data is a promising technique to visualize spine features and diagnose Adolescent Idiopathic Scoliosis. In this paper, we present a novel multi-task framework to reduce the scan noise in volume projection images and to segment different spine features simultaneously, which provides an appealing alternative for intelligent scoliosis assessment in clinical applications. Our proposed framework consists of two streams: i) A noise removal stream based on generative adversarial networks, which aims to achieve effective scan noise removal in a weakly-supervised manner, i.e., without paired noisy-clean samples for learning; ii) A spine segmentation stream, which aims to predict accurate bone masks. To establish the interaction between these two tasks, we propose a selective feature-sharing strategy to transfer only the beneficial features, while filtering out the useless or harmful information. We evaluate our proposed framework on both scan noise removal and spine segmentation tasks. The experimental results demonstrate that our proposed method achieves promising performance on both tasks, which provides an appealing approach to facilitating clinical diagnosis. [ABSTRACT FROM AUTHOR]

Deformable registration is a crucial step in many medical procedures such as image-guided surgery and radiation therapy. Most recent learning-based methods focus on improving the accuracy by optimizing the non-linear spatial correspondence between the input images. Therefore, these methods are computationally expensive and require modern graphic cards for real-time deployment. In this paper, we introduce a new Light-weight Deformable Registration network that significantly reduces the computational cost while achieving competitive accuracy. In particular, we propose a new adversarial learning with distilling knowledge algorithm that successfully leverages meaningful information from the effective but expensive teacher network to the student network. We design the student network such as it is light-weight and well suitable for deployment on a typical CPU. The extensively experimental results on different public datasets show that our proposed method achieves state-of-the-art accuracy while significantly faster than recent methods. We further show that the use of our adversarial learning algorithm is essential for a time-efficiency deformable registration method. Finally, our source code and trained models are available at https://github.com/aioz-ai/LDR_ALDK. [ABSTRACT FROM AUTHOR]

Segmenting anatomical structures in medical images has been successfully addressed with deep learning methods for a range of applications. However, this success is heavily dependent on the quality of the image that is being segmented. A commonly neglected point in the medical image analysis community is the vast amount of clinical images that have severe image artefacts due to organ motion, movement of the patient and/or image acquisition related issues. In this paper, we discuss the implications of image motion artefacts on cardiac MR segmentation and compare a variety of approaches for jointly correcting for artefacts and segmenting the cardiac cavity. The method is based on our recently developed joint artefact detection and reconstruction method, which reconstructs high quality MR images from k-space using a joint loss function and essentially converts the artefact correction task to an under-sampled image reconstruction task by enforcing a data consistency term. In this paper, we propose to use a segmentation network coupled with this in an end-to-end framework. Our training optimises three different tasks: 1) image artefact detection, 2) artefact correction and 3) image segmentation. We train the reconstruction network to automatically correct for motion-related artefacts using synthetically corrupted cardiac MR k-space data and uncorrected reconstructed images. Using a test set of 500 2D+time cine MR acquisitions from the UK Biobank data set, we achieve demonstrably good image quality and high segmentation accuracy in the presence of synthetic motion artefacts. We showcase better performance compared to various image correction architectures. [ABSTRACT FROM AUTHOR]

The article discusses various reports published within the issue, including one by A. Bistoquet, J. Oshinski, and O. Škrinjar on cardiac motion tracking algorithms and another by V. Grau, H. Becher, and J. A. Noble on cardiac images.

This paper explores the competency of the time domain ultra-wideband (UWB)-circular synthetic aperture radar (CSAR) to image the breast and detect tumors. The image reconstruction is performed using a time domain global back projection technique adapted to the circular trajectory data acquisition. This paper also proposes a sectional image reconstruction method to compensate for the group velocity changes in different layers of a multilayer medium. Experiments on an advanced breast phantom examines the suitability of this technique for breast tumor imaging. The advanced breast phantom is designed based on a MRI of a real patient, fabricated using 3D printing technology, and filled with liquids that emulate normal and cancerous tissues. The measurement results, compared with MRI imaging of the phantom, demonstrate the suitability of the UWB-CSAR method for breast tumor imaging. This method can be a tool for early diagnosis as well as for treatment monitoring during chemotherapy or radiotherapy. [ABSTRACT FROM AUTHOR]

Previous work has shown that total variation superiorization (TVS) improves reconstructed image quality in proton computed tomography (pCT). The structure of the TVS algorithm has evolved since then and this paper investigated if this new algorithmic structure provides additional benefits to pCT image quality. Structural and parametric changes introduced to the original TVS algorithm included: (1) inclusion or exclusion of TV reduction requirement, (2) a variable number, $N$ , of TV perturbation steps per feasibility-seeking iteration, and (3) introduction of a perturbation kernel $0< \alpha < 1$. The structural change of excluding the TV reduction requirement check tended to have a beneficial effect for $3\le N\le 6$ and allows full parallelization of the TVS algorithm. Repeated perturbations per feasibility-seeking iterations reduced total variation (TV) and material dependent standard deviations for $3\le N\le 6$. The perturbation kernel $\alpha $ , effectively equal to $\alpha =0.5$ in the original TVS algorithm, reduced TV and standard deviations as $\alpha $ was increased beyond $\alpha =0.5$ , but negatively impacted reconstructed relative stopping power (RSP) values for $\alpha >0.75$. The reductions in TV and standard deviations allowed feasibility-seeking with a larger relaxation parameter $\lambda $ than previously used, without the corresponding increases in standard deviations experienced with the original TVS algorithm. This paper demonstrates that the modifications related to the evolution of the original TVS algorithm provide benefits in terms of both pCT image quality and computational efficiency for appropriately chosen parameter values. [ABSTRACT FROM AUTHOR]

In this paper, we propose a multi-scale, multi-kernel shape, compactly supported kernel bundle framework for stationary velocity field-based image registration (Wendland kernel bundle stationary velocity field, wKB-SVF). We exploit the possibility of directly choosing kernels to construct a reproducing kernel Hilbert space (RKHS) instead of imposing it from a differential operator. The proposed framework allows us to minimize computational cost without sacrificing the theoretical foundations of SVF-based diffeomorphic registration. In order to recover deformations occurring at different scales, we use compactly supported Wendland kernels at multiple scales and orders to parameterize the velocity fields, and the framework allows simultaneous optimization over all scales. The performance of wKB-SVF is extensively compared to the 14 non-rigid registration algorithms presented in a recent comparison paper. On both MGH10 and CUMC12 datasets, the accuracy of wKB-SVF is improved when compared to other registration algorithms. In a disease-specific application for intra-subject registration, atrophy scores estimated using the proposed registration scheme separates the diagnostic groups of Alzheimer's and normal controls better than the state-of-the-art segmentation technique. Experimental results show that wKB-SVF is a robust, flexible registration framework that allows theoretically well-founded and computationally efficient multi-scale representation of deformations and is equally well-suited for both inter- and intra-subject image registration. [ABSTRACT FROM AUTHOR]

Since the seminal paper by Mitra et al., diffusion MR has been widely used in order to estimate surface-to-volume ratios. In this paper, we generalize Mitra’s formula for arbitrary diffusion encoding waveforms, including recently developed q-space trajectory encoding sequences. We show that the surface-to-volume ratio can be significantly misestimated using the original Mitra’s formula without taking into account the applied gradient profile. In order to obtain more accurate estimation in anisotropic samples, we propose an efficient and robust optimization algorithm to design diffusion gradient waveforms with prescribed features. Our results are supported by Monte Carlo simulations. [ABSTRACT FROM AUTHOR]

Optical coherence tomography (OCT) has revolutionized diagnosis and prognosis of ophthalmic diseases by visualization and measurement of retinal layers. To speed up the quantitative analysis of disease biomarkers, an increasing number of automatic segmentation algorithms have been proposed to estimate the boundary locations of retinal layers. While the performance of these algorithms has significantly improved in recent years, a critical question to ask is how far we are from a theoretical limit to OCT segmentation performance. In this paper, we present the Cramèr–Rao lower bounds (CRLBs) for the problem of OCT layer segmentation. In deriving the CRLBs, we address the important problem of defining statistical models that best represent the intensity distribution in each layer of the retina. Additionally, we calculate the bounds under an optimal affine bias, reflecting the use of prior knowledge in many segmentation algorithms. Experiments using in vivo images of human retina from a commercial spectral domain OCT system are presented, showing potential for improvement of automated segmentation accuracy. Our general mathematical model can be easily adapted for virtually any OCT system. Furthermore, the statistical models of signal and noise developed in this paper can be utilized for the future improvements of OCT image denoising, reconstruction, and many other applications. [ABSTRACT FROM AUTHOR]

The basic emission process associated with positron emission tomography (PET) imaging is Poisson in nature. Reconstructed images inherit some aspects of this—regional variability is typically proportional to the regional mean. Iterative reconstruction using expectation–maximization (EM), widely used in clinical imaging now, imposes positivity constraints that impact noise properties. This paper is motivated by the analysis of data from a physical phantom study of a PET/CT scanner in routine clinical use. Both traditional filtered back-projection (FBP) and EM reconstructions of the images are considered. FBP images are quite Gaussian, but the EM reconstructions exhibit Gamma-like skewness. The Gamma structure has implications for how reconstructed PET images might be processed statistically. Post-reconstruction inference—model fitting and diagnostics for regions of interest are of particular interest. Although the relevant Gamma parameterization is not within the framework of generalized linear models (GLM), iteratively re-weighted least squares (IRLS) techniques, which are often used to find the maximum likelihood estimates of a GLM, can be adapted for analysis in this setting. This paper highlights the use of a Gamma-based probability transform in producing normalized residuals as model diagnostics. The approach is demonstrated for quality assurance analyses associated with physical phantom studies—recovering estimates of local bias and variance characteristics in an operational scanner. Numerical simulations show that when the Gamma assumption is reasonable, gains in efficiency are obtained. This paper shows that the adaptation of standard analysis methods to accommodate the Gamma structure is straightforward and beneficial. [ABSTRACT FROM PUBLISHER]

A fully automated 2-D+time myocardial segmentation framework is proposed for cardiac magnetic resonance (CMR) blood-oxygen-level-dependent (BOLD) data sets. Ischemia detection with CINE BOLD CMR relies on spatio-temporal patterns in myocardial intensity, but these patterns also trouble supervised segmentation methods, the de facto standard for myocardial segmentation in cine MRI. Segmentation errors severely undermine the accurate extraction of these patterns. In this paper, we build a joint motion and appearance method that relies on dictionary learning to find a suitable subspace. Our method is based on variational pre-processing and spatial regularization using Markov random fields, to further improve performance. The superiority of the proposed segmentation technique is demonstrated on a data set containing cardiac phase-resolved BOLD MR and standard CINE MR image sequences acquired in baseline and ischemic condition across ten canine subjects. Our unsupervised approach outperforms even supervised state-of-the-art segmentation techniques by at least 10% when using Dice to measure accuracy on BOLD data and performs at par for standard CINE MR. Furthermore, a novel segmental analysis method attuned for BOLD time series is utilized to demonstrate the effectiveness of the proposed method in preserving key BOLD patterns. [ABSTRACT FROM AUTHOR]

In this paper, we develop a new weakly supervised learning algorithm to learn to segment cancerous regions in histopathology images. This paper is under a multiple instance learning (MIL) framework with a new formulation, deep weak supervision (DWS); we also propose an effective way to introduce constraints to our neural networks to assist the learning process. The contributions of our algorithm are threefold: 1) we build an end-to-end learning system that segments cancerous regions with fully convolutional networks (FCNs) in which image-to-image weakly-supervised learning is performed; 2) we develop a DWS formulation to exploit multi-scale learning under weak supervision within FCNs; and 3) constraints about positive instances are introduced in our approach to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. The proposed algorithm, abbreviated as DWS-MIL, is easy to implement and can be trained efficiently. Our system demonstrates the state-of-the-art results on large-scale histopathology image data sets and can be applied to various applications in medical imaging beyond histopathology images, such as MRI, CT, and ultrasound images. [ABSTRACT FROM AUTHOR]

In fluorescence optical tomography, many works in the literature focus on the linear reconstruction problem to obtain the fluorescent yield or the linearized reconstruction problem to obtain the absorption coefficient. The nonlinear reconstruction problem, to reconstruct the fluorophore absorption coefficient, is of interest in imaging studies as it presents the possibility of better reconstructions owing to a more appropriate model. Accurate and computationally efficient forward models are also critical in the reconstruction process. The \textSPN approximation to the radiative transfer equation (RTE) is gaining importance for tomographic reconstructions owing to its computational advantages over the full RTE while being more accurate and applicable than the commonly used diffusion approximation. This paper presents Gauss–Newton-based fully nonlinear reconstruction for the \textSP3 approximated fluorescence optical tomography problem with respect to shape as well as the conventional finite-element method-based representations. The contribution of this paper is the Frechet derivative calculations for this problem and demonstration of reconstructions in both representations. For the shape reconstructions, radial-basis-function represented level-set-based shape representations are used. We present reconstructions for tumor-mimicking test objects in scattering and absorption dominant settings, respectively, for moderately noisy data sets in order to demonstrate the viability of the formulation. Comparisons are presented between the nonlinear and linearized reconstruction schemes in an element wise setting to illustrate the benefits of using the former especially for absorption dominant media. [ABSTRACT FROM AUTHOR]

Although deep networks have been shown to perform very well on a variety of medical imaging tasks, inference in the presence of pathology presents several challenges to common models. These challenges impede the integration of deep learning models into real clinical workflows, where the customary process of cascading deterministic outputs from a sequence of image-based inference steps (e.g. registration, segmentation) generally leads to an accumulation of errors that impacts the accuracy of downstream inference tasks. In this paper, we propose that by embedding uncertainty estimates across cascaded inference tasks, performance on the downstream inference tasks should be improved. We demonstrate the effectiveness of the proposed approach in three different clinical contexts: (i) We demonstrate that by propagating T2 weighted lesion segmentation results and their associated uncertainties, subsequent T2 lesion detection performance is improved when evaluated on a proprietary large-scale, multi-site, clinical trial dataset acquired from patients with Multiple Sclerosis. (ii) We show an improvement in brain tumour segmentation performance when the uncertainty map associated with a synthesised missing MR volume is provided as an additional input to a follow-up brain tumour segmentation network, when evaluated on the publicly available BraTS-2018 dataset. (iii) We show that by propagating uncertainties from a voxel-level hippocampus segmentation task, the subsequent regression of the Alzheimer’s disease clinical score is improved. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis systems (CADs) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel end-to-end deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predicts diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional consideration of important shape and boundary knowledge. One path, called the Locality Preserving Learner (LPL), is devoted to hierarchically extracting and exploiting intrinsic features of the input. Whereas the other path, called the Conditional Graph Learner (CGL), focuses on generating geometrical features via modeling pixel-wise image to mask correlations. By integrating the two learners, both the cancer semantics and cancer representations are well learned, and the component learning paths in return complement each other, contributing an improvement to the mass segmentation and cancer classification problem at the same time. In addition, by integrating an automatic detection set-up, the DualCoreNet achieves fully automatic breast cancer diagnosis practically. Experimental results show that in benchmark DDSM dataset, DualCoreNet has outperformed other related works in both segmentation and classification tasks, achieving 92.27% DI coefficient and 0.85 AUC score. In another benchmark INbreast dataset, DualCoreNet achieves the best mammography segmentation (93.69% DI coefficient) and competitive classification performance (0.93 AUC score). [ABSTRACT FROM AUTHOR]

This paper proposes a new method for joint design of radiofrequency (RF) and gradient waveforms in Magnetic Resonance Imaging (MRI), and applies it to the design of 3D spatially tailored saturation and inversion pulses. The joint design of both waveforms is characterized by the ODE Bloch equations, to which there is no known direct solution. Existing approaches therefore typically rely on simplified problem formulations based on, e.g., the small-tip approximation or constraining the gradient waveforms to particular shapes, and often apply only to specific objective functions for a narrow set of design goals (e.g., ignoring hardware constraints). This paper develops and exploits an auto-differentiable Bloch simulator to directly compute Jacobians of the (Bloch-simulated) excitation pattern with respect to RF and gradient waveforms. This approach is compatible with arbitrary sub-differentiable loss functions, and optimizes the RF and gradients directly without restricting the waveform shapes. For computational efficiency, we derive and implement explicit Bloch simulator Jacobians (approximately halving computation time and memory usage). To enforce hardware limits (peak RF, gradient, and slew rate), we use a change of variables that makes the 3D pulse design problem effectively unconstrained; we then optimize the resulting problem directly using the proposed auto-differentiation framework. We demonstrate our approach with two kinds of 3D excitation pulses that cannot be easily designed with conventional approaches: Outer-volume saturation (90° flip angle), and inner-volume inversion. [ABSTRACT FROM AUTHOR]

Digital topology and geometry refers to the use of topologic and geometric properties and features for images defined in digital grids. Such methods have been widely used in many medical imaging applications, including image segmentation, visualization, manipulation, interpolation, registration, surface-tracking, object representation, correction, quantitative morphometry etc. Digital topology and geometry play important roles in medical imaging research by enriching the scope of target outcomes and by adding strong theoretical foundations with enhanced stability, fidelity, and efficiency. This paper presents a comprehensive yet compact survey on results, principles, and insights of methods related to digital topology and geometry with strong emphasis on understanding their roles in various medical imaging applications. Specifically, this paper reviews methods related to distance analysis and path propagation, connectivity, surface-tracking, image segmentation, boundary and centerline detection, topology preservation and local topological properties, skeletonization, and object representation, correction, and quantitative morphometry. A common thread among the topics reviewed in this paper is that their theory and algorithms use the principle of digital path connectivity, path propagation, and neighborhood analysis. [ABSTRACT FROM AUTHOR]

This paper describes a fast-forward electromagnetic solver (FFS) for the image reconstruction algorithm of our microwave tomography system. Our apparatus is a preclinical prototype of a biomedical imaging system, designed for the purpose of early breast cancer detection. It operates in the 3–6-GHz frequency band using a circular array of probe antennas immersed in a matching liquid; it produces image reconstructions of the permittivity and conductivity profiles of the breast under examination. Our reconstruction algorithm solves the electromagnetic (EM) inverse problem and takes into account the real EM properties of the probe antenna array as well as the influence of the patient’s body and that of the upper metal screen sheet. This FFS algorithm is much faster than conventional EM simulation solvers. In comparison, in the same PC, the CST solver takes ~45 min, while the FFS takes ~1 s of effective simulation time for the same EM model of a numerical breast phantom. [ABSTRACT FROM PUBLISHER]

Glaucoma is the leading reason for irreversible blindness. Early detection and timely treatment of glaucoma are essential for preventing visual field loss or even blindness. In clinical practice, Optical Coherence Tomography (OCT) and Visual Field (VF) exams are two widely-used and complementary techniques for diagnosing glaucoma. OCT provides quantitative measurements of the optic nerve head (ONH) structure, while VF test is the functional assessment of peripheral vision. In this paper, we propose a Deep Relation Transformer (DRT) to perform glaucoma diagnosis with OCT and VF information combined. A novel deep reasoning mechanism is proposed to explore implicit pairwise relations between OCT and VF information in global and regional manners. With the pairwise relations, a carefully-designed deep transformer mechanism is developed to enhance the representation with complementary information for each modal. Based on reasoning and transformer mechanisms, three successive modules are designed to extract and collect valuable information for glaucoma diagnosis, the global relation module, the guided regional relation module, and the interaction transformer module, namely. Moreover, we build a large dataset, namely ZOC-OCT&VF dataset, which includes 1395 OCT-VF pairs for developing and evaluating our DRT. We conduct extensive experiments to validate the effectiveness of the proposed method. Experimental results show that our method achieves 88.3% accuracy and outperforms the existing single-modal approaches with a large margin. The codes and dataset will be publicly available in the future. [ABSTRACT FROM AUTHOR]

Automated airway segmentation is a prerequisite for pre-operative diagnosis and intra-operative navigation for pulmonary intervention. Due to the small size and scattered spatial distribution of peripheral bronchi, this is hampered by a severe class imbalance between foreground and background regions, which makes it challenging for CNN-based methods to parse distal small airways. In this paper, we demonstrate that this problem is arisen by gradient erosion and dilation of the neighborhood voxels. During back-propagation, if the ratio of the foreground gradient to background gradient is small while the class imbalance is local, the foreground gradients can be eroded by their neighborhoods. This process cumulatively increases the noise information included in the gradient flow from top layers to the bottom ones, limiting the learning of small structures in CNNs. To alleviate this problem, we use group supervision and the corresponding WingsNet to provide complementary gradient flows to enhance the training of shallow layers. To further address the intra-class imbalance between large and small airways, we design a General Union loss function that obviates the impact of airway size by distance-based weights and adaptively tunes the gradient ratio based on the learning process. Extensive experiments on public datasets demonstrate that the proposed method can predict the airway structures with higher accuracy and better morphological completeness than the baselines. [ABSTRACT FROM AUTHOR]

The aim of this paper is to provide a comprehensive overview of the MICCAI 2020 AutoImplant Challenge. The approaches and publications submitted and accepted within the challenge will be summarized and reported, highlighting common algorithmic trends and algorithmic diversity. Furthermore, the evaluation results will be presented, compared and discussed in regard to the challenge aim: seeking for low cost, fast and fully automated solutions for cranial implant design. Based on feedback from collaborating neurosurgeons, this paper concludes by stating open issues and post-challenge requirements for intra-operative use. The codes can be found at https://github.com/Jianningli/tmi. [ABSTRACT FROM AUTHOR]

Adequate medical images are often indispensable in contemporary deep learning-based medical imaging studies, although the acquisition of certain image modalities may be limited due to several issues including high costs and patients issues. However, thanks to recent advances in deep learning techniques, the above tough problem can be substantially alleviated by medical images synthesis, by which various modalities including T1/T2/DTI MRI images, PET images, cardiac ultrasound images, retinal images, and so on, have already been synthesized. Unfortunately, the arterial spin labeling (ASL) image, which is an important fMRI indicator in dementia diseases diagnosis nowadays, has never been comprehensively investigated for the synthesis purpose yet. In this paper, ASL images have been successfully synthesized from structural magnetic resonance images for the first time. Technically, a novel unbalanced deep discriminant learning-based model equipped with new ResNet sub-structures is proposed to realize the synthesis of ASL images from structural magnetic resonance images. The extensive experiments have been conducted. Comprehensive statistical analyses reveal that: 1) this newly introduced model is capable to synthesize ASL images that are similar towards real ones acquired by actual scanning; 2) synthesized ASL images obtained by the new model have demonstrated outstanding performance when undergoing rigorous tests of region-based and voxel-based corrections of partial volume effects, which are essential in ASL images processing; and 3) it is also promising that the diagnosis performance of dementia diseases can be significantly improved with the help of synthesized ASL images obtained by the new model, based on a multi-modal MRI dataset containing 355 demented patients in this paper. [ABSTRACT FROM AUTHOR]