Search

We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between $O(N^2)$ image pairs in $O(N~\log~N)$ operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genotyping are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC=0.97. Software is provided for efficient fine-grained curation of large, generic image datasets., 10 pages, 8 figures

We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison to common multi-atlas segmentation, while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with highly variable field-of-view., Comment: Accepted for publication at IEEE Transactions on Medical Imaging

Prostate cancer is the second most prevalent cancer in men worldwide. Deep neural networks have been successfully applied for prostate cancer diagnosis in magnetic resonance images (MRI). Pathology results from biopsy procedures are often used as ground truth to train such systems. There are several sources of noise in creating ground truth from biopsy data including sampling and registration errors. We propose: 1) A fully convolutional neural network (FCN) to produce cancer probability maps across the whole prostate gland in MRI; 2) A Gaussian weighted loss function to train the FCN with sparse biopsy locations; 3) A probabilistic framework to model biopsy location uncertainty and adjust cancer probability given the deep model predictions. We assess the proposed method on 325 biopsy locations from 203 patients. We observe that the proposed loss improves the area under the receiver operating characteristic curve and the biopsy location adjustment improves the sensitivity of the models.

Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.

PURPOSE We present SlicerDMRI, an open-source software suite that enables research using diffusion magnetic resonance imaging (dMRI), the only modality that can map the white matter connections of the living human brain. SlicerDMRI enables analysis and visualization of dMRI data and is aimed at the needs of clinical research users. SlicerDMRI is built upon and deeply integrated with 3D Slicer, a National Institutes of Health–supported open-source platform for medical image informatics, image processing, and three-dimensional visualization. Integration with 3D Slicer provides many features of interest to cancer researchers, such as real-time integration with neuronavigation equipment, intraoperative imaging modalities, and multimodal data fusion. One key application of SlicerDMRI is in neurosurgery research, where brain mapping using dMRI can provide patient-specific maps of critical brain connections as well as insight into the tissue microstructure that surrounds brain tumors. PATIENTS AND METHODS In this article, we focus on a demonstration of SlicerDMRI as an informatics tool to enable end-to-end dMRI analyses in two retrospective imaging data sets from patients with high-grade glioma. Analyses demonstrated here include conventional diffusion tensor analysis, advanced multifiber tractography, automated identification of critical fiber tracts, and integration of multimodal imagery with dMRI. RESULTS We illustrate the ability of SlicerDMRI to perform both conventional and advanced dMRI analyses as well as to enable multimodal image analysis and visualization. We provide an overview of the clinical rationale for each analysis along with pointers to the SlicerDMRI tools used in each. CONCLUSION SlicerDMRI provides open-source and clinician-accessible research software tools for dMRI analysis. SlicerDMRI is available for easy automated installation through the 3D Slicer Extension Manager.

Background Brain metastases are manually identified during stereotactic radiosurgery (SRS) treatment planning, which is time consuming and potentially challenging. Purpose To develop and investigate deep learning (DL) methods for detecting brain metastasis with MRI to aid in treatment planning for SRS. Materials and Methods In this retrospective study, contrast material-enhanced three-dimensional T1-weighted gradient-echo MRI scans from patients who underwent gamma knife SRS from January 2011 to August 2018 were analyzed. Brain metastases were manually identified and contoured by neuroradiologists and treating radiation oncologists. DL single-shot detector (SSD) algorithms were constructed and trained to map axial MRI slices to a set of bounding box predictions encompassing metastases and associated detection confidences. Performances of different DL SSDs were compared for per-lesion metastasis-based detection sensitivity and positive predictive value (PPV) at a 50% confidence threshold. For the highest-performing model, detection performance was analyzed by using free-response receiver operating characteristic analysis. Results Two hundred sixty-six patients (mean age, 60 years ± 14 [standard deviation]; 148 women) were randomly split into 80% training and 20% testing groups (212 and 54 patients, respectively). For the testing group, sensitivity of the highest-performing (baseline) SSD was 81% (95% confidence interval [CI]: 80%, 82%; 190 of 234) and PPV was 36% (95% CI: 35%, 37%; 190 of 530). For metastases measuring at least 6 mm, sensitivity was 98% (95% CI: 97%, 99%; 130 of 132) and PPV was 36% (95% CI: 35%, 37%; 130 of 366). Other models (SSD with a ResNet50 backbone, SSD with focal loss, and RetinaNet) yielded lower sensitivities of 73% (95% CI: 72%, 74%; 171 of 234), 77% (95% CI: 76%, 78%; 180 of 234), and 79% (95% CI: 77%, 81%; 184 of 234), respectively, and lower PPVs of 29% (95% CI: 28%, 30%; 171 of 581), 26% (95% CI: 26%, 26%; 180 of 681), and 13% (95% CI: 12%, 14%; 184 of 1412). Conclusion Deep-learning single-shot detector models detected nearly all brain metastases that were 6 mm or larger with limited false-positive findings using postcontrast T1-weighted MRI. © RSNA, 2020 See also the editorial by Kikinis and Wells in this issue.

Neuroimaging studies typically adopt a common feature space for all data, which may obscure aspects of neuroanatomy only observable in subsets of a population, e.g. cortical folding patterns unique to individuals or shared by close relatives. Here, we propose to model individual variability using a distinctive keypoint signature: a set of unique, localized patterns, detected automatically in each image by a generic saliency operator. The similarity of an image pair is then quantified by the proportion of keypoints they share using a novel Jaccard-like measure of set overlap. Experiments demonstrate the keypoint method to be highly efficient and accurate, using a set of 7536 T1-weighted MRIs pooled from four public neuroimaging repositories, including twins, non-twin siblings, and 3334 unique subjects. All same-subject image pairs are identified by a similarity threshold despite confounds including aging and neurodegenerative disease progression. Outliers reveal previously unknown data labeling inconsistencies, demonstrating the usefulness of the keypoint signature as a computational tool for curating large neuroimage datasets.

This paper presents an efficient approach to quantifying image registration uncertainty based on a low-dimensional representation of geometric deformations. In contrast to previous methods, we develop a Bayesian diffeomorphic registration framework in a bandlimited space, rather than a high-dimensional image space. We show that a dense posterior distribution on deformation fields can be fully characterized by much fewer parameters, which dramatically reduces the computational complexity of model inferences. To further avoid heavy computation loads introduced by random sampling algorithms, we approximate a marginal posterior by using Laplace’s method at the optimal solution of log-posterior distribution. Experimental results on both 2D synthetic data and real 3D brain magnetic resonance imaging (MRI) scans demonstrate that our method is significantly faster than the state-of-the-art diffeomorphic registration uncertainty quantification algorithms, while producing comparable results.

To investigate the heating of EEG electrodes during magnetic resonance imaging (MRI) scans and to better understand the underlying physical mechanisms with a focus on the antenna effect.Gold cup and conductive plastic electrodes were placed on small watermelons with fiberoptic probes used to measure electrode temperature changes during a variety of 1.5T and 3T MRI scans. A subset of these experiments was repeated on a healthy human volunteer.The differences between gold and plastic electrodes did not appear to be practically significant. For both electrode types, we observed heating below 4°C for straight wires whose lengths were multiples of ½ the radiofrequency (RF) wavelength and stronger heating (over 15°C) for wire lengths that were odd multiples of ¼ RF wavelength, consistent with the antenna effect.The antenna effect, which has received little attention so far in the context of EEG-MRI safety, can play as significant a role as the loop effect (from electromagnetic induction) in the heating of EEG electrodes, and therefore wire lengths that are odd multiples of ¼ RF wavelength should be avoided. These results have important implications for the design of EEG electrodes and MRI studies as they help to minimize the risk to patients undergoing MRI with EEG electrodes in place.

PURPOSE: The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI–ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS: We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS: Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION: We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.

PURPOSE: Brain shift during tumor resection can progressively invalidate the accuracy of neuronavigation systems and affect neurosurgeons’ ability to achieve optimal resections. This paper compares two methods that have been presented in the literature to compensate for brain shift: a thin-plate spline deformation model and a finite element method (FEM). For this comparison, both methods are driven by identical sparse data. Specifically, both methods are driven by displacements between automatically detected and matched feature points from intraoperative 3D ultrasound (iUS). Both methods have been shown to be fast enough for intraoperative brain shift correction (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018; Luo et al. in J Med Imaging (Bellingham) 4(3):035003, 2017). However, the spline method requires no preprocessing and ignores physical properties of the brain while the FEM method requires significant preprocessing and incorporates patient-specific physical and geometric constraints. The goal of this work was to explore the relative merits of these methods on recent clinical data. METHODS: Data acquired during 19 sequential tumor resections in Brigham and Women’s Hospital’s Advanced Multi-modal Image-Guided Operating Suite between December 2017 and October 2018 were considered for this retrospective study. Of these, 15 cases and a total of 24 iUS to iUS image pairs met inclusion requirements. Automatic feature detection (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018) was used to detect and match features in each pair of iUS images. Displacements between matched features were then used to drive both the spline model and the FEM method to compensate for brain shift between image acquisitions. The accuracies of the resultant deformation models were measured by comparing the displacements of manually identified landmarks before and after deformation. RESULTS: The mean initial subcortical registration error between preoperative MRI and the first iUS image averaged 5.3 ± 0.75 mm. The mean subcortical brain shift, measured using displacements between manually identified landmarks in pairs of iUS images, was 2.5 ± 1.3 mm. Our results showed that FEM was able to reduce subcortical registration error by a small but statistically significant amount (from 2.46 to 2.02 mm). A large variability in the results of the spline method prevented us from demonstrating either a statistically significant reduction in subcortical registration error after applying the spline method or a statistically significant difference between the results of the two methods. CONCLUSIONS: In this study, we observed less subcortical brain shift than has previously been reported in the literature (Frisken et al., in: Miller (ed) Biomechanics of the brain, Springer, Cham, 2019). This may be due to the fact that we separated out the initial misregistration between preoperative MRI and the first iUS image from our brain shift measurements or it may be due to modern neurosurgical practices designed to reduce brain shift, including reduced craniotomy sizes and better control of intracranial pressure with the use of mannitol and other medications. It appears that the FEM method and its use of geometric and biomechanical constraints provided more consistent brain shift correction and better correction farther from the driving feature displacements than the simple spline model. The spline-based method was simpler and tended to give better results for small deformations. However, large variability in the spline results and relatively small brain shift prevented this study from demonstrating a statistically significant difference between the results of the two methods.

Multimodal integration between mass spectrometry imaging (MSI) and radiology-established modalities such as magnetic resonance imaging (MRI) would allow the investigations of key questions in complex biological systems such as the central nervous system. Such integration would provide complementary multiscale data to bridge the gap between molecular and anatomical phenotypes, potentially revealing new insights into molecular mechanisms underlying anatomical pathologies presented on MRI. Automatic coregistration between 3D MSI/MRI is a computationally challenging process due to dimensional complexity, MSI data sparsity, lack of direct spatial-correspondences, and nonlinear tissue deformation. Here, we present a new computational approach based on stochastic neighbor embedding to nonlinearly align 3D MSI to MRI data, identify and reconstruct biologically relevant molecular patterns in 3D, and fuse the MSI datacube to the MRI space. We demonstrate our method using multimodal high-spectral resolution matrix-assisted laser desorption ionization (MALDI) 9.4 T MSI and 7 T in vivo MRI data, acquired from a patient-derived, xenograft mouse brain model of glioblastoma following administration of the EGFR inhibitor drug of Erlotinib. Results show the distribution of some identified molecular ions of the EGFR inhibitor erlotinib, a phosphatidylcholine lipid, and cholesterol, which were reconstructed in 3D and mapped to the MRI space. The registration quality was evaluated on two normal mouse brains using the Dice coefficient for the regions of brainstem, hippocampus, and cortex. The method is generic and can therefore be applied to hyperspectral images from different mass spectrometers and integrated with other established in vivo imaging modalities such as computed tomography (CT) and positron emission tomography (PET).

The ability to detect neuronal currents with high spatiotemporal resolution using magnetic resonance imaging (MRI) is important for studying human brain function in both health and disease. While significant progress has been made, we still lack evidence showing that it is possible to measure an MR signal time-locked to neuronal currents with a temporal waveform matching concurrently recorded local field potentials (LFPs). Also lacking is evidence that such MR data can be used to image current distribution in active tissue. Since these two results are lacking even in vitro, we obtained these data in an intact isolated whole cerebellum of turtle during slow neuronal activity mediated by metabotropic glutamate receptors using a gradient-echo EPI sequence (TR = 100 ms) at 4.7 T. Our results show that it is possible (1) to reliably detect an MR phase shift time course matching that of the concurrently measured LFP evoked by stimulation of a cerebellar peduncle, (2) to detect the signal in single voxels of 0.1 mm3, (3) to determine the spatial phase map matching the magnetic field distribution predicted by the LFP map, (4) to estimate the distribution of neuronal current in the active tissue from a group-average phase map, and (5) to provide a quantitatively accurate theoretical account of the measured phase shifts. The peak values of the detected MR phase shifts were 0.27–0.37°, corresponding to local magnetic field changes of 0.67–0.93 nT (for TE = 26 ms). Our work provides an empirical basis for future extensions to in vivo imaging of neuronal currents.

Image-guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application, and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperenial MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-dimensional fully convolutional neural network model was developed, trained and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80 mm average in needle tip detection, and 0.98ᵒ in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared to the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.

Purpose: The brain undergoes significant structural change over the course of neurosurgery, including highly nonlinear deformation and resection. It can be informative to recover the spatial mapping between structures identified in preoperative surgical planning and the intraoperative state of the brain. We present a novel feature-based method for achieving robust, fully automatic deformable registration of intraoperative neurosurgical ultrasound images. Methods: A sparse set of local image feature correspondences is first estimated between ultrasound image pairs, after which rigid, affine and thin-plate spline models are used to estimate dense mappings throughout the image. Correspondences are derived from 3D features, distinctive generic image patterns that are automatically extracted from 3D ultrasound images and characterized in terms of their geometry (i.e., location, scale, and orientation) and a descriptor of local image appearance. Feature correspondences between ultrasound images are achieved based on a nearest-neighbor descriptor matching and probabilistic voting model similar to the Hough transform. Results: Experiments demonstrate our method on intraoperative ultrasound images acquired before and after opening of the dura mater, during resection and after resection in nine clinical cases. A total of 1620 automatically extracted 3D feature correspondences were manually validated by eleven experts and used to guide the registration. Then, using manually labeled corresponding landmarks in the pre- and post-resection ultrasound images, we show that our feature-based registration reduces the mean target registration error from an initial value of 3.3 to 1.5 mm. Conclusions: This result demonstrates that the 3D features promise to offer a robust and accurate solution for 3D ultrasound registration and to correct for brain shift in image-guided neurosurgery., National Institutes of Health (U.S.) (Grant P41-EB015898-09), National Institutes of Health (U.S.) (Grant P41-EB015902), National Institutes of Health (U.S.) (Grant R01-NS049251), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant PD/BD/105869/2014), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant IDMEC/LAETA UID/EMS/50022/2013)

Intraoperative brain deformation reduces the effectiveness of using preoperative images for intraoperative surgical guidance. We propose an algorithm for deformable registration of intraoperative ultrasound (US) and preoperative magnetic resonance (MR) images in the context of brain tumor resection. From each image voxel, a set of multi-scale and multi-orientation Gabor attributes is extracted from which optimal components are selected to establish a distinctive morphological signature of the anatomical and geometric context of its surroundings. To match the attributes across image pairs, we assign higher weights – higher mutual-saliency values - to those voxels more likely to establish reliable correspondences across images. The correlation coefficient is used as the similarity measure to evaluate effectiveness of the algorithm for multi-modal registration. Free-form deformation and discrete optimization are chosen as the deformation model and optimization strategy, respectively. Experiments demonstrate our methodology on registering preoperative T2-FLAIR MR to intraoperative US in 22 clinical cases. Using manually labelled corresponding landmarks between preoperative MR and intraoperative US images, we show that the mean target registration error decreases from an initial value of 5.37 ± 4.27 mm to 3.35 ± 1.19 mm after registration.

To determine the detection rate, clinical relevance, Gleason grade, and location of prostate cancer ( PCa prostate cancer ) diagnosed with and the safety of an in-bore transperineal 3-T magnetic resonance (MR) imaging-guided prostate biopsy in a clinically heterogeneous patient population.This prospective retrospectively analyzed study was HIPAA compliant and institutional review board approved, and informed consent was obtained. Eighty-seven men (mean age, 66.2 years ± 6.9) underwent multiparametric endorectal prostate MR imaging at 3 T and transperineal MR imaging-guided biopsy. Three subgroups of patients with at least one lesion suspicious for cancer were included: men with no prior PCa prostate cancer diagnosis, men with PCa prostate cancer who were undergoing active surveillance, and men with treated PCa prostate cancer and suspected recurrence. Exclusion criteria were prior prostatectomy and/or contraindication to 3-T MR imaging. The transperineal MR imaging-guided biopsy was performed in a 70-cm wide-bore 3-T device. Overall patient biopsy outcomes, cancer detection rates, Gleason grade, and location for each subgroup were evaluated and statistically compared by using χ(2) and one-way analysis of variance followed by Tukey honestly significant difference post hoc comparisons.Ninety biopsy procedures were performed with no serious adverse events, with a mean of 3.7 targets sampled per gland. Cancer was detected in 51 (56.7%) men: 48.1% (25 of 52) with no prior PCa prostate cancer , 61.5% (eight of 13) under active surveillance, and 72.0% (18 of 25) in whom recurrence was suspected. Gleason pattern 4 or higher was diagnosed in 78.1% (25 of 32) in the no prior PCa prostate cancer and active surveillance groups. Gleason scores were not assigned in the suspected recurrence group. MR targets located in the anterior prostate had the highest cancer yield (40 of 64, 62.5%) compared with those for the other parts of the prostate (P.001).In-bore 3-T transperineal MR imaging-guided biopsy, with a mean of 3.7 targets per gland, allowed detection of many clinically relevant cancers, many of which were located anteriorly.

Using electroencephalography (EEG) to elucidate the spontaneous activation of brain resting-state networks (RSNs) is nontrivial as the signal of interest is of low amplitude and it is difficult to distinguish the underlying neural sources. Using the principles of electric field topographical analysis, it is possible to estimate the meta-stable states of the brain (i.e., the resting-state topographies, so-called microstates). We estimated seven resting-state topographies explaining the EEG data set with k-means clustering (N = 164, 256 electrodes). Using a method specifically designed to localize the sources of broadband EEG scalp topographies by matching sensor and source space temporal patterns, we demonstrated that we can estimate the EEG RSNs reliably by measuring the reproducibility of our findings. After subtracting their mean from the seven EEG RSNs, we identified seven state-specific networks. The mean map includes regions known to be densely anatomically and functionally connected (superior frontal, superior parietal, insula, and anterior cingulate cortices). While the mean map can be interpreted as a “router,” crosslinking multiple functional networks, the seven state-specific RSNs partly resemble and extend previous functional magnetic resonance imaging-based networks estimated as the hemodynamic correlates of four canonical EEG microstates.

This work presents an initial exploration of joint cortical surface and diffusion MRI analysis for neurosurgical patient data. We propose a groupwise cortical modeling strategy that performs an embedding of cortical points from a healthy population and a method for transferring the embedding (with associated information of anatomical label) to patient datasets for cortical parcellation prediction. Our proposed method correlates cortical surfaces based on groupwise white matter connectivity characteristics via a fiber clustering scheme. Unlike other parcellation methods, correspondence of cortical surface vertices is not required. Thus the proposed method can be applied to datasets of patients with brain tumors, using an approximate cortical surface such as a white matter/gray matter boundary derived from diffusion anisotropy. Our initial results on patient data showed good overlap of functional ground truth (subject-specific functional MRI activation areas) with predicted cortical parcels, with 10 of 13 activations overlapping an anatomically corresponding prediction.

Prostate cancer (PCa) remains a leading cause of cancer mortality among American men. Multi-parametric magnetic resonance imaging (mpMRI) is widely used to assist with detection of PCa and characterization of its aggressiveness. Computer-aided diagnosis (CADx) of PCa in MRI can be used as clinical decision support system to aid radiologists in interpretation and reporting of mpMRI. We report on the development of a convolution neural network (CNN) model to support CADx in PCa based on the appearance of prostate tissue in mpMRI, conducted as part of the SPIE-AAPM-NCI PROSTATEx challenge. The performance of different combinations of mpMRI inputs to CNN was assessed and the best result was achieved using DWI and DCE-MRI modalities together with the zonal information of the finding. On the test set, the model achieved an area under the receiver operating characteristic curve of 0.80.

Magnetic Resonance Imaging (MRI) is widely used in routine clinical diagnosis and treatment. However, variations in MRI acquisition protocols result in different appearances of normal and diseased tissue in the images. Convolutional neural networks (CNNs), which have shown to be successful in many medical image analysis tasks, are typically sensitive to the variations in imaging protocols. Therefore, in many cases, networks trained on data acquired with one MRI protocol, do not perform satisfactorily on data acquired with different protocols. This limits the use of models trained with large annotated legacy datasets on a new dataset with a different domain which is often a recurring situation in clinical settings. In this study, we aim to answer the following central questions regarding domain adaptation in medical image analysis: Given a fitted legacy model, 1) How much data from the new domain is required for a decent adaptation of the original network?; and, 2) What portion of the pre-trained model parameters should be retrained given a certain number of the new domain training samples? To address these questions, we conducted extensive experiments in white matter hyperintensity segmentation task. We trained a CNN on legacy MR images of brain and evaluated the performance of the domain-adapted network on the same task with images from a different domain. We then compared the performance of the model to the surrogate scenarios where either the same trained network is used or a new network is trained from scratch on the new dataset.The domain-adapted network tuned only by two training examples achieved a Dice score of 0.63 substantially outperforming a similar network trained on the same set of examples from scratch.

Purpose Reliably detecting MRI signals in the brain that are more tightly coupled to neural activity than blood-oxygen-level-dependent fMRI signals could not only prove valuable for basic scientific research but could also enhance clinical applications such as epilepsy presurgical mapping. This endeavor will likely benefit from an improved understanding of the behavior of ionic currents, the mediators of neural activity, in the presence of the strong magnetic fields that are typical of modern-day MRI scanners. Theory Of the various mechanisms that have been proposed to explain the behavior of ionic volume currents in a magnetic field, only one—magnetohydrodynamic flow—predicts a slow evolution of signals, on the order of a minute for normal saline in a typical MRI scanner. Methods This prediction was tested by scanning a volume-current phantom containing normal saline with gradient-echo-planar imaging at 3 T. Results Greater signal changes were observed in the phase of the images than in the magnitude, with the changes evolving on the order of a minute. Conclusion These results provide experimental support for the MHD flow hypothesis. Furthermore, MHD-driven cerebrospinal fluid flow could provide a novel fMRI contrast mechanism. Magn Reson Med 74:1145–1155, 2015. © 2014 Wiley Periodicals, Inc.

International audience; In this paper, we present a graph-based concurrent brain tumor segmentation and atlas to diseased patient registration framework. Both segmentation and registration problems are modeled using a unified pairwise discrete Markov Random Field model on a sparse grid superimposed to the image domain. Segmentation is addressed based on pattern classification techniques, while registration is performed by maximizing the similarity between volumes and is modular with respect to the matching criterion. The two problems are coupled by relaxing the registration term in the tumor area, corresponding to areas of high classification score and high dissimilarity between volumes. In order to overcome the main shortcomings of discrete approaches regarding appropriate sampling of the solution space as well as important memory requirements, content driven samplings of the discrete displacement set and the sparse grid are considered, based on the local segmentation and registration uncertainties recovered by the min marginal energies. State of the art results on a substantial low-grade glioma database demonstrate the potential of our method, while our proposed approach shows maintained performance and strongly reduced complexity of the model.

Intraoperative tissue deformation, known as brain shift, decreases the benefit of using preoperative images to guide neurosurgery. Non-rigid registration of preoperative magnetic resonance (MR) to intraoperative ultrasound (iUS) has been proposed as a means to compensate for brain shift. We focus on the initial registration from MR to predurotomy iUS. We present a method that builds on previous work to address the need for accuracy and generality of MR-iUS registration algorithms in multi-site clinical data. High-dimensional texture attributes were used instead of image intensities for image registration and the standard difference-based attribute matching was replaced with correlation-based attribute matching. A strategy that deals explicitly with the large field-of-view mismatch between MR and iUS images was proposed. Key parameters were optimized across independent MR-iUS brain tumor datasets acquired at 3 institutions, with a total of 43 tumor patients and 758 reference landmarks for evaluating the accuracy of the proposed algorithm. Despite differences in imaging protocols, patient demographics and landmark distributions, the algorithm is able to reduce landmark errors prior to registration in three data sets (5.37±4.27, 4.18±1.97 and 6.18±3.38 mm, respectively) to a consistently low level (2.28±0.71, 2.08±0.37 and 2.24±0.78 mm, respectively). This algorithm was tested against 15 other algorithms and it is competitive with the state-of-the-art on multiple datasets. We show that the algorithm has one of the lowest errors in all datasets (accuracy), and this is achieved while sticking to a fixed set of parameters for multi-site data (generality). In contrast, other algorithms/tools of similar performance need per-dataset parameter tuning (high accuracy but lower generality), and those that stick to fixed parameters have larger errors or inconsistent performance (generality but not the top accuracy). Landmark errors were further characterized according to brain regions and tumor types, a topic so far missing in the literature., National Institute of Health (Grants P41-EB015898, P41-EB015902 and R01-NS049251)

In settings where high-level inferences are made based on registered image data, the registration uncertainty can contain important information. In this article, we propose a Bayesian non-rigid registration framework where conventional dissimilarity and regularization energies can be included in the likelihood and the prior distribution on deformations respectively through the use of Boltzmann's distribution. The posterior distribution is characterized using Markov Chain Monte Carlo (MCMC) methods with the effect of the Boltzmann temperature hyper-parameters marginalized under broad uninformative hyper-prior distributions. The MCMC chain permits estimation of the most likely deformation as well as the associated uncertainty. On synthetic examples, we demonstrate the ability of the method to identify the maximum a posteriori estimate and the associated posterior uncertainty, and demonstrate that the posterior distribution can be non-Gaussian. Additionally, results from registering clinical data acquired during neurosurgery for resection of brain tumor are provided; we compare the method to single transformation results from a deterministic optimizer and introduce methods that summarize the high-dimensional uncertainty. At the site of resection, the registration uncertainty increases and the marginal distribution on deformations is shown to be multi-modal.

The delineation of functionally and structurally distinct regions as well as their connectivity can provide key knowledge towards understanding the brain’s behaviour and function. Cytoarchitecture has long been the gold standard for such parcellation tasks, but has poor scalability and cannot be mapped in vivo. Functional and diffusion magnetic resonance imaging allow in vivo mapping of brain’s connectivity and the parcellation of the brain based on local connectivity information. Several methods have been developed for single subject connectivity driven parcellation, but very few have tackled the task of group-wise parcellation, which is essential for uncovering group specific behaviours. In this paper, we propose a group-wise connectivity-driven parcellation method based on spectral clustering that captures local connectivity information at multiple scales and directly enforces correspondences between subjects. The method is applied to diffusion Magnetic Resonance Imaging driven parcellation on two independent groups of 50 subjects from the Human Connectome Project. Promising quantitative and qualitative results in terms of information loss, modality comparisons, group consistency and inter-group similarities demonstrate the potential of the method.

© Springer International Publishing AG 2016. Using image-based descriptors to investigate clinical hypotheses and therapeutic implications is challenging due to the notorious “curse of dimensionality” coupled with a small sample size. In this paper,we present a low-dimensional analysis of anatomical shape variability in the space of diffeomorphisms and demonstrate its benefits for clinical studies. To combat the high dimensionality of the deformation descriptors,we develop a probabilistic model of principal geodesic analysis in a bandlimited low-dimensional space that still captures the underlying variability of image data. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than models based on the high-dimensional state-of-the-art approaches such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA)., NIH (NIBIB-NAC-P41EB015902, NINDS-R01NS086905, NICHD-U01HD087211)

Purpose: This study introduces a probabilistic nonrigid registration method for use in image-guided prostate brachytherapy. Intraoperative imaging for prostate procedures, usually transrectal ultrasound(TRUS), is typically inferior to diagnostic-quality imaging of the pelvis such as endorectal magnetic resonance imaging(MRI).MRimages contain superior detail of the prostate boundaries and provide substructure features not otherwise visible. Previous efforts to register diagnostic prostate images with the intraoperative coordinate system have been deterministic and did not offer a measure of the registration uncertainty. The authors developed a Bayesian registration method to estimate the posterior distribution on deformations and provide a case-specific measure of the associated registration uncertainty. Methods: The authors adapted a biomechanical-based probabilistic nonrigid method to register diagnostic to intraoperative images by aligning a physician's segmentations of the prostate in the two images. The posterior distribution was characterized with a Markov Chain Monte Carlo method; the maximuma posteriori deformation and the associated uncertainty were estimated from the collection of deformation samples drawn from the posterior distribution. The authors validated the registration method using a dataset created from ten patients with MRI-guided prostate biopsies who had both diagnostic and intraprocedural 3 Tesla MRI scans. The accuracy and precision of the estimated posterior distribution on deformations were evaluated from two predictive distance distributions: between the deformed central zone-peripheral zone (CZ-PZ) interface and the physician-labeled interface, and based on physician-defined landmarks. Geometric margins on the registration of the prostate's peripheral zone were determined from the posterior predictive distance to the CZ-PZ interface separately for the base, mid-gland, and apical regions of the prostate. Results: The authors observed variation in the shape and volume of the segmented prostate in diagnostic and intraprocedural images. The probabilistic method allowed us to convey registration results in terms of posterior distributions, with the dispersion providing a patient-specific estimate of the registration uncertainty. The median of the predictive distance distribution between the deformed prostate boundary and the segmented boundary was ⩽3 mm (95th percentiles within ±4 mm) for all ten patients. The accuracy and precision of the internal deformation was evaluated by comparing the posterior predictive distance distribution for the CZ-PZ interface for each patient, with the median distance ranging from −0.6 to 2.4 mm. Posterior predictive distances between naturally occurring landmarks showed registration errors of ⩽5 mm in any direction. The uncertainty was not a global measure, but instead was local and varied throughout the registration region. Registration uncertainties were largest in the apical region of the prostate. Conclusions: Using a Bayesian nonrigid registration method, the authors determined the posterior distribution on deformations between diagnostic and intraprocedural MRimages and quantified the uncertainty in the registration results. The feasibility of this approach was tested and results were positive. The probabilistic framework allows us to evaluate both patient-specific and location-specific estimates of the uncertainty in the registration result. Although the framework was tested on MR-guided procedures, the preliminary results suggest that it may be applied to TRUS-guided procedures as well, where the addition of diagnostic MR information may have a larger impact on target definition and clinical guidance.

Traditional models of the human language circuitry encompass three cortical areas, Broca's, Geschwind's and Wernicke's, and their connectivity through white matter fascicles. The neural connectivity deep to these cortical areas remains poorly understood, as does the macroscopic functional organization of the cortico-subcortical language circuitry. In an effort to expand current knowledge, we combined functional MRI (fMRI) and diffusion tensor imaging to explore subject-specific structural and functional macroscopic connectivity, focusing on Broca's area. Fascicles were studied using diffusion tensor imaging fiber tracking seeded from volumes placed manually within the white matter. White matter fascicles and fMRI-derived clusters (antonym-generation task) of positive and negative blood-oxygen-level-dependent (BOLD) signal were co-registered with 3-D renderings of the brain in 12 healthy subjects. Fascicles connecting BOLD-derived clusters were analyzed within specific cortical areas: Broca's, with the pars triangularis, the pars opercularis, and the pars orbitaris; Geschwind's and Wernicke's; the premotor cortex, the dorsal supplementary motor area, the middle temporal gyrus, the dorsal prefrontal cortex and the frontopolar region. We found a functional connectome divisible into three systems-anterior, superior and inferior-around the insula, more complex than previously thought, particularly with respect to a new extended Broca's area. The extended Broca's area involves two new fascicles: the operculo-premotor fascicle comprised of well-organized U-shaped fibers that connect the pars opercularis with the premotor region; and (2) the triangulo-orbitaris system comprised of intermingled U-shaped fibers that connect the pars triangularis with the pars orbitaris. The findings enhance our understanding of language function.

We describe the validation of an anatomical brain atlas approach to the analysis of diffuse optical tomography (DOT). Using MRI data from 32 subjects, we compare the diffuse optical images of simulated cortical activation reconstructed using a registered atlas with those obtained using a subject’s true anatomy. The error in localization of the simulated cortical activations when using a registered atlas is due to a combination of imperfect registration, anatomical differences between atlas and subject anatomies and the localization error associated with diffuse optical image reconstruction. When using a subject-specific MRI, any localization error is due to diffuse optical image reconstruction only. In this study we determine that using a registered anatomical brain atlas results in an average localization error of approximately 18 mm in Euclidean space. The corresponding error when the subject’s own MRI is employed is 9.1 mm. In general, the cost of using atlas-guided DOT in place of subject-specific MRI-guided DOT is a doubling of the localization error. Our results show that despite this increase in error, reasonable anatomical localization is achievable even in cases where the subject-specific anatomy is unavailable.

Purpose: To develop and evaluate image registration methodology for automated re-identification of tumor-suspicious foci from preprocedural MR exams during MR-guided transperineal prostate core biopsy. Materials and Methods: A hierarchical approach for automated registration between planning and intra-procedural T2-weighted prostate MRI was developed and evaluated on the images acquired during 10 consecutive MR-guided biopsies. Registration accuracy was quantified at image-based landmarks and by evaluating spatial overlap for the manually segmented prostate and sub-structures. Registration reliability was evaluated by simulating initial mis-registration and analyzing the convergence behavior. Registration precision was characterized at the planned biopsy targets. Results: The total computation time was compatible with a clinical setting, being at most 2 min. Deformable registration led to a significant improvement in spatial overlap of the prostate and peripheral zone contours compared with both rigid and affine registration. Average in-slice landmark registration error was 1.3 ± 0.5 mm. Experiments simulating initial mis-registration resulted in an estimated average capture range of 6 mm and an average in-slice registration precision of ±0.3 mm. Conclusion: Our registration approach requires minimum user interaction and is compatible with the time constraints of our interventional clinical workflow. The initial evaluation shows acceptable accuracy, reliability and consistency of the method. J. Magn. Reson. Imaging 2012;36:987–992. © 2012 Wiley Periodicals, Inc.

We recently developed a functional neuroimaging technique called encephalographic magnetic resonance imaging (eMRI). Our method acquires rapid single-shot gradient-echo echo-planar MRI (repetition time=47 ms); it attempts to measure an MR signal more directly linked to neuronal electromagnetic activity than existing methods. To increase the likelihood of detecting such an MR signal, we recorded concurrent MRI and scalp electroencephalography (EEG) during fast (20-200 ms), localized, high-amplitude (50 μV on EEG) cortical discharges in a cohort of focal epilepsy patients. Seen on EEG as interictal spikes, these discharges occur in between seizures and induced easily detectable MR magnitude and phase changes concurrent with the spikes with a lag of milliseconds to tens of milliseconds. Due to the time scale of the responses, localized changes in blood flow or hemoglobin oxygenation are unlikely to cause the MR signal changes that we observed. While the precise underlying mechanisms are unclear, in this study, we empirically investigate one potentially important confounding variable - motion. Head motion in the scanner affects both EEG and MR recording. It can produce brief "spike-like" artifacts on EEG and induce large MR signal changes similar to our interictal spike-related signal changes. In order to explore the possibility that interictal spikes were associated with head motions (although such an association had never been reported), we had previously tracked head position in epilepsy patients during interictal spikes and explicitly demonstrated a lack of associated head motion. However, that study was performed outside the MR scanner, and the root-mean-square error in the head position measurement was 0.7 mm. The large inaccuracy in this measurement therefore did not definitively rule out motion as a possible signal generator. In this study, we instructed healthy subjects to make deliberate brief (500 ms) head motions inside the MR scanner and imaged these head motions with concurrent EEG and MRI. We compared these artifactual MR and EEG data to genuine interictal spikes. While per-voxel MR and per-electrode EEG time courses for the motion case can mimic the corresponding time courses associated with a genuine interictal spike, head motion can be unambiguously differentiated from interictal spikes via scalp EEG potential maps. Motion induces widespread changes in scalp potential, whereas interictal spikes are localized and have a regional fall-off in amplitude. These findings make bulk head motion an unlikely generator of the large spike-related MR signal changes that we had observed. Further work is required to precisely identify the underlying mechanisms.

Image registration is the process of transforming images acquired at different time points, or with different imaging modalities, into the same coordinate system. It is an essential part of any neurosurgical planning and navigation system because it facilitates combining images with important complementary structural and functional information to improve the information basis on which a surgeon makes critical decisions. Magnetic Resonance Imaging (MRI) can be used to generate both structural and functional neurological images and thus plays an important role in planning neurosurgical procedures (Figure 1). High-resolution anatomical MR images can be used to discriminate between healthy and pathological tissue, functional MRI (fMRI) can be used to identify the location, and extent, of important cognitive areas of the brain while Diffusion Tensor Imaging (DTI) can be used to identify the white matter connectivity of the brain. If important cognitive areas are injured or removed during surgery, it can have an adverse effect on the patient's quality of life post surgery. Figure 1 Anatomical and functional MR information acquired pre-operatively48. (A) An anatomical T1 scan of a patient with a lesion in the left frontal region. (B) Extracted white matter tracts using tractography from DTI. (C) Speech areas found with fMRI overlaid ... It is especially important to put these complementary images into the same coordinate system when functional areas are located adjacent to a tumor. A map of important anatomical structures and surrounding functional areas can then be built and used by the surgeon to plan the surgical procedure1. Because the brain is enclosed by the skull, a rigid transformation which consists of a translation and rotation is adequate to align these images. The importance of fMRI in planning of tumor resections is underlined by the findings of Petrella et al.2 where the effect of therapeutic decision making in patients with brain tumors was studied. Treatment plans before and after seeing the fMRI differed in 19 out of 49 patients, with a more aggressive approach recommended after imaging in 18 patients. The availability of fMRI resulted in reduced surgical time in 22 patients who underwent surgery, a more aggressive resection in 6 patients and a smaller resection in 2 patients. Other modalities than MRI have been and are also currently being used in neurosurgical navigation systems. Positron Emission Tomography (PET) allows analyzing function and metabolism of the brain. It is mainly used to localize the most malignant parts of brain tumors, but has also been used to locate eloquent areas of the cortex. However, it does not provide useful structural information and is therefore combined with high resolution structural images, e.g. MRI and Computed Tomography (CT), to accurately locate the functional areas in relation to anatomical structures3. For planning of neurosurgical procedures, PET images are usually combined with MRI because they provide the best tissue contrast. fMRI has now superseded PET as a functional imaging modality for locating eloquent cortex because it eliminates any radiation exposure. Registration also plays an important role in other neurosurgical guidance applications besides tumor removal, for instance in resection of suspected epileptogenic centers. The epileptogenic centers are located using information from subdural electrodes and the location of these electrodes extracted from CT. By registering the CT with structural MRI, the centers can be precisely located within the brain anatomy and used to plan the resection4. Because of brain-shift, tissue resection and retraction, the pre-operative image information does not necessarily reflect the intra-operative anatomy the surgeon sees during surgery, even after rigid registration. Brain-shift, a deformation of the brain tissue that occurs after the craniotomy and opening of the dura mater, is caused by various combined factors: cerebrospinal fluid (CSF) leakage, gravity, edema, tumor mass effect and administration of osmotic diuretics5,6. A clinical example of brain-shift is shown in Figure 2. Reported deformations of the brain tissue caused by brain-shift are up to 24mm7–9, and further deformations may be induced because of tissue manipulation during the resection5,10. Figure 2 Pre- and intra-operative MR images that shows brain-shift and resection. (A) Pre- operative image. (B) Intra-operative image. Notice the cavity which is a result of the tumor resection. (C) Absolute difference image of the images in (A) and (B). Notice ... As a way of acquiring up to date anatomical information during surgery, intra-operative MRI has been introduced into the surgical workflow11,12. Intra-operative imaging during neuro-surgical procedures helps identify any residual tumor tissue and leads to a significant increase in the extent of tumor removal and survival times13. However, intra-operative images are often of a degraded quality compared to the pre-operative images. Many factors lead to reduced quality in intra-operative MRI: fast acquisition protocols which are used to reduce the surgical time, intra-operative MRI scanners often operate on lower magnetic field strengths than diagnostic MRI scanners, surface coils are used instead of head coils, and disturbances to the main magnetic field caused by the surgery. Ultrasound (US) is another imaging modality used to acquire up-to-date images during neurosurgery 14. Compared to MRI, acquisition is faster and does not require the patient to move, but image quality and soft tissue discrimination is reduced. US can identify functional information such as blood flow, but not cognitive functions such as motor, visual and language cortices. Reliable and accurate acquisition of intra-operative fMRI is in general not feasible, either because of factors such as time constraint, the requirement that the patient be conscious during acquisition of fMRI, and disturbances of the magnetic field from the surgical procedure1. How then can we accurately and robustly provide the surgeon with the location of these important eloquent areas in relation to the intra-operative anatomy? Non-rigid transformations have a high degree of freedom and are capable of accommodating the local brain deformations that occur during surgery. Non-rigid intra-operative registration has therefore been introduced as a way of mapping the pre-operative functional information into the intra-operative space. In practice, a transformation cascade is constructed to facilitate intra-operative navigation of the pre-operative image data (see Figure 3). First, the functional images, typically fMRI and DTI, are rigidly registered to a high-resolution pre-operative anatomical image. Next, the pre-operative anatomical image is non-rigidly registered with the intra-operative image. Finally, to facilitate navigation of these images, the intra-operative image is registered with the physical patient space by estimating a rigid transformation from corresponding (fiducial) landmarks identified in the image and on the patient. Most commercial neuro-surgical navigation systems register the pre-operative images to the intra-operative space using fiducial markers and rigid registration and do not take into account the non-rigid deformations caused by brain-shift15. Figure 3 Transformation cascade. Three steps are needed to map the functional images, fMRI and DTI, into the intra-operative physical space. First, rigid, or affine, transforms, RfMRI and RDTI are estimated to align the functional images with the pre-operative ... Brigham and Women’s Hospital (BWH), an affiliate of Harvard Medical School, has been one of the pioneers in developing intra-operative registration methods for aligning pre-and intra-operative images of the brain12. In this paper we present a general overview of intra-operative registration and highlight some of the recent developments at BWH. The rest of the paper is organized as follows. First we introduce patient-to-image registration followed by a discussion of the error involved in such estimations. Next, we provide an introduction of image-to-image registration and the specific challenges of intra-operative registration and, finally, present an overview of the research that has been carried out to handle these challenges.

Neuronal currents produce local electromagnetic fields that can potentially modulate the phase of the magnetic resonance signal and thus provide a contrast mechanism tightly linked to neuronal activity. Previous work has demonstrated the feasibility of direct MRI of neuronal activity in phantoms and cell culture, but in vivo efforts have yielded inconclusive, conflicting results. The likelihood of detecting and validating such signals can be increased with (i) fast gradient-echo echo-planar imaging, with acquisition rates sufficient to resolve neuronal activity, (ii) subjects with epilepsy, who frequently experience stereotypical electromagnetic discharges between seizures, expressed as brief, localized, high-amplitude spikes (interictal discharges), and (iii) concurrent electroencephalography. This work demonstrates that both MR magnitude and phase show large-amplitude changes concurrent with electroencephalography spikes. We found a temporal derivative relationship between MR phase and scalp electroencephalography, suggesting that the MR phase changes may be tightly linked to local cerebral activity. We refer to this manner of MR acquisition, designed explicitly to track the electroencephalography, as encephalographic MRI (eMRI). Potential extension of this technique into a general purpose functional neuroimaging tool requires further study of the MR signal changes accompanying lower amplitude neuronal activity than those discussed here.

This paper presents feature-based morphometry (FBM), a new, fully data-driven technique for discovering patterns of group-related anatomical structure in volumetric imagery. In contrast to most morphometry methods which assume one-to-one correspondence between subjects, FBM explicitly aims to identify distinctive anatomical patterns that may only be present in subsets of subjects, due to disease or anatomical variability. The image is modeled as a collage of generic, localized image features that need not be present in all subjects. Scale-space theory is applied to analyze image features at the characteristic scale of underlying anatomical structures, instead of at arbitrary scales such as global or voxel-level. A probabilistic model describes features in terms of their appearance, geometry, and relationship to subject groups, and is automatically learned from a set of subject images and group labels. Features resulting from learning correspond to group-related anatomical structures that can potentially be used as image biomarkers of disease or as a basis for computer-aided diagnosis. The relationship between features and groups is quantified by the likelihood of feature occurrence within a specific group vs. the rest of the population, and feature significance is quantified in terms of the false discovery rate. Experiments validate FBM clinically in the analysis of normal (NC) and Alzheimer's (AD) brain images using the freely available OASIS database. FBM automatically identifies known structural differences between NC and AD subjects in a fully data-driven fashion, and an equal error classification rate of 0.80 is achieved for subjects aged 60-80 years exhibiting mild AD (CDR=1).

To investigate the association of behavioral and psychological symptoms of dementia (BPSD) in Alzheimer's disease (AD) and magnetic resonance imaging (MRI) measures of brain atrophy and white matter hyperintensities (WMH).Thirty-seven patients with probable AD received the Neuropsychiatric Inventory (NPI), the Mini Mental Status Exam (MMSE), and an MRI scan as part of their initial evaluation at the Outpatient Memory Diagnostic Clinic at McLean Hospital. MRI-based volumetric measurements of whole brain atrophy, hippocampal volumes, and WMH were obtained. Analysis of covariance models, using age as a covariate and the presence of specific BPSD as independent variables, were used to test for differences in whole brain volumes, hippocampal volumes and WMH volumes.Increased WMH were associated with symptoms of anxiety, aberrant motor behavior, and night time disturbance, while symptoms of disinhibition were linked to lower WMH volume. No associations were found for whole brain or hippocampal volumes and BPSD.These findings suggest that white matter changes are associated with the presence of BPSD in AD.

In the present report, estimates of test–retest and between‐site reliability of fMRI assessments were produced in the context of a multicenter fMRI reliability study (FBIRN Phase 1, http://www.nbirn.net). Five subjects were scanned on 10 MRI scanners on two occasions. The fMRI task was a simple block design sensorimotor task. The impulse response functions to the stimulation block were derived using an FIR‐deconvolution analysis with FMRISTAT. Six functionally‐derived ROIs covering the visual, auditory and motor cortices, created from a prior analysis, were used. Two dependent variables were compared: percent signal change and contrast‐to‐noise‐ratio. Reliability was assessed with intraclass correlation coefficients derived from a variance components analysis. Test–retest reliability was high, but initially, between‐site reliability was low, indicating a strong contribution from site and site‐by‐subject variance. However, a number of factors that can markedly improve between‐site reliability were uncovered, including increasing the size of the ROIs, adjusting for smoothness differences, and inclusion of additional runs. By employing multiple steps, between‐site reliability for 3T scanners was increased by 123%. Dropping one site at a time and assessing reliability can be a useful method of assessing the sensitivity of the results to particular sites. These findings should provide guidance toothers on the best practices for future multicenter studies. Hum Brain Mapp, 2008. © 2007 Wiley‐Liss, Inc.

We propose two software tools for non-rigid registration of MRI and transrectal ultrasound (TRUS) images of the prostate. Our ultimate goal is to develop an open-source solution to support MRI-TRUS fusion image guidance of prostate interventions, such as targeted biopsy for prostate cancer detection and focal therapy. It is widely hypothesized that image registration is an essential component in such systems.The two non-rigid registration methods are: (1) a deformable registration of the prostate segmentation distance maps with B-spline regularization and (2) a finite element-based deformable registration of the segmentation surfaces in the presence of partial data. We evaluate the methods retrospectively using clinical patient image data collected during standard clinical procedures. Computation time and Target Registration Error (TRE) calculated at the expert-identified anatomical landmarks were used as quantitative measures for the evaluation.The presented image registration tools were capable of completing deformable registration computation within 5 min. Average TRE was approximately 3 mm for both methods, which is comparable with the slice thickness in our MRI data. Both tools are available under nonrestrictive open-source license.We release open-source tools that may be used for registration during MRI-TRUS-guided prostate interventions. Our tools implement novel registration approaches and produce acceptable registration results. We believe these tools will lower the barriers in development and deployment of interventional research solutions and facilitate comparison with similar tools.

Functional Magnetic Resonance Imaging (fMRI) studies are derived from a time series of Echo-Planar images (EPIs). Compared to conventional Magnetic Resonance Images (MRIs), EPIs are of relatively poor quality for discerning anatomic features and are often registered with corresponding MRIs to map brain activity to neuroanatomy. In this paper we demonstrate the utility of a technique to register an EPI-MRI pair by minimizing the discrepancy between its joint intensity probability mass function (PMF) and a previously learned one for a properly registered EPI-MRI pair, using the Kullback-Leibler Distance (KLD). In probing experiments Joint Entropy (JE) and Mutual Information showed significant bias relative to KLD along the axial direction and JE along a rotation axis. A comparison of searches using random starting poses showed KLD to have lower final pose errors than JE. Results of variation on parameters of the KLD based EPI-MRI registration technique are also presented.

Magnetic Resonance (MR) imaging provides excellent image quality at a high cost and low frame rate. Ultrasound (US) provides poor image quality at a low cost and high frame rate. We propose an instance-based learning system to obtain the best of both worlds: high quality MR images at high frame rates from a low cost single-element US sensor. Concurrent US and MRI pairs are acquired during a relatively brief offine learning phase involving the US transducer and MR scanner. High frame rate, high quality MR imaging of respiratory organ motion is then predicted from US measurements, even after stopping MRI acquisition, using a probabilistic kernel regression framework. Experimental results show predicted MR images to be highly representative of actual MR images.

In this article, we report efforts to integrate a number of state-of-the-art technologies for MRI-guided neurosurgery at the Brigham and Women's Hospital (BWH) in Boston. These include advanced intraoperative imaging, image registration, visualization, navigation, minimally invasive ablative therapies, and robotics. This is part of a multidisciplinary Image-Guided Therapy Program that comprises several key research thrusts, including the surgical planning laboratory, magnetic resonance therapy (MRT), focused ultrasound surgery, thermal ablation, and neurosurgery

A statistical model is presented that combines the registration of an atlas with the segmentation of magnetic resonance images. We use an Expectation Maximization-based algorithm to find a solution within the model, which simultaneously estimates image artifacts, anatomical labelmaps, and a structure-dependent hierarchical mapping from the atlas to the image space. The algorithm produces segmentations for brain tissues as well as their substructures. We demonstrate the approach on a set of 22 magnetic resonance images. On this set of images, the new approach performs significantly better than similar methods which sequentially apply registration and segmentation.

To prospectively investigate the factors--including subject, brain hemisphere, study site, field strength, imaging unit vendor, imaging run, and examination visit--affecting the reproducibility of functional magnetic resonance (MR) imaging activations based on a repeated sensory-motor (SM) task.The institutional review boards of all participating sites approved this HIPAA-compliant study. All subjects gave informed consent. Functional MR imaging data were repeatedly acquired from five healthy men aged 20-29 years who performed the same SM task at 10 sites. Five 1.5-T MR imaging units, four 3.0-T units, and one 4.0-T unit were used. The subjects performed bilateral finger tapping on button boxes with a 3-Hz audio cue and a reversing checkerboard. In a block design, 15-second epochs of alternating baseline and tasks yielded 85 acquisitions per run. Functional MR images were acquired with block-design echo-planar or spiral gradient-echo sequences. Brain activation maps standardized in a unit-sphere for the left and right hemispheres of each subject were constructed. Areas under the receiver operating characteristic curve, intraclass correlation coefficients, multiple regression analysis, and paired Student t tests were used for statistical analyses.Significant factors were subject (P.005), k-space (P.005), and field strength (P = .02) for sensitivity and subject (P = .03) and k-space (P = .05) for specificity. At 1.5-T MR imaging, mean sensitivities ranged from 7% to 32% and mean specificities were higher than 99%. At 3.0 T, mean sensitivities and specificities ranged from 42% to 85% and from 96% to 99%, respectively. At 4.0 T, mean sensitivities and specificities ranged from 41% to 73% and from 95% to 99%, respectively. Mean areas under the receiver operating characteristic curve (+/- their standard errors) were 0.77 +/- 0.05 at 1.5 T, 0.90 +/- 0.09 at 3.0 T, and 0.95 +/- 0.02 at 4.0 T, with significant differences between the 1.5- and 3.0-T examinations and between the 1.5- and 4.0-T examinations (P.01 for both comparisons). Intraclass correlation coefficients ranged from 0.49 to 0.71.MR imaging at 3.0- and 4.0-T yielded higher reproducibility across sites and significantly better results than 1.5-T imaging. The effects of subject, k-space, and field strength on examination reproducibility were significant.

Characterizing the performance of image segmentation approaches has been a persistent challenge. Performance analysis is important since segmentation algorithms often have limited accuracy and precision. Interactive drawing of the desired segmentation by human raters has often been the only acceptable approach, and yet suffers from intra-rater and inter-rater variability. Automated algorithms have been sought in order to remove the variability introduced by raters, but such algorithms must be assessed to ensure they are suitable for the task. The performance of raters (human or algorithmic) generating segmentations of medical images has been difficult to quantify because of the difficulty of obtaining or estimating a known true segmentation for clinical data. Although physical and digital phantoms can be constructed for which ground truth is known or readily estimated, such phantoms do not fully reflect clinical images due to the difficulty of constructing phantoms which reproduce the full range of imaging characteristics and normal and pathological anatomical variability observed in clinical data. Comparison to a collection of segmentations by raters is an attractive alternative since it can be carried out directly on the relevant clinical imaging data. However, the most appropriate measure or set of measures with which to compare such segmentations has not been clarified and several measures are used in practice. We present here an expectation-maximization algorithm for simultaneous truth and performance level estimation (STAPLE). The algorithm considers a collection of segmentations and computes a probabilistic estimate of the true segmentation and a measure of the performance level represented by each segmentation. The source of each segmentation in the collection may be an appropriately trained human rater or raters, or may be an automated segmentation algorithm. The probabilistic estimate of the true segmentation is formed by estimating an optimal combination of the segmentations, weighting each segmentation depending upon the estimated performance level, and incorporating a prior model for the spatial distribution of structures being segmented as well as spatial homogeneity constraints. STAPLE is straightforward to apply to clinical imaging data, it readily enables assessment of the performance of an automated image segmentation algorithm, and enables direct comparison of human rater and algorithm performance.

The validity of brain tumour segmentation is an important issue in image processing because it has a direct impact on surgical planning. We examined the segmentation accuracy based on three two-sample validation metrics against the estimated composite latent gold standard, which was derived from several experts’ manual segmentations by an EM algorithm. The distribution functions of the tumour and control pixel data were parametrically assumed to be a mixture of two beta distributions with different shape parameters. We estimated the corresponding receiver operating characteristic curve, Dice similarity coefficient, and mutual information, over all possible decision thresholds. Based on each validation metric, an optimal threshold was then computed via maximization. We illustrated these methods on MR imaging data from nine brain tumour cases of three different tumour types, each consisting of a large number of pixels. The automated segmentation yielded satisfactory accuracy with varied optimal thresholds. The performances of these validation metrics were also investigated via Monte Carlo simulation. Extensions of incorporating spatial correlation structures using a Markov random field model were considered.

Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semi-automated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by bi-normal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnetic resonance imaging of brain tumors: meningiomas (0.924–0.984); astrocytomas (0.786–0.986); and other low-grade gliomas (0.896–0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific antigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

A multichannel statistical classifier for detecting prostate cancer was developed and validated by combining information from three different magnetic resonance (MR) methodologies: T2-weighted, T2-mapping, and line scan diffusion imaging (LSDI). From these MR sequences, four different sets of image intensities were obtained: T2-weighted (T2W) from T2-weighted imaging, Apparent Diffusion Coefficient (ADC) from LSDI, and proton density (PD) and T2 (T2 Map) from T2-mapping imaging. Manually segmented tumor labels from a radiologist, which were validated by biopsy results, served as tumor "ground truth." Textural features were extracted from the images using co-occurrence matrix (CM) and discrete cosine transform (DCT). Anatomical location of voxels was described by a cylindrical coordinate system. A statistical jack-knife approach was used to evaluate our classifiers. Single-channel maximum likelihood (ML) classifiers were based on 1 of the 4 basic image intensities. Our multichannel classifiers: support vector machine (SVM) and Fisher linear discriminant (FLD), utilized five different sets of derived features. Each classifier generated a summary statistical map that indicated tumor likelihood in the peripheral zone (PZ) of the prostate gland. To assess classifier accuracy, the average areas under the receiver operator characteristic (ROC) curves over all subjects were compared. Our best FLD classifier achieved an average ROC area of 0.839(+/-0.064), and our best SVM classifier achieved an average ROC area of 0.761(+/-0.043). The T2W ML classifier, our best single-channel classifier, only achieved an average ROC area of 0.599(+/-0.146). Compared to the best single-channel ML classifier, our best multichannel FLD and SVM classifiers have statistically superior ROC performance (P=0.0003 and 0.0017, respectively) from pairwise two-sided t-test. By integrating the information from multiple images and capturing the textural and anatomical features in tumor areas, summary statistical maps can potentially aid in image-guided prostate biopsy and assist in guiding and controlling delivery of localized therapy under image guidance.

Magnetic Resonance Imaging (MRI) and transrectal Ultrasound (TRUS) are both used in imaging interventions in men suspected of having and with prostate cancer for diagnosis as well as treatment. Due to the widespread availability and ease of use of TRUS, it is widely acknowledged that availability of spatially registered MRI/TRUS data could provide the optimal combination for characterization of prostate tissue and interventional guidance. To provide such spatially aligned data, we propose a device to support co-registered acquisition of MRI and TRUS data while maintaining a stable configuration (shape) of the prostate. We present the design and evaluation of a custom sleeve that can be introduced transrectally, and can accommodate both TRUS and endorectal MRI probes. Our experiments on a phantom have demonstrated that imaging with this sleeve did not compromise differentiation of internal structures and did not affect the quality of the MR acquisition. Reduction of the signal and contrast were however observed and quantified in the TRUS data. Further evaluation and modification of the device necessary for possible patient studies are discussed.

Intensity-based image registration requires resampling images on a common grid to evaluate the similarity function. The uncertainty of interpolation varies across the image, depending on the location of resampled points relative to the base grid. We propose to perform Bayesian inference with Gaussian processes, where the covariance matrix of the Gaussian process posterior distribution estimates the uncertainty in interpolation. The Gaussian process replaces a single image with a distribution over images that we integrate into a generative model for registration. Marginalization over resampled images leads to a new similarity measure that includes the uncertainty of the interpolation. We demonstrate that our approach increases the registration accuracy and propose an efficient approximation scheme that enables seamless integration with existing registration methods.