Search

Intraoperative ultrasound (iUS) imaging has the potential to improve surgical outcomes in brain surgery. However, its interpretation is challenging, even for expert neurosurgeons. In this work, we designed the first patient-specific framework that performs brain tumor segmentation in trackerless iUS. To disambiguate ultrasound imaging and adapt to the neurosurgeon's surgical objective, a patient-specific real-time network is trained using synthetic ultrasound data generated by simulating virtual iUS sweep acquisitions in pre-operative MR data. Extensive experiments performed in real ultrasound data demonstrate the effectiveness of the proposed approach, allowing for adapting to the surgeon's definition of surgical targets and outperforming non-patient-specific models, neurosurgeon experts, and high-end tracking systems. Our code is available at: \url{https://github.com/ReubenDo/MHVAE-Seg}., Comment: Early accept at MICCAI 2024 - code available at: https://github.com/ReubenDo/MHVAE-Seg

Self-supervised representation learning on image-text data facilitates crucial medical applications, such as image classification, visual grounding, and cross-modal retrieval. One common approach involves contrasting semantically similar (positive) and dissimilar (negative) pairs of data points. Drawing negative samples uniformly from the training data set introduces false negatives, i.e., samples that are treated as dissimilar but belong to the same class. In healthcare data, the underlying class distribution is nonuniform, implying that false negatives occur at a highly variable rate. To improve the quality of learned representations, we develop a novel approach that corrects for false negatives. Our method can be viewed as a variant of debiased contrastive learning that uses estimated sample-specific class probabilities. We provide theoretical analysis of the objective function and demonstrate the proposed approach on both image and paired image-text data sets. Our experiments illustrate empirical advantages of sample-specific debiasing., Comment: Machine Learning for Healthcare Conference 2023

AbstractMass spectrometry imaging (MSI) is an emerging technology that holds potential for improving, biomarker discovery, metabolomics research, pharmaceutical applications and clinical diagnosis. Despite many solutions being developed, the large data size and high dimensional nature of MSI, especially 3D datasets, still pose computational and memory complexities that hinder accurate identification of biologically relevant molecular patterns. Moreover, the subjectivity in the selection of parameters for conventional pre-processing approaches can lead to bias. Therefore, we assess if a probabilistic generative model based on a fully connected variational autoencoder can be used for unsupervised analysis and peak learning of MSI data to uncover hidden structures. The resulting msiPL method learns and visualizes the underlying non-linear spectral manifold, revealing biologically relevant clusters of tissue anatomy in a mouse kidney and tumor heterogeneity in human prostatectomy tissue, colorectal carcinoma, and glioblastoma mouse model, with identification of underlying m/z peaks. The method is applied for the analysis of MSI datasets ranging from 3.3 to 78.9 GB, without prior pre-processing and peak picking, and acquired using different mass spectrometers at different centers.

We introduce MHVAE, a deep hierarchical variational auto-encoder (VAE) that synthesizes missing images from various modalities. Extending multi-modal VAEs with a hierarchical latent structure, we introduce a probabilistic formulation for fusing multi-modal images in a common latent representation while having the flexibility to handle incomplete image sets as input. Moreover, adversarial learning is employed to generate sharper images. Extensive experiments are performed on the challenging problem of joint intra-operative ultrasound (iUS) and Magnetic Resonance (MR) synthesis. Our model outperformed multi-modal VAEs, conditional GANs, and the current state-of-the-art unified method (ResViT) for synthesizing missing images, demonstrating the advantage of using a hierarchical latent representation and a principled probabilistic fusion operation. Our code is publicly available \url{https://github.com/ReubenDo/MHVAE}., Comment: Accepted at MICCAI 2023

Image-text multimodal representation learning aligns data across modalities and enables important medical applications, e.g., image classification, visual grounding, and cross-modal retrieval. In this work, we establish a connection between multimodal representation learning and multiple instance learning. Based on this connection, we propose a generic framework for constructing permutation-invariant score functions with many existing multimodal representation learning approaches as special cases. Furthermore, we use the framework to derive a novel contrastive learning approach and demonstrate that our method achieves state-of-the-art results in several downstream tasks.

Neuroimaging-based prediction of neurocognitive measures is valuable for studying how the brain's structure relates to cognitive function. However, the accuracy of prediction using popular linear regression models is relatively low. We propose a novel deep regression method, namely TractoSCR, that allows full supervision for contrastive learning in regression tasks using diffusion MRI tractography. TractoSCR performs supervised contrastive learning by using the absolute difference between continuous regression labels (i.e. neurocognitive scores) to determine positive and negative pairs. We apply TractoSCR to analyze a large-scale dataset including multi-site harmonized diffusion MRI and neurocognitive data from 8735 participants in the Adolescent Brain Cognitive Development (ABCD) Study. We extract white matter microstructural measures using a fine parcellation of white matter tractography into fiber clusters. Using these measures, we predict three scores related to domains of higher-order cognition (general cognitive ability, executive function, and learning/memory). To identify important fiber clusters for prediction of these neurocognitive scores, we propose a permutation feature importance method for high-dimensional data. We find that TractoSCR improves the accuracy of neurocognitive score prediction compared to other state-of-the-art methods. We find that the most predictive fiber clusters are predominantly located within the superficial white matter and projection tracts, particularly the superficial frontal white matter and striato-frontal connections. Overall, our results demonstrate the utility of contrastive representation learning methods for regression, and in particular for improving neuroimaging-based prediction of higher-order cognitive abilities., Comment: 28 pages, 4 figures

In recent years, learning-based image registration methods have gradually moved away from direct supervision with target warps to instead use self-supervision, with excellent results in several registration benchmarks. These approaches utilize a loss function that penalizes the intensity differences between the fixed and moving images, along with a suitable regularizer on the deformation. In this paper, we argue that the relative failure of supervised registration approaches can in part be blamed on the use of regular U-Nets, which are jointly tasked with feature extraction, feature matching, and estimation of deformation. We introduce one simple but crucial modification to the U-Net that disentangles feature extraction and matching from deformation prediction, allowing the U-Net to warp the features, across levels, as the deformation field is evolved. With this modification, direct supervision using target warps begins to outperform self-supervision approaches that require segmentations, presenting new directions for registration when images do not have segmentations. We hope that our findings in this preliminary workshop paper will re-ignite research interest in supervised image registration techniques. Our code is publicly available from https://github.com/balbasty/superwarp.

© Springer International Publishing AG 2016. Using image-based descriptors to investigate clinical hypotheses and therapeutic implications is challenging due to the notorious “curse of dimensionality” coupled with a small sample size. In this paper,we present a low-dimensional analysis of anatomical shape variability in the space of diffeomorphisms and demonstrate its benefits for clinical studies. To combat the high dimensionality of the deformation descriptors,we develop a probabilistic model of principal geodesic analysis in a bandlimited low-dimensional space that still captures the underlying variability of image data. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than models based on the high-dimensional state-of-the-art approaches such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA).

© 1982-2012 IEEE. We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer the label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: 1) keypoint matching; 2) voting-based keypoint labeling; and 3) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison with common multi-atlas segmentation while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with a highly variable field-of-view.

Purpose Brain shift during tumor resection can progressively invalidate the accuracy of neuronavigation systems and affect neurosurgeons’ ability to achieve optimal resections. This paper compares two methods that have been presented in the literature to compensate for brain shift: a thin-plate spline deformation model and a finite element method (FEM). For this comparison, both methods are driven by identical sparse data. Specifically, both methods are driven by displacements between automatically detected and matched feature points from intraoperative 3D ultrasound (iUS). Both methods have been shown to be fast enough for intraoperative brain shift correction (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018; Luo et al. in J Med Imaging (Bellingham) 4(3):035003, 2017). However, the spline method requires no preprocessing and ignores physical properties of the brain while the FEM method requires significant preprocessing and incorporates patient-specific physical and geometric constraints. The goal of this work was to explore the relative merits of these methods on recent clinical data. Methods Data acquired during 19 sequential tumor resections in Brigham and Women’s Hospital’s Advanced Multi-modal Image-Guided Operating Suite between December 2017 and October 2018 were considered for this retrospective study. Of these, 15 cases and a total of 24 iUS to iUS image pairs met inclusion requirements. Automatic feature detection (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018) was used to detect and match features in each pair of iUS images. Displacements between matched features were then used to drive both the spline model and the FEM method to compensate for brain shift between image acquisitions. The accuracies of the resultant deformation models were measured by comparing the displacements of manually identified landmarks before and after deformation. Results The mean initial subcortical registration error

© 2017 Elsevier B.V. We present an efficient probabilistic model of anatomical variability in a linear space of initial velocities of diffeomorphic transformations and demonstrate its benefits in clinical studies of brain anatomy. To overcome the computational challenges of the high dimensional deformation-based descriptors, we develop a latent variable model for principal geodesic analysis (PGA) based on a low dimensional shape descriptor that effectively captures the intrinsic variability in a population. We define a novel shape prior that explicitly represents principal modes as a multivariate complex Gaussian distribution on the initial velocities in a bandlimited space. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than the state-of-the-art method such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA) that operate in the high dimensional image space.

© Springer Nature Switzerland AG 2018. This paper presents a novel approach to modeling the posterior distribution in image registration that is computationally efficient for large deformation diffeomorphic metric mapping (LDDMM). We develop a Laplace approximation of Bayesian registration models entirely in a bandlimited space that fully describes the properties of diffeomorphic transformations. In contrast to current methods, we compute the inverse Hessian at the mode of the posterior distribution of diffeomorphisms directly in the low dimensional frequency domain. This dramatically reduces the computational complexity of approximating posterior marginals in the high dimensional imaging space. Experimental results show that our method is significantly faster than the state-of-the-art diffeomorphic image registration uncertainty quantification algorithms, while producing comparable results. The efficiency of our method strengthens the feasibility in prospective clinical applications, e.g., real-time image-guided navigation for brain surgery., NIH (Grants P41EB015898 and P41EB015902)

© Springer Nature Switzerland AG 2018. This paper presents a novel approach to modeling the posterior distribution in image registration that is computationally efficient for large deformation diffeomorphic metric mapping (LDDMM). We develop a Laplace approximation of Bayesian registration models entirely in a bandlimited space that fully describes the properties of diffeomorphic transformations. In contrast to current methods, we compute the inverse Hessian at the mode of the posterior distribution of diffeomorphisms directly in the low dimensional frequency domain. This dramatically reduces the computational complexity of approximating posterior marginals in the high dimensional imaging space. Experimental results show that our method is significantly faster than the state-of-the-art diffeomorphic image registration uncertainty quantification algorithms, while producing comparable results. The efficiency of our method strengthens the feasibility in prospective clinical applications, e.g., real-time image-guided navigation for brain surgery.

We demonstrate an object tracking method for 3D images with fixed computational cost and state-of-the-art performance. Previous methods predicted transformation parameters from convolutional layers. We instead propose an architecture that does not include either flattening of convolutional features or fully connected layers, but instead relies on equivariant filters to preserve transformations between inputs and outputs (e.g. rot./trans. of inputs rotate/translate outputs). The transformation is then derived in closed form from the outputs of the filters. This method is useful for applications requiring low latency, such as real-time tracking. We demonstrate our model on synthetically augmented adult brain MRI, as well as fetal brain MRI, which is the intended use-case., Comment: MICCAI 2021 Early Accept

We propose and demonstrate a representation learning approach by maximizing the mutual information between local features of images and text. The goal of this approach is to learn useful image representations by taking advantage of the rich information contained in the free text that describes the findings in the image. Our method trains image and text encoders by encouraging the resulting representations to exhibit high local mutual information. We make use of recent advances in mutual information estimation with neural network discriminators. We argue that the sum of local mutual information is typically a lower bound on the global mutual information. Our experimental results in the downstream image classification tasks demonstrate the advantages of using local features for image-text representation learning., Comment: In Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2021

This paper presents an efficient approach to quantifying image registration uncertainty based on a low-dimensional representation of geometric deformations. In contrast to previous methods, we develop a Bayesian diffeomorphic registration framework in a bandlimited space, rather than a high-dimensional image space. We show that a dense posterior distribution on deformation fields can be fully characterized by much fewer parameters, which dramatically reduces the computational complexity of model inferences. To further avoid heavy computation loads introduced by random sampling algorithms, we approximate a marginal posterior by using Laplace's method at the optimal solution of log-posterior distribution. Experimental results on both 2D synthetic data and real 3D brain magnetic resonance imaging (MRI) scans demonstrate that our method is significantly faster than the state-of-the-art diffeomorphic registration uncertainty quantification algorithms, while producing comparable results.

We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at: https://github.com/RayRuizhiLiao/joint_chestxray., NIH/NIBIB/NAC (Grant P41EB015902)

Over the past decade, deep learning has become the gold standard for automatic medical image segmentation. Every segmentation task has an underlying uncertainty due to image resolution, annotation protocol, etc. Therefore, a number of methods and metrics have been proposed to quantify the uncertainty of neural networks mostly based on Bayesian deep learning, ensemble learning methods or output probability calibration. The aim of our research is to assess how reliable the different uncertainty metrics found in the literature are. We propose a quantitative and statistical comparison of uncertainty measures based on the relevance of the uncertainty map to predict misclassification. Four uncertainty metrics were compared over a set of 144 models. The application studied is the segmentation of the lumen and vessel wall of carotid arteries based on multiple sequences of magnetic resonance (MR) images in multi-center data.

The loss function of an unsupervised multimodal image registration framework has two terms, i.e., a metric for similarity measure and regularization. In the deep learning era, researchers proposed many approaches to automatically learn the similarity metric, which has been shown effective in improving registration performance. However, for the regularization term, most existing multimodal registration approaches still use a hand-crafted formula to impose artificial properties on the estimated deformation field. In this work, we propose a unimodal cyclic regularization training pipeline, which learns task-specific prior knowledge from simpler unimodal registration, to constrain the deformation field of multimodal registration. In the experiment of abdominal CT-MR registration, the proposed method yields better results over conventional regularization methods, especially for severely deformed local regions., Comment: 5 pages, 5 figures

Estimating mutual information between continuous random variables is often intractable and extremely challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual information. Although showing promise for this difficult problem, the variational methods have been theoretically and empirically proven to have serious statistical limitations: 1) many methods struggle to produce accurate estimates when the underlying mutual information is either low or high; 2) the resulting estimators may suffer from high variance. Our approach is based on training a classifier that provides the probability that a data sample pair is drawn from the joint distribution rather than from the product of its marginal distributions. Moreover, we establish a direct connection between mutual information and the average log odds estimate produced by the classifier on a test set, leading to a simple and accurate estimator of mutual information. We show theoretically that our method and other variational approaches are equivalent when they achieve their optimum, while our method sidesteps the variational bound. Empirical results demonstrate high accuracy of our approach and the advantages of our estimator in the context of representation learning. Our demo is available at https://github.com/RayRuizhiLiao/demi_mi_estimator.

We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at https://github.com/RayRuizhiLiao/joint_chestxray., Comment: The two first authors contributed equally. To be published in the proceedings of MICCAI 2020

Intraoperative tissue deformation, known as brain shift, decreases the benefit of using preoperative images to guide neurosurgery. Non-rigid registration of preoperative magnetic resonance (MR) to intraoperative ultrasound (iUS) has been proposed as a means to compensate for brain shift. We focus on the initial registration from MR to predurotomy iUS. We present a method that builds on previous work to address the need for accuracy and generality of MR-iUS registration algorithms in multi-site clinical data. High-dimensional texture attributes were used instead of image intensities for image registration and the standard difference-based attribute matching was replaced with correlation-based attribute matching. A strategy that deals explicitly with the large field-of-view mismatch between MR and iUS images was proposed. Key parameters were optimized across independent MR-iUS brain tumor datasets acquired at 3 institutions, with a total of 43 tumor patients and 758 reference landmarks for evaluating the accuracy of the proposed algorithm. Despite differences in imaging protocols, patient demographics and landmark distributions, the algorithm is able to reduce landmark errors prior to registration in three data sets (5.37±4.27, 4.18±1.97 and 6.18±3.38 mm, respectively) to a consistently low level (2.28±0.71, 2.08±0.37 and 2.24±0.78 mm, respectively). This algorithm was tested against 15 other algorithms and it is competitive with the state-of-the-art on multiple datasets. We show that the algorithm has one of the lowest errors in all datasets (accuracy), and this is achieved while sticking to a fixed set of parameters for multi-site data (generality). In contrast, other algorithms/tools of similar performance need per-dataset parameter tuning (high accuracy but lower generality), and those that stick to fixed parameters have larger errors or inconsistent performance (generality but not the top accuracy). Landmark errors were further characterized according to brain re, National Institute of Health (Grants P41-EB015898, P41-EB015902 and R01-NS049251)

Purpose: The brain undergoes significant structural change over the course of neurosurgery, including highly nonlinear deformation and resection. It can be informative to recover the spatial mapping between structures identified in preoperative surgical planning and the intraoperative state of the brain. We present a novel feature-based method for achieving robust, fully automatic deformable registration of intraoperative neurosurgical ultrasound images. Methods: A sparse set of local image feature correspondences is first estimated between ultrasound image pairs, after which rigid, affine and thin-plate spline models are used to estimate dense mappings throughout the image. Correspondences are derived from 3D features, distinctive generic image patterns that are automatically extracted from 3D ultrasound images and characterized in terms of their geometry (i.e., location, scale, and orientation) and a descriptor of local image appearance. Feature correspondences between ultrasound images are achieved based on a nearest-neighbor descriptor matching and probabilistic voting model similar to the Hough transform. Results: Experiments demonstrate our method on intraoperative ultrasound images acquired before and after opening of the dura mater, during resection and after resection in nine clinical cases. A total of 1620 automatically extracted 3D feature correspondences were manually validated by eleven experts and used to guide the registration. Then, using manually labeled corresponding landmarks in the pre- and post-resection ultrasound images, we show that our feature-based registration reduces the mean target registration error from an initial value of 3.3 to 1.5 mm. Conclusions: This result demonstrates that the 3D features promise to offer a robust and accurate solution for 3D ultrasound registration and to correct for brain shift in image-guided neurosurgery., National Institutes of Health (U.S.) (Grant P41-EB015898-09), National Institutes of Health (U.S.) (Grant P41-EB015902), National Institutes of Health (U.S.) (Grant R01-NS049251), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant PD/BD/105869/2014), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant IDMEC/LAETA UID/EMS/50022/2013)

Purpose: Matching points that are derived from features or landmarks in image data is a key step in some medical imaging applications. Since most robust point matching algorithms claim to be able to deal with outliers, users may place high confidence in the matching result and use it without further examination. However, for tasks such as feature-based registration in image-guided neurosurgery, even a few mismatches, in the form of invalid displacement vectors, could cause serious consequences. As a result, having an effective tool by which operators can manually screen all matches for outliers could substantially benefit the outcome of those applications. Methods: We introduce a novel variogram-based outlier screening method for vectors. The variogram is a powerful geostatistical tool for characterizing the spatial dependence of stochastic processes. Since the spatial correlation of invalid displacement vectors, which are considered as vector outliers, tends to behave differently than normal displacement vectors, they can be efficiently identified on the variogram. Results: We validate the proposed method on 9 sets of clinically acquired ultrasound data. In the experiment, potential outliers are flagged on the variogram by one operator and further evaluated by 8 experienced medical imaging researchers. The matching quality of those potential outliers is approximately 1.5 lower, on a scale from 1 (bad) to 5 (good), than valid displacement vectors. Conclusion: The variogram is a simple yet informative tool. While being used extensively in geostatistical analysis, it has not received enough attention in the medical imaging field. We believe there is a good deal of potential for clinically applying the proposed outlier screening method. By way of this paper, we also expect researchers to find variogram useful in other medical applications that involve motion vectors analyses., National Institutes of Health (U.S.) (Grant P41-EB015898-09), National Institutes of Health (U.S.) (Grant P41-EB015902)

We propose and demonstrate machine learning algorithms to assess the severity of pulmonary edema in chest x-ray images of congestive heart failure patients. Accurate assessment of pulmonary edema in heart failure is critical when making treatment and disposition decisions. Our work is grounded in a large-scale clinical dataset of over 300,000 x-ray images with associated radiology reports. While edema severity labels can be extracted unambiguously from a small fraction of the radiology reports, accurate annotation is challenging in most cases. To take advantage of the unlabeled images, we develop a Bayesian model that includes a variational auto-encoder for learning a latent representation from the entire image set trained jointly with a regressor that employs this representation for predicting pulmonary edema severity. Our experimental results suggest that modeling the distribution of images jointly with the limited labels improves the accuracy of pulmonary edema scoring compared to a strictly supervised approach. To the best of our knowledge, this is the first attempt to employ machine learning algorithms to automatically and quantitatively assess the severity of pulmonary edema in chest x-ray images.

Hashing is a popular approach for performing computationally efficient approximate nearest neighbor search by means of encoding data items into sequence of bits, such that the nearest neighbor search in the coding space is efficient and accurate. This thesis explores aspects of code-consistent training of hashing forests and deep learning models for end-to-end learning of hash codes and demonstrates that such hashing models can be leveraged to perform efficient and accurate large-scale content-based medical image retrieval., Hashing ist eine Methode zur laufzeit-effizienten Nearest-Neighbor- Suche, bei der man die Daten in einer Bitsequenz kodiert, so dass die Suche im Kodierungsraum effizient und präzise durchgeführt werden kann. Diese Dissertation untersucht Aspekte des code-konsistenten Trainings von Hashing Forests und Deep-Learning-Modellen für das end-to-end learning von Hashwerten und zeigt, dass diese Hashing-Modelle dazu genutzt werden können, effizientes und präzises inhaltsbasiertes Image Retrieval für medizinische Anwendungen in großem Maßstab durchzuführen.

We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison to common multi-atlas segmentation, while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with highly variable field-of-view., Comment: Accepted for publication at IEEE Transactions on Medical Imaging

The flowcharts and computer programs prepared for use in conjunction with the Procedures Manual for the Student Accountability Model (see JC 740 394) are presented. The programs provide an illustration of how one district applied the flowchart to its own hardware capabilities. The flowcharts are: Assign Course Priority, Assign Student Major, Assign Course Majors, Non-continuing Students, and Unduplicated Counts. The programs are: Student Count by Priority, Semester, and Unduplicated Counts, Year. The hardware requirements and a card layout form are provided. (DB)

This book constitutes the thoroughly refereed post-workshop proceedings of the International Workshop on Medical Computer Vision, MCV 2016, and of the International Workshop on Bayesian and grAphical Models for Biomedical Imaging, BAMBI 2016, held in Athens, Greece, in October 2016, held in conjunction with the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016. The 13 papers presented in MCV workshop and the 6 papers presented in BAMBI workshop were carefully reviewed and selected from numerous submissions. The goal of the MCV workshop is to explore the use of "big data” algorithms for harvesting, organizing and learning from large-scale medical imaging data sets and for general-purpose automatic understanding of medical images. The BAMBI workshop aims to highlight the potential of using Bayesian or random field graphical models for advancing research in biomedical image analysis.

We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ label maps. We introduce generative models for the inference of keypoint labels and for image segmentation, where keypoint matches are treated as a latent random variable and are marginalized out as part of the algorithm. We report segmentation results for abdominal organs in whole-body CT and in contrast-enhanced CT images. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm’s robustness enables the segmentation of scans with highly variable field-of-view., National Alliance for Medical Image Computing (U.S.) (U54-EB005149), National Center for Image Guided Therapy (P41-EB015898)

The field of biomedical image analysis has been enjoying fast growth in recent years due to the availability of large datasets of images in a wide variety of domains. As most volumetric imaging data can be tremendously noisy (e.g., ultrasound data) and results can be highly uncertain in every stage of inference, probabilistic models have the potential to greatly improve solutions to problems in biomedical imaging. Although probabilistic models have been developed and have shown great success in a wide variety of contexts in the field of computer vision, including object recognition, object/face detection, localization and classification, and image registration, there are still many significant questions to consider regarding their effective exploitation in the analysis of biomedical images. For example, objective selection of different models or estimates of parameter uncertainty may facilitate improved interpretability or plausibility. Also, improvements in the efficiency of inference techniques may allow hierarchical, or higher-order models to be tractably used to answer queries, reducing the dependence on subjective modeling choices. This special issue bridges the gap between research in computer vision, biomedical image analysis and machine learning by providing a platform for the exploration of probabilistic modeling approaches for difficult clinical problems within a variety of biomedical imaging contexts.

Laboratory-based mock crime studies have often been interpreted to mean that (i) eyewitness confidence in an identification made from a lineup is a weak indicator of accuracy and (ii) sequential lineups are diagnostically superior to traditional simultaneous lineups. Largely as a result, juries are increasingly encouraged to disregard eyewitness confidence, and up to 30% of law enforcement agencies in the United States have adopted the sequential procedure. We conducted a field study of actual eyewitnesses who were assigned to simultaneous or sequential photo lineups in the Houston Police Department over a 1-y period. Identifications were made using a three-point confidence scale, and a signal detection model was used to analyze and interpret the results. Our findings suggest that (i) confidence in an eyewitness identification from a fair lineup is a highly reliable indicator of accuracy and (ii) if there is any difference in diagnostic accuracy between the two lineup formats, it likely favors the simultaneous procedure.

OBJECTIVES: To introduce a multi-site assessment of oral health literacy and to describe preliminary analyses of the relationships between health literacy and selected oral health outcomes within the context of a comprehensive conceptual model. METHODS: Data for this analysis came from the Multi-Site Oral Health Literacy Research Study (MOHLRS), a federally funded investigation of health literacy and oral health. MOHLRS consisted of a broad survey, including several health literacy assessments, and measures of attitudes, knowledge, and other factors. The survey was administered to 922 initial care-seeking adult patients presenting to university-based dental clinics in California and Maryland. For this descriptive analysis, confidence filling out forms, word recognition, and reading comprehension comprised the health literacy assessments. Dental visits, oral health functioning, and dental self-efficacy were the outcomes. RESULTS: Overall, up to 21% of participants reported having difficulties with practical health literacy tasks. After controlling for sociodemographic confounders, no health literacy assessment was associated with dental visits or dental caries self-efficacy. However, confidence filling out forms and word recognition were each associated with oral health functioning and periodontal disease self-efficacy. CONCLUSIONS: Our analysis showed that dental school patients exhibit a range of health literacy abilities. It also revealed that the relationship between health literacy and oral health is not straightforward, depending on patient characteristics and the unique circumstances of the encounter. We anticipate future analyses of MOHLRS data will answer questions about the role that health literacy and various mediating factors play in explaining oral health disparities.

Spatial priors, such as probabilistic atlases, play an important role in MRI segmentation. However, the availability of comprehensive, reliable and suitable manual segmentations for atlas construction is limited. We therefore propose a method for joint segmentation of corresponding regions of interest in a collection of aligned images that does not require labeled training data. Instead, a latent atlas, initialized by at most a single manual segmentation, is inferred from the evolving segmentations of the ensemble. The algorithm is based on probabilistic principles but is solved using partial differential equations (PDEs) and energy minimization criteria. We evaluate the method on two datasets, segmenting subcortical and cortical structures in a multi-subject study and extracting brain tumors in a single-subject multi-modal longitudinal experiment. We compare the segmentation results to manual segmentations, when those exist, and to the results of a state-of-the-art atlas-based segmentation method. The quality of the results supports the latent atlas as a promising alternative when existing atlases are not compatible with the images to be segmented., National Institutes of Health (U.S.) (National Institute for Biomedical Imaging and Bioengineering (U.S.)/National Alliance for Medical Image Computing (U.S.) U54-EB005149), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Neuroimaging Analysis Center (U.S.) P41-RR13218), National Institutes of Health (U.S.) (National Institute of Neurological Disorders and Stroke (U.S.) R01-NS051826), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Biomedical Informatics Research Network U24-RR021382), National Science Foundation (U.S.) (CAREER Award 0642971), German Academy of Sciences Leopoldina (Fellowship LPDS 2009-10), Academy of Finland (Grant 133611)

There is increasing evidence that iron deposition occurs in specific regions of the brain in normal aging and neurodegenerative disorders such as Parkinson's, Huntington's, and Alzheimer's disease. Iron deposition changes the magnetic susceptibility of tissue, which alters the MR signal phase, and allows estimation of susceptibility differences using quantitative susceptibility mapping (QSM). We present a method for quantifying susceptibility by inversion of a perturbation model, or “QSIP.” The perturbation model relates phase to susceptibility using a kernel calculated in the spatial domain, in contrast to previous Fourier-based techniques. A tissue/air susceptibility atlas is used to estimate B[subscript 0] inhomogeneity. QSIP estimates in young and elderly subjects are compared to postmortem iron estimates, maps of the Field-Dependent Relaxation Rate Increase, and the L1-QSM method. Results for both groups showed excellent agreement with published postmortem data and in vivo FDRI: statistically significant Spearman correlations ranging from Rho=0.905 to Rho=1.00 were obtained. QSIP also showed improvement over FDRI and L1-QSM: reduced variance in susceptibility estimates and statistically significant group differences were detected in striatal and brainstem nuclei, consistent with age-dependent iron accumulation in these regions., National Institutes of Health (U.S.) (Grant P41EB015902), National Institutes of Health (U.S.) (Grant P41RR013218), National Institutes of Health (U.S.) (Grant P41EB015898), National Institutes of Health (U.S.) (Grant P41RR019703), National Institutes of Health (U.S.) (Grant T32EB0011680-06), National Institutes of Health (U.S.) (Grant K05AA017168), National Institutes of Health (U.S.) (Grant R01AA012388)

Intensity-based image registration requires resampling images on a common grid to evaluate the similarity function. The uncertainty of interpolation varies across the image, depending on the location of resampled points relative to the base grid. We propose to perform Bayesian inference with Gaussian processes, where the covariance matrix of the Gaussian process posterior distribution estimates the uncertainty in interpolation. The Gaussian process replaces a single image with a distribution over images that we integrate into a generative model for registration. Marginalization over resampled images leads to a new similarity measure that includes the uncertainty of the interpolation. We demonstrate that our approach increases the registration accuracy and propose an efficient approximation scheme that enables seamless integration with existing registration methods., Alexander von Humboldt-Stiftung, National Alliance for Medical Image Computing (U.S.) (U54-EB005149), Neuroimaging Analysis Center (U.S.) (P41-EB015902), National Center for Image-Guided Therapy (U.S.) (P41-EB015898)

In this paper, we analyze Markov Random Field (MRF) as a spatial regularizer in fMRI detection. The low signal-to-noise ratio (SNR) in fMRI images presents a serious challenge for detection algorithms, making regularization necessary to achieve good detection accuracy. Gaussian smoothing, traditionally employed to boost SNR, often produces over-smoothed activation maps. Recently, the use of MRF priors has been suggested as an alternative regularization approach. However, solving for an optimal configuration of the MRF is NP-hard in general. In this work, we investigate fast inference algorithms based on the Mean Field approximation in application to MRF priors for fMRI detection. Furthermore, we propose a novel way to incorporate anatomical information into the MRF-based detection framework and into the traditional smoothing methods. Intuitively speaking, the anatomical evidence increases the likelihood of activation in the gray matter and improves spatial coherency of the resulting activation maps within each tissue type. Validation using the receiver operating characteristic (ROC) analysis and the confusion matrix analysis on simulated data illustrates substantial improvement in detection accuracy using the anatomically guided MRF spatial regularizer. We further demonstrate the potential benefits of the proposed method in real fMRI signals of reduced length. The anatomically guided MRF regularizer enables significant reduction of the scan length while maintaining the quality of the resulting activation maps., National Institutes of Health (U.S.) (National Institute for Biomedical Imaging and Bioengineering (U.S.)/National Alliance for Medical Image Computing (U.S.) Grant U54-EB005149), National Science Foundation (U.S.) (Grant IIS 9610249), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Biomedical Informatics Research Network Grant U24-RR021382), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Neuroimaging Analysis Center (U.S.) Grant P41-RR13218), National Institutes of Health (U.S.) (National Institute of Neurological Disorders and Stroke (U.S.) Grant R01-NS051826), National Science Foundation (U.S.) (CAREER Grant 0642971), National Science Foundation (U.S.). Graduate Research Fellowship, National Center for Research Resources (U.S.) (FIRST-BIRN Grant), Neuroimaging Analysis Center (U.S.)

Shortest-path tractography (SPT) algorithms solve global optimization problems defined from local distance functions. As diffusion MRI data is inherently noisy, so are the voxelwise tensors from which local distances are derived. We extend Riemannian SPT by modeling the stochasticity of the diffusion tensor as a “random Riemannian metric”, where a geodesic is a distribution over tracts. We approximate this distribution with a Gaussian process and present a probabilistic numerics algorithm for computing the geodesic distribution. We demonstrate SPT improvements on data from the Human Connectome Project.