Search

ObjectiveThe P300 speller is a common brain-computer interface (BCI) application designed to communicate language by detecting event related potentials in a subject's electroencephalogram signal. Information about the structure of natural language can be valuable for BCI communication, but attempts to use this information have thus far been limited to rudimentary n-gram models. While more sophisticated language models are prevalent in natural language processing literature, current BCI analysis methods based on dynamic programming cannot handle their complexity.ApproachSampling methods can overcome this complexity by estimating the posterior distribution without searching the entire state space of the model. In this study, we implement sequential importance resampling, a commonly used particle filtering (PF) algorithm, to integrate a probabilistic automaton language model.Main resultThis method was first evaluated offline on a dataset of 15 healthy subjects, which showed significant increases in speed and accuracy when compared to standard classification methods as well as a recently published approach using a hidden Markov model (HMM). An online pilot study verified these results as the average speed and accuracy achieved using the PF method was significantly higher than that using the HMM method.SignificanceThese findings strongly support the integration of domain-specific knowledge into BCI classification to improve system performance.

Objectives: Mobile health (mHealth) regimens can improve health through the continuous monitoring of biometric parameters paired with appropriate interventions. However, adherence to monitoring tends to decay over time. Our randomized controlled trial sought to determine: (1) if a mobile app with gamification and financial incentives significantly increases adherence to mHealth monitoring in a population of heart failure patients; and (2) if activity data correlate with disease-specific symptoms., Materials and Methods: We recruited individuals with heart failure into a prospective 180-day monitoring study with 3 arms. All 3 arms included monitoring with a connected weight scale and an activity tracker. The second arm included an additional mobile app with gamification, and the third arm included the mobile app and a financial incentive awarded based on adherence to mobile monitoring., Results: We recruited 111 heart failure patients into the study. We found that the arm including the financial incentive led to significantly higher adherence to activity tracker (95% vs 72.2%, P = .01) and weight (87.5% vs 69.4%, P = .002) monitoring compared to the arm that included the monitoring devices alone. Furthermore, we found a significant correlation between daily steps and daily symptom severity., Discussion and Conclusion: Our findings indicate that mobile apps with added engagement features can be useful tools for improving adherence over time and may thus increase the impact of mHealth-driven interventions. Additionally, activity tracker data can provide passive monitoring of disease burden that may be used to predict future events., (© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Background: Numerous studies have explored image processing techniques aimed at enhancing ultrasound images to narrow the performance gap between low-quality portable devices and high-end ultrasound equipment. These investigations often use registered image pairs created by modifying the same image through methods like down sampling or adding noise, rather than using separate images from different machines. Additionally, they rely on organ-specific features, limiting the models' generalizability across various imaging conditions and devices. The challenge remains to develop a universal framework capable of improving image quality across different devices and conditions, independent of registration or specific organ characteristics., Objective: This study aims to develop a robust framework that enhances the quality of ultrasound images, particularly those captured with compact, portable devices, which are often constrained by low quality due to hardware limitations. The framework is designed to effectively process nonregistered ultrasound image pairs, a common challenge in medical imaging, across various clinical settings and device types. By addressing these challenges, the research seeks to provide a more generalized and adaptable solution that can be widely applied across diverse medical scenarios, improving the accessibility and quality of diagnostic imaging., Methods: A retrospective analysis was conducted by using a cycle-consistent generative adversarial network (CycleGAN) framework enhanced with perceptual loss to improve the quality of ultrasound images, focusing on nonregistered image pairs from various organ systems. The perceptual loss was integrated to preserve anatomical integrity by comparing deep features extracted from pretrained neural networks. The model's performance was evaluated against corresponding high-resolution images, ensuring that the enhanced outputs closely mimic those from high-end ultrasound devices. The model was trained and validated using a publicly available, diverse dataset to ensure robustness and generalizability across different imaging scenarios., Results: The advanced CycleGAN framework, enhanced with perceptual loss, significantly outperformed the previous state-of-the-art, stable CycleGAN, in multiple evaluation metrics. Specifically, our method achieved a structural similarity index of 0.2889 versus 0.2502 (P<.001), a peak signal-to-noise ratio of 15.8935 versus 14.9430 (P<.001), and a learned perceptual image patch similarity score of 0.4490 versus 0.5005 (P<.001). These results demonstrate the model's superior ability to enhance image quality while preserving critical anatomical details, thereby improving diagnostic usefulness., Conclusions: This study presents a significant advancement in ultrasound imaging by leveraging a CycleGAN model enhanced with perceptual loss to bridge the quality gap between images from different devices. By processing nonregistered image pairs, the model not only enhances visual quality but also ensures the preservation of essential anatomical structures, crucial for accurate diagnosis. This approach holds the potential to democratize high-quality ultrasound imaging, making it accessible through low-cost portable devices, thereby improving health care outcomes, particularly in resource-limited settings. Future research will focus on further validation and optimization for clinical use., (©Shreeram Athreya, Ashwath Radhachandran, Vedrana Ivezić, Vivek R Sant, Corey W Arnold, William Speier. Originally published in JMIR Biomedical Engineering (http://biomsedeng.jmir.org), 17.12.2024.)

The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for 'slice integration by vision transformer'), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications., (© 2024. The Author(s), under exclusive licence to Springer Nature Limited.)

Prostate cancer (PCa) was the most frequently diagnosed cancer among American men in 2023 [1]. The histological grading of biopsies is essential for diagnosis, and various deep learning-based solutions have been developed to assist with this task. Existing deep learning frameworks are typically applied to individual 2D cross-sections sliced from 3D biopsy tissue specimens. This process impedes the analysis of complex tissue structures such as glands, which can vary depending on the tissue slice examined. We propose a novel digital pathology data source called a "volumetric core," obtained via the extraction and co-alignment of serially sectioned tissue sections using a novel morphology-preserving alignment framework. We trained an attention-based multiple-instance learning (ABMIL) framework on deep features extracted from volumetric patches to automatically classify the Gleason Grade Group (GGG). To handle volumetric patches, we used a modified video transformer with a deep feature extractor pretrained using self-supervised learning. We ran our morphology preserving alignment framework to construct 10,210 volumetric cores, leaving out 30% for pretraining. The rest of the dataset was used to train ABMIL, which resulted in a 0.958 macro-average AUC, 0.671 F1 score, 0.661 precision, and 0.695 recall averaged across all five GGG significantly outperforming the 2D baselines., Competing Interests: COMPETING INTERESTS The authors declare no competing interests.

Background and Purpose: Following endovascular thrombectomy in patients with large-vessel occlusion stroke, successful recanalization from 1 attempt, known as the first-pass effect, has correlated favorably with long-term outcomes. Pretreatment imaging may contain information that can be used to predict the first-pass effect. Recently, applications of machine learning models have shown promising results in predicting recanalization outcomes, albeit requiring manual segmentation. In this study, we sought to construct completely automated methods using deep learning to predict the first-pass effect from pretreatment CT and MR imaging., Materials and Methods: Our models were developed and evaluated using a cohort of 326 patients who underwent endovascular thrombectomy at UCLA Ronald Reagan Medical Center from 2014 to 2021. We designed a hybrid transformer model with nonlocal and cross-attention modules to predict the first-pass effect on MR imaging and CT series., Results: The proposed method achieved a mean 0.8506 (SD, 0.0712) for cross-validation receiver operating characteristic area under the curve (ROC-AUC) on MR imaging and 0.8719 (SD, 0.0831) for cross-validation ROC-AUC on CT. When evaluated on the prospective test sets, our proposed model achieved a mean ROC-AUC of 0.7967 (SD, 0.0335) with a mean sensitivity of 0.7286 (SD, 0.1849) and specificity of 0.8462 (SD, 0.1216) for MR imaging and a mean ROC-AUC of 0.8051 (SD, 0.0377) with a mean sensitivity of 0.8615 (SD, 0.1131) and specificity 0.7500 (SD, 0.1054) for CT, respectively, representing the first classification of the first-pass effect from MR imaging alone and the first automated first-pass effect classification method in CT., Conclusions: Results illustrate that both nonperfusion MR imaging and CT from admission contain signals that can predict a successful first-pass effect following endovascular thrombectomy using our deep learning methods without requiring time-intensive manual segmentation., (© 2024 by American Journal of Neuroradiology.)

Background:  Heart failure is a complex clinical syndrome noted on approximately one in eight death certificates in the United States. Vital to reducing complications of heart failure and preventing hospital readmissions is adherence to heart failure self-care routines. Mobile health offers promising opportunities for enhancing self-care behaviors by facilitating tracking and timely reminders., Objectives:  We sought to investigate three characteristics of heart failure patients with respect to their heart failure self-care behaviors: (1) internet use to search for heart failure information; (2) familiarity with mobile health apps and devices; and (3) perceptions of using activity trackers or smartwatches to aid in their heart failure self-care., Methods:  Forty-nine heart failure patients were asked about their internet and mobile health usage. The structured interview included questions adapted from the Health Information National Trends Survey., Results:  Over 50% of the patients had utilized the internet to search for heart failure information in the past 12 months, experience using health-related apps, and thoughts that an activity tracker or smartwatch could help them manage heart failure. Qualitative analysis of the interviews revealed six themes: trust in their physicians, alternatives to mobile health apps, lack of need for mobile health devices, financial barriers to activity tracker and smartwatch ownership, benefits of tracking and reminders, and uncertainty of their potential due to lack of knowledge., Conclusion:  Trust in their physicians was a major factor for heart failure patients who reported not searching for health information on the internet. While those who used mobile health technologies found them useful, patients who did not use them were generally unaware of or unknowledgeable about them. Considering patients' preferences for recommendations from their physicians and tendency to search for heart failure information including treatment and management options, patient-provider discussions about mobile health may improve patient knowledge and impact their usage., Competing Interests: G.C.F. reports consulting for Abbott, Amgen, Bayer, Janssen, Medtronic, and Novartis., (Thieme. All rights reserved.)

Context: Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice., Evidence Acquisition: A literature search of PubMed and IEEE Xplore was conducted for English-language publications between January 1, 2015 and January 1, 2023, studying diagnostic tests on suspected thyroid nodules that used AI. We excluded articles without prospective or external validation, nonprimary literature, duplicates, focused on nonnodular thyroid conditions, not using AI, and those incidentally using AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence., Evidence Synthesis: A total of 61 studies were identified; all performed external validation, 16 studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding 5 high-level outcomes: (1) nodule localization, (2) ultrasound (US) risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from US and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and interobserver variability., Conclusion: Models predominantly used US images to predict malignancy. Of 4 Food and Drug Administration-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and revalidation to ensure appropriate clinical performance., (© The Author(s) 2024. Published by Oxford University Press on behalf of the Endocrine Society. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.)

Background and Purpose: The DWI-FLAIR mismatch is used to determine thrombolytic eligibility in patients with acute ischemic stroke when the time since stroke onset is unknown. Commercial software packages have been developed for automated DWI-FLAIR classification. We aimed to use e-Stroke software for automated classification of the DWI-FLAIR mismatch in a cohort of patients with acute ischemic stroke and in a comparative analysis with 2 expert neuroradiologists., Materials and Methods: In this retrospective study, patients with acute ischemic stroke who had MR imaging and known time since stroke onset were included. The DWI-FLAIR mismatch was evaluated by 2 neuroradiologists blinded to the time since stroke onset and automatically by the e-Stroke software. After 4 weeks, the neuroradiologists re-evaluated the MR images, this time equipped with automated predicted e-Stroke results as a computer-assisted tool. Diagnostic performances of e-Stroke software and the neuroradiologists were evaluated for prediction of DWI-FLAIR mismatch status., Results: A total of 157 patients met the inclusion criteria. A total of 82 patients (52%) had a time since stroke onset of ≤4.5 hours. By means of consensus reads, 81 patients (51.5%) had a DWI-FLAIR mismatch. The diagnostic accuracy (area under the curve/sensitivity/specificity) of e-Stroke software for the determination of the DWI-FLAIR mismatch was 0.72/90.0/53.9. The diagnostic accuracy (area under the curve/sensitivity/specificity) for neuroradiologists 1 and 2 was 0.76/69.1/84.2 and 0.82/91.4/73.7, respectively; both significantly ( P  < .05) improved to 0.83/79.0/86.8 and 0.89/92.6/85.5, respectively, following the use of e-Stroke predictions as a computer-assisted tool. The interrater agreement (κ) for determination of DWI-FLAIR status was improved from 0.49 to 0.57 following the use of the computer-assisted tool., Conclusions: This automated quantitative approach for DWI-FLAIR mismatch provides results comparable with those of human experts and can improve the diagnostic accuracies of expert neuroradiologists in the determination of DWI-FLAIR status., (© 2024 by American Journal of Neuroradiology.)

Objective: Automated methods for prostate segmentation on MRI are typically developed under ideal scanning and anatomical conditions. This study evaluates three different prostate segmentation AI algorithms in a challenging population of patients with prior treatments, variable anatomic characteristics, complex clinical history, or atypical MRI acquisition parameters., Materials and Methods: A single institution retrospective database was queried for the following conditions at prostate MRI: prior prostate-specific oncologic treatment, transurethral resection of the prostate (TURP), abdominal perineal resection (APR), hip prosthesis (HP), diversity of prostate volumes (large ≥ 150 cc, small ≤ 25 cc), whole gland tumor burden, magnet strength, noted poor quality, and various scanners (outside/vendors). Final inclusion criteria required availability of axial T2-weighted (T2W) sequence and corresponding prostate organ segmentation from an expert radiologist. Three previously developed algorithms were evaluated: (1) deep learning (DL)-based model, (2) commercially available shape-based model, and (3) federated DL-based model. Dice Similarity Coefficient (DSC) was calculated compared to expert. DSC by model and scan factors were evaluated with Wilcox signed-rank test and linear mixed effects (LMER) model., Results: 683 scans (651 patients) met inclusion criteria (mean prostate volume 60.1 cc [9.05-329 cc]). Overall DSC scores for models 1, 2, and 3 were 0.916 (0.707-0.971), 0.873 (0-0.997), and 0.894 (0.025-0.961), respectively, with DL-based models demonstrating significantly higher performance (p < 0.01). In sub-group analysis by factors, Model 1 outperformed Model 2 (all p < 0.05) and Model 3 (all p < 0.001). Performance of all models was negatively impacted by prostate volume and poor signal quality (p < 0.01). Shape-based factors influenced DL models (p < 0.001) while signal factors influenced all (p < 0.001)., Conclusion: Factors affecting anatomical and signal conditions of the prostate gland can adversely impact both DL and non-deep learning-based segmentation models., (© 2024. This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.)

An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9888 images, and annotated by collaborating radiologists. Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any UNet architecture variant to improve image-level nodule detection. Of the evaluated multitask models, a UNet with a ImageNet pretrained encoder and AD achieved the highest F1 score of 0.839 and image-wide Dice similarity coefficient of 0.808 on the hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound., Competing Interests: Declaration of competing interest This work was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award number R21EB030691. This is mentioned in the manuscript text under the Funding section. There are no other declarations of interest., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Understanding imaging research experiences, challenges, and strategies for academic radiology departments during and after COVID-19 is critical to prepare for future disruptive events. We summarize key insights and programmatic initiatives at major academic hospitals across the world, based on literature review and meetings of the Radiological Society of North America Vice Chairs of Research (RSNA VCR) group. Through expert discussion and case studies, we provide suggested guidelines to maintain and grow radiology research in the postpandemic era., (Copyright © 2023 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.)

Rationale and Objectives: Early prostate cancer detection and staging from MRI is extremely challenging for both radiologists and deep learning algorithms, but the potential to learn from large and diverse datasets remains a promising avenue to increase their performance within and across institutions. To enable this for prototype-stage algorithms, where the majority of existing research remains, we introduce a flexible federated learning framework for cross-site training, validation, and evaluation of custom deep learning prostate cancer detection algorithms., Materials and Methods: We introduce an abstraction of prostate cancer groundtruth that represents diverse annotation and histopathology data. We maximize use of this groundtruth if and when they are available using UCNet, a custom 3D UNet that enables simultaneous supervision of pixel-wise, region-wise, and gland-wise classification. We leverage these modules to perform cross-site federated training using 1400+ heterogeneous multi-parameteric prostate MRI exams from two University hospitals., Results: We observe a positive result, with significant improvements in cross-site generalization performance with negligible intra-site performance degradation for both lesion segmentation and per-lesion binary classification of clinically-significant prostate cancer. Cross-site lesion segmentation performance intersection-over-union (IoU) improved by 100%, while cross-site lesion classification performance overall accuracy improved by 9.5-14.8%, depending on the optimal checkpoint selected by each site., Conclusion: Federated learning can improve the generalization performance of prostate cancer detection models across institutions while protecting patient health information and institution-specific code and data. However, even more data and participating institutions are likely required to improve the absolute performance of prostate cancer classification models. To enable adoption of federated learning with limited re-engineering of federated components, we open-source our FLtools system at https://federated.ucsf.edu, including examples that can be easily adapted to other medical imaging deep learning projects., Competing Interests: Declaration of Competing Interest Authors have no competing interests to declare., (Copyright © 2023. Published by Elsevier Inc.)

Objective: Gadolinium-based contrast agents (GBCAs) have been widely used to better visualize disease in brain magnetic resonance imaging (MRI). However, gadolinium deposition within the brain and body has raised safety concerns about the use of GBCAs. Therefore, the development of novel approaches that can decrease or even eliminate GBCA exposure while providing similar contrast information would be of significant use clinically., Methods: In this work, we present a deep learning based approach for contrast-enhanced T1 synthesis on brain tumor patients. A 3D high-resolution fully convolutional network (FCN), which maintains high resolution information through processing and aggregates multi-scale information in parallel, is designed to map pre-contrast MRI sequences to contrast-enhanced MRI sequences. Specifically, three pre-contrast MRI sequences, T1, T2 and apparent diffusion coefficient map (ADC), are utilized as inputs and the post-contrast T1 sequences are utilized as target output. To alleviate the data imbalance problem between normal tissues and the tumor regions, we introduce a local loss to improve the contribution of the tumor regions, which leads to better enhancement results on tumors., Results: Extensive quantitative and visual assessments are performed, with our proposed model achieving a PSNR of 28.24 dB in the brain and 21.2 dB in tumor regions., Conclusion and Significance: Our results suggest the potential of substituting GBCAs with synthetic contrast images generated via deep learning.

Purpose: Interstitial cystitis/bladder pain syndrome is a debilitating chronic condition that disproportionately affects women at a ratio of 5:1. We sought to capture women's experiences with interstitial cystitis/bladder pain syndrome by conducting a large-scale digital ethnographic analysis of anonymous posts on Internet forums., Materials and Methods: Online posts were identified using condition-specific keywords and data mining extraction services. Once posts were identified, a random sample of 200 online posts was coded and analyzed by hand using qualitative methods. A Latent Dirichlet Allocation probabilistic topic model was applied to the complete dataset to substantiate the qualitative analysis and allow for further thematic discovery., Results: A total of 6,842 posts written by 3,902 unique users from 224 websites were identified. There was a significant overlap between the hand coding and Latent Dirichlet Allocation themes. Our analysis yielded the following themes: online community engagement, triggers and disease etiologies, medical comorbidities, quality of life impact, patient experience with medical care, and alternative therapies and self-management strategies. Additionally, our population appeared to have a high burden of nonurological associated syndromes. We identified barriers to patient-centered care and found that online peer support was important for women., Conclusions: Our digital ethnographic analysis is a novel application of qualitative methods using online sources. Social media analytics appears to capture a broader patient population than that typically included in clinic-based qualitative studies, such as patient interviews and focus groups. Understanding patient behaviors and concerns are important to guide strategies for improving care and the overall experience with this difficult-to-treat condition.

Background and Purpose: Treatment of acute ischemic stroke is heavily contingent upon time, as there is a strong relationship between time clock and tissue progression. Work has established imaging biomarker assessments as surrogates for time since stroke (TSS), namely, by comparing signal mismatch between diffusion-weighted imaging (DWI) and fluid-attenuated inversion recovery (FLAIR) imaging. Our goal was to develop an automatic technique for determining TSS from imaging that does not require subspecialist radiology expertise., Methods: Using 772 patients (66 ± 9 years, 319 women), we developed and externally evaluated a deep learning network for classifying TSS from MR images and compared algorithm predictions to neuroradiologist assessments of DWI-FLAIR mismatch. Models were trained to classify TSS within 4.5 hours and performance metrics with confidence intervals were reported on both internal and external evaluation sets., Results: Three board-certified neuroradiologists' DWI-FLAIR mismatch assessments, based on majority vote, yielded a sensitivity of .62, a specificity of .86, and a Fleiss' kappa of .46 when used to classify TSS. The deep learning method performed similarly to radiologists and outperformed previously reported methods, with the best model achieving an average evaluation accuracy, sensitivity, and specificity of .726, .712, and .741, respectively, on an internal cohort and .724, .757, and .679, respectively, on an external cohort., Conclusion: Our model achieved higher generalization performance on external evaluation datasets than the current state-of-the-art for TSS classification. These results demonstrate the potential of automatic assessment of onset time from imaging without the need for expertly trained radiologists., (© 2022 American Society of Neuroimaging.)

Deep neural networks, in particular convolutional networks, have rapidly become a popular choice for analyzing histopathology images. However, training these models relies heavily on a large number of samples manually annotated by experts, which is cumbersome and expensive. In addition, it is difficult to obtain a perfect set of labels due to the variability between expert annotations. This paper presents a novel active learning (AL) framework for histopathology image analysis, named PathAL. To reduce the required number of expert annotations, PathAL selects two groups of unlabeled data in each training iteration: one "informative" sample that requires additional expert annotation, and one "confident predictive" sample that is automatically added to the training set using the model's pseudo-labels. To reduce the impact of the noisy-labeled samples in the training set, PathAL systematically identifies noisy samples and excludes them to improve the generalization of the model. Our model advances the existing AL method for medical image analysis in two ways. First, we present a selection strategy to improve classification performance with fewer manual annotations. Unlike traditional methods focusing only on finding the most uncertain samples with low prediction confidence, we discover a large number of high confidence samples from the unlabeled set and automatically add them for training with assigned pseudo-labels. Second, we design a method to distinguish between noisy samples and hard samples using a heuristic approach. We exclude the noisy samples while preserving the hard samples to improve model performance. Extensive experiments demonstrate that our proposed PathAL framework achieves promising results on a prostate cancer Gleason grading task, obtaining similar performance with 40% fewer annotations compared to the fully supervised learning scenario. An ablation study is provided to analyze the effectiveness of each component in PathAL, and a pathologist reader study is conducted to validate our proposed algorithm.

Bone age assessment (BAA) is clinically important as it can be used to diagnose endocrine and metabolic disorders during child development. Existing deep learning based methods for classifying bone age use the global image as input, or exploit local information by annotating extra bounding boxes or key points. However, training with the global image underutilizes discriminative local information, while providing extra annotations is expensive and subjective. In this paper, we propose an attention-guided approach to automatically localize the discriminative regions for BAA without any extra annotations. Specifically, we first train a classification model to learn the attention maps of the discriminative regions, finding the hand region, the most discriminative region (the carpal bones), and the next most discriminative region (the metacarpal bones). Guided by those attention maps, we then crop the informative local regions from the original image and aggregate different regions for BAA. Instead of taking BAA as a general regression task, which is suboptimal due to the label ambiguity problem in the age label space, we propose using joint age distribution learning and expectation regression, which makes use of the ordinal relationship among hand images with different individual ages and leads to more robust age estimation. Extensive experiments are conducted on the RSNA pediatric bone age data set. Without using extra manual annotations, our method achieves competitive results compared with existing state-of-the-art deep learning-based methods that require manual annotation. Code is available at https://github.com/chenchao666/Bone-Age-Assessment.

Purpose: Multidisciplinary oncology meetings, or tumor boards (TBs), ensure and facilitate communication between specialties regarding the management of cancer cases to improve patient care. The organization of TB and the preparation and presentation of patient cases are typically inefficient processes that require the exchange of patient information via e-mail, the hunting for data and images in the electronic health record, and the copying and pasting of patient data into desktop presentation software., Methods: We implemented a standards-based electronic health record-integrated application that automated several aspects of TB organization and preparation. We hypothesized that this application would increase the efficiency of TB preparation, reduce errors in patient entry, and enhance communication with the clinical team. Our experimental design used a prospective evaluation by pathologists who were timed in preparing for weekly TBs using both the new application and the conventional method. In addition, patient data entry errors associated with each method were tracked, and TB attendees completed a survey evaluating satisfaction with the new application., Results: The total time savings for TB preparation using the digital TB application over the conventional method was 5 hours and 19 minutes, representing a 45% reduction in preparation time ( P < .01). Survey results showed that 91% of respondents preferred the digital method and believed that it improved the flow of the TB meeting. In addition, most believed that the digital method had an impact on subsequent patient care., Conclusion: This study provides further evidence that new electronic systems have the potential to significantly improve the overall TB paradigm by optimizing and enhancing case organization, preparation, and presentation., Competing Interests: Alex NoboriEmployment: Southern California Permanente Medical Group Dieter EnzmannPatents, Royalties, Other Intellectual Property: Machine learning for chest tube detection (Inst) Sarah DryTravel, Accommodations, Expenses: NewTom (I) Scott NelsonConsulting or Advisory Role: DePuy Orthopaedics, Zimmer OrthopedicsExpert Testimony: Zimmer BioMetNo other potential conflicts of interest were reported.

Treating acute ischemic stroke (AIS) patients is a time-sensitive endeavor, as therapies target areas experiencing ischemia to prevent irreversible damage to brain tissue. Depending on how an AIS is progressing, thrombolytics such as tissue-plasminogen activator (tPA) may be administered within a short therapeutic window. The underlying conditions for optimal treatment are varied. While previous clinical guidelines only permitted tPA to be administered to patients with a known onset within 4.5 hours, clinical trials demonstrated that patients with signal intensity differences between diffusion-weighted imaging (DWI) and fluid-attenuated inversion recovery (FLAIR) sequences in an MRI study can benefit from thrombolytic therapy. This intensity difference, known as DWI-FLAIR mismatch, is prone to high inter-reader variability. Thus, a paradigm exists where onset time serves as a weak proxy for DWI-FLAIR mismatch. In this study, we sought to detect DWI-FLAIR mismatch in an automated fashion, and we compared this to assessments done by three expert neuroradiologists. Our approach involved training a deep learning model on MRI to classify tissue clock and leveraging time clock as a weak proxy label to supplement training in a semi-supervised learning (SSL) framework. We evaluate our deep learning model by testing it on an unseen dataset from an external institution. In total, our proposed framework was able to improve detection of DWI-FLAIR mismatch, achieving a top ROC-AUC of 74.30%. Our study illustrated that incorporating clinical proxy information into SSL can improve model optimization by increasing the fidelity of unlabeled samples included in the training process.

Objective: To assess women's knowledge, patient experience, and treatment decision making regarding overactive bladder (OAB) using digital ethnography., Methods: Online posts were identified using a data mining service. Two hundred randomized posts were reviewed and coded using grounded theory. We then applied a latent Dirichlet allocation (LDA) probabilistic topic modeling process to review the entire collection of identified posts., Results: A total of 2618 posts by 1867 unique users from 203 different websites were identified. Our analysis yielded six themes: the impact of OAB on quality of life, patient-physician interactions, online engagement, symptom management, patient knowledge acquisition, and alternative therapies., Conclusion: Overall, online communities are a source of support for women to self-manage the OAB symptom complex and help overcome treatment pathway challenges. Digital ethnography provides insight into patient knowledge and barriers to patient-centered care, which are important to improve patient outreach. Additionally, we identify similar findings to prior work, indicating the reliability of studying social media., (© 2021. The International Urogynecological Association.)

Purpose: The appropriate number of systematic biopsy cores to retrieve during magnetic resonance imaging (MRI)-targeted prostate biopsy is not well defined. We aimed to demonstrate a biopsy sampling approach that reduces required core count while maintaining diagnostic performance., Materials and Methods: We collected data from a cohort of 971 men who underwent MRI-ultrasound fusion targeted biopsy for suspected prostate cancer. A regional targeted biopsy (RTB) was evaluated retrospectively; only cores within 2 cm of the margin of a radiologist-defined region of interest were considered part of the RTB. We compared detection rates for clinically significant prostate cancer (csPCa) and cancer upgrading rate on final whole mount pathology after prostatectomy between RTB, combined, MRI-targeted, and systematic biopsy., Results: A total of 16,459 total cores from 971 men were included in the study data sets, of which 1,535 (9%) contained csPCa. The csPCa detection rates for systematic, MRI-targeted, combined, and RTB were 27.0% (262/971), 38.3% (372/971), 44.8% (435/971), and 44.0% (427/971), respectively. Combined biopsy detected significantly more csPCa than systematic and MRI-targeted biopsy (p <0.001 and p=0.004, respectively) but was similar to RTB (p=0.71), which used on average 3.8 (22%) fewer cores per patient. In 102 patients who underwent prostatectomy, there was no significant difference in upgrading rates between RTB and combined biopsy (p=0.84)., Conclusions: A RTB approach can maintain state-of-the-art detection rates while requiring fewer retrieved cores. This result informs decision making about biopsy site selection and total retrieved core count.

Advancements in machine learning algorithms have had a beneficial impact on representation learning, classification, and prediction models built using electronic health record (EHR) data. Effort has been put both on increasing models' overall performance as well as improving their interpretability, particularly regarding the decision-making process. In this study, we present a temporal deep learning model to perform bidirectional representation learning on EHR sequences with a transformer architecture to predict future diagnosis of depression. This model is able to aggregate five heterogenous and high-dimensional data sources from the EHR and process them in a temporal manner for chronic disease prediction at various prediction windows. We applied the current trend of pretraining and fine-tuning on EHR data to outperform the current state-of-the-art in chronic disease prediction, and to demonstrate the underlying relation between EHR codes in the sequence. The model generated the highest increases of precision-recall area under the curve (PRAUC) from 0.70 to 0.76 in depression prediction compared to the best baseline model. Furthermore, the self-attention weights in each sequence quantitatively demonstrated the inner relationship between various codes, which improved the model's interpretability. These results demonstrate the model's ability to utilize heterogeneous EHR data to predict depression while achieving high accuracy and interpretability, which may facilitate constructing clinical decision support systems in the future for chronic disease screening and early detection.

Purpose: Developing large-scale datasets with research-quality annotations is challenging due to the high cost of refining clinically generated markup into high precision annotations. We evaluated the direct use of a large dataset with only clinically generated annotations in development of high-performance segmentation models for small research-quality challenge datasets., Materials and Methods: We used a large retrospective dataset from our institution comprised of 1,620 clinically generated segmentations, and two challenge datasets (PROMISE12: 50 patients, ProstateX-2: 99 patients). We trained a 3D U-Net convolutional neural network (CNN) segmentation model using our entire dataset, and used that model as a template to train models on the challenge datasets. We also trained versions of the template model using ablated proportions of our dataset, and evaluated the relative benefit of those templates for the final models. Finally, we trained a version of the template model using an out-of-domain brain cancer dataset, and evaluated the relevant benefit of that template for the final models. We used five-fold cross-validation (CV) for all training and evaluation across our entire dataset., Results: Our model achieves state-of-the-art performance on our large dataset (mean overall Dice 0.916, average Hausdorff distance 0.135 across CV folds). Using this model as a pre-trained template for refining on two external datasets significantly enhanced performance (30% and 49% enhancement in Dice scores respectively). Mean overall Dice and mean average Hausdorff distance were 0.912 and 0.15 for the ProstateX-2 dataset, and 0.852 and 0.581 for the PROMISE12 dataset. Using even small quantities of data to train the template enhanced performance, with significant improvements using 5% or more of the data., Conclusion: We trained a state-of-the-art model using unrefined clinical prostate annotations and found that its use as a template model significantly improved performance in other prostate segmentation tasks, even when trained with only 5% of the original dataset., Competing Interests: LSM and AMP report a financial interest in Avenda Health outside the submitted work. BT reports IP-related royalties from Philips outside the submitted work. The NIH has cooperative research and development agreements with NVIDIA, Philips, Siemens, Xact Robotics, Celsion Corp, and Boston Scientific outside the submitted work. The NIH has research partnerships with Angiodynamics, ArciTrax, and Exact Imaging outside the submitted work. CWA has received research equipment from NVIDIA Corporation, outside the submitted work. No commercial funding or equipment was used in the execution of this study. No other authors have competing interests to disclose.

Objective: To demonstrate enabling multi-institutional training without centralizing or sharing the underlying physical data via federated learning (FL)., Materials and Methods: Deep learning models were trained at each participating institution using local clinical data, and an additional model was trained using FL across all of the institutions., Results: We found that the FL model exhibited superior performance and generalizability to the models trained at single institutions, with an overall performance level that was significantly better than that of any of the institutional models alone when evaluated on held-out test sets from each institution and an outside challenge dataset., Discussion: The power of FL was successfully demonstrated across 3 academic institutions while avoiding the privacy risk associated with the transfer and pooling of patient data., Conclusion: Federated learning is an effective methodology that merits further study to enable accelerated development of models across institutions, enabling greater generalizability in clinical use., (© The Author(s) 2021. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Treatment of acute ischemic strokes (AIS) is largely contingent upon the time since stroke onset (TSS). However, TSS may not be readily available in up to 25% of patients with unwitnessed AIS. Current clinical guidelines for patients with unknown TSS recommend the use of MRI to determine eligibility for thrombolysis, but radiology assessments have high inter-reader variability. In this work, we present deep learning models that leverage MRI diffusion series to classify TSS based on clinically validated thresholds. We propose an intra-domain task-adaptive transfer learning method, which involves training a model on an easier clinical task (stroke detection) and then refining the model with different binary thresholds of TSS. We apply this approach to both 2D and 3D CNN architectures with our top model achieving an ROC-AUC value of 0.74, with a sensitivity of 0.70 and a specificity of 0.81 for classifying TSS < 4.5 h. Our pretrained models achieve better classification metrics than the models trained from scratch, and these metrics exceed those of previously published models applied to our dataset. Furthermore, our pipeline accommodates a more inclusive patient cohort than previous work, as we did not exclude imaging studies based on clinical, demographic, or image processing criteria. When applied to this broad spectrum of patients, our deep learning model achieves an overall accuracy of 75.78% when classifying TSS < 4.5 h, carrying potential therapeutic implications for patients with unknown TSS., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Large numbers of histopathological images have been digitized into high resolution whole slide images, opening opportunities in developing computational image analysis tools to reduce pathologists' workload and potentially improve inter- and intra-observer agreement. Most previous work on whole slide image analysis has focused on classification or segmentation of small pre-selected regions-of-interest, which requires fine-grained annotation and is non-trivial to extend for large-scale whole slide analysis. In this paper, we proposed a multi-resolution multiple instance learning model that leverages saliency maps to detect suspicious regions for fine-grained grade prediction. Instead of relying on expensive region- or pixel-level annotations, our model can be trained end-to-end with only slide-level labels. The model is developed on a large-scale prostate biopsy dataset containing 20,229 slides from 830 patients. The model achieved 92.7% accuracy, 81.8% Cohen's Kappa for benign, low grade (i.e. Grade group 1) and high grade (i.e. Grade group ≥ 2) prediction, an area under the receiver operating characteristic curve (AUROC) of 98.2% and an average precision (AP) of 97.4% for differentiating malignant and benign slides. The model obtained an AUROC of 99.4% and an AP of 99.8% for cancer detection on an external dataset., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Recent developments in machine learning algorithms have enabled models to exhibit impressive performance in healthcare tasks using electronic health record (EHR) data. However, the heterogeneous nature and sparsity of EHR data remains challenging. In this work, we present a model that utilizes heterogeneous data and addresses sparsity by representing diagnoses, procedures, and medication codes with temporal Hierarchical Clinical Embeddings combined with Topic modeling (HCET) on clinical notes. HCET aggregates various categories of EHR data and learns inherent structure based on hospital visits for an individual patient. We demonstrate the potential of the approach in the task of predicting depression at various time points prior to a clinical diagnosis. We found that HCET outperformed all baseline methods with a highest improvement of 0.07 in precision-recall area under the curve (PRAUC). Furthermore, applying attention weights across EHR data modalities significantly improved the performance as well as the model's interpretability by revealing the relative weight for each data modality. Our results demonstrate the model's ability to utilize heterogeneous EHR information to predict depression, which may have future implications for screening and early detection.

In this study, we propose the Ductal Instance-Oriented Pipeline (DIOP) that contains a duct-level instance segmentation model, a tissue-level semantic segmentation model, and three-levels of features for diagnostic classification. Based on recent advancements in instance segmentation and the Mask RCNN model, our duct-level segmenter tries to identify each ductal individual inside a microscopic image; then, it extracts tissue-level information from the identified ductal instances. Leveraging three levels of information obtained from these ductal instances and also the histopathology image, the proposed DIOP outperforms previous approaches (both feature-based and CNN-based) in all diagnostic tasks; for the four-way classification task, the DIOP achieves comparable performance to general pathologists in this unique dataset. The proposed DIOP only takes a few seconds to run in the inference time, which could be used interactively on most modern computers. More clinical explorations are needed to study the robustness and generalizability of this system in the future.

Background: Around 50% of hospital readmissions due to heart failure are preventable, with lack of adherence to prescribed self-care as a driving factor. Remote tracking and reminders issued by mobile health devices could help to promote self-care, which could potentially reduce these readmissions., Objective: We sought to investigate two factors: (1) feasibility of enrolling heart failure patients in a remote monitoring regimen that uses wireless sensors and patient-reported outcome measures; and (2) their adherence to using the study devices and completing patient-reported outcome measures., Methods: Twenty heart failure patients participated in piloting a remote monitoring regimen. Data collection included: (1) physical activity using wrist-worn activity trackers; (2) body weight using bathroom scales; (3) medication adherence using smart pill bottles; and (4) patient -reported outcomes using patient-reported outcome measures., Results: We evaluated 150 hospitalized heart failure patients and enrolled 20 individuals. Two factors contributed to 50% (65/130) being excluded from the study: smartphone ownership and patient discharge. Over the course of the study, 60.0% of the subjects wore the activity tracker for at least 70% of the hours, and 45.0% used the scale for more than 70% of the days. The pill bottle was used less than 10% of the days by 55.0% of the subjects., Conclusions: Our method of recruiting heart failure patients prior to hospital discharge may not be feasible as the enrollment rate was low. Once enrolled, the majority of subjects maintained a high adherence to wearing the activity tracker but low adherence to using the pill bottle and completing the follow-up surveys. Scale usage was fair, but it received positive reviews from most subjects. Given the observed usage and feedback, we suggest mobile health-driven interventions consider including an activity tracker and bathroom scale. We also recommend administering a shorter survey more regularly and through an easier interface., Competing Interests: GCF reports consulting for Abbott, Amgen, Bayer, Janssen, Medtronic, and Novartis.

Purpose: Prostate cancer (PCa) is the most common solid organ cancer and second leading cause of death in men. Multiparametric magnetic resonance imaging (mpMRI) enables detection of the most aggressive, clinically significant PCa (csPCa) tumors that require further treatment. A suspicious region of interest (ROI) detected on mpMRI is now assigned a Prostate Imaging-Reporting and Data System (PIRADS) score to standardize interpretation of mpMRI for PCa detection. However, there is significant inter-reader variability among radiologists in PIRADS score assignment and a minimal input semi-automated artificial intelligence (AI) system is proposed to harmonize PIRADS scores with mpMRI data. Approach: The proposed deep learning model (the seed point model) uses a simulated single-click seed point as input to annotate the lesion on mpMRI. This approach is in contrast to typical medical AI-based approaches that require annotation of the complete lesion. The mpMRI data from 617 patients used in this study were prospectively collected at a major tertiary U.S. medical center. The model was trained and validated to classify whether an mpMRI image had a lesion with a PIRADS score greater than or equal to PIRADS 4. Results: The model yielded an average receiver-operator characteristic (ROC) area under the curve (ROC-AUC) of 0.704 over a 10-fold cross-validation, which is significantly higher than the previously published benchmark. Conclusions: The proposed model could aid in PIRADS scoring of mpMRI, providing second reads to promote quality as well as offering expertise in environments that lack a radiologist with training in prostate mpMRI interpretation. The model could help identify tumors with a higher PIRADS for better clinical management and treatment of PCa patients at an early stage., (© 2020 Society of Photo-Optical Instrumentation Engineers (SPIE).)

Diagnostic radiology (DxR), having had successful serial co-evolutions with imaging equipment and PACS, is faced with another. With a backdrop termed "globotics transition," it should create an IT and informatics infrastructure capable of integrating artificial intelligence (AI) into current critical communication functions of PACS and incorporating functions currently residing in balkanized products. DxR will face the challenge of adopting sustaining and disruptive AI innovations simultaneously. In this co-evolution, a major selection force for AI will be increasing the flow of information and patients; "increasing" means faster flow over larger areas defined by geography and content. Larger content includes a broader spectrum of imaging and nonimaging information streams that facilitate medical decision making. Evolution to faster flow will gravitate toward a hierarchical IT architecture consisting of many small channels feeding into fewer larger channels, something potentially difficult for current PACS. Smartphone-like architecture optimized for communication and integration could provide a large-channel backbone and many smaller feeding channels for basic functions, as well as those needing to innovate rapidly. New, more flexible architectures stimulate market competition in which DxR could act as an artificial selection force to influence development of faster increased flow in current PACS companies, in disruptors such as consolidated AI companies, or in entirely new entrants like Apple or Google. In this co-evolution, DxR should be able to stimulate design of a modern communication medium that increases the flow of information and decreases the time and energy necessary to absorb it, thereby creating even more indispensable clinical value for itself., (Copyright © 2020 American College of Radiology. All rights reserved.)

Objective: To detail the development of an electronic report that graphically conveys all relevant information from targeted prostate biopsy., Methods: The Urology Integrated Diagnostic Report (Uro-IDR) is based on a published framework (RadPath) which enables the compilation of diagnostic data from urology, radiology, and pathology. Each component of the Uro-IDR is generated by the contributing clinician, is assembled in one document, and provides correlation of the 3 inputs at a glance. Upon completion, the Uro-IDR is automatically linked to the electronic medical record as an interactive file and can also be downloaded for offline sharing as a PDF., Results: At our institution, 1638 individual Uro-IDRs were generated between June 2016 and April 2019. There were 5715 views of these documents via the EMR. The average turnaround time for the creation of an individual report decreased from nearly 8 days at the time of its launch to 2 days after 6 months of use. The average time for report generation was 22 seconds for the pathologist and 69 seconds for the radiologist. An instructive video is linked to this article., Conclusion: The Uro-IDR has proven to be a feasible, efficient, clinically useful form to concisely transmit key information about targeted prostate biopsy to both clinicians and patients., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Constructing statistical models using personal sensor data could allow for tracking health status over time, thereby enabling the possibility of early intervention. The goal of this study was to use machine learning algorithms to classify patient-reported outcomes (PROs) using activity tracker data in a cohort of patients with stable ischemic heart disease (SIHD). A population of 182 patients with SIHD were monitored over a period of 12 weeks. Each subject received a Fitbit Charge 2 device to record daily activity data, and each subject completed eight Patient-Reported Outcomes Measurement Information Systems short form at the end of each week as a self-assessment of their health status. Two models were built to classify PRO scores using activity tracker data. The first model treated each week independently, whereas the second used a hidden Markov model (HMM) to take advantage of correlations between successive weeks. Retrospective analysis compared the classification accuracy of the two models and the importance of each feature. In the independent model, a random forest classifier achieved a mean area under curve (AUC) of 0.76 for classifying the physical function PRO. The HMM model achieved significantly better AUCs for all PROs (p < 0.05) other than Fatigue and Sleep Disturbance, with a highest mean AUC of 0.79 for the physical function-short form 10a. Our study demonstrates the ability of activity tracker data to classify health status over time. These results suggest that patient outcomes can be monitored in real time using activity trackers.

We describe the protocol, design, and methodology of the Prediction, Risk, and Evaluation of Major Adverse Cardiac Events (PRE-MACE) study as a multicomponent remote patient monitoring in cardiology. Using biosensor, biomarkers, and patient-reported outcomes in participants with stable ischemic heart disease, the PRE-MACE study is designed to measure cross-sectional correlations and establish the ability of remote monitoring to predict major adverse cardiovascular event (MACE) biomarkers and incident MACE at baseline and 12-month follow-up. It will further assess the adherence and cost-effectiveness of remote monitoring and blood sampling over the initial months. Despite medication and lifestyle changes, patients with cardiovascular disease can experience MACE due to undertreatment, poor adherence, or failure to recognize clinical or biochemical changes that presage MACE. Identifying patients using remote monitoring to detect MACE forerunners has potential to improve outcomes, avoid MACE, and reduce resource utilization. Data collection will include: (1) continuous remote monitoring using wearable biosensors; (2) biomarker measurements using plasma and at-home micro-sampling blood collection; and (3) patient-reported outcomes to monitor perceived stress, anxiety, depression, and health-related quality of life. Two hundred participants will be followed for 90 days with a subset ( n  = 80) monitored for 180 days. All participants will be followed up for MACE at 12 months.The PRE-MACE study will utilize remote monitoring with biosensors, biomarkers, and patient-reported outcomes to identify intermediate biomarkers of MACE in patients with stable ischemic heart disease. If shown to be effective, this intervention can be utilized between health visits to predict MACE and reduce financial impact of MACE., Competing Interests: Competing interestsThe authors declare no competing interests.

Social media presents a rich opportunity to gather health information with limited intervention through the analysis of completely unstructured and unlabeled microposts. We sought to estimate the health-related quality of life (HRQOL) of Twitter users using automated semantic processing methods. We collected tweets from 878 Twitter users recruited through online solicitation and in-person contact with patients. All participants completed the four-item Centers for Disease Control Healthy Days Questionnaire at the time of enrollment and 30 days later to measure "ground truth" HRQOL. We used a combination of document frequency analysis, sentiment analysis, topic analysis, and concept mapping to extract features from tweets, which we then used to estimate dichotomized HRQOL ("high" vs. "low") using logistic regression. Binary HRQOL status was estimated with moderate performance (AUC = 0.64). This result indicates that free-range social media data only offers a window into HRQOL, but does not afford direct access to current health status.

Current clinical practice relies on clinical history to determine the time since stroke (TSS) onset. Imaging-based determination of acute stroke onset time could provide critical information to clinicians in deciding stroke treatment options, such as thrombolysis. The patients with unknown or unwitnessed TSS are usually excluded from thrombolysis, even if their symptoms began within the therapeutic window. In this paper, we demonstrate a machine learning approach for TSS classification using routinely acquired imaging sequences. We develop imaging features from the magnetic resonance (MR) images and train machine learning models to classify the TSS. We also propose a deep-learning model to extract hidden representations for the MR perfusion-weighted images and demonstrate classification improvement by incorporating these additional deep features. The cross-validation results show that our best classifier achieved an area under the curve of 0.765, with a sensitivity of 0.788 and a negative predictive value of 0.609, outperforming existing methods. We show that the features generated by our deep-learning algorithm correlate with the MR imaging features, and validate the robustness of the model on imaging parameter variations (e.g., year of imaging). This paper advances magnetic resonance imaging analysis one-step-closer to an operational decision support tool for stroke treatment guidance.

Prostate cancer is the most common and second most deadly form of cancer in men in the United States. The classification of prostate cancers based on Gleason grading using histological images is important in risk assessment and treatment planning for patients. Here, we demonstrate a new region-based convolutional neural network framework for multi-task prediction using an epithelial network head and a grading network head. Compared with a single-task model, our multi-task model can provide complementary contextual information, which contributes to better performance. Our model is achieved a state-of-the-art performance in epithelial cells detection and Gleason grading tasks simultaneously. Using fivefold cross-validation, our model is achieved an epithelial cells detection accuracy of 99.07% with an average area under the curve of 0.998. As for Gleason grading, our model is obtained a mean intersection over union of 79.56% and an overall pixel accuracy of 89.40%.

Automated Gleason grading is an important preliminary step for quantitative histopathological feature extraction. Different from the traditional task of classifying small pre-selected homogeneous regions, semantic segmentation provides pixel-wise Gleason predictions across an entire slide. Deep learning-based segmentation models can automatically learn visual semantics from data, which alleviates the need for feature engineering. However, performance of deep learning models is limited by the scarcity of large-scale fully annotated datasets, which can be both expensive and time-consuming to create. One way to address this problem is to leverage external weakly labeled datasets to augment models trained on the limited data. In this paper, we developed an expectation maximization-based approach constrained by an approximated prior distribution in order to extract useful representations from a large number of weakly labeled images generated from low-magnification annotations. This method was utilized to improve the performance of a model trained on a limited fully annotated dataset. Our semi-supervised approach trained with 135 fully annotated and 1800 weakly annotated tiles achieved a mean Jaccard Index of 49.5% on an independent test set, which was 14% higher than the initial model trained only on the fully annotated dataset., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

Gleason grading of histological images is important in risk assessment and treatment planning for prostate cancer patients. Much research has been done in classifying small homogeneous cancer regions within histological images. However, semi-supervised methods published to date depend on pre-selected regions and cannot be easily extended to an image of heterogeneous tissue composition. In this paper, we propose a multi-scale U-Net model to classify images at the pixel-level using 224 histological image tiles from radical prostatectomies of 20 patients. Our model was evaluated by a patient-based 10-fold cross validation, and achieved a mean Jaccard index of 65.8% across 4 classes (stroma, Gleason 3, Gleason 4 and benign glands), and 75.5% for 3 classes (stroma, benign glands, prostate cancer), outperforming other methods.