Search

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

This paper addresses the challenge of grading visual features in lumbar spine MRI using Deep Learning. Such a method is essential for the automatic quantification of structural changes in the spine, which is valuable for understanding low back pain. Multiple recent studies investigated different architecture designs, and the most recent success has been attributed to the use of transformer architectures. In this work, we argue that with a well-tuned three-stage pipeline comprising semantic segmentation, localization, and classification, convolutional networks outperform the state-of-the-art approaches. We conducted an ablation study of the existing methods in a population cohort, and report performance generalization across various subgroups. Our code is publicly available to advance research on disc degeneration and low back pain., Comment: 5 pages, under review

The eigendecomposition of a matrix is the central procedure in probabilistic models based on matrix factorization, for instance principal component analysis and topic models. Quantifying the uncertainty of such a decomposition based on a finite sample estimate is essential to reasoning under uncertainty when employing such models. This paper tackles the challenge of computing confidence bounds on the individual entries of eigenvectors of a covariance matrix of fixed dimension. Moreover, we derive a method to bound the entries of the inverse covariance matrix, the so-called precision matrix. The assumptions behind our method are minimal and require that the covariance matrix exists, and its empirical estimator converges to the true covariance. We make use of the theory of U-statistics to bound the $L_2$ perturbation of the empirical covariance matrix. From this result, we obtain bounds on the eigenvectors using Weyl's theorem and the eigenvalue-eigenvector identity and we derive confidence intervals on the entries of the precision matrix using matrix inversion perturbation bounds. As an application of these results, we demonstrate a new statistical test, which allows us to test for non-zero values of the precision matrix. We compare this test to the well-known Fisher-z test for partial correlations, and demonstrate the soundness and scalability of the proposed statistical test, as well as its application to real-world data from medical and physics domains., arXiv admin note: text overlap with arXiv:1604.01733

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflect the domain interest, thus failing to adequately measure scientific progress and hindering translation of ML techniques into practice. To overcome this, our large international expert consortium created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint - a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), data set and algorithm output. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as a classification task at image, object or pixel level, namely image-level classification, object detection, semantic segmentation, and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool, which also provides a point of access to explore weaknesses, strengths and specific recommendations for the most common validation metrics. The broad applicability of our framework across domains is demonstrated by an instantiation for various biological and medical image analysis use cases., Shared first authors: Lena Maier-Hein, Annika Reinke

This paper tackles the challenge of forensic medical image matching (FMIM) using deep neural networks (DNNs). FMIM is a particular case of content-based image retrieval (CBIR). The main challenge in FMIM compared to the general case of CBIR, is that the subject to whom a query image belongs may be affected by aging and progressive degenerative disorders, making it difficult to match data on a subject level. CBIR with DNNs is generally solved by minimizing a ranking loss, such as Triplet loss (TL), computed on image representations extracted by a DNN from the original data. TL, in particular, operates on triplets: anchor, positive (similar to anchor) and negative (dissimilar to anchor). Although TL has been shown to perform well in many CBIR tasks, it still has limitations, which we identify and analyze in this work. In this paper, we introduce (i) the AdaTriplet loss -- an extension of TL whose gradients adapt to different difficulty levels of negative samples, and (ii) the AutoMargin method -- a technique to adjust hyperparameters of margin-based losses such as TL and our proposed loss dynamically. Our results are evaluated on two large-scale benchmarks for FMIM based on the Osteoarthritis Initiative and Chest X-ray-14 datasets. The codes allowing replication of this study have been made publicly available at \url{https://github.com/Oulu-IMEDS/AdaTriplet}., 15 pages, 6 figures, accepted as a conference paper at MICCAI 2022

This paper tackles the problem of developing active learning (AL) methods in the context of knee osteoarthritis (OA) diagnosis from X-ray images. OA is known to be a huge burden for society, and its associated costs are constantly rising. Automatic diagnostic methods can potentially reduce these costs, and Deep Learning (DL) methodology may be its key enabler. To date, there have been numerous studies on knee OA severity grading using DL, and all but one of them assume a large annotated dataset available for model development. In contrast, our study shows one can develop a knee OA severity grading model using AL from as little as 50 samples randomly chosen from a pool of unlabeled data. The main insight of this work is that the performance of AL improves when the model developer leverages the consistency regularization technique, commonly applied in semi-supervised learning.

In medical applications, deep learning methods are designed to automate diagnostic tasks. However, a clinically relevant question that practitioners usually face, is how to predict the future trajectory of a disease (prognosis). Current methods for such a problem often require domain knowledge, and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many forecasting problem from multimodal data. Inspired by a clinical decision-making process with two agents — a radiologist and a general practitioner, we model a prognosis prediction problem with two transformer-based components that share information between each other. The first block in this model aims to analyze the imaging data, and the second block leverages the internal representations of the first one as inputs, also fusing them with auxiliary patient data. We show the effectiveness of our method in predicting the development of structural knee osteoarthritis changes over time. Our results show that the proposed method outperforms the state-of-the-art baselines in terms of various performance metrics. In addition, we empirically show that the existence of the multi-agent transformers with depths of 2 is sufficient to achieve good performances. Our code is publicly available at https://github.com/MIPT-Oulu/CLIMAT.

Objective: To investigate the value of ultrasonographic data in predicting total knee replacement (TKR). Design: Data from the Musculoskeletal Pain in Ullensaker study (MUST) was linked to the Norwegian Arthroplasty Register to form a 5–7 year prospective cohort study of 630 persons (69% women, mean (SD) age 64 (8.7) years). We examined the predictive power of ultrasound (US) features, i.e. osteophytes, meniscal extrusion, synovitis in the suprapatellar recess, femoral cartilage thickness, and quality for future knee osteoarthritis (OA) surgery. We investigated 4 main settings for multivariate predictive modeling: 1) clinical predictors (age, sex, body mass index, knee injury, familial OA and workload), 2) radiographic data (assessed by the Kellgren Lawrence grade, KL) with clinical predictors, 3) US features and clinical predictors. Finally, we also considered an ensemble of models 2) and 3) and used it as our fifth model. All models were compared using the Average Precision (AP) and the Area Under Receiver Operating Characteristic Curve (AUC) metrics. Results: Clinical predictors yielded AP of 0.11 (95% confidence interval [CI] 0.05–0.23) and AUC of 0.69 (0.58–0.79). Clinical predictors with KL grade yielded AP of 0.20 (0.12–0.33) and AUC of 0.81 (0.67–0.90). The clinical variables with ultrasound yielded AP of 0.17 (0.08–0.30) and AUC of 0.79 (0.69–0.86). Conclusions: Ultrasonographic examination of the knee may provide added value to basic clinical and demographic descriptors when predicting TKR. While it does not achieve the same predictive performance as radiography, it can provide additional value to the radiographic examination.

Accurate prediction of knee osteoarthritis (KOA) progression from structural MRI has a potential to enhance disease understanding and support clinical trials. Prior art focused on manually designed imaging biomarkers, which may not fully exploit all disease-related information present in MRI scan. In contrast, our method learns relevant representations from raw data end-to-end using Deep Learning, and uses them for progression prediction. The method employs a 2D CNN to process the data slice-wise and aggregate the extracted features using a Transformer. Evaluated on a large cohort (n=4,866), the proposed method outperforms conventional 2D and 3D CNN-based models and achieves average precision of $0.58\pm0.03$ and ROC AUC of $0.78\pm0.01$. This paper sets a baseline on end-to-end KOA progression prediction from structural MRI. Our code is publicly available at https://github.com/MIPT-Oulu/OAProgressionMR., $\copyright$ 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Active learning is usually applied to acquire labels of informative data points in supervised learning, to maximize accuracy in a sample-efficient way. However, maximizing the accuracy is not the end goal when the results are used for decision-making, for example in personalized medicine or economics. We argue that when acquiring samples sequentially, separating learning and decision-making is sub-optimal, and we introduce an active learning strategy which takes the down-the-line decision problem into account. Specifically, we introduce a novel active learning criterion which maximizes the expected information gain on the posterior distribution of the optimal decision. We compare our targeted active learning strategy to existing alternatives on both simulated and real data, and show improved performance in decision-making accuracy.

The treatment of patients with ERBB2 (HER2)-positive breast cancer with anti-ERBB2 therapy is based on the detection of ERBB2 gene amplification or protein overexpression. Machine learning (ML) algorithms can predict the amplification of ERBB2 based on tumor morphological features, but it is not known whether ML-derived features can predict survival and efficacy of anti-ERBB2 treatment. In this study, we trained a deep learning model with digital images of hematoxylin–eosin (H&E)-stained formalin-fixed primary breast tumor tissue sections, weakly supervised by ERBB2 gene amplification status. The gene amplification was determined by chromogenic in situ hybridization (CISH). The training data comprised digitized tissue microarray (TMA) samples from 1,047 patients. The correlation between the deep learning–predicted ERBB2 status, which we call H&E-ERBB2 score, and distant disease-free survival (DDFS) was investigated on a fully independent test set, which included whole-slide tumor images from 712 patients with trastuzumab treatment status available. The area under the receiver operating characteristic curve (AUC) in predicting gene amplification in the test sets was 0.70 (95% CI, 0.63–0.77) on 354 TMA samples and 0.67 (95% CI, 0.62–0.71) on 712 whole-slide images. Among patients with ERBB2-positive cancer treated with trastuzumab, those with a higher than the median morphology–based H&E-ERBB2 score derived from machine learning had more favorable DDFS than those with a lower score (hazard ratio [HR] 0.37; 95% CI, 0.15–0.93; P = 0.034). A high H&E-ERBB2 score was associated with unfavorable survival in patients with ERBB2-negative cancer as determined by CISH. ERBB2-associated morphology correlated with the efficacy of adjuvant anti-ERBB2 treatment and can contribute to treatment-predictive information in breast cancer. publishedVersion

Wrist Fracture is the most common type of fracture with a high incidence rate. Conventional radiography (i.e. X-ray imaging) is used for wrist fracture detection routinely, but occasionally fracture delineation poses issues and an additional confirmation by computed tomography (CT) is needed for diagnosis. Recent advances in the field of Deep Learning (DL), a subfield of Artificial Intelligence (AI), have shown that wrist fracture detection can be automated using Convolutional Neural Networks. However, previous studies did not pay close attention to the difficult cases which can only be confirmed via CT imaging. In this study, we have developed and analyzed a state-of-the-art DL-based pipeline for wrist (distal radius) fracture detection—DeepWrist, and evaluated it against one general population test set, and one challenging test set comprising only cases requiring confirmation by CT. Our results reveal that a typical state-of-the-art approach, such as DeepWrist, while having a near-perfect performance on the general independent test set, has a substantially lower performance on the challenging test set—average precision of 0.99 (0.99–0.99) versus 0.64 (0.46–0.83), respectively. Similarly, the area under the ROC curve was of 0.99 (0.98–0.99) versus 0.84 (0.72–0.93), respectively. Our findings highlight the importance of a meticulous analysis of DL-based models before clinical use, and unearth the need for more challenging settings for testing medical AI systems.

To evaluate the acoustic emissions (AE) and kinematic instability (KI) of the osteoarthritic (OA) knee joints, and to compare these signals to radiographic findings. Sixty-six female and 43 male participants aged 44–67 were recruited. On radiography, joint-space narrowing, osteophytes and Kellgren–Lawrence (KL) grade were evaluated. Based on radiography, 54 subjects (the study group) were diagnosed with radiographic OA (KL-grade ≥ 2) while the remaining 55 subjects (KL-grade

Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In this study, we propose the Semixup algorithm, a semi-supervised learning (SSL) approach to leverage unlabeled data. Semixup relies on consistency regularization using in- and out-of-manifold samples, together with interpolated consistency. On an independent test set, our method significantly outperformed other state-of-the-art SSL methods in most cases. Finally, when compared to a well-tuned fully supervised baseline that yielded a balanced accuracy (BA) of $70.9\pm0.8%$ on the test set, Semixup had comparable performance -- BA of $71\pm0.8%$ $(p=0.368)$ while requiring $6$ times less labeled data. These results show that our proposed SSL method allows building fully automatic OA severity assessment tools with datasets that are available outside research settings., 11 main, 03 supplementary pages. The manuscript was accepted to IEEE Transactions on Medical Imaging in August 2020

Osteoarthritis (OA) is the most common musculoskeletal disorder in the world, affecting hand, hip, and knee joints. At the final stage, OA leads to joint replacement, causing an immense burden at the individual and societal levels. Multiple risk factors that can lead to OA are known; however, the etiology of OA and the underlying mechanisms of OA progression are not currently known. OA is currently diagnosed by a clinical examination and, when necessary, confirmed by imaging — a radiographic evaluation. However, these conventional tools are not sensitive to detect the early stages of OA, which makes the development of preventive measures for further disease progression difficult. Therefore, there is a need for other methods that could allow for the early diagnosis of OA. As such, computer vision-based techniques provide quantitative biomarkers that allow for an automatic and systematic assessment of OA severity from images. In recent years, the rapid development of computer vision and machine learning methods have merged into a new field — deep learning (DL). DL allows for one to formulate the problems of computer vision and other fields in a machine learning fashion. In the medical field, DL has made a tremendous impact and allowed to approach for human-level decision-making accuracy in diagnostic and prognostic tasks compared with the traditional computer vision-based methods. The focus of this thesis is on the development of DL-based methods for fully automatic knee OA severity diagnosis and the prediction of its progression. Multiple new methods for localizing the region of interest, landmark localization, knee OA severity assessment, and OA progression prediction are proposed. The results exceeded the state-of-the-art or formed completely new benchmarks for the evaluation of diagnostic and predictive model performance in OA. The main conclusion is that DL yields excellent performance in the diagnostics of OA and in the prediction of its progression. All the source codes of all the developed methods and the annotations for some of the datasets have been made publicly available. Tiivistelmä Nivelrikko on maailman yleisin käden, lonkan ja polven niveliin vaikuttava liikuntaelinsairaus. Viimekädessä nivelrikko johtaa tekonivelleikkauksiin, aiheuttaen merkittävää rasitetta niin yksilö- kuin yhteiskunnallisella tasolla. Monia nivelrikolle altistavia tekijöitä on jo tunnistettu, mutta kaikkia nivelrikon syitä ja vaikutusmekanismeja nivelrikon etenemisessä ei tunneta. Nivelrikko diagnosoidaan kliinisellä tutkimuksella ja vahvistetaan/varmistetaan tarvittaessa tehtävällä kuvantamistutkimuksella — tekemällä radiografinen arviointi. Nämä perinteiset työkalut eivät kuitenkaan ole riittävän herkkiä nivelrikon varhaisten vaiheiden havaitsemiseen, ja tämä hankaloittaa sairauden kehittymistä ehkäisevien toimenpiteiden kehittämistä. Näistä syistä johtuen tarvitaan muita menetelmiä, jotka mahdollistavat nivelrikon varhaisen diagnosoinnin. Konenäkömenetelmät sellaisenaan tuottavat kvantitatiivisia biologisia indikaattoreita jotka mahdollistavat automaattisen ja järjestelmällisen nivelrikon vakavuusarvion tekemisen kuvamateriaalista. Viime vuosina konenäkö- ja koneoppimismenetelmien nopea kehitys on synnyttänyt uuden syväoppimisen haaran. Syväoppiminen mahdollistaa konenäkö- ja muiden ongelmien määrittelyn koneoppimisongelman tavoin. Verrattuna perinteisiin lääketieteessä käytettyihin tietokonenäkömenetelmiin, syväoppiminen on mahdollistanut ihmisen suorituskykyä lähestyvät toteutukset lääketieteen diagnostisissa ja prognostisissa tehtävissä ja niiden vaikutus alan kehitykselle on ollut merkittävä. Tämän väitöskirja keskittyy kehittämään syväoppimismenetelmiä täysautomaattiseen polven nivelrikon vakavuuden diagnosointiin ja taudin kehittymisen ennustamiseen. Työssä ehdotetaan/esitetään useita uusia menetelmiä kohdealueen paikallistamiseen, maamerkkien paikallistamiseen, polven nivelrikon vakavuuden arviointiin ja nivelrikon etenemisen ennustamiseen. Työn tulokset ylittävät viimeisintä tekniikkaa edustavat ratkaisut tai muodostavat täysin uuden mittarin diagnostisten ja ennustavien menetelmien suorituskyvyn evaluoinnille nivelrikon kontekstissa. Työn keskeisimpänä johtopäätöksenä esitetään, että syväoppimisella on mahdollista saavuttaa erittäin hyvä suorituskyky nivelrikon diagnosoinnissa ja sen etenemisen ennustamisessa. Kaikki työssä kehitetyt menetelmät lähdekoodeineen sekä annotoinnit osalle tutkimuksessa käytetyistä aineistoista on saatettu avoimesti saataville.

Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In this study, we propose the Semixup algorithm, a semi-supervised learning (SSL) approach to leverage unlabeled data. Semixup relies on consistency regularization using in- and out-of-manifold samples, together with interpolated consistency. On an independent test set, our method significantly outperformed other state-of-the-art SSL methods in most cases. Finally, when compared to a well-tuned fully supervised baseline that yielded a balanced accuracy (BA) of 70.9 ± 0.8% on the test set, Semixup had comparable performance - BA of 71 ± 0.8% (p = 0.368) while requiring 6 times less labeled data. These results show that our proposed SSL method allows building fully automatic OA severity assessment tools with datasets that are available outside research settings.

Background: In horses, osteoarthritis (OA) mostly affects metacarpophalangeal and metatarsophalangeal (fetlock) joints. The current modalities used for diagnosis of equine limb disorders lack ability to detect early OA. Here, we propose a new alternative approach to assess experimental cartilage damage in fetlock joint using Acoustic Emissions (AE). Objectives: To evaluate the potential of AE technique in diagnosing OA and see how AE signals changes with increasing severity of OA. Study design: An in vitro experimental study. Methods: A total of 16 distal limbs (8 forelimbs and 8 hindlimbs) from six Finn horses were collected from an abattoir and fitted in a custom‐made frame allowing fetlock joint bending. Eight fetlock joints were opened, and cartilage surface was progressively damaged mechanically three times using sandpaper to mimic mild, moderate and severe OA. The remaining eight fetlock joints were opened and closed without any mechanical procedure, serving as controls. Before cartilage alteration, synovial fluid was aspirated, mixed with phosphate‐buffered saline solution, and then reinjected before suturing for constant joint lubrication. For each simulated condition of OA severity, a force was applied to the frame and then released to mimic joint flexion and extension. AE signals were acquired using air microphones. Results: A strong association was found between the joint condition and the power of AE signals analysed in 1.5–6 kHz range. The signal from both forelimb and hindlimb joints followed a similar pattern for increased cartilage damage. There were statistically significant differences between each joint condition progressively (generalised linear mixed model, P

ObjectiveTo develop and validate a machine learning (ML) approach for automatic three-dimensional (3D) histopathological grading of osteochondral samples imaged with contrast-enhanced micro-computed tomography (CEμCT).DesignOsteochondral cores from 24 total knee arthroplasty patients and 2 asymptomatic cadavers (n = 34, Ø = 2 mm; n = 45, Ø = 4 mm) were imaged using CEμCT with phosphotungstic acid-staining. Volumes-of-interest (VOI) in surface (SZ), deep (DZ) and calcified (CZ) zones were extracted depthwise and subjected to dimensionally reduced Local Binary Pattern-textural feature analysis. Regularized Ridge and Logistic regression (LR) models were trained zone-wise against the manually assessed semi-quantitative histopathological CEμCT grades (Ø = 2 mm samples). Models were validated using nested leave-one-out cross-validation and an independent test set (Ø = 4 mm samples). The performance was assessed using Spearman’s correlation, Average Precision (AP) and Area under the Receiver Operating Characteristic Curve (AUC).ResultsHighest performance on cross-validation was observed for SZ, both on Ridge regression (ρ = 0.68, p < 0.0001) and LR (AP = 0.89, AUC = 0.92). The test set evaluations yielded decreased Spearman’s correlations on all zones. For LR, performance was almost similar in SZ (AP = 0.89, AUC = 0.86), decreased in CZ (AP = 0.71→0.62, AUC = 0.77→0.63) and increased in DZ (AP = 0.50→0.83, AUC = 0.72→0.72).ConclusionWe showed that the ML-based automatic 3D histopathological grading of osteochondral samples is feasible from CEμCT. The developed method can be directly applied by OA researchers since the grading software and all source codes are publicly available.

Key PointsQuestionIs the gap between brain age predicted from MRI and chronological age associated with incident dementia in a general population of Dutch adults?FindingsBrain age was predicted using a deep learning model, using MRI-derived grey matter density maps. In a population based study including 5496 participants, the observed gap was significantly associated with the risk of dementia.MeaningThe gap between MRI-brain predicted and chronological age is potentially a biomarker for dementia risk screening.AbstractImportanceThe gap between predicted brain age using magnetic resonance imaging (MRI) and chronological age may serve as biomarker for early-stage neurodegeneration and potentially as a risk indicator for dementia. However, owing to the lack of large longitudinal studies, it has been challenging to validate this link.ObjectiveWe aimed to investigate the utility of such a gap as a risk biomarker for incident dementia in a general Dutch population, using a deep learning approach for predicting brain age based on MRI-derived grey matter maps.DesignData was collected from participants of the cohort-based Rotterdam Study who underwent brain magnetic resonance imaging between 2006 and 2015. This study was performed in a longitudinal setting and all participant were followed up for incident dementia until 2016.SettingThe Rotterdam Study is a prospective population-based study, initiated in 1990 in the suburb Ommoord of in Rotterdam, the Netherlands.ParticipantsAt baseline, 5496 dementia- and stroke-free participants (mean age 64.67±9.82, 54.73% women) were scanned and screened for incident dementia. During 6.66±2.46 years of follow-up, 159 people developed dementia.Main outcomes and measuresWe built a convolutional neural network (CNN) model to predict brain age based on its MRI. Model prediction performance was measured in mean absolute error (MAE). Reproducibility of prediction was tested using the intraclass correlation coefficient (ICC) computed on a subset of 80 subjects. Logistic regressions and Cox proportional hazards were used to assess the association of the age gap with incident dementia, adjusted for years of education, ApoEε4 allele carriership, grey matter volume and intracranial volume. Additionally, we computed the attention maps of CNN, which shows which brain regions are important for age prediction.ResultsMAE of brain age prediction was 4.45±3.59 years and ICC was 0.97 (95% confidence interval CI=0.96-0.98). Logistic regression and Cox proportional hazards models showed that the age gap was significantly related to incident dementia (odds ratio OR=1.11 and 95% confidence intervals CI=1.05-1.16; hazard ratio HR=1.11 and 95% CI=1.06-1.15, respectively). Attention maps indicated that grey matter density around the amygdalae and hippocampi primarily drive the age estimation.Conclusion and relevanceWe show that the gap between predicted and chronological brain age is a biomarker associated with risk of dementia development. This suggests that it can be used as a biomarker, complimentary to those that are known, for dementia risk screening.

Breast cancer is one of the main causes of death worldwide. Histopathological cellularity assessment of residual tumors in post-surgical tissues is used to analyze a tumor’s response to a therapy. Correct cellularity assessment increases the chances of getting an appropriate treatment and facilitates the patient’s survival. In current clinical practice, tumor cellularity is manually estimated by pathologists; this process is tedious and prone to errors or low agreement rates between assessors. In this work, we evaluated three strong novel Deep Learning-based approaches for automatic assessment of tumor cellularity from post-treated breast surgical specimens stained with hematoxylin and eosin. We validated the proposed methods on the BreastPathQ SPIE challenge dataset that consisted of 2395 image patches selected from whole slide images acquired from 64 patients. Compared to expert pathologist scoring, our best performing method yielded the Cohen’s kappa coefficient of 0.69 (vs. 0.42 previously known in literature) and the intra-class correlation coefficient of 0.89 (vs. 0.83). Our results suggest that Deep Learning-based methods have a significant potential to alleviate the burden on pathologists, enhance the diagnostic workflow, and, thereby, facilitate better clinical outcomes in breast cancer treatment.

Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition settings remains an open problem. In this study, we investigated two modern regularization techniques — mixup and adversarial unsupervised domain adaptation (UDA) — to improve the robustness of DL-based knee cartilage segmentation to new MRI acquisition settings. Our validation setup included two datasets produced by different MRI scanners and using distinct data acquisition protocols. We assessed the robustness of automatic segmentation by comparing mixup and UDA approaches to a strong baseline method at different OA severity stages and, additionally, in relation to anatomical locations. Our results showed that for moderate changes in knee MRI data acquisition settings both approaches may provide notable improvements in the robustness, which are consistent for all stages of the disease and affect the clinically important areas of the knee joint. However, mixup may be considered as a recommended approach, since it is more computationally efficient and does not require additional data from the target acquisition setup.

This paper addresses the challenge of localization of anatomical landmarks in knee X-ray images at different stages of osteoarthritis (OA). Landmark localization can be viewed as regression problem, where the landmark position is directly predicted by using the region of interest or even full-size images leading to large memory footprint, especially in case of high resolution medical images. In this work, we propose an efficient deep neural networks framework with an hourglass architecture utilizing a soft-argmax layer to directly predict normalized coordinates of the landmark points. We provide an extensive evaluation of different regularization techniques and various loss functions to understand their influence on the localization performance. Furthermore, we introduce the concept of transfer learning from low-budget annotations, and experimentally demonstrate that such approach is improving the accuracy of landmark localization. Compared to the prior methods, we validate our model on two datasets that are independent from the train data and assess the performance of the method for different stages of OA severity. The proposed approach demonstrates better generalization performance compared to the current state-of-the-art.