Search

Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In this study, we propose the Semixup algorithm, a semi-supervised learning (SSL) approach to leverage unlabeled data. Semixup relies on consistency regularization using in- and out-of-manifold samples, together with interpolated consistency. On an independent test set, our method significantly outperformed other state-of-the-art SSL methods in most cases. Finally, when compared to a well-tuned fully supervised baseline that yielded a balanced accuracy (BA) of $70.9\pm0.8%$ on the test set, Semixup had comparable performance -- BA of $71\pm0.8%$ $(p=0.368)$ while requiring $6$ times less labeled data. These results show that our proposed SSL method allows building fully automatic OA severity assessment tools with datasets that are available outside research settings., 11 main, 03 supplementary pages. The manuscript was accepted to IEEE Transactions on Medical Imaging in August 2020

Osteoarthritis (OA) is the most common musculoskeletal disorder in the world, affecting hand, hip, and knee joints. At the final stage, OA leads to joint replacement, causing an immense burden at the individual and societal levels. Multiple risk factors that can lead to OA are known; however, the etiology of OA and the underlying mechanisms of OA progression are not currently known. OA is currently diagnosed by a clinical examination and, when necessary, confirmed by imaging — a radiographic evaluation. However, these conventional tools are not sensitive to detect the early stages of OA, which makes the development of preventive measures for further disease progression difficult. Therefore, there is a need for other methods that could allow for the early diagnosis of OA. As such, computer vision-based techniques provide quantitative biomarkers that allow for an automatic and systematic assessment of OA severity from images. In recent years, the rapid development of computer vision and machine learning methods have merged into a new field — deep learning (DL). DL allows for one to formulate the problems of computer vision and other fields in a machine learning fashion. In the medical field, DL has made a tremendous impact and allowed to approach for human-level decision-making accuracy in diagnostic and prognostic tasks compared with the traditional computer vision-based methods. The focus of this thesis is on the development of DL-based methods for fully automatic knee OA severity diagnosis and the prediction of its progression. Multiple new methods for localizing the region of interest, landmark localization, knee OA severity assessment, and OA progression prediction are proposed. The results exceeded the state-of-the-art or formed completely new benchmarks for the evaluation of diagnostic and predictive model performance in OA. The main conclusion is that DL yields excellent performance in the diagnostics of OA and in the prediction of its progression. All the source codes of all the developed methods and the annotations for some of the datasets have been made publicly available. Tiivistelmä Nivelrikko on maailman yleisin käden, lonkan ja polven niveliin vaikuttava liikuntaelinsairaus. Viimekädessä nivelrikko johtaa tekonivelleikkauksiin, aiheuttaen merkittävää rasitetta niin yksilö- kuin yhteiskunnallisella tasolla. Monia nivelrikolle altistavia tekijöitä on jo tunnistettu, mutta kaikkia nivelrikon syitä ja vaikutusmekanismeja nivelrikon etenemisessä ei tunneta. Nivelrikko diagnosoidaan kliinisellä tutkimuksella ja vahvistetaan/varmistetaan tarvittaessa tehtävällä kuvantamistutkimuksella — tekemällä radiografinen arviointi. Nämä perinteiset työkalut eivät kuitenkaan ole riittävän herkkiä nivelrikon varhaisten vaiheiden havaitsemiseen, ja tämä hankaloittaa sairauden kehittymistä ehkäisevien toimenpiteiden kehittämistä. Näistä syistä johtuen tarvitaan muita menetelmiä, jotka mahdollistavat nivelrikon varhaisen diagnosoinnin. Konenäkömenetelmät sellaisenaan tuottavat kvantitatiivisia biologisia indikaattoreita jotka mahdollistavat automaattisen ja järjestelmällisen nivelrikon vakavuusarvion tekemisen kuvamateriaalista. Viime vuosina konenäkö- ja koneoppimismenetelmien nopea kehitys on synnyttänyt uuden syväoppimisen haaran. Syväoppiminen mahdollistaa konenäkö- ja muiden ongelmien määrittelyn koneoppimisongelman tavoin. Verrattuna perinteisiin lääketieteessä käytettyihin tietokonenäkömenetelmiin, syväoppiminen on mahdollistanut ihmisen suorituskykyä lähestyvät toteutukset lääketieteen diagnostisissa ja prognostisissa tehtävissä ja niiden vaikutus alan kehitykselle on ollut merkittävä. Tämän väitöskirja keskittyy kehittämään syväoppimismenetelmiä täysautomaattiseen polven nivelrikon vakavuuden diagnosointiin ja taudin kehittymisen ennustamiseen. Työssä ehdotetaan/esitetään useita uusia menetelmiä kohdealueen paikallistamiseen, maamerkkien paikallistamiseen, polven nivelrikon vakavuuden arviointiin ja nivelrikon etenemisen ennustamiseen. Työn tulokset ylittävät viimeisintä tekniikkaa edustavat ratkaisut tai muodostavat täysin uuden mittarin diagnostisten ja ennustavien menetelmien suorituskyvyn evaluoinnille nivelrikon kontekstissa. Työn keskeisimpänä johtopäätöksenä esitetään, että syväoppimisella on mahdollista saavuttaa erittäin hyvä suorituskyky nivelrikon diagnosoinnissa ja sen etenemisen ennustamisessa. Kaikki työssä kehitetyt menetelmät lähdekoodeineen sekä annotoinnit osalle tutkimuksessa käytetyistä aineistoista on saatettu avoimesti saataville.

Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In this study, we propose the Semixup algorithm, a semi-supervised learning (SSL) approach to leverage unlabeled data. Semixup relies on consistency regularization using in- and out-of-manifold samples, together with interpolated consistency. On an independent test set, our method significantly outperformed other state-of-the-art SSL methods in most cases. Finally, when compared to a well-tuned fully supervised baseline that yielded a balanced accuracy (BA) of 70.9 ± 0.8% on the test set, Semixup had comparable performance - BA of 71 ± 0.8% (p = 0.368) while requiring 6 times less labeled data. These results show that our proposed SSL method allows building fully automatic OA severity assessment tools with datasets that are available outside research settings.

Background: In horses, osteoarthritis (OA) mostly affects metacarpophalangeal and metatarsophalangeal (fetlock) joints. The current modalities used for diagnosis of equine limb disorders lack ability to detect early OA. Here, we propose a new alternative approach to assess experimental cartilage damage in fetlock joint using Acoustic Emissions (AE). Objectives: To evaluate the potential of AE technique in diagnosing OA and see how AE signals changes with increasing severity of OA. Study design: An in vitro experimental study. Methods: A total of 16 distal limbs (8 forelimbs and 8 hindlimbs) from six Finn horses were collected from an abattoir and fitted in a custom‐made frame allowing fetlock joint bending. Eight fetlock joints were opened, and cartilage surface was progressively damaged mechanically three times using sandpaper to mimic mild, moderate and severe OA. The remaining eight fetlock joints were opened and closed without any mechanical procedure, serving as controls. Before cartilage alteration, synovial fluid was aspirated, mixed with phosphate‐buffered saline solution, and then reinjected before suturing for constant joint lubrication. For each simulated condition of OA severity, a force was applied to the frame and then released to mimic joint flexion and extension. AE signals were acquired using air microphones. Results: A strong association was found between the joint condition and the power of AE signals analysed in 1.5–6 kHz range. The signal from both forelimb and hindlimb joints followed a similar pattern for increased cartilage damage. There were statistically significant differences between each joint condition progressively (generalised linear mixed model, P

ObjectiveTo develop and validate a machine learning (ML) approach for automatic three-dimensional (3D) histopathological grading of osteochondral samples imaged with contrast-enhanced micro-computed tomography (CEμCT).DesignOsteochondral cores from 24 total knee arthroplasty patients and 2 asymptomatic cadavers (n = 34, Ø = 2 mm; n = 45, Ø = 4 mm) were imaged using CEμCT with phosphotungstic acid-staining. Volumes-of-interest (VOI) in surface (SZ), deep (DZ) and calcified (CZ) zones were extracted depthwise and subjected to dimensionally reduced Local Binary Pattern-textural feature analysis. Regularized Ridge and Logistic regression (LR) models were trained zone-wise against the manually assessed semi-quantitative histopathological CEμCT grades (Ø = 2 mm samples). Models were validated using nested leave-one-out cross-validation and an independent test set (Ø = 4 mm samples). The performance was assessed using Spearman’s correlation, Average Precision (AP) and Area under the Receiver Operating Characteristic Curve (AUC).ResultsHighest performance on cross-validation was observed for SZ, both on Ridge regression (ρ = 0.68, p < 0.0001) and LR (AP = 0.89, AUC = 0.92). The test set evaluations yielded decreased Spearman’s correlations on all zones. For LR, performance was almost similar in SZ (AP = 0.89, AUC = 0.86), decreased in CZ (AP = 0.71→0.62, AUC = 0.77→0.63) and increased in DZ (AP = 0.50→0.83, AUC = 0.72→0.72).ConclusionWe showed that the ML-based automatic 3D histopathological grading of osteochondral samples is feasible from CEμCT. The developed method can be directly applied by OA researchers since the grading software and all source codes are publicly available.

Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilises raw radiographic data, clinical examination results and previous medical history of the patient. We validated this approach on an independent test set of 3,918 knee images from 2,129 subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78–0.81) and Average Precision (AP) of 0.68 (0.66–0.70). In contrast, a reference approach, based on logistic regression, yielded AUC of 0.75 (0.74–0.77) and AP of 0.62 (0.60–0.64). The proposed method could significantly improve the subject selection process for OA drug-development trials and help the development of personalised therapeutic plans.

Key PointsQuestionIs the gap between brain age predicted from MRI and chronological age associated with incident dementia in a general population of Dutch adults?FindingsBrain age was predicted using a deep learning model, using MRI-derived grey matter density maps. In a population based study including 5496 participants, the observed gap was significantly associated with the risk of dementia.MeaningThe gap between MRI-brain predicted and chronological age is potentially a biomarker for dementia risk screening.AbstractImportanceThe gap between predicted brain age using magnetic resonance imaging (MRI) and chronological age may serve as biomarker for early-stage neurodegeneration and potentially as a risk indicator for dementia. However, owing to the lack of large longitudinal studies, it has been challenging to validate this link.ObjectiveWe aimed to investigate the utility of such a gap as a risk biomarker for incident dementia in a general Dutch population, using a deep learning approach for predicting brain age based on MRI-derived grey matter maps.DesignData was collected from participants of the cohort-based Rotterdam Study who underwent brain magnetic resonance imaging between 2006 and 2015. This study was performed in a longitudinal setting and all participant were followed up for incident dementia until 2016.SettingThe Rotterdam Study is a prospective population-based study, initiated in 1990 in the suburb Ommoord of in Rotterdam, the Netherlands.ParticipantsAt baseline, 5496 dementia- and stroke-free participants (mean age 64.67±9.82, 54.73% women) were scanned and screened for incident dementia. During 6.66±2.46 years of follow-up, 159 people developed dementia.Main outcomes and measuresWe built a convolutional neural network (CNN) model to predict brain age based on its MRI. Model prediction performance was measured in mean absolute error (MAE). Reproducibility of prediction was tested using the intraclass correlation coefficient (ICC) computed on a subset of 80 subjects. Logistic regressions and Cox proportional hazards were used to assess the association of the age gap with incident dementia, adjusted for years of education, ApoEε4 allele carriership, grey matter volume and intracranial volume. Additionally, we computed the attention maps of CNN, which shows which brain regions are important for age prediction.ResultsMAE of brain age prediction was 4.45±3.59 years and ICC was 0.97 (95% confidence interval CI=0.96-0.98). Logistic regression and Cox proportional hazards models showed that the age gap was significantly related to incident dementia (odds ratio OR=1.11 and 95% confidence intervals CI=1.05-1.16; hazard ratio HR=1.11 and 95% CI=1.06-1.15, respectively). Attention maps indicated that grey matter density around the amygdalae and hippocampi primarily drive the age estimation.Conclusion and relevanceWe show that the gap between predicted and chronological brain age is a biomarker associated with risk of dementia development. This suggests that it can be used as a biomarker, complimentary to those that are known, for dementia risk screening.

Breast cancer is one of the main causes of death worldwide. Histopathological cellularity assessment of residual tumors in post-surgical tissues is used to analyze a tumor’s response to a therapy. Correct cellularity assessment increases the chances of getting an appropriate treatment and facilitates the patient’s survival. In current clinical practice, tumor cellularity is manually estimated by pathologists; this process is tedious and prone to errors or low agreement rates between assessors. In this work, we evaluated three strong novel Deep Learning-based approaches for automatic assessment of tumor cellularity from post-treated breast surgical specimens stained with hematoxylin and eosin. We validated the proposed methods on the BreastPathQ SPIE challenge dataset that consisted of 2395 image patches selected from whole slide images acquired from 64 patients. Compared to expert pathologist scoring, our best performing method yielded the Cohen’s kappa coefficient of 0.69 (vs. 0.42 previously known in literature) and the intra-class correlation coefficient of 0.89 (vs. 0.83). Our results suggest that Deep Learning-based methods have a significant potential to alleviate the burden on pathologists, enhance the diagnostic workflow, and, thereby, facilitate better clinical outcomes in breast cancer treatment.

Cardiothoratic ratio (CTR) estimated from chest radiographs is a marker indicative of cardiomegaly, the presence of which is in the criteria for heart failure diagnosis. Existing methods for automatic assessment of CTR are driven by Deep Learning-based segmentation. However, these techniques produce only point estimates of CTR but clinical decision making typically assumes the uncertainty. In this paper, we propose a novel method for chest X-ray segmentation and CTR assessment in an automatic manner. In contrast to the previous art, we, for the first time, propose to estimate CTR with uncertainty bounds. Our method is based on Deep Convolutional Neural Network with Feature Pyramid Network (FPN) decoder. We propose two modifications of FPN: replace the batch normalization with instance normalization and inject the dropout which allows to obtain the Monte-Carlo estimates of the segmentation maps at test time. Finally, using the predicted segmentation mask samples, we estimate CTR with uncertainty. In our experiments we demonstrate that the proposed method generalizes well to three different test sets. Finally, we make the annotations produced by two radiologists for all our datasets publicly available., Comment: Roman Solovyev and Iaroslav Melekhov contributed equally. Timo Lesonen and Elias Vaattovaara contributed equally

Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition settings remains an open problem. In this study, we investigated two modern regularization techniques — mixup and adversarial unsupervised domain adaptation (UDA) — to improve the robustness of DL-based knee cartilage segmentation to new MRI acquisition settings. Our validation setup included two datasets produced by different MRI scanners and using distinct data acquisition protocols. We assessed the robustness of automatic segmentation by comparing mixup and UDA approaches to a strong baseline method at different OA severity stages and, additionally, in relation to anatomical locations. Our results showed that for moderate changes in knee MRI data acquisition settings both approaches may provide notable improvements in the robustness, which are consistent for all stages of the disease and affect the clinically important areas of the knee joint. However, mixup may be considered as a recommended approach, since it is more computationally efficient and does not require additional data from the target acquisition setup.

This paper addresses the challenge of localization of anatomical landmarks in knee X-ray images at different stages of osteoarthritis (OA). Landmark localization can be viewed as regression problem, where the landmark position is directly predicted by using the region of interest or even full-size images leading to large memory footprint, especially in case of high resolution medical images. In this work, we propose an efficient deep neural networks framework with an hourglass architecture utilizing a soft-argmax layer to directly predict normalized coordinates of the landmark points. We provide an extensive evaluation of different regularization techniques and various loss functions to understand their influence on the localization performance. Furthermore, we introduce the concept of transfer learning from low-budget annotations, and experimentally demonstrate that such approach is improving the accuracy of landmark localization. Compared to the prior methods, we validate our model on two datasets that are independent from the train data and assess the performance of the method for different stages of OA severity. The proposed approach demonstrates better generalization performance compared to the current state-of-the-art.

Three-dimensional (3D) semi-quantitative grading of pathological features in articular cartilage (AC) offers significant improvements in basic research of osteoarthritis (OA). We have earlier developed the 3D protocol for imaging of AC and its structures which includes staining of the sample with a contrast agent (phosphotungstic acid, PTA) and a consequent scanning with micro-computed tomography. Such a protocol was designed to provide X-ray attenuation contrast to visualize AC structure. However, at the same time, this protocol has one major disadvantage: the loss of contrast at the tidemark (calcified cartilage interface, CCI). An accurate segmentation of CCI can be very important for understanding the etiology of OA and ex-vivo evaluation of tidemark condition at early OA stages. In this paper, we present the first application of Deep Learning to PTA-stained osteochondral samples that allows to perform tidemark segmentation in a fully-automatic manner. Our method is based on U-Net trained using a combination of binary cross-entropy and soft-Jaccard loss. On cross-validation, this approach yielded intersection over the union of 0.59, 0.70, 0.79, 0.83 and 0.86 within 15 \(\upmu \)m, 30 \(\upmu \)m, 45 \(\upmu \)m, 60 \(\upmu \)m. and 75 \(\upmu \)m padded zones around the tidemark, respectively. Our codes and the dataset that consisted of 35 PTA-stained human AC samples are made publicly available together with the segmentation masks to facilitate the development of biomedical image segmentation methods.

Summary Background In horses, osteoarthritis (OA) mostly affects metacarpophalangeal and metatarsophalangeal (fetlock) joints. The current modalities used for diagnosis of equine limb disorders lack ability to detect early OA. Here, we propose a new alternative approach to assess experimental cartilage damage in fetlock joint using Acoustic Emissions (AE). Objectives To evaluate the potential of AE technique in diagnosing OA and see how AE signals changes with increasing severity of OA. Study design An in vitro experimental study. Methods A total of 16 distal limbs (8 forelimbs and 8 hindlimbs) from six Finn horses were collected from an abattoir and fitted in a custom‐made frame allowing fetlock joint bending. Eight fetlock joints were opened, and cartilage surface was progressively damaged mechanically three times using sandpaper to mimic mild, moderate and severe OA. The remaining eight fetlock joints were opened and closed without any mechanical procedure, serving as controls. Before cartilage alteration, synovial fluid was aspirated, mixed with phosphate‐buffered saline solution, and then reinjected before suturing for constant joint lubrication. For each simulated condition of OA severity, a force was applied to the frame and then released to mimic joint flexion and extension. AE signals were acquired using air microphones. Results A strong association was found between the joint condition and the power of AE signals analysed in 1.5–6 kHz range. The signal from both forelimb and hindlimb joints followed a similar pattern for increased cartilage damage. There were statistically significant differences between each joint condition progressively (generalised linear mixed model, P

Knee osteoarthritis (OA) is the most common musculoskeletal disorder affecting all populations. One common knee OA symptom is instability; thus its assessment could allow diagnosing and following-up of the disease without using conventional imaging techniques, such as plain radiography or magnetic resonance imaging (MRI). Knee kinematic measurements using accelerometers could provide a low-cost and non-invasive option to quantify knee instability. The aim of this study was to assess the relationships between kinematic data, instability parameters derived from the imaging techniques, goniometer-based measurements, and radiological OA stage. The right knees of 66 females (44–67 years) were examined using MRI, plain radiography, and goniometer-based angle measurement. Kellgren–Lawrence (KL) grade and the joint line convergence angle (JLCA) were determined from the radiographs. Cartilage thickness and OA score (MOAKS) were derived from the MRI. A ratio between lateral and medial cartilage thicknesses was calculated from the average thickness of segmented cartilage over the weight bearing area (MRIratio). Accelerometers attached to thigh and shank were used to record kinematic signals during a one-leg-stand test. Power of the accelerometer signals along the anatomical longitudinal axis (Pacc) was used as a measure of knee instability. Finally, Spearman’s correlations between the acquired parameters and KL grade/MOAKS scores were calculated. Leave-one-out cross-validation and logistic regression were used to discriminate OA subjects (KL ≥ 2). All the instability parameters (Pacc, JLCA and MRIratio), except the goniometer angle, showed significant correlations with KL grading (rho = 0.32–0.64, p < 0.01) and MOAKS composite score (rho = 0.35–0.56, p < 0.01). Both Pacc and JLCA showed higher areas under the ROC curve to discriminate OA (AUC = 0.76 and AUC = 0.78) than MRIratio and goniometer angle (AUC = 0.55 and AUC = 0.56). Our results demonstrate the clinical potential of kinematic knee instability measurements using low-cost accelerometers. Such approach could become a potential new tool in OA diagnostics.