Search

BackgroundDisability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking.MethodsData of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. TRIPOD guidelines were followed.Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expended disability status scale, treatment, relapse information, and MS course.To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated on their area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error.FindingsA temporal attention model was the best model. It achieved a ROC-AUC of 0·71 ± 0·01, an AUC-PR of 0·26 ± 0·02, a Brier score of 0·1 ± 0·01 and an expected calibration error of 0·07 ± 0·04. The history of disability progression is more predictive for future disability progression than the treatment or relapses.InterpretationGood discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This makes these models ready for a clinical impact study. All our preprocessing and model code is available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS.

Patients with the ‘aggressive’ form of MS accrue disability at an accelerated rate, typically reaching EDSS >= 6 within 10 years of symptom onset. Several clinicodemographic factors have been associated with aggressive MS, but less research has focused on clinical markers that are present in the first year of disease. The development of early predictive models of aggressive MS is essential to optimise treatment in this MS subtype. We evaluated whether patients who will develop aggressive MS can be identified based on early clinical markers, and to replicate this analysis in an independent cohort. Patient data were obtained from MSBase. Inclusion criteria were (a) first recorded disability score (EDSS) within 12 months of symptom onset, (b) at least 2 recorded EDSS scores, and (c) at least 10 years of observation time. Patients were classified as having ‘aggressive MS’ if they: (a) reached EDSS >= 6 within 10 years of symptom onset, (b) EDSS >=6 was confirmed and sustained over >=6 months, and (c) EDSS >=6 was sustained until the end of follow-up. Clinical predictors included patient variables (sex, age at onset, baseline EDSS, disease duration at first visit) and recorded relapses in the first 12 months since disease onset (count, pyramidal signs, bowel-bladder symptoms, cerebellar signs, incomplete relapse recovery, steroid administration, hospitalisation). Predictors were evaluated using Bayesian Model Averaging (BMA). Independent validation was performed using data from the Swedish MS Registry. Of the 2,403 patients identified, 145 were classified as having aggressive MS (6%). BMA identified three statistical predictors: age > 35 at symptom onset, EDSS >= 3 in the first year, and the presence of pyramidal signs in the first year. This model significantly predicted aggressive MS (AUC = .80, 95% CIs = .75, .84). The presence of all three signs was strongly predictive, with 32% of such patients meeting aggressive disease criteria. The absence of all three signs was associated with a 1.4% risk. Of the 556 eligible patients in the Swedish MS Registry cohort, 34 (6%) met criteria for aggressive MS. The combination of all three signs was also predictive in this cohort (AUC = .75, 95% CIs = .66, .84). Taken together, these findings suggest that older age at symptom onset, greater disability during the first year, and pyramidal signs in the first year are early indicators of aggressive MS.