Search

• StrokeBERT is a disease-specific BERT-based model pre-trained on real world evidence (RWE) from cerebrovascular disease-related corpora. • The model was evaluated and validated in larger, multiple-center datasets by two independent empirical tasks (stenosis detection and stroke recurrence prediction). • Disease-specific BERT model improves results of various disease-specific language processing tasks compared to similar BERT-models pre-trained on the general domain corpora. Effectively utilizing disease-relevant text information from unstructured clinical notes for medical research presents many challenges. BERT (Bidirectional Encoder Representation from Transformers) related models such as BioBERT and ClinicalBERT, pre-trained on biomedical corpora and general clinical information, have shown promising performance in various biomedical language processing tasks. This study aims to explore whether a BERT-based model pre-trained on disease-related clinical information can be more effective for cerebrovascular disease-relevant research. This study proposed the StrokeBERT which was initialized from BioBERT and pre-trained on large-scale cerebrovascular disease related clinical text information. The pre-trained corpora contained 113,590 discharge notes, 105,743 radiology reports, and 38,199 neurological reports. Two real-world empirical clinical tasks were conducted to validate StrokeBERT's performance. The first task identified extracranial and intracranial artery stenosis from two independent sets of radiology angiography reports. The second task predicted the risk of recurrent ischemic stroke based on patients' first discharge information. In stenosis detection, StrokeBERT showed improved performance on targeted carotid arteries, with an average AUC compared to that of ClinicalBERT of 0.968 ± 0.021 and 0.956 ± 0.018, respectively. In recurrent ischemic stroke prediction, after 10-fold cross-validation on 1,700 discharge information, StrokeBERT presented better prediction ability (AUC±SD = 0.838 ± 0.017) than ClinicalBERT (AUC±SD = 0.808 ± 0.045). The attention scores of StrokeBERT showed better ability to detect and associate cerebrovascular disease related terms than current BERT based models. This study shows that a disease-specific BERT model improved the performance and accuracy of various disease-specific language processing tasks and can readily be fine-tuned to advance cerebrovascular disease research and further developed for clinical applications. [ABSTRACT FROM AUTHOR]

• Using a nationwide prospective stroke registry to evaluate several machine learning approaches for prediction of stroke outcomes. • Over two hundred clinical variables are screened to identify important features that predicts stroke outcome. • The follow-up data is important which can further improve the predictive models' performance. • Error analysis shows that most prediction errors come from more severe stroke patients. Being able to predict functional outcomes after a stroke is highly desirable for clinicians. This allows clinicians to set reasonable goals with patients and relatives, and to reach shared after-care decisions for recovery or rehabilitation. The aim of this study was to apply various machine learning (ML) methods for 90-day stroke outcome predictions, using a nationwide disease registry. This study used the Taiwan Stroke Registry (TSR) which has prospectively collected data from stroke patients since 2006. Three known ML models (support vector machine, random forest, and artificial neural network), and a hybrid artificial neural network were implemented and evaluated by 10-time repeated hold-out with 10-fold cross-validation. ML techniques present over 0.94 AUC in both ischemic and hemorrhagic stroke using preadmission and inpatient data. By adding follow-up data, the prediction ability improved to 0.97 AUC. We screened 206 clinical variables to identify 17 important features from the ischemic stroke dataset and 22 features from the hemorrhagic stroke dataset without losing much performance. Error analysis revealed that most prediction errors come from more severe stroke patients. The study showed that ML techniques trained from large, cross-reginal registry datasets were able to predict functional outcome after stroke with high accuracy. The follow-up data is important which can further improve the predictive models' performance. With similar performances among different ML techniques, the algorithm's characteristics and performance on severe stroke patients will be the primary focus when we further develop inference models and artificial intelligence tools for potential medical. Image, graphical abstract [ABSTRACT FROM AUTHOR]