Search

Historically, spine surgeons used expected postoperative survival of 3-months to help select candidates for operative intervention in spinal metastasis. However, this cutoff has been challenged by the development of minimally invasive techniques, novel biologics, and advanced radiotherapy. Recent studies have suggested that a life expectancy of 6 weeks may be enough to achieve significant improvements in postoperative health-related quality of life.The purpose of this study was to develop a model capable of predicting 6-week mortality in patients with spinal metastases treated with radiation or surgery.A retrospective review was conducted at five large tertiary centers in the United States and Taiwan.The development cohort consisted of 3,001 patients undergoing radiotherapy and/or surgery for spinal metastases from one institution. The validation institutional cohort consisted of 1,303 patients from four independent, external institutions.The primary outcome was 6-week mortality.Five models were considered to predict 6-week mortality, and the model with the best performance across discrimination, calibration, decision-curve analysis, and overall performance was integrated into an open access web-based application.The most important variables for prediction of 6-week mortality were albumin, primary tumor histology, absolute lymphocyte, three or more spine metastasis, and ECOG score. The elastic-net penalized logistic model was chosen as the best performing model with AUC 0.84 on evaluation in the independent testing set. On external validation in the 1,303 patients from the four independent institutions, the model retained good discriminative ability with an area under the curve of 0.81. The model is available here: https://sorg-apps.shinyapps.io/spinemetssurvival/.While this study does not advocate for the use of a 6-week life expectancy as criteria for considering operative management, the algorithm developed and externally validated in this study may be helpful for preoperative planning, multidisciplinary management, and shared decision-making in spinal metastasis patients with shorter life expectancy.

Spinal epidural abscess is a rare but severe condition with high rates of postoperative adverse events.The objective of the study was to identify independent prognostic factors for reoperation using two datasets: an institutional and national database.Retrospective Review.Database 1: Review of five medical centers from 1993 to 2016. Database 2: The National Surgical Quality Improvement Program (NSQIP) was queried between 2012 and 2016.Thirty-day and ninety-day reoperation rate.Two independent datasets were reviewed to identify patients with spinal epidural abscesses undergoing spinal surgery. Multivariate analyses were used to determine independent prognostic factors for reoperation while including factors identified in bivariate analyses.Overall, 642 patients underwent surgery for a spinal epidural abscess in the institutional cohort, with a 90-day unplanned reoperation rate of 19.9%. In the NSQIP database, 951 patients were identified with a 30-day unplanned reoperation rate of 12.3%. On multivariate analysis in the NSQIP database, cervical spine abscess was the only factor that reached significance for 30-day reoperation (OR=1.71, 95% CI=1.11-2.63, p=.02, Area under the curve (AUC)=0.61). On multivariate analysis in the institutional cohort, independent prognostic factors for 30-day reoperation were: preoperative urinary incontinence, ventral location of abscess relative to thecal sac, cervical abscess, preoperative wound infection, and leukocytosis (AUC=0.65). Ninety-day reoperation rate also found hypoalbuminemia as a significant predictor (AUC=0.66).Six novel independent prognostic factors were identified for 90-day reoperation after surgery for a spinal epidural abscess. The multivariable analysis fairly predicts reoperation, indicating that there may be additional factors that need to be uncovered in future studies. The risk factors delineated in this study through the use of two large cohorts of spinal epidural abscess patients can be used to improve preoperative risk stratification and patient management.

Well-performing survival prediction models (SPMs) help patients and healthcare professionals to choose treatment aligning with prognosis. This retrospective study aims to investigate the prognostic impacts of laboratory data and to compare the performances of Metastases location, Elderly, Tumor primary, Sex, Sickness/comorbidity, and Site of radiotherapy (METSSS) model, New England Spinal Metastasis Score (NESMS), and Skeletal Oncology Research Group machine learning algorithm (SORG-MLA) for spinal metastases (SM).From 2010 to 2018, patients who received radiotherapy (RT) for SM at a tertiary center were enrolled and the data were retrospectively collected. Multivariate logistic and Cox-proportional-hazard regression analyses were used to assess the association between laboratory values and survival. The area under receiver-operating characteristics curve (AUROC), calibration analysis, Brier score, and decision curve analysis were used to evaluate the performance of SPMs.A total of 2786 patients were included for analysis. The 90-day and 1-year survival rates after RT were 70.4% and 35.7%, respectively. Higher albumin, hemoglobin, or lymphocyte count were associated with better survival, while higher alkaline phosphatase, white blood cell count, neutrophil count, neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, or international normalized ratio were associated with poor prognosis. SORG-MLA has the best discrimination (AUROC 90-day, 0.78; 1-year 0.76), best calibrations, and the lowest Brier score (90-day 0.16; 1-year 0.18). The decision curve of SORG-MLA is above the other two competing models with threshold probabilities from 0.1 to 0.8.Laboratory data are of prognostic significance in survival prediction after RT for SM. Machine learning-based model SORG-MLA outperforms statistical regression-based model METSSS model and NESMS in survival predictions.

Background Incidental durotomy is an intraoperative complication in spine surgery that can lead to postoperative complications, increased length of stay, and higher healthcare costs. Natural language processing (NLP) is an artificial intelligence method that assists in understanding free-text notes that may be useful in the automated surveillance of adverse events in orthopaedic surgery. A previously developed NLP algorithm is highly accurate in the detection of incidental durotomy on internal validation and external validation in an independent cohort from the same country. External validation in a cohort with linguistic differences is required to assess the transportability of the developed algorithm, referred to geographical validation. Ideally, the performance of a prediction model, the NLP algorithm, is constant across geographic regions to ensure reproducibility and model validity.Question/purpose Can we geographically validate an NLP algorithm for the automated detection of incidental durotomy across three independent cohorts from two continents?Methods Patients 18 years or older undergoing a primary procedure of (thoraco)lumbar spine surgery were included. In Massachusetts, between January 2000 and June 2018, 1000 patients were included from two academic and three community medical centers. In Maryland, between July 2016 and November 2018, 1279 patients were included from one academic center, and in Australia, between January 2010 and December 2019, 944 patients were included from one academic center. The authors retrospectively studied the free-text operative notes of included patients for the primary outcome that was defined as intraoperative durotomy. Incidental durotomy occurred in 9% (93 of 1000), 8% (108 of 1279), and 6% (58 of 944) of the patients, respectively, in the Massachusetts, Maryland, and Australia cohorts. No missing reports were observed. Three datasets (Massachusetts, Australian, and combined Massachusetts and Australian) were divided into training and holdout test sets in an 80:20 ratio. An extreme gradient boosting (an efficient and flexible tree-based algorithm) NLP algorithm was individually trained on each training set, and the performance of the three NLP algorithms (respectively American, Australian, and combined) was assessed by discrimination via area under the receiver operating characteristic curves (AUC-ROC; this measures the model's ability to distinguish patients who obtained the outcomes from those who did not), calibration metrics (which plot the predicted and the observed probabilities) and Brier score (a composite of discrimination and calibration). In addition, the sensitivity (true positives, recall), specificity (true negatives), positive predictive value (also known as precision), negative predictive value, Fl-score (composite of precision and recall), positive likelihood ratio, and negative likelihood ratio were calculated.Results The combined NLP algorithm (the combined Massachusetts and Australian data) achieved excellent performance on independent testing data from Australia (AUC-ROC 0.97 [95% confidence interval 0.87 to 0.99]), Massachusetts (AUC-ROC 0.99 [95% CI 0.80 to 0.99]) and Maryland (AUC-ROC 0.95 [95% CI 0.93 to 0.97]). The NLP developed based on the Massachusetts cohort had excellent performance in the Maryland cohort (AUC-ROC 0.97 [95% CI 0.95 to 0.99]) but worse performance in the Australian cohort (AUC-ROC 0.74 [95% CI 0.70 to 0.77]).Conclusion We demonstrated the clinical utility and reproducibility of an NLP algorithm with combined datasets retaining excellent performance in individual countries relative to algorithms developed in the same country alone for detection of incidental durotomy. Further multi-institutional, international collaborations can facilitate the creation of universal NLP algorithms that improve the quality and safety of orthopaedic surgery globally. The combined NLP algorithm has been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/nlp_incidental_durotomy/. Clinicians and researchers can use the tool to help incorporate the model in evaluating spine registries or quality and safety departments to automate detection of incidental durotomy and optimize prevention efforts.

Preoperative prediction of prolonged postoperative opioid prescription helps identify patients for increased surveillance after surgery. The SORG machine learning model has been developed and successfully tested using 5,413 patients from the United States (US) to predict the risk of prolonged opioid prescription after surgery for lumbar disc herniation. However, external validation is an often-overlooked element in the process of incorporating prediction models in current clinical practice. This cannot be stressed enough in prediction models where medicolegal and cultural differences may play a major role.The authors aimed to investigate the generalizability of the US citizens prediction model SORG to a Taiwanese patient cohort.Retrospective study at a large academic medical center in Taiwan.Of 1,316 patients who were 20 years or older undergoing initial operative management for lumbar disc herniation between 2010 and 2018.The primary outcome of interest was prolonged opioid prescription defined as continuing opioid prescription to at least 90 to 180 days after the first surgery for lumbar disc herniation at our institution.Baseline characteristics were compared between the external validation cohort and the original developmental cohorts. Discrimination (area under the receiver operating characteristic curve and the area under the precision-recall curve), calibration, overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithm in the validation cohort. This study had no funding source or conflict of interests.Overall, 1,316 patients were identified with sustained postoperative opioid prescription in 41 (3.1%) patients. The validation cohort differed from the development cohort on several variables including 93% of Taiwanese patients receiving NSAIDS preoperatively compared with 22% of US citizens patients, while 30% of Taiwanese patients received opioids versus 25% in the US. Despite these differences, the SORG prediction model retained good discrimination (area under the receiver operating characteristic curve of 0.76 and the area under the precision-recall curve of 0.33) and good overall performance (Brier score of 0.028 compared with null model Brier score of 0.030) while somewhat overestimating the chance of prolonged opioid use (calibration slope of 1.07 and calibration intercept of -0.87). Decision-curve analysis showed the SORG model was suitable for clinical use.Despite differences at baseline and a very strict opioid policy, the SORG algorithm for prolonged opioid use after surgery for lumbar disc herniation has good discriminative abilities and good overall performance in a Han Chinese patient group in Taiwan. This freely available digital application can be used to identify high-risk patients and tailor prevention policies for these patients that may mitigate the long-term adverse consequence of opioid dependence: https://sorg-apps.shinyapps.io/lumbardiscopioid/.

Purpose Mortality prediction in elderly femoral neck fracture patients is valuable in treatment decision-making. A previously developed and internally validated clinical prediction model shows promise in identifying patients at risk of 90-day and 2-year mortality. Validation in an independent cohort is required to assess the generalizability; especially in geographically distinct regions. Therefore we questioned, is the SORG Orthopaedic Research Group (SORG) femoral neck fracture mortality algorithm externally valid in an Israeli cohort to predict 90-day and 2-year mortality? Methods We previously developed a prediction model in 2022 for estimating the risk of mortality in femoral neck fracture patients using a multicenter institutional cohort of 2,478 patients from the USA. The model included the following input variables that are available on clinical admission: age, male gender, creatinine level, absolute neutrophil, hemoglobin level, international normalized ratio (INR), congestive heart failure (CHF), displaced fracture, hemiplegia, chronic obstructive pulmonary disease (COPD), history of cerebrovascular accident (CVA) and beta-blocker use. To assess the generalizability, we used an intercontinental institutional cohort from the Sheba Medical Center in Israel (level I trauma center), queried between June 2008 and February 2022. Generalizability of the model was assessed using discrimination, calibration, Brier score, and decision curve analysis. Results The validation cohort included 2,033 patients, aged 65 years or above, that underwent femoral neck fracture surgery. Most patients were female 64.8% (n = 1317), the median age was 81 years (interquartile range = 75–86), and 80.4% (n = 1635) patients sustained a displaced fracture (Garden III/IV). The 90-day mortality was 9.4% (n = 190) and 2-year mortality was 30.0% (n = 610). Despite numerous baseline differences, the model performed acceptably to the validation cohort on discrimination (c-statistic 0.67 for 90-day, 0.67 for 2-year), calibration, Brier score, and decision curve analysis. Conclusions The previously developed SORG femoral neck fracture mortality algorithm demonstrated good performance in an independent intercontinental population. Current iteration should not be relied on for patient care, though suggesting potential utility in assessing patients at low risk for 90-day or 2-year mortality. Further studies should evaluate this tool in a prospective setting and evaluate its feasibility and efficacy in clinical practice. The algorithm can be freely accessed: https://sorg-apps.shinyapps.io/hipfracturemortality/. Level of evidence Level III, Prognostic study.

➤ As the Patient-Reported Outcomes Measurement Information System (PROMIS) is increasingly utilized in orthopaedic research and clinical practice, there is not a consensus regarding the minimum clinically important difference (MCID) calculation. ➤ The varied MCID calculation methods can lead to a range of possible values, which limits the translatability of research efforts. ➤ The completion rate and follow-up period also influence MCID values and should be reported alongside study results.

Background and purpose — Advancements in software and hardware have enabled the rise of clinical prediction models based on machine learning (ML) in orthopedic surgery. Given their growing popularity and their likely implementation in clinical practice we evaluated which outcomes these new models have focused on and what methodologies are being employed. Material and methods — We performed a systematic search in PubMed, Embase, and Cochrane Library for studies published up to June 18, 2020. Studies reporting on non-ML prediction models or non-orthopedic outcomes were excluded. After screening 7,138 studies, 59 studies reporting on 77 prediction models were included. We extracted data regarding outcome, study design, and reported performance metrics. Results — Of the 77 identified ML prediction models the most commonly reported outcome domain was medical management (17/77). Spinal surgery was the most commonly involved orthopedic subspecialty (28/77). The most frequently employed algorithm was neural networks (42/77). Median size of datasets was 5,507 (IQR 635–26,364). The median area under the curve (AUC) was 0.80 (IQR 0.73–0.86). Calibration was reported for 26 of the models and 14 provided decision-curve analysis. Interpretation — ML prediction models have been developed for a wide variety of topics in orthopedics. Topics regarding medical management were the most commonly studied. Heterogeneity between studies is based on study size, algorithm, and time-point of outcome. Calibration and decision-curve analysis were generally poorly reported.

BACKGROUND Intraoperative vascular injury (VI) may be an unavoidable complication of anterior lumbar spine surgery; however, vascular injury has implications for quality and safety reporting as this intraoperative complication may result in serious bleeding, thrombosis, and postoperative stricture. PURPOSE The purpose of this study was to (1) develop machine learning algorithms for preoperative prediction of VI and (2) develop natural language processing (NLP) algorithms for automated surveillance of intraoperative VI from free-text operative notes. PATIENT SAMPLE Adult patients, 18 years or age or older, undergoing anterior lumbar spine surgery at two academic and three community medical centers were included in this analysis. OUTCOME MEASURES The primary outcome was unintended VI during anterior lumbar spine surgery. METHODS Manual review of free-text operative notes was used to identify patients who had unintended VI. The available population was split into training and testing cohorts. Five machine learning algorithms were developed for preoperative prediction of VI. An NLP algorithm was trained for automated detection of intraoperative VI from free-text operative notes. Performance of the NLP algorithm was compared to current procedural terminology and international classification of diseases codes. RESULTS In all, 1035 patients underwent anterior lumbar spine surgery and the rate of intraoperative VI was 7.2% (n=75). Variables used for preoperative prediction of VI were age, male sex, body mass index, diabetes, L4-L5 exposure, and surgery for infection (discitis, osteomyelitis). The best performing machine learning algorithm achieved c-statistic of 0.73 for preoperative prediction of VI ( https://sorg-apps.shinyapps.io/lumbar_vascular_injury/ ). For automated detection of intraoperative VI from free-text notes, the NLP algorithm achieved c-statistic of 0.92. The NLP algorithm identified 18 of the 21 patients (sensitivity 0.86) who had a VI whereas current procedural terminologyand international classification of diseases codes identified 6 of the 21 (sensitivity 0.29) patients. At this threshold, the NLP algorithm had a specificity of 0.93, negative predictive value of 0.99, positive predictive value of 0.51, and F1-score of 0.64. CONCLUSION Relying on administrative procedural and diagnosis codes may underestimate the rate of unintended intraoperative VI in anterior lumbar spine surgery. External and prospective validation of the algorithms presented here may improve quality and safety reporting.

BACKGROUND CONTEXT Surgical decompression and stabilization in the setting of spinal metastasis is performed to relieve pain and preserve functional status. These potential benefits must be weighed against the risks of perioperative morbidity and mortality. Accurate prediction of a patient's postoperative survival is a crucial component of patient counseling. PURPOSE To externally validate the SORG machine learning algorithms for prediction of 90-day and 1-year mortality after surgery for spinal metastasis. STUDY DESIGN/SETTING Retrospective, cohort study PATIENT SAMPLE Patients 18 years or older at a tertiary care medical center treated surgically for spinal metastasis OUTCOME MEASURES Mortality within 90 days of surgery, mortality within 1 year of surgery METHODS This is a retrospective cohort study of 298 adult patients at a tertiary care medical center treated surgically for spinal metastasis between 2004 and 2020. Baseline characteristics of the validation cohort were compared to the derivation cohort for the SORG algorithms. The following metrics were used to assess the performance of the algorithms: discrimination, calibration, overall model performance, and decision curve analysis. RESULTS Sixty-one patients died within 90 days of surgery and 133 died within 1 year of surgery. The validation cohort differed significantly from the derivation cohort. The SORG algorithms for 90-day mortality and 1-year mortality performed excellently with respect to discrimination; the algorithm for 1-year mortality was well-calibrated. At both postoperative time points, the SORG algorithms showed greater net benefit than the default strategies of changing management for no patients or for all patients. CONCLUSIONS With an independent, contemporary, and geographically distinct population, we report successful external validation of SORG algorithms for preoperative risk prediction of 90-day and 1-year mortality after surgery for spinal metastasis. By providing accurate prediction of intermediate and long-term mortality risk, these externally validated algorithms may inform shared decision-making with patients in determining management of spinal metastatic disease.

Background Context Accurately predicting the survival of patients with spinal metastases is important for guiding surgical intervention. The SORG machine-learning (ML) algorithm for the 90-day and one-year mortality of patients with metastatic cancer to the spine has been multiply validated, with a high degree of accuracy in both internal and external validation studies. However, prior external validations were conducted using patient groups located on the east coast of the United States, representing a generally homogeneous population. The aim of this study was to externally validate the SORG algorithms with a Taiwanese population. Study Design/Setting Retrospective study at a single tertiary care center in Taiwan Patient Sample Four hundred and twenty-seven patients who underwent surgery for metastatic spine disease from November 1, 2010 to December 31, 2018 Outcome Measures 90-day and one-year mortality Methods The baseline characteristics of our validation cohort were compared with those of the previously published developmental and external validation cohorts. Discrimination (c-statistic and receiver operating curve), calibration (calibration plot, intercept, and slope), overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithms in this cohort. Results Ninety-day and one-year mortality rates were 110 of 427 (26%) and 256 of 427 (60%), respectively. The external validation cohort and the developmental cohort differed in body mass index (BMI), preoperative performance status, American Spinal Injury Association impairment scale, primary tumor histology and in several laboratory measurements. The SORG ML algorithm for 90-day and 1-year mortality demonstrated a high level of discriminative ability (c-statistics of 0.73 [95% confidence interval [CI], 0.67–0.78] and 0.74 [95% CI, 0.69−0.79]), overall performance, and had a positive net benefit throughout the range of threshold probabilities in decision curve analysis. The algorithm for 1-year mortality had a calibration intercept of 0.08, representing a good calibration. However, the 90-day mortality algorithm underestimated mortality for the lowest predicted probabilities, with an overall intercept of 0.81. Conclusions The SORG algorithms for predicting 90-day and 1-year mortality in patients with spinal metastatic disease generally performed well on international external validation in a predominately Taiwanese population. However, 90-day mortality was underestimated in this group. Whether this inconsistency was due to different primary tumor characteristics, body mass index, selection bias or other factors remains unclear, and may be better understood with further validative works that utilize international and/or diverse populations.

Study Design Retrospective cohort study Objective The aim of this study was to determine the relative importance and predicative power of the Hospital Frailty Risk Score (HFRS) on unplanned 30-day readmission after surgical intervention for metastatic spinal column tumors. Methods All adult patients undergoing surgery for metastatic spinal column tumor were identified in the Nationwide Readmission Database from the years 2016 to 2018. Patients were categorized into 3 cohorts based on the criteria of the HFRS: Low(Results There were 4346 patients included. The proportion of patients who required any readmission were higher among the Intermediate and High frailty cohorts when compared to the Low frailty cohort ( Low:33.9% vs. Intermediate:39.3% vs. High:39.2%, P < .001). An RF classifier was trained to predict 30-day readmission on all features (AUC = .60) and architecturally equivalent model trained using only ten features with highest MDG (AUC = .59). Both models found frailty to have the highest importance in predicting risk of readmission. On multivariate regression analysis, Intermediate frailty [ OR:1.32, CI(1.06,1.64), P = .012] was found to be an independent predictor of unplanned 30-day readmission. Conclusion Our study utilizes machine learning approaches and predictive modeling to identify frailty as a significant risk-factor that contributes to unplanned 30-day readmission after spine surgery for metastatic spinal column metastases.

BACKGROUND The Skeletal Oncology Research Group machine-learning algorithms (SORG-MLAs) estimate 90-day and 1-year survival in patients with long-bone metastases undergoing surgical treatment and have demonstrated good discriminatory ability on internal validation. However, the performance of a prediction model could potentially vary by race or region, and the SORG-MLA must be externally validated in an Asian cohort. Furthermore, the authors of the original developmental study did not consider the Eastern Cooperative Oncology Group (ECOG) performance status, a survival prognosticator repeatedly validated in other studies, in their algorithms because of missing data. QUESTIONS/PURPOSES (1) Is the SORG-MLA generalizable to Taiwanese patients for predicting 90-day and 1-year mortality? (2) Is the ECOG score an independent factor associated with 90-day and 1-year mortality while controlling for SORG-MLA predictions? METHODS All 356 patients who underwent surgery for long-bone metastases between 2014 and 2019 at one tertiary care center in Taiwan were included. Ninety-eight percent (349 of 356) of patients were of Han Chinese descent. The median (range) patient age was 61 years (25 to 95), 52% (184 of 356) were women, and the median BMI was 23 kg/m2 (13 to 39 kg/m2). The most common primary tumors were lung cancer (33% [116 of 356]) and breast cancer (16% [58 of 356]). Fifty-five percent (195 of 356) of patients presented with a complete pathologic fracture. Intramedullary nailing was the most commonly performed type of surgery (59% [210 of 356]), followed by plate screw fixation (23% [81 of 356]) and endoprosthetic reconstruction (18% [65 of 356]). Six patients were lost to follow-up within 90 days; 30 were lost to follow-up within 1 year. Eighty-five percent (301 of 356) of patients were followed until death or for at least 2 years. Survival was 82% (287 of 350) at 90 days and 49% (159 of 326) at 1 year. The model's performance metrics included discrimination (concordance index [c-index]), calibration (intercept and slope), and Brier score. In general, a c-index of 0.5 indicates random guess and a c-index of 0.8 denotes excellent discrimination. Calibration refers to the agreement between the predicted outcomes and the actual outcomes, with a perfect calibration having an intercept of 0 and a slope of 1. The Brier score of a prediction model must be compared with and ideally should be smaller than the score of the null model. A decision curve analysis was then performed for the 90-day and 1-year prediction models to evaluate their net benefit across a range of different threshold probabilities. A multivariate logistic regression analysis was used to evaluate whether the ECOG score was an independent prognosticator while controlling for the SORG-MLA's predictions. We did not perform retraining/recalibration because we were not trying to update the SORG-MLA algorithm in this study. RESULTS The SORG-MLA had good discriminatory ability at both timepoints, with a c-index of 0.80 (95% confidence interval 0.74 to 0.86) for 90-day survival prediction and a c-index of 0.84 (95% CI 0.80 to 0.89) for 1-year survival prediction. However, the calibration analysis showed that the SORG-MLAs tended to underestimate Taiwanese patients' survival (90-day survival prediction: calibration intercept 0.78 [95% CI 0.46 to 1.10], calibration slope 0.74 [95% CI 0.53 to 0.96]; 1-year survival prediction: calibration intercept 0.75 [95% CI 0.49 to 1.00], calibration slope 1.22 [95% CI 0.95 to 1.49]). The Brier score of the 90-day and 1-year SORG-MLA prediction models was lower than their respective null model (0.12 versus 0.16 for 90-day prediction; 0.16 versus 0.25 for 1-year prediction), indicating good overall performance of SORG-MLAs at these two timepoints. Decision curve analysis showed SORG-MLAs provided net benefits when threshold probabilities ranged from 0.40 to 0.95 for 90-day survival prediction and from 0.15 to 1.0 for 1-year prediction. The ECOG score was an independent factor associated with 90-day mortality (odds ratio 1.94 [95% CI 1.01 to 3.73]) but not 1-year mortality (OR 1.07 [95% CI 0.53 to 2.17]) after controlling for SORG-MLA predictions for 90-day and 1-year survival, respectively. CONCLUSION SORG-MLAs retained good discriminatory ability in Taiwanese patients with long-bone metastases, although their actual survival time was slightly underestimated. More international validation and incremental value studies that address factors such as the ECOG score are warranted to refine the algorithms, which can be freely accessed online at https://sorg-apps.shinyapps.io/extremitymetssurvival/. LEVEL OF EVIDENCE Level III, therapeutic study.

As cost-effective measures become increasingly implemented in the US healthcare system, changes in patient-reported outcome measure (PROM) scores can be utilized to indicate patient satisfaction following procedures including total knee arthroplasty (TKA). The primary aim of this study was to develop and evaluate machine learning algorithms to predict achievement of the minimal clinically important difference (MCID) for the Knee Injury and Osteoarthritis Outcome Score-Physical Function Short Form (KOOS-PS) at 1-year following TKA. A retrospective review of primary TKA patients between 2016 and 2018 was performed. Variables considered for prediction included demographics and preoperative PROMs. The KOOS-PS MCID was calculated via a distribution-based method. Five machine learning algorithms were developed and tested by discrimination, calibration, Brier score, and decision curve analysis. Among the 744 patients who met the inclusion criteria, 385 (72.8%) patients achieved the MCID. The elastic-net penalized logistic regression model was selected as the best performing model (c-statistic 0.77, calibration intercept -0.02, calibration slope 1.15, and Brier score 0.14). The most important variables for MCID achievement were preoperative KOOS-PS score, preoperative VAS Pain, preoperative opioid use, preoperative PROMIS global mental health score, age, and sex. Algorithms were incorporated into an open-access digital application available at https://sorg-apps.shinyapps.io/tka_koos_mcid/. This study is the first to predict the probability of achieving the KOOS-PS MCID following TKA using a machine learning-based approach. The results were used to develop a clinical decision aid based on commonly collected predictive variables to preoperatively predict an individual patient's likelihood of attaining an acceptable outcome following TKA.

Background and purpose — External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Material and methods — We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting. Results — We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43–89), with 6 items being reported in less than 4/18 of the studies. Interpretation — Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.

Purpose: Effects of clockwise torque rotation onto proximal femoral fracture fixation have been subject of ongoing debate: fixated right-sided trochanteric fractures seem more rotationally stable than left-sided fractures in the biomechanical setting, but this theoretical advantage has not been demonstrated in the clinical setting to date. The purpose of this study was to identify a difference in early reoperation rate between patients undergoing surgery for left- versus right-sided proximal femur fractures using cephalomedullary nailing (CMN). Materials and methods: The American College of Surgeons National Surgical Quality Improvement Program was queried from 2016-2019 to identify patients aged 50 years and older undergoing CMN for a proximal femoral fracture. The primary outcome was any unplanned reoperation within 30 days following surgery. The difference was calculated using a Chi-square test, and observed power calculated using post-hoc power analysis. Results: In total, of 20,122 patients undergoing CMN for proximal femoral fracture management, 1.8% (n=371) had to undergo an unplanned reoperation within 30 days after surgery. Overall, 208 (2.0%) were left-sided and 163 (1.7%) right-sided fractures (p=0.052, risk ratio [RR] 1.22, 95% confidence interval [CI] 1.00–1.50), odds ratio [OR] 1.23 (95%CI 1.00–1.51), power 49.2% (α=0.05). Conclusion: This study shows a higher risk of reoperation for left-sided compared to right-sided proximal femur fractures after CMN in a large sample size. Although results may be underpowered and statistically insignificant, this finding might substantiate the hypothesis that clockwise rotation during implant insertion and (postoperative) weightbearing may lead to higher reoperation rates. Level of evidence: Therapeutic level II.

In the last 5 years, artificial intelligence (AI) algorithms have made rapid advances for diagnosis and prognosis in fields ranging from dermatology to anesthesiology. How do we make sense of the rise of AI in healthcare and specifically in spine? How much of what we see today is “hype” and what will remain when the dust settles? In this special issue, several reviews and original articles help us understand the state of AI in healthcare today, the avenues for future progress, and the implications for spine care. Continued engagement, skepticism, and collaboration with technical experts will allow for the development of AI systems that complement and expand our abilities to diagnose, predict, and operate.

BACKGROUND CONTEXT Patient-Reported Outcome Measurement Information System (PROMIS) scores are increasingly utilized in clinical care. However, it is unclear if PROMIS can discriminate surgeon performance on an individual level. PURPOSE The purpose of this study was to examine surgeon-level variance in rates of achieving minimal clinically important difference (MCID) after lumbar decompression. PATIENT SAMPLE This is a prospective, observational cohort study performed across a healthcare enterprise (two academic medical centers and three community centers). Patients 18 years or older undergoing one- to two-level primary decompression for lumbar disc herniation (LDH) or lumbar spinal stenosis (LSS) were included. OUTCOME MEASURES The primary outcome was achievement of MCID, using a distribution-based method, on paired PROMIS physical function scores. METHODS Descriptive statistics were generated to examine the baseline characteristics of the study cohort. Bivariate analyses were used to examine the impact of surgeon-level variance on rates of MCID. Multivariable analyses were used to examine the risk-adjusted impact of surgeon-level variance on rates of MCID. RESULTS Overall, 636 patients treated by nine surgeons were included. The median patient age was 58 [interquartile range (IQR): 46–70] and 62.3% (n=396) were female. Among all patients, 56.9% (n=362) underwent surgery for LDH. The overall rate of achieving MCID was 75.8% (n=482). Of the surgeons, the median years in practice were 12 (range 4–31) and 55.6% (n=5) were in academic practice settings. On bivariate analysis, patients treated by one of the surgeons had lower rates of achieving MICD (odds ratio=0.37, 95% confidence interval: 0.15–0.91, p=.03). However, on multivariable analysis adjusting for operative indication (LDH vs. LSS), body mass index, number of comorbidities, percent unemployment in patient zip code, and preoperative PROMIS physical function scores, all surgeons were equally likely to obtain MCID. CONCLUSIONS In this cohort, variance in PROMIS scores after primary lumbar decompression is influenced by patient-related factors and not by individual surgeon. Adequate risk adjustment is needed if ascertaining clinical improvement on an individual surgeon basis. Level of Evidence 2

Purpose To (1) determine the diagnostic efficacy of artificial intelligence (AI) methods for detecting anterior cruciate ligament (ACL) and meniscus tears and to (2) compare the efficacy to human clinical experts. Methods PubMed, OVID/Medline, and Cochrane libraries were queried in November 2019 for research articles pertaining to AI use for detection of ACL and meniscus tears. Information regarding AI model, prediction accuracy/area under the curve (AUC), sample sizes of testing/training sets, and imaging modalities were recorded. Results A total of 11 AI studies were identified: 5 investigated ACL tears, 5 investigated meniscal tears, and 1 investigated both. The AUC of AI models for detecting ACL tears ranged from 0.895 to 0.980, and the prediction accuracy ranged from 86.7% to 100%. Of these studies, 3 compared AI models to clinical experts. Two found no significant differences in diagnostic capability, whereas one found that radiologists had a significantly greater sensitivity for detecting ACL tears (P = .002) and statistically similar specificity and accuracy. Of the 5 studies investigating the meniscus, the AUC for AI models ranged from 0.847 to 0.910 and prediction accuracy ranged from 75.0% to 90.0%. Of these studies, 2 compared AI models with clinical experts. One found no significant differences in diagnostic accuracy, whereas one found that the AI model had a significantly lower specificity (P = .003) and accuracy (P = .015) than radiologists. Two studies reported that the addition of AI models significantly increased the diagnostic performance of clinicians compared to their efforts without these models. Conclusions AI prediction capabilities were excellent and may enhance the diagnosis of ACL and meniscal pathology; however, AI did not outperform clinical experts. Clinical Relevance AI models promise to improve diagnosing certain pathologies as well as or better than human experts, are excellent for detecting ACL and meniscus tears, and may enhance the diagnostic capabilities of human experts; however, when compared with these experts, they may not offer any significant advantage.

BACKGROUND: Machine learning (ML) is a subdomain of artificial intelligence that enables computers to abstract patterns from data without explicit programming. A myriad of impactful ML applications already exists in orthopaedics ranging from predicting infections after surgery to diagnostic imaging. However, no systematic reviews that we know of have compared, in particular, the performance of ML models with that of clinicians in musculoskeletal imaging to provide an up-to-date summary regarding the extent of applying ML to imaging diagnoses. By doing so, this review delves into where current ML developments stand in aiding orthopaedists in assessing musculoskeletal images. QUESTIONS/PURPOSES: This systematic review aimed (1) to compare performance of ML models versus clinicians in detecting, differentiating, or classifying orthopaedic abnormalities on imaging by (A) accuracy, sensitivity, and specificity, (B) input features (for example, plain radiographs, MRI scans, ultrasound), (C) clinician specialties, and (2) to compare the performance of clinician-aided versus unaided ML models. METHODS: A systematic review was performed in PubMed, Embase, and the Cochrane Library for studies published up to October 1, 2019, using synonyms for machine learning and all potential orthopaedic specialties. We included all studies that compared ML models head-to-head against clinicians in the binary detection of abnormalities in musculoskeletal images. After screening 6531 studies, we ultimately included 12 studies. We conducted quality assessment using the Methodological Index for Non-randomized Studies (MINORS) checklist. All 12 studies were of comparable quality, and they all clearly included six of the eight critical appraisal items (study aim, input feature, ground truth, ML versus human comparison, performance metric, and ML model description). This justified summarizing the findings in a quantitative form by calculating the median absolute improvement of the ML models compared with clinicians for the following metrics of performance: accuracy, sensitivity, and specificity. RESULTS: ML models provided, in aggregate, only very slight improvements in diagnostic accuracy and sensitivity compared with clinicians working alone and were on par in specificity (3% (interquartile range [IQR] -2.0% to 7.5%), 0.06% (IQR -0.03 to 0.14), and 0.00 (IQR -0.048 to 0.048), respectively). Inputs used by the ML models were plain radiographs (n = 8), MRI scans (n = 3), and ultrasound examinations (n = 1). Overall, ML models outperformed clinicians more when interpreting plain radiographs than when interpreting MRIs (17 of 34 and 3 of 16 performance comparisons, respectively). Orthopaedists and radiologists performed similarly to ML models, while ML models mostly outperformed other clinicians (outperformance in 7 of 19, 7 of 23, and 6 of 10 performance comparisons, respectively). Two studies evaluated the performance of clinicians aided and unaided by ML models; both demonstrated considerable improvements in ML-aided clinician performance by reporting a 47% decrease of misinterpretation rate (95% confidence interval [CI] 37 to 54; p < 0.001) and a mean increase in specificity of 0.048 (95% CI 0.029 to 0.068; p < 0.001) in detecting abnormalities on musculoskeletal images. CONCLUSIONS: At present, ML models have comparable performance to clinicians in assessing musculoskeletal images. ML models may enhance the performance of clinicians as a technical supplement rather than as a replacement for clinical intelligence. Future ML-related studies should emphasize how ML models can complement clinicians, instead of determining the overall superiority of one versus the other. This can be accomplished by improving transparent reporting, diminishing bias, determining the feasibility of implantation in the clinical setting, and appropriately tempering conclusions. LEVEL OF EVIDENCE: Level III, diagnostic study.

BACKGROUND Surgical site infections are a major driver of morbidity and increased costs in the postoperative period after spine surgery. Current tools for surveillance of these adverse events rely on prospective clinical tracking, manual retrospective chart review, or administrative procedural and diagnosis codes. PURPOSE The purpose of this study was to develop natural language processing (NLP) algorithms for automated reporting of postoperative wound infection requiring reoperation after lumbar discectomy. PATIENT SAMPLE Adult patients undergoing discectomy at two academic and three community medical centers between January 1, 2000 and July 31, 2019 for lumbar disc herniation. OUTCOME MEASURES Reoperation for wound infection within 90 days after surgery METHODS Free-text notes of patients who underwent surgery from January 1, 2000 to December 31, 2015 were used for algorithm training. Free-text notes of patients who underwent surgery after January 1, 2016 were used for algorithm testing. Manual chart review was used to label which patients had reoperation for wound infection. An extreme gradient-boosting NLP algorithm was developed to detect reoperation for postoperative wound infection. RESULTS Overall, 5,860 patients were included in this study and 62 (1.1%) had a reoperation for wound infection. In patients who underwent surgery after January 1, 2016 (n=1,377), the NLP algorithm detected 15 of the 16 patients (sensitivity=0.94) who had reoperation for infection. In comparison, current procedural terminology and international classification of disease codes detected 12 of these 16 patients (sensitivity=0.75). At a threshold of 0.05, the NLP algorithm had positive predictive value of 0.83 and F1-score of 0.88. CONCLUSION Temporal validation of the algorithm developed in this study demonstrates a proof-of-concept application of NLP for automated reporting of adverse events after spine surgery. Adapting this methodology for other procedures and outcomes in spine and orthopedics has the potential to dramatically improve and automatize quality and safety reporting.

IMPORTANCE Preoperative determination of the potential for postoperative opioid dependence in previously naive patients undergoing elective spine surgery may facilitate targeted interventions. OBJECTIVE The purpose of this study was to develop supervised machine learning algorithms for preoperative prediction of prolonged opioid prescription use in opioid-naive patients following lumbar spine surgery. DESIGN Retrospective review of clinical registry data. Variables considered for prediction included demographics, insurance status, preoperative medications, surgical factors, laboratory values, comorbidities, and neighborhood characteristics. Five supervised machine learning algorithms were developed and assessed by discrimination, calibration, Brier score, and decision curve analysis. SETTING One healthcare entity (two academic medical centers, three community hospitals), 2000 to 2018. PARTICIPANTS Opioid-naive patients undergoing decompression and/or fusion for lumbar disk herniation, stenosis, and spondylolisthesis. MAIN OUTCOME Sustained prescription opioid use exceeding 90 days after surgery. RESULTS Overall, of 8,435 patients included, 359 (4.3%) were found to have prolonged postoperative opioid prescriptions. The elastic-net penalized logistic regression achieved the best performance in the independent testing set not used for algorithm development with c-statistic=0.70, calibration intercept=0.06, calibration slope=1.02, and Brier score=0.039. The five most important factors for prolonged opioid prescriptions were use of instrumented spinal fusion, preoperative benzodiazepine use, preoperative antidepressant use, preoperative gabapentin use, and uninsured status. Individual patient-level explanations were provided for the algorithm predictions and the algorithms were incorporated into an open access digital application available here: https://sorg-apps.shinyapps.io/lumbaropioidnaive/ . CONCLUSION AND RELEVANCE The clinician decision aid developed in this study may be helpful to preoperatively risk-stratify opioid-naive patients undergoing lumbar spine surgery. The tool demonstrates moderate discriminative capacity for identifying those at greatest risk of prolonged prescription opioid use. External validation is required to further support the potential utility of this tool in practice.

Background A preoperative estimation of survival is critical for deciding on the operative management of metastatic bone disease of the extremities. Several tools have been developed for this purpose, but there is room for improvement. Machine learning is an increasingly popular and flexible method of prediction model building based on a data set. It raises some skepticism, however, because of the complex structure of these models. Questions/purposes The purposes of this study were (1) to develop machine learning algorithms for 90-day and 1-year survival in patients who received surgical treatment for a bone metastasis of the extremity, and (2) to use these algorithms to identify those clinical factors (demographic, treatment related, or surgical) that are most closely associated with survival after surgery in these patients. Methods All 1090 patients who underwent surgical treatment for a long-bone metastasis at two institutions between 1999 and 2017 were included in this retrospective study. The median age of the patients in the cohort was 63 years (interquartile range [IQR] 54 to 72 years), 56% of patients (610 of 1090) were female, and the median BMI was 27 kg/m (IQR 23 to 30 kg/m). The most affected location was the femur (70%), followed by the humerus (22%). The most common primary tumors were breast (24%) and lung (23%). Intramedullary nailing was the most commonly performed type of surgery (58%), followed by endoprosthetic reconstruction (22%), and plate screw fixation (14%). Missing data were imputed using the missForest methods. Features were selected by random forest algorithms, and five different models were developed on the training set (80% of the data): stochastic gradient boosting, random forest, support vector machine, neural network, and penalized logistic regression. These models were chosen as a result of their classification capability in binary datasets. Model performance was assessed on both the training set and the validation set (20% of the data) by discrimination, calibration, and overall performance. Results We found no differences among the five models for discrimination, with an area under the curve ranging from 0.86 to 0.87. All models were well calibrated, with intercepts ranging from -0.03 to 0.08 and slopes ranging from 1.03 to 1.12. Brier scores ranged from 0.13 to 0.14. The stochastic gradient boosting model was chosen to be deployed as freely available web-based application and explanations on both a global and an individual level were provided. For 90-day survival, the three most important factors associated with poorer survivorship were lower albumin level, higher neutrophil-to-lymphocyte ratio, and rapid growth primary tumor. For 1-year survival, the three most important factors associated with poorer survivorship were lower albumin level, rapid growth primary tumor, and lower hemoglobin level. Conclusions Although the final models must be externally validated, the algorithms showed good performance on internal validation. The final models have been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/extremitymetssurvival/. Pending external validation, clinicians may use this tool to predict survival for their individual patients to help in shared treatment decision making. Level of evidence Level III, therapeutic study.

BACKGROUND CONTEXT Preoperative survival estimation in spinal metastatic disease helps determine the appropriateness of invasive management. The SORG ML 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease were previously developed in a single institutional sample but remain to be externally validated. PURPOSE The purpose of this study was to externally validate these algorithms in an independent population from another institution. STUDY DESIGN/SETTING Retrospective study at a large, tertiary care center. PATIENT SAMPLE Patients 18 years or older who underwent surgery between 2003 and 2016. OUTCOME MEASURES Ninety-day and 1-year mortality. METHODS Baseline characteristics of the validation cohort were compared to the developmental cohort for the SORG ML algorithms. Discrimination (c-statistic and receiver operating curve), calibration (calibration slope, intercept, calibration plot, and observed proportions by predicted risk groups), overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithms in the validation cohort. RESULTS Overall, 176 patients underwent surgery for spinal metastatic disease, of which 44 (22.7%) experienced 90-day mortality and 99 (56.2%) experienced 1-year mortality. The validation cohort differed significantly from the developmental cohort on primary tumor histology, metastatic tumor burden, previous systemic therapy, overall comorbidity burden, and preoperative laboratory characteristics. Despite these differences, the SORG ML algorithms generalized well to the validation cohort on discrimination (c-statistic 0.75–0.81 for 90-day mortality and 0.77–0.78 for 1-year mortality), calibration, Brier score, and decision curve analysis. CONCLUSION and RELEVANCE Initial results from external validation of the SORG ML 90-day and 1-year algorithms for survival prediction in spinal metastatic disease suggest potential utility of these digital decision aids in clinical practice. Further studies are needed to validate or refute these algorithms in large patient samples from prospective, international, multi-institutional trials.

Artificial Intelligence is gaining traction in medicine for its ease of use and advancements in technology. This study evaluates the current literature on the use of artificial intelligence in adult spinal deformity.

Artificial Intelligence is gaining traction in medicine for its ease of use and advancements in technology. This study evaluates the current literature on the use of artificial intelligence in adult spinal deformity.

Complications after total hip arthroplasty (THA) may result in readmission or reoperation and impose a significant cost on the healthcare system. Understanding which patients are at-risk for complications can potentially allow for targeted interventions to decrease complication rates through pursuing preoperative health optimization. The purpose of the current was to develop and internally validate machine learning (ML) algorithms capable of performing patient-specific predictions of all-cause complications within two years of primary THA.This was a retrospective case-control study of clinical registry data from 616 primary THA patients from one large academic and two community hospitals. The primary outcome was all-cause complications at a minimum of 2-years after primary THA. Recursive feature elimination was applied to identify preoperative variables with the greatest predictive value. Five ML algorithms were developed on the training set using tenfold cross-validation and internally validated on the independent testing set of patients. Algorithms were assessed by discrimination, calibration, Brier score, and decision curve analysis to quantify performance.The observed complication rate was 16.6%. The stochastic gradient boosting model achieved the best performance with an AUC = 0.88, calibration intercept = 0.1, calibration slope = 1.22, and Brier score = 0.09. The most important factors for predicting complications were age, drug allergies, prior hip surgery, smoking, and opioid use. Individual patient-level explanations were provided for the algorithm predictions and incorporated into an open access digital application: https://sorg-apps.shinyapps.io/tha_complication/ CONCLUSIONS: The stochastic boosting gradient algorithm demonstrated good discriminatory capacity for identifying patients at high-risk of experiencing a postoperative complication and proof-of-concept for creating office-based applications from ML that can perform real-time prediction. However, this clinical utility of the current algorithm is unknown and definitions of complications broad. Further investigation on larger data sets and rigorous external validation is necessary prior to the assessment of clinical utility with respect to risk-stratification of patients undergoing primary THA.III, therapeutic study.

Objectives: Recovery after shoulder arthroscopy can be impaired by sustained postoperative opioid use, yet there are few validated risk calculators for this outcome. The purpose of this study is to develop and validate a machine learning algorithm that can reliably and effectively predict sustained opioid use in patients following elective shoulder arthroscopy. Methods: A retrospective review of an institutional outcomes database was performed at a tertiary academic medical center to identify adult patients who underwent shoulder arthroscopy between January 1, 2014 and October 1, 2019. Extended postoperative opioid consumption was defined as opioid consumption at least 150 days following surgery. Five machine learning algorithms were developed to predict this outcome. Performance of the algorithms were assessed through discrimination, calibration, and decision curve analysis. Results: Overall, of the 1504 patients included, 132 (8.8%) demonstrated sustained postoperative opioid consumption. The factors determined for prediction of prolonged postoperative opioid prescriptions were preoperative opioid use, age, median percentage of population in patient zip code living at the federal poverty line, preoperative pain score, duration of symptoms, BMI. The random forest model achieved the best performance based on discrimination (AUC = 0.78), calibration, and decision curve analysis. This model was integrated into a web-based open-access application able to provide both predictions and explanations. Conclusions: If externally validated in independent populations, the algorithm developed presently could effectively guide preoperative screening in patients at high risk for extended postoperative opioid prescriptions. Early identification and interdisciplinary counseling in high-risk cases can optimize both resource allocation and surgical outcomes.

Introduction Numerous prognostication models have been developed to estimate survival in patients with extremity metastatic bone disease, but few include albumin despite albumin's role in malnutrition and inflammation. The purpose of this study was to examine two independent datasets to determine the value for albumin in prognosticating survival in this population. Materials and methods Extremity metastatic bone disease patients undergoing surgical management were identified from two independent populations. Population 1: Retrospective chart review at two tertiary care centers. Population 2: A large, national, North American multicenter surgical registry with 30-day follow-up. Bivariate and multivariate analyses were used to examine albumin's value for prognostication at 1-, 3-, and 12-month after surgery. Results In Population 1, 1,090 patients were identified with 1-, 3-, and 12-month mortality rates of 95 (8.8%), 305 (28.9%), and 639 (62.0%), respectively. In Population 2, 1,675 patients were identified with one-month postoperative mortality rates of 148 (8.8%). In both populations, hypoalbuminemia was an independent prognostic factor for mortality at 30 days. In the institutional set, hypoalbuminemia was additionally associated with 3- and 12-month mortality. Conclusions Hypoalbuminemia is a marker for mortality in extremity metastatic bone disease. Further consideration of this marker could improve existing prognostication models in this population. Level of evidence III.

This was a retrospective cohort study.To characterize the variability in cost for anterior cervical discectomy and fusion (ACDF) constructs and to identify key predictors of procedural cost.ACDF is commonly performed for surgical treatment of cervical radiculopathy and myelopathy. Numerous biomechanical constructs and graft/biological options are available, with most demonstrating relatively equivalent clinical results. Despite the substantial focus on value in spine care, the differences and contributions to procedural cost in ACDF have not been well defined.We evaluated the records of patients who underwent a single level ACDF from 2016 to 2020 at 4 hospitals in a major metropolitan area. We abstracted demographics, insurance status, operative time, diagnosis, surgeon, institution, and components of procedural costs. Costs based on construct were compared using multivariable adjusted analyses using negative binomial regression. The primary outcome measures were cost differences between ACDF techniques.Two hundred sixty-four patients were included, with procedures by 13 surgeons across 4 institutions. The total procedural cost for ACDF had a mean of US$2317 with wide variation (range, US$967-US$7370). Multivariable analysis revealed body mass index and use of polyether ether ketone to be correlated with increased cost while carbon fiber and autograft correlated with decreased cost. When comparing standalone device constructs to cases with anterior instrumentation (plate/screws), the total cost was significantly higher in the plate/screw group (US$2686±US$921 vs. US$1466±US$878, P0.001).We encountered wide variation in procedural costs associated with ACDF, including as much as an 8-fold difference in the cost of constructs. The most important drivers included instrumentation type and implant materials. Here, we identify potential targets of opportunity for health care organizations that are looking to reduce variance in procedural expenditures to improve health care savings associated with the performance of ACDF.

BACKGROUND The prediction of survival is valuable to optimize treatment of metastatic long-bone disease. The Skeletal Oncology Research Group (SORG) machine-learning (ML) algorithm has been previously developed and internally validated. The purpose of this study was to determine if the SORG ML algorithm accurately predicts 90-day and 1-year survival in an external metastatic long-bone disease patient cohort. METHODS A retrospective review of 264 patients who underwent surgery for long-bone metastases between 2003 and 2019 was performed. Variables used in the stochastic gradient boosting SORG algorithm were age, sex, primary tumor type, visceral/brain metastases, systemic therapy, and 10 preoperative laboratory values. Model performance was calculated by discrimination, calibration, and overall performance. RESULTS The SORG ML algorithms retained good discriminative ability (area under the cure [AUC]: 0.83; 95% confidence interval [CI]: 0.76-0.88 for 90-day mortality and AUC: 0.84; 95% CI: 0.79-0.88 for 1-year mortality), calibration, overall performance, and decision curve analysis. CONCLUSION The previously developed ML algorithms demonstrated good performance in the current study, thereby providing external validation. The models were incorporated into an accessible application (https://sorg-apps.shinyapps.io/extremitymetssurvival/) that may be freely utilized by clinicians in helping predict survival for individual patients and assist in informative decision-making discussion before operative management of long bone metastatic lesions.

PURPOSE Although the bulk of patient-generated health data are increasing exponentially, their use is impeded because most data come in unstructured format, namely as free-text clinical reports. A variety of natural language processing (NLP) methods have emerged to automate the processing of free text ranging from statistical to deep learning–based models; however, the optimal approach for medical text analysis remains to be determined. The aim of this study was to provide a head-to-head comparison of novel NLP techniques and inform future studies about their utility for automated medical text analysis. PATIENTS AND METHODS Magnetic resonance imaging reports of patients with brain metastases treated in two tertiary centers were retrieved and manually annotated using a binary classification (single metastasis v two or more metastases). Multiple bag-of-words and sequence-based NLP models were developed and compared after randomly splitting the annotated reports into training and test sets in an 80:20 ratio. RESULTS A total of 1,479 radiology reports of patients diagnosed with brain metastases were retrieved. The least absolute shrinkage and selection operator (LASSO) regression model demonstrated the best overall performance on the hold-out test set with an area under the receiver operating characteristic curve of 0.92 (95% CI, 0.89 to 0.94), accuracy of 83% (95% CI, 80% to 87%), calibration intercept of –0.06 (95% CI, –0.14 to 0.01), and calibration slope of 1.06 (95% CI, 0.95 to 1.17). CONCLUSION Among various NLP techniques, the bag-of-words approach combined with a LASSO regression model demonstrated the best overall performance in extracting binary outcomes from free-text clinical reports. This study provides a framework for the development of machine learning-based NLP models as well as a clinical vignette of patients diagnosed with brain metastases.

Background Context In-hospital and short-term mortality in patients with spinal epidural abscess (SEA) remains unacceptably high despite diagnostic and therapeutic advancements. Forecasting this potentially avoidable consequence at the time of admission could improve patient management and counseling. Few studies exist to meet this need, and none have explored methodologies such as machine learning. Purpose The purpose of this study was to develop machine learning algorithms for prediction of in-hospital and 90-day postdischarge mortality in SEA. Study Design/Setting Retrospective, case-control study at two academic medical centers and three community hospitals from 1993 to 2016. Patients Sample Adult patients with an inpatient admission for radiologically confirmed diagnosis of SEA. Outcome Measures In-hospital and 90-day postdischarge mortality. Methods Five machine learning algorithms (elastic-net penalized logistic regression, random forest, stochastic gradient boosting, neural network, and support vector machine) were developed and assessed by discrimination, calibration, overall performance, and decision curve analysis. Results Overall, 1,053 SEA patients were identified in the study, with 134 (12.7%) experiencing in-hospital or 90-day postdischarge mortality. The stochastic gradient boosting model achieved the best performance across discrimination, c-statistic=0.89, calibration, and decision curve analysis. The variables used for prediction of 90-day mortality, ranked by importance, were age, albumin, platelet count, neutrophil to lymphocyte ratio, hemodialysis, active malignancy, and diabetes. The final algorithm was incorporated into a web application available here: https://sorg-apps.shinyapps.io/seamortality/ . Conclusions Machine learning algorithms show promise on internal validation for prediction of 90-day mortality in SEA. Future studies are needed to externally validate these algorithms in independent populations.

OBJECTIVENonroutine discharge after elective spine surgery increases healthcare costs, negatively impacts patient satisfaction, and exposes patients to additional hospital-acquired complications. Therefore, prediction of nonroutine discharge in this population may improve clinical management. The authors previously developed a machine learning algorithm from national data that predicts risk of nonhome discharge for patients undergoing surgery for lumbar disc disorders. In this paper the authors externally validate their algorithm in an independent institutional population of neurosurgical spine patients.METHODSMedical records from elective inpatient surgery for lumbar disc herniation or degeneration in the Transitional Care Program at Brigham and Women’s Hospital (2013–2015) were retrospectively reviewed. Variables included age, sex, BMI, American Society of Anesthesiologists (ASA) class, preoperative functional status, number of fusion levels, comorbidities, preoperative laboratory values, and discharge disposition. Nonroutine discharge was defined as postoperative discharge to any setting other than home. The discrimination (c-statistic), calibration, and positive and negative predictive values (PPVs and NPVs) of the algorithm were assessed in the institutional sample.RESULTSOverall, 144 patients underwent elective inpatient surgery for lumbar disc disorders with a nonroutine discharge rate of 6.9% (n = 10). The median patient age was 50 years and 45.1% of patients were female. Most patients were ASA class II (66.0%), had 1 or 2 levels fused (80.6%), and had no diabetes (91.7%). The median hematocrit level was 41.2%. The neural network algorithm generalized well to the institutional data, with a c-statistic (area under the receiver operating characteristic curve) of 0.89, calibration slope of 1.09, and calibration intercept of −0.08. At a threshold of 0.25, the PPV was 0.50 and the NPV was 0.97.CONCLUSIONSThis institutional external validation of a previously developed machine learning algorithm suggests a reliable method for identifying patients with lumbar disc disorder at risk for nonroutine discharge. Performance in the institutional cohort was comparable to performance in the derivation cohort and represents an improved predictive value over clinician intuition. This finding substantiates initial use of this algorithm in clinical practice. This tool may be used by multidisciplinary teams of case managers and spine surgeons to strategically invest additional time and resources into postoperative plans for this population.

BACKGROUND CONTEXT Data regarding risk of failure of nonoperative management in spinal epidural abscess (SEA) are limited. Given the potential for deterioration with treatment failure, a tool that predicts the probability of failure would be of great clinical utility. PURPOSE We primarily aim to build a machine learning model using independent predictors of nonoperative management failure. Secondarily, we aim to develop an open-access web-based application that provides a patient-specific probability of treatment failure. STUDY DESIGN/SETTING Retrospective, case-control study. PATIENT SAMPLE Patients 18 years or older diagnosed with SEA at 2 academic medical centers and 3 community hospitals. OUTCOME MEASURES Failure of nonoperative management. METHODS This is a retrospective cohort study of 367 patients with SEA initially managed nonoperatively between 1993 and 2016. The primary outcome was failure of nonoperative management defined as neurologic deterioration, worsened back and/or radicular pain, or persistent symptoms despite initiation of antibiotic therapy. Five machine learning algorithms were developed and assessed by discrimination, calibration, and overall performance. RESULTS Ninety-nine (27%) patients failed nonoperative management. Factors determined for prediction of nonoperative management were: motor deficit, diabetes, ventral component of abscess relative to thecal sac, history of compression or pathologic vertebral fracture, sensory deficit, active malignancy, and involvement of 3 or more vertebral levels. The elastic-net penalized logistic regression model was chosen as the final model given its superior discrimination, calibration, and overall model performance. This model was incorporated into an open access web application. CONCLUSION By building a discriminative and well-calibrated model in a user-friendly and open-access digital interface, we hope to provide a prognostic tool that can be used to inform clinical decision-making in real-time.

Neutrophil to lymphocyte ratio and platelet to lymphocyte ratio have been previously identified as markers for overall survival in oncology but remain heretofore unexplored in spinal epidural abscess (SEA).The purpose of this study was to determine the impact of these routinely collected assessments on 90-day mortality in SEA.Retrospective, case-control study.Patients 18 years or older diagnosed with SEA at 2 academic medical centers and 3 community hospitals.Ninety-day postdischarge and in-hospital mortality.Complete blood count with differential obtained on the day immediately preceding or on the day of admission was used to calculate platelet to lymphocyte and neutrophil to lymphocyte ratios. Multivariate analyses were used to determine if these ratios were independent risk factors for 90-day mortality.For 1,053 SEA patients included in the study, the rate of 90-day mortality was 134 (12.7%). The rate of 90-day mortality with neutrophil to lymphocyte ratio (≥8) was (20.5%) compared to (8.1%) with neutrophil to lymphocyte ratio8. Neutrophil to lymphocyte ratio was positively associated with bacteremia, elevated erythrocyte sedimentation rate, and concurrent systemic infections (endocarditis, meningitis) and negatively associated with duration of symptoms prior to presentation. On multivariate analysis, elevated neutrophil to lymphocyte remained an independent risk factor for 90-day mortality (odds ratio=2.62, 95% confidence interval=1.66-4.17, p.001). Platelet to lymphocyte ratio was not associated with 90-day mortality.Absolute neutrophil to lymphocyte ratio is a routinely collected but overlooked biomarker in patients with spinal epidural abscess that is a novel independent risk factor for 90-day mortality.

The foot and ankle surgery literature lacks an analysis of studies published in general medical journals. Studies published in general medical journals have high visibility and significant impact on decision making by medical practitioners and health care policy makers. Knowledge of the subject matter and methodologic characteristics of foot and ankle publications being read by general medical practitioners and the lay public is important to the practicing foot and ankle surgeon. A systematic review of foot and ankle studies published from 2000 to 2017 in 5 high-impact general medical journals was undertaken. Data extracted included study topic, study design, study demographics, and methodologic characteristics of randomized controlled trials (RCTs). There were 47 foot and ankle studies published in the selected medical journals examined during the study time period. Most common topics were diabetic foot ulcers (n = 25 [53.2%]), plantar fasciitis (n = 6 [12.8%]), and Achilles tendinopathy (n = 5 [10.6%]). The most common study types were reviews (n = 24 [51.1%]), RCTs (n = 12 [25.4%], and prospective studies (n = 6 [12.8%]). Of the published RCTs, the most common journal of publication was the Journal of the American Medical Association (n = 7 [58.3%]). The median number of citations for RCTs published before 2016 was 242.5. Of the small number of RCTs that compared operative with nonoperative treatment, the majority supported nonoperative management (n = 3 [75%]). Diabetic foot ulcers, plantar fasciitis, and Achilles tendinopathy are the most commonly published foot and ankle topics in general medical journals.

BACKGROUND We developed a machine learning algorithm to predict the survival of patients with chondrosarcoma. The algorithm demonstrated excellent discrimination and calibration on internal validation in a derivation cohort based on data from the Surveillance, Epidemiology, and End Results (SEER) registry. However, the algorithm has not been validated in an independent external dataset. QUESTIONS/PURPOSES Does the Skeletal Oncology Research Group (SORG) algorithm accurately predict 5-year survival in an independent patient population surgically treated for chondrosarcoma? METHODS The SORG algorithm was developed using the SEER registry, which contains demographic data, tumor characteristics, treatment, and outcome values; and includes approximately 30% of the cancer patients in the United States. The SEER registry was ideal for creating the derivation cohort, and consequently the SORG algorithm, because of the high number of eligible patients and the availability of most (explanatory) variables of interest. Between 1992 to 2013, 326 patients were treated surgically for extracranial chondrosarcoma of the bone at two tertiary care referral centers. Of those, 179 were accounted for at a minimum of 5 years after diagnosis in a clinical note at one of the two institutions, unless they died earlier, and were included in the validation cohort. In all, 147 (45%) did not meet the minimum 5 years of followup at the institution and were not included in the validation of the SORG algorithm. The outcome (survival at 5 years) was checked for all 326 patients in the Social Security death index and were included in the supplemental validation cohort, to also ascertain validity for patients with less than 5 years of institutional followup. Variables used in the SORG algorithm to predict 5-year survival including sex, age, histologic subtype, tumor grade, tumor size, tumor extension, and tumor location were collected manually from medical records. The tumor characteristics were collected from the postoperative musculoskeletal pathology report. Predicted probabilities of 5-year survival were calculated for each patient in the validation cohort using the SORG algorithm, followed by an assessment of performance using the same metrics as used for internal validation, namely: discrimination, calibration, and overall performance. Discrimination was calculated using the concordance statistic (or the area under the Receiver Operating Characteristic (ROC) curve) to determine how well the algorithm discriminates between the outcome, which ranges from 0.5 (no better than a coin-toss) to 1.0 (perfect discrimination). Calibration was assessed using the calibration slope and intercept from a calibration plot to measure the agreement between predicted and observed outcomes. A perfect calibration plot should show a 45° upwards line. Overall performance was determined using the Brier score, ranging from 0 (excellent prediction) to 1 (worst prediction). The Brier score was compared with the null-model Brier score, which showed the performance of a model that ignored all the covariates. A Brier score lower than the null model Brier score indicated greater performance of the algorithm. For the external validation an F1-score was added to measure the overall accuracy of the algorithm, which ranges between 0 (total failure of an algorithm) and 1 (perfect algorithm).The 5-year survival was lower in the validation cohort than it was in the derivation cohort from SEER (61.5% [110 of 179] versus 76% [1131 of 1544] ; p < 0.001). This difference was driven by higher proportion of dedifferentiated chondrosarcoma in the institutional population than in the derivation cohort (27% [49 of 179] versus 9% [131 of 1544]; p < 0.001). Patients in the validation cohort also had larger tumor sizes, higher grades, and nonextremity tumor locations than did those in the derivation cohort. These differences between the study groups emphasize that the external validation is performed not only in a different patient cohort, but also in terms of disease characteristics. Five-year survival was not different for both patient groups between subpopulations of patients with conventional chondrosarcomas and those with dedifferentiated chondrosarcomas. RESULTS The concordance statistic for the validation cohort was 0.87 (95% CI, 0.80-0.91). Evaluation of the algorithm's calibration in the institutional population resulted in a calibration slope of 0.97 (95% CI, 0.68-1.3) and calibration intercept of -0.58 (95% CI, -0.20 to -0.97). Finally, on overall performance, the algorithm had a Brier score of 0.152 compared with a null-model Brier score of 0.237 for a high level of overall performance. The F1-score was 0.836. For the supplementary validation in the total of 326 patients, the SORG algorithm had a validation of 0.89 (95% CI, 0.85-0.93). The calibration slope was 1.13 (95% CI, 0.87-1.39) and the calibration intercept was -0.26 (95% CI, -0.57 to 0.06). The Brier score was 0.11, with a null-model Brier score of 0.19. The F1-score was 0.901. CONCLUSIONS On external validation, the SORG algorithm retained good discriminative ability and overall performance but overestimated 5-year survival in patients surgically treated for chondrosarcoma. This internet-based tool can help guide patient counseling and shared decision making. LEVEL OF EVIDENCE Level III, prognostic study.

Purpose: An excessive amount of total hospitalization is caused by delays due to patients waiting to be placed in a rehabilitation facility or skilled nursing facility (RF/SNF). An accurate preoperative prediction of who would need a RF/SNF place after surgery could reduce costs and allow more efficient organizational planning. We aimed to develop a machine learning algorithm that predicts non-home discharge after elective surgery for lumbar spinal stenosis. Methods: We used the American College of Surgeons National Surgical Quality Improvement Program to select patient that underwent elective surgery for lumbar spinal stenosis between 2009 and 2016. The primary outcome measure for the algorithm was non-home discharge. Four machine learning algorithms were developed to predict non-home discharge. Performance of the algorithms was measured with discrimination, calibration, and an overall performance score. Results: We included 28,600 patients with a median age of 67 (interquartile range 58–74). The non-home discharge rate was 18.2%. Our final model consisted of the following variables: age, sex, body mass index, diabetes, functional status, ASA class, level, fusion, preoperative hematocrit, and preoperative serum creatinine. The neural network was the best model based on discrimination (c-statistic = 0.751), calibration (slope = 0.933; intercept = 0.037), and overall performance (Brier score = 0.131). Conclusions: A machine learning algorithm is able to predict discharge placement after surgery for lumbar spinal stenosis with both good discrimination and calibration. Implementing this type of algorithm in clinical practice could avert risks associated with delayed discharge and lower costs. Graphical abstract: These slides can be retrieved under Electronic Supplementary Material.[Figure not available: see fulltext.].