Search

Abstract Background Two neonatal mortality prediction models, the Neonatal Essential Treatment Score (NETS) which uses treatments prescribed at admission and the Score for Essential Neonatal Symptoms and Signs (SENSS) which uses basic clinical signs, were derived in high-mortality, low-resource settings to utilise data more likely to be available in these settings. In this study, we evaluate the predictive accuracy of two neonatal prediction models for all-cause in-hospital mortality. Methods We used retrospectively collected routine clinical data recorded by duty clinicians at admission from 16 Kenyan hospitals used to externally validate and update the SENSS and NETS models that were initially developed from the data from the largest Kenyan maternity hospital to predict in-hospital mortality. Model performance was evaluated by assessing discrimination and calibration. Discrimination, the ability of the model to differentiate between those with and without the outcome, was measured using the c-statistic. Calibration, the agreement between predictions from the model and what was observed, was measured using the calibration intercept and slope (with values of 0 and 1 denoting perfect calibration). Results At initial external validation, the estimated mortality risks from the original SENSS and NETS models were markedly overestimated with calibration intercepts of − 0.703 (95% CI − 0.738 to − 0.669) and − 1.109 (95% CI − 1.148 to − 1.069) and too extreme with calibration slopes of 0.565 (95% CI 0.552 to 0.577) and 0.466 (95% CI 0.451 to 0.480), respectively. After model updating, the calibration of the model improved. The updated SENSS and NETS models had calibration intercepts of 0.311 (95% CI 0.282 to 0.350) and 0.032 (95% CI − 0.002 to 0.066) and calibration slopes of 1.029 (95% CI 1.006 to 1.051) and 0.799 (95% CI 0.774 to 0.823), respectively, while showing good discrimination with c-statistics of 0.834 (95% CI 0.829 to 0.839) and 0.775 (95% CI 0.768 to 0.782), respectively. The overall calibration performance of the updated SENSS and NETS models was better than any existing neonatal in-hospital mortality prediction models externally validated for settings comparable to Kenya. Conclusion Few prediction models undergo rigorous external validation. We show how external validation using data from multiple locations enables model updating and improving their performance and potential value. The improved models indicate it is possible to predict in-hospital mortality using either treatments or signs and symptoms derived from routine neonatal data from low-resource hospital settings also making possible their use for case-mix adjustment when contrasting similar hospital settings.

Abstract Background Effective coverage requires that those in need can access skilled care supported by adequate resources. There are, however, few studies of effective coverage of facility-based neonatal care in low-income settings, despite the recognition that improving newborn survival is a global priority. Methods We used a detailed retrospective review of medical records for neonatal admissions to public, private not-for-profit (mission) and private-for-profit (private) sector facilities providing 24×7 inpatient neonatal care in Nairobi City County to estimate the proportion of small and sick newborns receiving nationally recommended care across six process domains. We used our findings to explore the relationship between facility measures of structure and process and estimate effective coverage. Results Of 33 eligible facilities, 28 (four public, six mission and 18 private), providing an estimated 98.7% of inpatient neonatal care in the county, agreed to partake. Data from 1184 admission episodes were collected. Overall performance was lowest (weighted mean score 0.35 [95% confidence interval or CI: 0.22–0.48] out of 1) for correct prescription of fluid and feed volumes and best (0.86 [95% CI: 0.80–0.93]) for documentation of demographic characteristics. Doses of gentamicin, when prescribed, were at least 20% higher than recommended in 11.7% cases. Larger (often public) facilities tended to have higher process and structural quality scores compared with smaller, predominantly private, facilities. We estimate effective coverage to be 25% (estimate range: 21–31%). These newborns received high-quality inpatient care, while almost half (44.5%) of newborns needed care but did not receive it and a further 30.4% of newborns received an inadequate service. Conclusions Failure to receive services and gaps in quality of care both contribute to a shortfall in effective coverage in Nairobi City County. Three-quarters of small and sick newborns do not have access to high-quality facility-based care. Substantial improvements in effective coverage will be required to tackle high neonatal mortality in this urban setting with high levels of poverty.