Search

Human experts scoring sleep according to the American Academy of Sleep Medicine (AASM) rules are forced to select, for every 30-second epoch, one out of five stages, even if the characteristics of the neurological signals are ambiguous, a very common occurrence in clinical studies. Moreover, experts cannot score sleep in studies where these signals have not been recorded, such as in home sleep apnea testing (HSAT). In this topic review we describe how artificial intelligence can provide consistent and reliable scoring of sleep stages based on neurological signals recorded in polysomnography (PSG) and on cardiorespiratory signals recorded in HSAT. We also show how estimates of sleep stage probabilities, usually displayed as hypnodensity graph, can be used to quantify sleep stage ambiguity and stability. As an example of the application of hypnodensity in the characterization of sleep disordered breathing (SDB), we compared 49 patients with sleep apnea to healthy controls and revealed a severity-depending increase in ambiguity and decrease in stability during non-rapid eye movement (NREM) sleep. Moreover, using autoscoring of cardiorespiratory signals, we show how HSAT-derived apnea-hypopnea index and hypoxic burden are well correlated with the PSG indices in 80 patients, showing how using this technology can truly enable HSATs as alternatives to PSG to diagnose SDB.

Introduction In a recent analysis of PSG studies with six, nine and twelve human scorers, we showed that sleep stage ambiguity is the rule rather than the exception, and that sleep stage probabilities (SSPs) calculated with artificial intelligence provide an excellent estimate of this ambiguity. Here we investigate clinical and demographic factors driving this ambiguity and evaluate potential benefits of using hypnodensity in addition to the traditional hypnogram for sleep assessment. Methods SSPs were determined in 195 healthy subjects aged 20–95 years and in 49 apnea patients aged 29–73 years (AHI: 5.8–105.5 events/hour). In addition to the recommended sleep parameters, we derived the percentage of ambiguous sleep stages (SSP ≤ 0.95), the mean amount of ambiguity per sleep stage (1-SSP), sleep stage continuity (absolute difference in SSP between two epochs) and sleep stage stability (two adjacent epochs belonging to the same sleep stage with SSP > 0.95 each) as well as NREM sleep depth (weighted average of NREM SSPs). Results The percentages of ambiguous NREM epochs (±SD) increased significantly with age (Pearson’s r=0.72; p< 0.01) from 40±9% in young healthy subjects (20-39 years, n=61) to 50±9% in middle-aged healthy subjects (40-59 years, n=59) and to 62±11% in older healthy subjects (≥ 60 years, n=75). The percentages of ambiguous REM epochs did not change significantly across age (r=0.1). Apnea patients showed an increased NREM ambiguity (77±15% compared to 53±10% in age- and sex-matched healthy controls; p< 0.01), while REM ambiguity was only slightly increased (p< 0.05). Furthermore, sleep stage continuity, stability, and NREM sleep depth decrease significantly with age and are significantly lower in apnea patients than in healthy controls (p< 0.01). Conclusion Artificial intelligence-based autoscoring shows ambiguity in 40% of NREM epochs in young healthy subjects, increasing to 77% in apnea patients. Assigning a single sleep stage to each epoch and presenting sleep architecture as a traditional hypnogram may be misleading, especially for older subjects and patients with sleep-disordered breathing. A hypnodensity chart representing sleep stage probabilities reflects this sleep staging ambiguity, provides all the information contained in a hypnogram, and offers insights into sleep ambiguity, continuity, stability, and sleep depth. Support (if any)

Conventionally, sleep and associated events are scored visually by trained technologists according to the rules summarized in the American Academy of Sleep Medicine Manual. Since its first publication in 2007, the manual was continuously updated; the most recent version as of this writing was published in 2020. Human expert scoring is considered as gold standard, even though there is increasing evidence of limited interrater reliability between human scorers. Significant advances in machine learning have resulted in powerful methods for addressing complex classification problems such as automated scoring of sleep and associated events. Evidence is increasing that these autoscoring systems deliver performance comparable to manual scoring and offer several advantages to visual scoring: (1) avoidance of the rather expensive, time-consuming, and difficult visual scoring task that can be performed only by well-trained and experienced human scorers, (2) attainment of consistent scoring results, and (3) proposition of added value such as scoring in real time, sleep stage probabilities per epoch (hypnodensity), estimates of signal quality and sleep/wake-related features, identifications of periods with clinically relevant ambiguities (confidence trends), configurable sensitivity and rule settings, as well as cardiorespiratory sleep staging for home sleep apnea testing. This chapter describes the development of autoscoring systems since the first attempts in the 1970s up to the most recent solutions based on deep neural network approaches which achieve an accuracy that allows to use the autoscoring results directly for review and interpretation by a physician.

Study Objectives To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers. Methods We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6–12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule. Results The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01). Conclusions Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment.

Introduction There have been significant advances in machine learning in recent years. This means that powerful methods are now available for classification problems, such as scoring sleep stages from neurological or cardiorespiratory signals. In the present work, validation studies for both applications are presented. Methods To determine the 5 sleep stages from the neurological signals, 54 sleep-wake-related features were calculated and classified by a bidirectional long short-term memory (LSTM) network which had been trained on 1956 manual scorings of 588 PSGs from 294 subjects (supervised deep learning). To determine the 4 stages (wake, light sleep, deep sleep, REM) from cardiorespiratory signals, a convolutional neural network combined with LSTM layers was used for feature extraction and classification. This network had been trained on 685 PSGs from 391 subjects (Bakker et al. JCSM 2021). The networks obtained were validated in 428 PSGs with one and 10 PSGs with 12 manual scorings (neurological staging) as well as in 2 two datasets, each containing 296 ambulatory recordings (cardiorespiratory staging). Results Cohen’s kappa between autoscoring based on neurological signals and manual scoring was 0.74 (95%-confidence interval: 0.74-0.74) for the 428 PSGs. The intraclass correlation coefficient (ICC) for absolute agreement between autoscoring and manual scoring was for the AHI 0.97 (0.96-0.98), for the arousal index 0.79 (0.67-0.86) and for the PLMSI 0.91 (0.88-0.93). The ICC between the sleep stage probabilities derived from the 12 manual scorings and the artificial intelligence (AI) derived hypnodensity was 0.91 (0.91-0.91). Cohen’s kappa values for the cardiorespiratory sleep staging were 0.68 (0.68-0.68) and 0.64 (0.63-0.64) for the 2 datasets with 296 ambulatory recordings each. Conclusion All metrics from the PSG validation studies show substantial (Cohen’s kappa > 0.6) as well as good to excellent agreement (ICC > 0.75 or > 0.90) compared to manual scorings. As an added value of the AI-supported PSG evaluation, the probabilities of the sleep stages per epoch are determined (hypnodensity graph). The valid estimation of the sleep stages from cardiorespiratory signals by means of AI may result in improved clinical interpretation of home sleep apnea tests, which are increasingly used in the sleep-disordered breathing diagnostic pathway. Support (If Any)