Artificial intelligence based software facilitates spirometry quality control in asthma and COPD clinical trials.

Rationale: Acquiring high-quality spirometry data in clinical trials is important, particularly when using forced expiratory volume in 1 s or forced vital capacity as primary end-points. In addition to quantitative criteria, the American Thoracic Society (ATS)/European Respiratory Society (ERS) standards include subjective evaluation which introduces inter-rater variability and potential mistakes. We explored the value of artificial intelligence (AI)-based software (ArtiQ.QC) to assess spirometry quality and compared it to traditional over-reading control.
Methods: A random sample of 2000 sessions (8258 curves) was selected from Chiesi COPD and asthma trials (n=1000 per disease). Acceptability using the 2005 ATS/ERS standards was determined by over-reader review and by ArtiQ.QC. Additionally, three respiratory physicians jointly reviewed a subset of curves (n=150).
Results: The majority of curves (n=7267, 88%) were of good quality. The AI agreed with over-readers in 91% of cases, with 97% sensitivity and 93% positive predictive value. Performance was significantly better in the asthma group. In the revised subset, n=50 curves were repeated to assess intra-rater reliability (κ=0.83, 0.86 and 0.80 for each of the three reviewers). All reviewers agreed on 63% of 100 unique tests (κ=0.5). When reviewers set the consensus (gold standard), individual agreement with it was 88%, 94% and 70%. The agreement between AI and "gold-standard" was 73%; over-reader agreement was 46%.
Conclusion: AI-based software can be used to measure spirometry data quality with comparable accuracy as experts. The assessment is a subjective exercise, with intra- and inter-rater variability even when the criteria are defined very precisely and objectively. By providing consistent results and immediate feedback to the sites, AI may benefit clinical trial conduct and variability reduction.
Competing Interests: Conflict of interest: E. Topole, S. Biondaro, I. Montagna and S. Corre are all employees of Chiesi Farmaceutici S.p.A. K. Ray was an employee of ArtiQ NV, Belgium at the time the study was performed. M. Topalovic is a founder of ArtiQ NV, Belgium. M. Corradi received research funds and professional fees from Chiesi Farmaceutici S.p.A. B. Graham received professional fees from Chiesi Farmaceutici S.p.A., MGC Diagnostics, Vyaire Medical and the Lung Association of Saskatchewan; and has a patent application underway related to pulmonary function testing.
(Copyright ©The authors 2023.)