Search

Importance: The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow., Objectives: To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle., Evidence Review: PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores., Findings: The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%)., Conclusions and Relevance: The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

Background: Diagnostics are increasingly shifting to patients' home environment, facilitated by new digital technologies. Digital diagnostics (diagnostic services enabled by digital technologies) can be a tool to better respond to the challenges faced by primary care systems while aligning with patients' and healthcare professionals' needs. However, it needs to be clarified how to determine the success of these interventions., Objectives: We aim to provide practical guidance to facilitate the adequate development and implementation of digital diagnostics., Strategy: Here, we propose the quadruple aim (better patient experiences, health outcomes and professional satisfaction at lower costs) as a framework to determine the contribution of digital diagnostics in primary care. Using this framework, we critically analyse the advantages and challenges of digital diagnostics in primary care using scientific literature and relevant casuistry., Results: Two use cases address the development process and implementation in the Netherlands: a patient portal for reporting laboratory results and digital diagnostics as part of hybrid care, respectively. The third use case addresses digital diagnostics for sexually transmitted diseases from an international perspective., Conclusions: We conclude that although evidence is gathering, the often-expected value of digital diagnostics needs adequate scientific evidence. We propose striving for evidence-based 'responsible digital diagnostics' (sustainable, ethically acceptable, and socially desirable digital diagnostics). Finally, we provide a set of conditions necessary to achieve it. The analysis and actionable guidance provided can improve the chance of success of digital diagnostics interventions and overall, the positive impact of this rapidly developing field.