Search

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

PurposeThe Electronic Medical Record Search Engine (EMERSE) is a software tool built to aid research spanning cohort discovery, population health, and data abstraction for clinical trials. EMERSE is now live at three academic medical centers, with additional sites currently working on implementation. In this report, we describe how EMERSE has been used to support cancer research based on a variety of metrics.MethodsWe identified peer-reviewed publications that used EMERSE through online searches as well as through direct e-mails to users based on audit logs. These logs were also used to summarize use at each of the three sites. Search terms for two of the sites were characterized using the natural language processing tool MetaMap to determine to which semantic types the terms could be mapped.ResultsWe identified a total of 326 peer-reviewed publications that used EMERSE through August 2019, although this is likely an underestimation of the true total based on the use log analysis. Oncology-related research comprised nearly one third (n = 105; 32.2%) of all research output. The use logs showed that EMERSE had been used by multiple people at each site (nearly 3,500 across all three) who had collectively logged into the system > 100,000 times. Many user-entered search queries could not be mapped to a semantic type, but the most common semantic type for terms that did match was "disease or syndrome," followed by "pharmacologic substance."ConclusionEMERSE has been shown to be a valuable tool for supporting cancer research. It has been successfully deployed at other sites, despite some implementation challenges unique to each deployment environment.

Background: Research on social determinants of genetic testing uptake is limited, particularly among unaffected patients with inherited cancer susceptibility. Methods: We conducted a secondary analysis of the Broadening the Reach, Impact, and Delivery of Genetic Services (BRIDGE) trial at University of Utah Health and NYU Langone Health, involving 2,760 unaffected patients meeting genetic testing criteria for inherited cancer susceptibility and who were initially randomized to either an automated chatbot or an enhanced standard of care (SOC) genetic services delivery model. We used encounters from the electronic health record (EHR) to measure the uptake of genetic counseling and testing, including dichotomous measures of (1) whether participants initiated pre-test cancer genetic services, (2) completed pre-test cancer genetic services, (3) had genetic testing ordered, and (4) completed genetic testing. We merged zip codes from the EHR to construct census tract-weighted social measures of the Social Vulnerability Index. Multilevel models estimated associations between social vulnerability and genetic services utilization. We tested whether intervention condition (i.e., chatbot vs. SOC) moderated the association of social vulnerability with genetic service utilization. Covariates included study arm, study site, age, sex, race/ethnicity, language preference, rural residence, having a recorded primary care provider, and number of algorithm criteria met. Results: Patients living in areas of medium socioeconomic status (SES) vulnerability had lower odds of initiating pre-test genetic services (adjusted OR [aOR] = 0.81, 95% CI: 0.67, 0.98) compared to patients living in low SES vulnerability areas. Patients in medium household vulnerability areas had a lower likelihood of completing pre-test genetic services (aOR = 0.80, 95% CI: 0.66–0.97) and having genetic testing ordered (aOR = 0.79, 95% CI: 0.63–0.99) relative to patients in low household vulnerability areas. We did not find that social vulnerability associations varied by intervention condition. Conclusions: These results underscore the importance of investigating social and structural mechanisms as potential pathways to increasing genetic testing uptake among patients with increased inherited risk of cancer. Census information is publicly available but seldom used to assess social determinants of genetic testing uptake among unaffected populations. Existing and future cohort studies can incorporate census data to derive analytic insights for clinical scientists. Trial registration: BRIDGE was registered as NCT03985852 on June 6, 2019 at clinicaltrials.gov. [ABSTRACT FROM AUTHOR]

Hypertension is among the most important risk factors for cardiovascular disease, chronic kidney disease, and dementia. The artificial intelligence (AI) field is advancing quickly, and there has been little discussion on how AI could be leveraged for improving the diagnosis and management of hypertension. AI technologies, including machine learning tools, could alter the way we diagnose and manage hypertension, with potential impacts for improving individual and population health. The development of successful AI tools in public health and health care systems requires diverse types of expertise with collaborative relationships between clinicians, engineers, and data scientists. Unbiased data sources, management, and analyses remain a foundational challenge. From a diagnostic standpoint, machine learning tools may improve the measurement of blood pressure and be useful in the prediction of incident hypertension. To advance the management of hypertension, machine learning tools may be useful to find personalized treatments for patients using analytics to predict response to antihypertension medications and the risk for hypertension-related complications. However, there are real-world implementation challenges to using AI tools in hypertension. Herein, we summarize key findings from a diverse group of stakeholders who participated in a workshop held by the National Heart, Lung, and Blood Institute in March 2023. Workshop participants presented information on communication gaps between clinical medicine, data science, and engineering in health care; novel approaches to estimating BP, hypertension risk, and BP control; and real-world implementation challenges and issues. [ABSTRACT FROM AUTHOR]

With the advent of interoperability standards such as FHIR, SMART, CDS Hooks, and CQL, interoperable clinical decision support (CDS) holds great promise for improving healthcare. In 2018, the Agency for Healthcare Research and Quality (AHRQ)-sponsored Patient-Centered CDS Learning Network (PCCDS LN) chartered a Technical Framework Working Group (TechFWG) to identify barriers, facilitators, and potential solutions for interoperable CDS, with a specific focus on addressing the opioid epidemic. Through an open, multi-stakeholder process that engaged 54 representatives from healthcare, industry, and academia, the TechFWG identified barriers in 6 categories: regulatory environment, data integration, scalability, business case, effective and useful CDS, and care planning and coordination. Facilitators and key recommendations were also identified for overcoming these barriers. The key insights were also extrapolated to CDS-facilitated care improvement outside of the specific opioid use case. If applied broadly, the recommendations should help advance the availability and impact of interoperable CDS delivered at scale.

Interventions designed to address COVID-19 needed to be rapidly scaled up to the population level, and to address health equity by reaching historically marginalized populations most affected by the pandemic (e.g., racial/ethnic minorities and rural and low socioeconomic status populations). From February 2021 to June 2022, SCALE-UP Utah used text messaging interventions to reach 107 846 patients from 28 clinics within seven safety-net health care systems. Interventions provided informational and motivational messaging regarding COVID-19 testing and vaccination, and were developed using extensive community partner input. (Am J Public Health. 2024;114(11):1207–1211. https://doi.org/10.2105/AJPH.2024.307770) [ABSTRACT FROM AUTHOR]

Maintaining a healthy weight postintentional weight loss is crucial for preventing chronic health conditions, yet many regain weight postintervention. Electronic health record (EHR) portals offer a promising avenue for weight management interventions, leveraging patient-primary care relationships. Our previous research demonstrated that coaching alongside self-monitoring improves weight maintenance compared to monitoring alone. Integrating weight management into routine clinical practice by training existing staff could enhance scalability and sustainability. However, challenges such as inconsistent staff qualifications and high coach turnover rates could affect intervention effectiveness. Standardizing services, training, and coaching continuity seem crucial for success. To report on developing, testing, and evaluating an EHR-based coaching training program for clinical staff, guided by an implementation tool for the MAINTAIN PRIME study. Conducted across 14 University of Utah primary care sites, we developed, tested, and evaluated a coaching training for clinical staff. Guided by a planning model and the Predisposing, Enabling, and Reinforcing (PER) tool, stakeholders actively participated in planning, ensuring alignment with clinic priorities. All clinical staff were invited to participate voluntarily. Evaluation measures included staff interest, training effectiveness, confidence, and readiness. Data collection utilized REDCap, with survey results analyzed using descriptive statistics. Despite increased clinical workload and reassignments posed by coronavirus disease 2019, we were able to train 39 clinical staff, with 34 successfully coaching patients. Feedback indicated high readiness and positive perceptions of coaching feasibility. Coaches reported satisfaction with training, support, and enjoyed establishing connections with patients. The PER strategies allowed us to implement a well-received training program found effective by primary care coaches. [ABSTRACT FROM AUTHOR]

Background: Individuals with high risk for lung cancer may benefit from lung cancer screening, but there are associated risks as well as benefits. Shared decision‐making (SDM) tools with personalized information may provide key support for patients. Understanding patient perspectives on educational tools to facilitate SDM for lung cancer screening may support tool development. Aim: This study aimed to explore patient perspectives related to a SDM tool for lung cancer screening using a qualitative approach. Methods: We elicited patient perspectives by showing a provider‐facing SDM tool. Focus group interviews that ranged in duration from 1.5 to 2 h were conducted with 23 individuals with high risk for lung cancer. Data were interpreted inductively using thematic analysis to identify patients' thoughts on and desires for a patient‐facing SDM tool. Results: The findings highlight that patients would like to have educational information related to lung cancer screening. We identified several key themes to be considered in the future development of patient‐facing tools: barriers to acceptance, preference against screening and seeking empowerment. One further theme illustrated effects of patient–provider relationship as a limitation to meeting lung cancer screening information needs. Participants also noted several suggestions for the design of technology decision aids. Conclusion: These findings suggest that patients desire additional information on lung cancer screening in advance of clinical visits. However, there are several issues that must be considered in the design and development of technology to meet the information needs of patients for lung cancer screening decisions. Patient or Public Contribution: Patients, service users, caregivers or members of the public were not involved in the study design, conduct, analysis or interpretation of the data. However, clinical experts in health communication provided detailed feedback on the study protocol, including the focus group approach. The study findings contribute to a better understanding of patient expectations for lung cancer screening decisions and may inform future development of tools for SDM. [ABSTRACT FROM AUTHOR]

Objectives Patient care using genetics presents complex challenges. Clinical decision support (CDS) tools are a potential solution because they provide patient-specific risk assessments and/or recommendations at the point of care. This systematic review evaluated the literature on CDS systems which have been implemented to support genetically guided precision medicine (GPM). Materials and Methods A comprehensive search was conducted in MEDLINE and Embase, encompassing January 1, 2011–March 14, 2023. The review included primary English peer-reviewed research articles studying humans, focused on the use of computers to guide clinical decision-making and delivering genetically guided, patient-specific assessments, and/or recommendations to healthcare providers and/or patients. Results The search yielded 3832 unique articles. After screening, 41 articles were identified that met the inclusion criteria. Alerts and reminders were the most common form of CDS used. About 27 systems were integrated with the electronic health record; 2 of those used standards-based approaches for genomic data transfer. Three studies used a framework to analyze the implementation strategy. Discussion Findings include limited use of standards-based approaches for genomic data transfer, system evaluations that do not employ formal frameworks, and inconsistencies in the methodologies used to assess genetic CDS systems and their impact on patient outcomes. Conclusion We recommend that future research on CDS system implementation for genetically GPM should focus on implementing more CDS systems, utilization of standards-based approaches, user-centered design, exploration of alternative forms of CDS interventions, and use of formal frameworks to systematically evaluate genetic CDS systems and their effects on patient care. [ABSTRACT FROM AUTHOR]

Objectives To enhance the Business Process Management (BPM)+ Healthcare language portfolio by incorporating knowledge types not previously covered and to improve the overall effectiveness and expressiveness of the suite to improve Clinical Knowledge Interoperability. Methods We used the BPM+ Health and Object Management Group (OMG) standards development methodology to develop new languages, following a gap analysis between existing BPM+ Health languages and clinical practice guideline knowledge types. Proposal requests were developed based on these requirements, and submission teams were formed to respond to them. The resulting proposals were submitted to OMG for ratification. Results The BPM+ Health family of languages, which initially consisted of the Business Process Model and Notation, Decision Model and Notation, and Case Model and Notation, was expanded by adding 5 new language standards through the OMG. These include Pedigree and Provenance Model and Notation for expressing epistemic knowledge, Knowledge Package Model and Notation for supporting packaging knowledge, Shared Data Model and Notation for expressing ontic knowledge, Party Model and Notation for representing entities and organizations, and Specification Common Elements, a language providing a standard abstract and reusable library that underpins the 4 new languages. Discussion and conclusion In this effort, we adopted a strategy of separation of concerns to promote a portfolio of domain-agnostic, independent, but integrated domain-specific languages for authoring medical knowledge. This strategy is a practical and effective approach to expressing complex medical knowledge. These new domain-specific languages offer various knowledge-type options for clinical knowledge authors to choose from without potentially adding unnecessary overhead or complexity. [ABSTRACT FROM AUTHOR]

Objectives To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design. Materials and Methods Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements. Results Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist. Conclusion User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk. [ABSTRACT FROM AUTHOR]