Search

Objectives: To examine changes in technology-related errors (TREs), their manifestations and underlying mechanisms at 3 time points after the implementation of computerized provider order entry (CPOE) in an electronic health record; and evaluate the clinical decision support (CDS) available to mitigate the TREs at 5-years post-CPOE., Materials and Methods: Prescribing errors (n = 1315) of moderate, major, or serious potential harm identified through review of 35 322 orders at 3 time points (immediately, 1-year, and 4-years post-CPOE) were assessed to identify TREs at a tertiary pediatric hospital. TREs were coded using the Technology-Related Error Mechanism classification. TRE rates, percentage of prescribing errors that were TREs, and mechanism rates were compared over time. Each TRE was tested in the CPOE 5-years post-implementation to assess the availability of CDS to mitigate the error., Results: TREs accounted for 32.5% (n = 428) of prescribing errors; an adjusted rate of 1.49 TREs/100 orders (95% confidence interval [CI]: 1.06, 1.92). At 1-year post-CPOE, the rate of TREs was 40% lower than immediately post (incident rate ratio [IRR]: 0.60; 95% CI: 0.41, 0.89). However, at 4-years post, the TRE rate was not significantly different to baseline (IRR: 0.80; 95% CI: 0.59, 1.08). "New workflows required by the CPOE" was the most frequent TRE mechanism at all time points. CDS was available to mitigate 32.7% of TREs., Discussion: In a pediatric setting, TREs persisted 4-years post-CPOE with no difference in the rate compared to immediately post-CPOE., Conclusion: Greater attention is required to address TREs to enhance the safety benefits of systems., (© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Limited research has focused on understanding if and how evidence of health information technology (HIT) effectiveness drives the selection and implementation of technologies in practice. This study aimed to explore the views of senior hospital staff on the role evidence plays in the selection and implementation of HIT, with a particular focus on clinical decision support (CDS) alerts in electronic medication management systems. A qualitative descriptive design was used. Twenty senior hospital staff from six Australian hospitals in New South Wales and Queensland took part in a semistructured interview. Interviews were audio-recorded and transcribed, and a general inductive content analysis approach was used to identify themes. Participants acknowledged the importance of an evidence base, but reported that selection of CDS alerts, and HIT more broadly, was rarely underpinned by evidence that technologies improve patient care. Instead, investments in technologies were guided by the expectation that benefits will be achieved, bolstered by vendor assurances, and a perception that implementation of HIT is unavoidable. Postponing implementation of a technology until an evidence base is available was not always feasible. Although some technologies were seen as not requiring an evidence base, stakeholders viewed evidence as extremely valuable for informing decisions about selection of CDS alerts. In the absence of evidence, evaluation or monitoring of technologies postimplementation is critical, particularly to identify new errors or risks associated with HIT implementation and use. Increased transparency from vendors, with technology evaluation outcomes made directly available to healthcare organizations, may result in less reliance on logic, intuition, and vendor assertions and more evidence-based selection of HIT., (© The Author(s) 2023. Published by Oxford University Press on behalf of International Society for Quality in Health Care.)

Objective: To understand and synthesize factors influencing user acceptance of digital interventions used for antimicrobial prescribing and monitoring in hospitals., Materials and Methods: A meta-synthesis was conducted to identify qualitative studies that explored user acceptance of digital interventions for antimicrobial prescribing and/or monitoring in hospitals. Databases were searched and qualitative data were extracted and systematically classified using the unified theory of acceptance and use of technology (UTAUT) model., Results: Fifteen qualitative studies met the inclusion criteria. Eleven papers used interviews and four used focus groups. Most digital interventions evaluated in studies were decision support for prescribing (n = 13). Majority of perceptions were classified in the UTAUT performance expectancy domain in perceived usefulness and relative advantage constructs. Key facilitators in this domain included systems being trusted and credible sources of information, improving performance of tasks and increasing efficiency. Reported barriers were that interventions were not considered useful for all settings or patient conditions. Facilitating conditions was the second largest domain, which highlights the importance of users having infrastructure to support system use. Digital interventions were viewed positively if they were compatible with values, needs, and experiences of users., Conclusions: User perceptions that drive users to accept and utilize digital interventions for antimicrobial prescribing and monitoring were predominantly related to performance expectations and facilitating conditions. To ensure digital interventions for antimicrobial prescribing are accepted and used, we recommend organizations ensure systems are evaluated and benefits are conveyed to users, that utility meets expectations, and that appropriate infrastructure is in place to support use., (© The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Background: Antimicrobial stewardship (AMS) programmes in hospitals support optimal antimicrobial use by utilizing strategies such as restriction policies and education. Several systematic reviews on digital interventions supporting AMS have been conducted but they have focused on specific interventions and outcomes., Objectives: To provide a systematic overview and synthesis of evidence on the effectiveness of digital interventions to improve antimicrobial prescribing and monitoring in hospitals., Methods: Multiple databases were searched from 2010 onwards. Review papers were eligible if they included studies that examined the effectiveness of AMS digital interventions in an inpatient hospital setting. Papers were excluded if they were not systematic reviews, were limited to a paediatric setting, or were not in English., Results: Eight systematic reviews were included for data extraction. A large number of digital interventions were evaluated, with a strong focus on clinical decision support. Due to the heterogeneity of the interventions and outcome measures, a meta-analysis could not be performed. The majority of reviews reported that digital interventions reduced antimicrobial use and improved antimicrobial appropriateness. The impact of digital interventions on clinical outcomes was inconsistent., Conclusions: Digital interventions reduce antimicrobial use and improve antimicrobial appropriateness in hospitals, but no firm conclusions can be drawn about the degree to which different types of digital interventions achieve these outcomes. Evaluation of sociotechnical aspects of digital intervention implementation is limited, despite the critical role that user acceptance, uptake and feasibility play in ensuring improvements in AMS are achieved with digital health., (© The Author(s) 2022. Published by Oxford University Press on behalf of British Society for Antimicrobial Chemotherapy.)

Objective: To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches., Materials and Methods: Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included., Results: Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate., Conclusions: A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals., (© The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

Objective: The efficiency-thoroughness trade-off (ETTO) principle proposes that people and organizations are often required to make a trade-off between being efficient and being thorough, as it is difficult to be both efficient and thorough at the same time. This study aimed to compare pre- electronic medication management system (EMMS) expectation of how an EMMS is likely to impact on efficiency and thoroughness to post-EMM experiences of an EMMS and the ETTO., Design: Qualitative interview study., Setting: A paediatric oncology cancer centre in a large paediatric tertiary teaching hospital in Sydney, Australia., Participants: Forty-four semi-structured interviews with doctors, nurses and pharmacists six months prior to and two years following implementation of an EMMS., Results: Prior to EMM implementation, staff identified a number of areas of work where both efficiency and thoroughness were expected to improve with EMM. These included ease of accessibility of the medication record, and organization and legibility of medication information. Following EMMS implementation, staff reported improvements in these areas. However, the EMMS was perceived to drive thoroughness (safety) benefits at the expense of efficiency (time). Measures to improve safety in the EMMS enforced processes that required time, such as medication double-checking procedures., Conclusions: Overall, staff were aware of the competitive interplay between thoroughness and efficiency and reported that introduction of an EMMS had imposed processes that favoured improvements in thoroughness at the expense of efficiency., (© The Author(s) 2020. Published by Oxford University Press on behalf of International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Background: Chronic disease patients are adopting self-management techniques, such as using mobile health applications (apps). As GPs are the main caregivers of chronic disease patients, obtaining GP perspectives regarding patient use of mobile health apps is vital in understanding longer term value and feasibility of these apps., Objective: The aim was to determine GP perceptions of their current and potential roles in the use of health apps by their patients and how patient-focused apps affect patient management., Methods: Ten GPs participated in semi-structured, face-to-face interviews, which focused on their perceptions of, and involvement in, the use of patient-focused health apps. Interviews were transcribed verbatim and thematically analysed by two independent reviewers., Results: GPs found that apps complemented their role in patient management as additional sources of medical information of their patients. They perceived that patient-focused apps would be part of their future practices; however they noted that further work was required to incorporate them into their current practices. Currently, the main role of GPs was in promoting apps to patients. Suggestions for further engagement in mobile health included regularly reviewing patient data entered into health apps during consultations., Conclusion: GPs view patient-focused health apps positively, particularly to support them in providing patient care. Discussing information recorded in apps during consultations and frequent promotion of apps are feasible ways to integrate apps into their current work practices. Further studies involving evaluations of apps in improving health care delivery and patient communication in general practice are required., (© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Quantifying alert override has been the focus of much research in health informatics, with override rate traditionally viewed as a surrogate inverse indicator for alert effectiveness. However, relying on alert override to assess computerized alerts assumes that alerts are being read and determined to be irrelevant by users. Our research suggests that this is unlikely to be the case when users are experiencing alert overload. We propose that over time, alert override becomes habitual. The override response is activated by environmental cues and repeated automatically, with limited conscious intention. In this paper we outline this new perspective on understanding alert override. We present evidence consistent with the notion of alert override as a habitual behavior and discuss implications of this novel perspective for future research on alert override, a common and persistent problem accompanying decision support system implementation., (© The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com.)

Objectives: To (i) compare medication errors identified at audit and observation with medication incident reports; (ii) identify differences between two hospitals in incident report frequency and medication error rates; (iii) identify prescribing error detection rates by staff., Design: Audit of 3291 patient records at two hospitals to identify prescribing errors and evidence of their detection by staff. Medication administration errors were identified from a direct observational study of 180 nurses administering 7451 medications. Severity of errors was classified. Those likely to lead to patient harm were categorized as 'clinically important'., Setting: Two major academic teaching hospitals in Sydney, Australia., Main Outcome Measures: Rates of medication errors identified from audit and from direct observation were compared with reported medication incident reports., Results: A total of 12 567 prescribing errors were identified at audit. Of these 1.2/1000 errors (95% CI: 0.6-1.8) had incident reports. Clinically important prescribing errors (n = 539) were detected by staff at a rate of 218.9/1000 (95% CI: 184.0-253.8), but only 13.0/1000 (95% CI: 3.4-22.5) were reported. 78.1% (n = 421) of clinically important prescribing errors were not detected. A total of 2043 drug administrations (27.4%; 95% CI: 26.4-28.4%) contained ≥ 1 errors; none had an incident report. Hospital A had a higher frequency of incident reports than Hospital B, but a lower rate of errors at audit., Conclusions: Prescribing errors with the potential to cause harm frequently go undetected. Reported incidents do not reflect the profile of medication errors which occur in hospitals or the underlying rates. This demonstrates the inaccuracy of using incident frequency to compare patient risk or quality performance within or across hospitals. New approaches including data mining of electronic clinical information systems are required to support more effective medication error detection and mitigation., (© The Author 2015. Published by Oxford University Press in association with the International Society for Quality in Health Care.)

Objectives: To compare the manifestations, mechanisms, and rates of system-related errors associated with two electronic prescribing systems (e-PS). To determine if the rate of system-related prescribing errors is greater than the rate of errors prevented., Methods: Audit of 629 inpatient admissions at two hospitals in Sydney, Australia using the CSC MedChart and Cerner Millennium e-PS. System related errors were classified by manifestation (eg, wrong dose), mechanism, and severity. A mechanism typology comprised errors made: selecting items from drop-down menus; constructing orders; editing orders; or failing to complete new e-PS tasks. Proportions and rates of errors by manifestation, mechanism, and e-PS were calculated., Results: 42.4% (n=493) of 1164 prescribing errors were system-related (78/100 admissions). This result did not differ by e-PS (MedChart 42.6% (95% CI 39.1 to 46.1); Cerner 41.9% (37.1 to 46.8)). For 13.4% (n=66) of system-related errors there was evidence that the error was detected prior to study audit. 27.4% (n=135) of system-related errors manifested as timing errors and 22.5% (n=111) wrong drug strength errors. Selection errors accounted for 43.4% (34.2/100 admissions), editing errors 21.1% (16.5/100 admissions), and failure to complete new e-PS tasks 32.0% (32.0/100 admissions). MedChart generated more selection errors (OR=4.17; p=0.00002) but fewer new task failures (OR=0.37; p=0.003) relative to the Cerner e-PS. The two systems prevented significantly more errors than they generated (220/100 admissions (95% CI 180 to 261) vs 78 (95% CI 66 to 91))., Conclusions: System-related errors are frequent, yet few are detected. e-PS require new tasks of prescribers, creating additional cognitive load and error opportunities. Dual classification, by manifestation and mechanism, allowed identification of design features which increase risk and potential solutions. e-PS designs with fewer drop-down menu selections may reduce error risk.

Objectives: To determine the frequency with which computerized alerts occur and the proportion triggered as a result of prescribers not utilizing e-prescribing system functions., Methods: An audit of electronic inpatient medication charts at a teaching hospital in Sydney, Australia, was conducted to identify alerts fired, to categorize the system functions used by prescribers, and to assess if use of short-cut system functions could have prevented the alerts., Results: Of the 2209 active orders reviewed, 600 (27.2%) triggered at least one alert. Therapeutic duplication alerts were the most frequent (n=572). One third of these (20.2% of all alerts) was 'technically preventable' and would not have fired if prescribers had used a short-cut system function to prescribe. Under-utilized system functions included the option to 'MODIFY' existing orders and use of the 'AND' function for concurrent orders. Pregnancy alerts, set for women aged between 12 and 55 years, were triggered for 43% of drugs ordered for this group., Conclusion: Developers of decision support systems should test the extent to which technically preventable alerts may arise when prescribers fail to use system functions as designed. Designs which aim to improve the efficiency of the prescribing process but which do not align with the cognitive processes of users may fail to achieve this desired outcome and produce unexpected consequences such as triggering unnecessary alerts and user frustration. Ongoing user training to support effective use of e-prescribing system functions and modifications to the mechanisms underlying alert generation are needed to ensure that prescribers are presented with fewer but more meaningful alerts.

Objective: To assess whether a low level of decision support within a hospital computerized provider order entry system has an observable influence on the medication ordering process on ward-rounds and to assess prescribers' views of the decision support features., Methods: 14 specialty teams (46 doctors) were shadowed by the investigator while on their ward-rounds and 16 prescribers from these teams were interviewed., Results: Senior doctors were highly influential in prescribing decisions during ward-rounds but rarely used the computerized provider order entry system. Junior doctors entered the majority of medication orders into the system, nearly always ignored computerized alerts and never raised their occurrence with other doctors on ward-rounds. Interviews with doctors revealed that some decision support features were valued but most were not perceived to be useful., Discussion and Conclusion: The computerized alerts failed to target the doctors who were making the prescribing decisions on ward-rounds. Senior doctors were the decision makers, yet the junior doctors who used the system received the alerts. As a result, the alert information was generally ignored and not incorporated into the decision-making processes on ward-rounds. The greatest value of decision support in this setting may be in non-ward-round situations where senior doctors are less influential. Identifying how prescribing systems are used during different clinical activities can guide the design of decision support that effectively supports users in different situations. If confirmed, the findings reported here present a specific focus and user group for designers of medication decision support.