Search

Advances in smartphone technology have allowed people to access mental healthcare via digital apps from wherever and whenever they choose. University students experience a high burden of mental health concerns. Although these apps improve mental health symptoms, user engagement has remained low. Studies have shown that users can be subgrouped based on unique characteristics that just-in-time adaptive interventions (JITAIs) can use to improve engagement. To date, however, no studies have examined the effect of the COVID-19 pandemic on these subgroups. Here, we use machine learning to examine user subgroup characteristics across three COVID-19-specific timepoints: during lockdown, immediately following lockdown, and three months after lockdown ended. We demonstrate that there are three unique subgroups of university students who access mental health apps. Two of these, with either higher or lower mental well-being, were defined by characteristics that were stable across COVID-19 timepoints. The third, situational well-being, had characteristics that were timepoint-dependent, suggesting that they are highly influenced by traumatic stressors and stressful situations. This subgroup also showed feelings and behaviours consistent with burnout. Overall, our findings clearly suggest that user subgroups are unique: they have different characteristics and therefore likely have different mental healthcare goals. Our findings also highlight the importance of including questions and additional interventions targeting traumatic stress(ors), reason(s) for use, and burnout in JITAI-style mental health apps to improve engagement.

Background Evaluations of computerized provider order entry (CPOE) systems have revealed that reductions in certain types of medication errors occur simultaneously with the emergence of system-related errors – errors that are unlikely or not possible to occur with the use of paper-based medication charts. System-related errors appear to persist many years post-implementation of CPOE, although little is known about whether the types and rates of system-related errors that occur immediately following CPOE implementation are similar to those that endure or emerge after years of system use. Objective To analyze and synthesize the literature on system-related errors, specifically in relation to the length of time that CPOE systems have been in use, to determine what is currently known about how system-related errors change over time. Methods A literature search was undertaken using the PubMed database to identify English language articles published between January 2005 and March 2020 that provided original data on system-related errors resulting from CPOE system use. Studies were included if they provided results on system-related errors and information relating to the length of time that CPOE had been in use. Results Thirty-one studies met the inclusion criteria for this narrative review. System-related errors were identified and described during short, medium and long-term use of CPOE systems, but no single study examined how errors changed over time. In comparing findings across studies, results suggest that system-related errors persist with long-term use of CPOE systems, although likely to occur at a reduced rate. Conclusions This review has highlighted a significant gap in knowledge on how system-related errors change over time. Determining what and when system-related errors occur and the system factors that contribute to their occurrence at different time points after CPOE implementation is necessary for the future prevention and mitigation of these errors.

Background Despite growing evidence that deprescribing can improve clinical outcomes, quality of life and reduce the likelihood of adverse drug events, the practice is not widespread, particularly in hospital settings. Clinical risk assessment tools, like the Drug Burden Index (DBI), can help prioritise patients for medication review and prioritise medications to deprescribe, but are not integrated within routine care. The aim of this study was to conduct formative usability testing of a computerised decision support (CDS) tool, based on DBI, to identify modifications required to the tool prior to trialling in practice. Methods Our CDS tool comprised a DBI MPage in the electronic medical record (clinical workspace) that facilitated review of a patient’s DBI and medication list, access to deprescribing resources, and the ability to deprescribe. Two rounds of scenario-based formative usability testing with think-aloud protocol were used. Seventeen end-users participated in the testing, including junior and senior doctors, and pharmacists. Results Participants expressed positive views about the DBI CDS tool but testing revealed a number of clear areas for improvement. These primarily related to terminology used (i.e. what is a DBI and how is it calculated?), and consistency of functionality and display. A key finding was that users wanted the CDS tool to look and function in a similar way to other decision support tools in the electronic medical record. Modifications were made to the CDS tool in response to user feedback. Conclusion Usability testing proved extremely useful for identifying components of our CDS tool that were confusing, difficult to locate or to understand. We recommend usability testing be adopted prior to implementation of any digital health intervention. We hope our revised CDS tool equips clinicians with the knowledge and confidence to consider discontinuation of inappropriate medications in routine care of hospitalised patients. In the next phase of our project, we plan to pilot test the tool in practice to evaluate its uptake and effectiveness in supporting deprescribing in routine hospital care.

Objective To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches. Materials and methods Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included. Results Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate. Conclusions A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

Limited research has focused on understanding if and how evidence of health information technology (HIT) effectiveness drives the selection and implementation of technologies in practice. This study aimed to explore the views of senior hospital staff on the role evidence plays in the selection and implementation of HIT, with a particular focus on clinical decision support (CDS) alerts in electronic medication management systems. A qualitative descriptive design was used. Twenty senior hospital staff from six Australian hospitals in New South Wales and Queensland took part in a semistructured interview. Interviews were audio-recorded and transcribed, and a general inductive content analysis approach was used to identify themes. Participants acknowledged the importance of an evidence base, but reported that selection of CDS alerts, and HIT more broadly, was rarely underpinned by evidence that technologies improve patient care. Instead, investments in technologies were guided by the expectation that benefits will be achieved, bolstered by vendor assurances, and a perception that implementation of HIT is unavoidable. Postponing implementation of a technology until an evidence base is available was not always feasible. Although some technologies were seen as not requiring an evidence base, stakeholders viewed evidence as extremely valuable for informing decisions about selection of CDS alerts. In the absence of evidence, evaluation or monitoring of technologies postimplementation is critical, particularly to identify new errors or risks associated with HIT implementation and use. Increased transparency from vendors, with technology evaluation outcomes made directly available to healthcare organizations, may result in less reliance on logic, intuition, and vendor assertions and more evidence-based selection of HIT.

Summary Objective: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institutional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries. Methods: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to identify and discuss shared themes. Results: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices. Conclusion: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowledge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered.

Background Electronic medication management (eMM) has been shown to reduce medication errors; however, new safety risks have also been introduced that are associated with system use. No research has specifically examined the changes made to eMM systems to mitigate these risks. Objectives To (1) identify system-related medication errors or workflow blocks that were the target of eMM system updates, including the types of medications involved, and (2) describe and classify the system enhancements made to target these risks. Methods In this retrospective qualitative study, documents detailing updates made from November 2014 to December 2019 to an eMM system were reviewed. Medication-related updates were classified according to “rationale for changes” and “changes made to the system.” Results One hundred and seventeen updates, totaling 147 individual changes, were made to the eMM system over the 4-year period. The most frequent reasons for changes being made to the eMM were to prevent medication errors (24% of reasons), optimize workflow (22%), and support “work as done” on paper (16%). The most frequent changes made to the eMM were options added to lists (14% of all changes), extra information made available on the screen (8%), and the wording or phrasing of text modified (8%). Approximately a third of the updates (37%) related to high-risk medications. The reasons for system changes appeared to vary over time, as eMM functionality and use expanded. Conclusion To our knowledge, this is the first study to systematically review and categorize system updates made to overcome new safety risks associated with eMM use. Optimization of eMM is an ongoing process, which changes over time as users become more familiar with the system and use is expanded to more sites. Continuous monitoring of the system is necessary to detect areas for improvement and capitalize on the benefits an electronic system can provide.

Electronic medication management (eMM) systems can have a significant impact on efficiency and safety. There is limited evidence on the effects of eMM implementation on the physical location of work. The objective of this study was to evaluate the impact of eMM and associated hardware implementation on the location of tasks performed by doctors and nurses. 41.5 hours of observation were conducted in the oncology ward of a paediatric hospital. Tasks, locations and resources used were recorded pre and post eMM implementation. Results showed that a wider variety of locations were used to conduct tasks following eMM implementation. Post-eMM, more tasks were performed in the hallway, where medication trolleys with attached laptops were situated, and in patient rooms where additional computers were installed, providing more opportunities for patient/carer and clinician interaction. The findings from this study reveal the impact that computer placement has on the location of work for doctors and nurses, and the importance of planning hardware placement for eMM implementation.

Electronic medication management (eMM) systems can have a significant impact on efficiency and safety. There is limited evidence on the effects of eMM implementation on the physical location of work. The objective of this study was to evaluate the impact of eMM and associated hardware implementation on the location of tasks performed by doctors and nurses. 41.5 hours of observation were conducted in the oncology ward of a paediatric hospital. Tasks, locations and resources used were recorded pre and post eMM implementation. Results showed that a wider variety of locations were used to conduct tasks following eMM implementation. Post-eMM, more tasks were performed in the hallway, where medication trolleys with attached laptops were situated, and in patient rooms where additional computers were installed, providing more opportunities for patient/carer and clinician interaction. The findings from this study reveal the impact that computer placement has on the location of work for doctors and nurses, and the importance of planning hardware placement for eMM implementation.

IntroductionThere is a clear need for improved care quality and quality monitoring in aged care. Aged care providers collect an abundance of data, yet rarely are these data integrated and transformed in real-time into actionable information to support evidence-based care, nor are they shared with older people and informal caregivers. This protocol describes the co-design and testing of a dashboard in residential aged care facilities (nursing or care homes) and community-based aged care settings (formal care provided at home or in the community). The dashboard will comprise integrated data to provide an ‘at-a-glance’ overview of aged care clients, indicators to identify clients at risk of fall-related hospitalisations and poor quality of life, and evidence-based decision support to minimise these risks. Longer term plans for dashboard implementation and evaluation are also outlined.MethodsThis mixed-method study will involve (1) co-designing dashboard features with aged care staff, clients, informal caregivers and general practitioners (GPs), (2) integrating aged care data silos and developing risk models, and (3) testing dashboard prototypes with users. The dashboard features will be informed by direct observations of routine work, interviews, focus groups and co-design groups with users, and a community forum. Multivariable discrete time survival models will be used to develop risk indicators, using predictors from linked historical aged care and hospital data. Dashboard prototype testing will comprise interviews, focus groups and walk-through scenarios using a think-aloud approach with staff members, clients and informal caregivers, and a GP workshop.Ethics and disseminationThis study has received ethical approval from the New South Wales (NSW) Population & Health Services Research Ethics Committee and Macquarie University’s Human Research Ethics Committee. The research findings will be presented to the aged care provider who will share results with staff members, clients, residents and informal caregivers. Findings will be disseminated as peer-reviewed journal articles, policy briefs and conference presentations.

Objective To identify factors that prevent and promote uptake of medication-related computerized decision support systems (CDSS) in hospitals, based on the perceptions of prescribers. Materials and methods Databases Medline, Embase, CINAHL, PubMed and PsycINFO and the top five medical informatics journals were searched. English papers published after 2002, which used a qualitative approach to examine prescriber views of CDSS in hospitals were included. Qualitative data were extracted and mapped to the three domains of the HOT-fit framework (human, organization, and technology). Results Factors preventing CDSS uptake were perceived threats to autonomy, CDSS conflicting with personal prescribing preferences, and mistrust of CDSS information. Factors promoting CDSS uptake were perceptions that CDSS improves safety and efficiency, and is easy to use. With respect to medication alerts, large numbers of irrelevant alerts reportedly led to alerts being ignored. When using order sentences/order sets, long lists of options led to excessive scrolling or clicks, and resulted in users opting for free text ordering. Discussion and conclusion To promote medication-related CDSS uptake, it is recommended that prescribers’ perspectives on CDSS usability and integration into workflow be sought during the design phase, that evidence on CDSS’ effectiveness to improve safety be provided to prescribers, and that system information be kept up to date. To improve alert uptake, organizations should ensure that alerts and minimal and relevant. To improve uptake of order sentences/order sets, organizations should minimise the number of options available to prescribers. Future work should focus on exploring prescriber perceptions of other types of CDSS.

This study aimed to assess drug–drug interaction alert interfaces and to examine the relationship between compliance with human factors principles and user-preferences of alerts. Three reviewers independently evaluated drug–drug interaction alert interfaces in seven electronic systems using the Instrument-for-Evaluating-Human-Factors-Principles-in-Medication-Related-Decision-Support-Alerts (I-MeDeSA). Fifty-three doctors and pharmacists completed a survey to rate the alert interfaces from best to worst and reported on liked and disliked features. Human factors compliance and user-preferences of alerts were compared. Statistical analysis revealed no significant association between I-MeDeSA scores and user-preferences. However, the strengths and weaknesses of drug–drug interaction alerts from users’ perspectives were in-line with the human factors constructs evaluated by the I-MeDeSA. I-MeDeSA in its current form, is unable to identify alerts that are preferred by the users. The design principles assessed by I-MeDeSA appear to be sound, but its arbitrary allocation of points to each human factors construct may not reflect the relative importance that the end-users place on different aspects of alert design.

The WOMBAT pilot selection test is widely used to select candidates for pilot training programs. Despite use in many countries, little information is available regarding the predictive validity of the test. This study was designed to test the ability of the WOMBAT test to predict performance outcomes in a sample of ab-initio pilots. Sixty students commenced the study in 3 cohorts, and completed the WOMBAT test before their performance in the training program was evaluated through flight time to solo, flight time to licence level achievement, and instructor ratings of performance. Higher WOMBAT total scores were significantly related to reduced time to solo, achieving flight licences, as well as some early ratings of performance by flight instructors. Further research now needs to examine the exact nature of the skills and abilities that the test indexes in order to further improve pilot selection and training procedures.

Background Recently, attention has shifted to improving the design of computerized alerts via the incorporation of human factors design principles. The Instrument for Evaluating Human Factors Principles in Medication-Related Decision Support Alerts (I-MeDeSA) is a tool developed in the United States to guide improvements to alert design and facilitate selection of electronic systems with superior design. In this study, we aimed to determine the reliability, ease of use and usefulness of I-MeDeSA for assessing drug-drug interaction (DDI) alerts in an Australian context. Methods Using the I-MeDeSA, three reviewers independently evaluated DDI alert interfaces of seven electronic systems used in Australia. Inter-rater reliability was assessed and reviewers met to discuss difficulties in using I-MeDeSA and the tool’s usefulness. Results Inter-rater reliability was high (Krippendorff’s alpha = 0.76), however, ambiguous wording and the inclusion of conditional items impacted ease of use. A number of items were not relevant to Australian implementations and as a result, most systems achieved an I-MeDeSA score of less than 50%. Conclusions The I-MeDeSA proved to be reliable, but item wording and structure made application difficult. Future studies should investigate potential modifications to the I-MeDeSA to improve ease of use and increase applicability to a variety of system configurations. Electronic supplementary material The online version of this article (10.1186/s12911-018-0666-y) contains supplementary material, which is available to authorized users.

Background and objectives Deprescribing, the supervised withdrawal of inappropriate medications, intends to manage polypharmacy, which is prevalent in older patients. This study aimed to examine general practitioner (GP) perceptions of communication processes between clinicians in hospital and GPs in the community about deprescribing decisions made in hospital. Method Focus groups and interviews were held with 15 GPs, exploring deprescribing in hospitals, communication of deprescribing information, and the format of communications. Sessions were audio-taped, transcribed and analysed using an inductive approach. Results GPs stated that they should be involved in deprescribing decisions, especially for older complex patients, because of their good knowledge of their patients. Barriers to effective communication included the acute nature of hospital stays and lack of time. Facilitators included long-term relationships of GPs with their patients and engaged patients. GPs preferred communication of deprescribing decisions to be over the telephone while the patient was still in hospital, and with a concise, electronic discharge summary at the time of discharge. GPs indicated that rationale for medication changes and recommended follow-up actions were crucial in a discharge summary to enable care post-discharge. Conclusions GPs welcome increased communication with hospital clinicians regarding deprescribing decisions made while patients are in hospital. Communication needs to be timely, transparent, succinct and accessible. Lack of time and difficulties contacting hospital clinicians challenge this process. This article is protected by copyright. All rights reserved.

BACKGROUND It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems. OBJECTIVE This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems. METHODS The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability. RESULTS The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity. CONCLUSIONS This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.

Objective The efficiency–thoroughness trade-off (ETTO) principle proposes that people and organizations are often required to make a trade-off between being efficient and being thorough, as it is difficult to be both efficient and thorough at the same time. This study aimed to compare pre- electronic medication management system (EMMS) expectation of how an EMMS is likely to impact on efficiency and thoroughness to post-EMM experiences of an EMMS and the ETTO. Design Qualitative interview study. Setting A paediatric oncology cancer centre in a large paediatric tertiary teaching hospital in Sydney, Australia. Participants Forty-four semi-structured interviews with doctors, nurses and pharmacists six months prior to and two years following implementation of an EMMS. Results Prior to EMM implementation, staff identified a number of areas of work where both efficiency and thoroughness were expected to improve with EMM. These included ease of accessibility of the medication record, and organization and legibility of medication information. Following EMMS implementation, staff reported improvements in these areas. However, the EMMS was perceived to drive thoroughness (safety) benefits at the expense of efficiency (time). Measures to improve safety in the EMMS enforced processes that required time, such as medication double-checking procedures. Conclusions Overall, staff were aware of the competitive interplay between thoroughness and efficiency and reported that introduction of an EMMS had imposed processes that favoured improvements in thoroughness at the expense of efficiency.

Background It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems. Objective This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems. Methods The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability. Results The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity. Conclusions This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.