Search

IntroductionUniversity Hospitals Leicester has codeveloped, with Nervecentre, an Electronic Prescribing and Medicines Administration System that meets specific clinical and interoperability demands of the National Health Service (NHS).MethodsThe system was developed through a frontline-led and agile approach with a project team consisting of clinicians, Information Technology (IT) specialists and the vendor’s representatives over an 18-month period.ResultsThe system was deployed successfully with more than a thousand transcriptions during roll-out. Despite the high caseload and novelty of the system, there was no increase in error rates within the first 3 months of roll-out. Healthcare professionals perceived the new system as efficient with improved clinical workflow, and safe through an integrated medication alert system.DiscussionThis case study demonstrates how NHS trusts can successfully co-develop, with vendors, new IT systems which meet interoperability standards such as Fast Healthcare Interoperability Resources, while improving front line clinical experience.ConclusionAlternative methods to the ‘big bang’ deployment of IT projects, such as ‘gradual implementation’, must be demonstrated and evaluated for their ability to deliver digital transformation projects in the NHS successfully.

BMJ Health & Care Informatics (BMJHCI) is launching a partnership programme, where patients write articles and serve as peer reviewers on both patient-written and researcher-written articles. This article outlines the programme and describes the importance of public involvement in research and implementation in digital health. We think patients and carers should be funded to participate at this stage of research as well as other stages of research. The quality of peer review can be greatly improved by recruiting patients to peer review and improve readability and understanding of scientific literature and to ensure that research and other articles appropriately include what matters most to patients. Just as real-time communication is two-way communication, both healthcare providers and patients should have a voice in the literature, and involving patients in journals is an important step toward amplifying and supporting the balance of perspectives. Patients are the whole purpose of research and practice in health and care, so this rightly includes their role in the publication and review of health informatics literature as well as the publication of their own perspectives regarding access and delivery of healthcare. Patients and carers can provide valuable insights into research articles, and they can also serve as effective peer reviewers. The BMJHCI is excited to kick off the new partnership programme and encourages all interested patients and carers to apply to participate as authors and/or reviewers.

Systematic reviews show that clinical decision support systems (CDSSs) can improve the quality of clinical decisions and healthcare processes[1][1] and patient outcomes[2][2]; although caution has been expressed as to balancing the risks of using CDSSs (eg, alert fatigue) when only small or moderate

ObjectivesThe value of healthcare data is being increasingly recognised, including the need to improve health dataset utility. There is no established mechanism for evaluating healthcare dataset utility making it difficult to evaluate the effectiveness of activities improving the data. To describe the method for generating and involving the user community in developing a proposed framework for evaluation and communication of healthcare dataset utility for given research areas.MethodsAn initial version of a matrix to review datasets across a range of dimensions was developed based on previous published findings regarding healthcare data. This was used to initiate a design process through interviews and surveys with data users representing a broad range of user types and use cases, to help develop a focused framework for characterising datasets.ResultsFollowing 21 interviews, 31 survey responses and testing on 43 datasets, five major categories and 13 subcategories were identified as useful for a dataset, including Data Model, Completeness and Linkage. Each sub-category was graded to facilitate rapid and reproducible evaluation of dataset utility for specific use-cases. Testing of applicability to >40 existing datasets demonstrated potential usefulness for subsequent evaluation in real-world practice.DiscussionThe research has developed an evidenced-based initial approach for a framework to understand the utility of a healthcare dataset. It is likely to require further refinement following wider application and additional categories may be required.ConclusionThe process has resulted in a user-centred designed framework for objectively evaluating the likely utility of specific healthcare datasets, and therefore, should be of value both for potential users of health data, and for data custodians to identify the areas to provide the optimal value for data curation investment.

ObjectiveGastrointestinal (GI) bleeding commonly requires intensive care unit (ICU) in cases of potentialhaemodynamiccompromise or likely urgent intervention. However, manypatientsadmitted to the ICU stop bleeding and do not require further intervention, including blood transfusion. The present work proposes an artificial intelligence (AI) solution for the prediction of rebleeding in patients with GI bleeding admitted to ICU.MethodsA machine learning algorithm was trained and tested using two publicly available ICU databases, the Medical Information Mart for Intensive Care V.1.4 database and eICU Collaborative Research Database using freedom from transfusion as a proxy for patients who potentially did not require ICU-level care. Multiple initial observation time frames were explored using readily available data including labs, demographics and clinical parameters for a total of 20 covariates.ResultsThe optimal model used a 5-hour observation period to achieve an area under the curve of the receiving operating curve (ROC-AUC) of greater than 0.80. The model was robust when tested against both ICU databases with a similar ROC-AUC for all.ConclusionsThe potential disruptive impact of AI in healthcare innovation is acknowledge, but awareness of AI-related risk on healthcare applications and current limitations should be considered before implementation and deployment. The proposed algorithm is not meant to replace but to inform clinical decision making. Prospective clinical trial validation as a triage tool is warranted.

ObjectivesPrior research has reported an increased risk of fatality for patients with cancer, but most studies investigated the risk by comparing cancer to non-cancer patients among COVID-19 infections, where cancer might have contributed to the increased risk. This study is to understand COVID-19’s imposed HR of fatality while controlling for covariates, such as age, sex, metastasis status and cancer type.MethodsWe conducted survival analyses of 4606 cancer patients with COVID-19 test results from 16 March to 11 October 2020 in UK Biobank and estimated the overall HR of fatality with and without COVID-19 infection. We also examined the HRs of 13 specific cancer types with at least 100 patients using a stratified analysis.ResultsCOVID-19 resulted in an overall HR of 7.76 (95% CI 5.78 to 10.40, p−10) by following 4606 patients with cancer for 21 days after the tests. The HR varied among cancer type, with over a 10-fold increase in fatality rate (false discovery rate ≤0.02) for melanoma, haematological malignancies, uterine cancer and kidney cancer. Although COVID-19 imposed a higher risk for localised versus distant metastasis cancers, those of distant metastases yielded higher overall fatality rates due to their multiplicative effects.DiscussionThe results confirmed prior reports for the increased risk of fatality for patients with COVID-19 plus hematological malignancies and demonstrated similar findings of COVID-19 on melanoma, uterine, and kidney cancers.ConclusionThe results highlight the heightened risk that COVID-19 imposes on localised and haematological cancer patients and the necessity to vaccinate uninfected patients with cancer promptly, particularly for the cancer types most influenced by COVID-19. Results also suggest the importance of timely care for patients with localised cancer, whether they are infected by COVID-19 or not.

ObjectivesUnstructured free-text patient feedback contains rich information, and analysing these data manually would require a lot of personnel resources which are not available in most healthcare organisations.To undertake a systematic review of the literature on the use of natural language processing (NLP) and machine learning (ML) to process and analyse free-text patient experience data.MethodsDatabases were systematically searched to identify articles published between January 2000 and December 2019 examining NLP to analyse free-text patient feedback. Due to the heterogeneous nature of the studies, a narrative synthesis was deemed most appropriate. Data related to the study purpose, corpus, methodology, performance metrics and indicators of quality were recorded.ResultsNineteen articles were included. The majority (80%) of studies applied language analysis techniques on patient feedback from social media sites (unsolicited) followed by structured surveys (solicited). Supervised learning was frequently used (n=9), followed by unsupervised (n=6) and semisupervised (n=3). Comments extracted from social media were analysed using an unsupervised approach, and free-text comments held within structured surveys were analysed using a supervised approach. Reported performance metrics included the precision, recall and F-measure, with support vector machine and Naïve Bayes being the best performing ML classifiers.ConclusionNLP and ML have emerged as an important tool for processing unstructured free text. Both supervised and unsupervised approaches have their role depending on the data source. With the advancement of data analysis tools, these techniques may be useful to healthcare organisations to generate insight from the volumes of unstructured free-text data.

ObjectivesWe describe a hospital’s implementation of predictive models to optimise emergency response to the COVID-19 pandemic.MethodsWe were tasked to construct and evaluate COVID-19 driven predictive models to identify possible planning and resource utilisation scenarios. We used system dynamics to derive a series of chain susceptible, infected and recovered (SIR) models. We then built a discrete event simulation using the system dynamics output and bootstrapped electronic medical record data to approximate the weekly effect of tuning surgical volume on hospital census. We evaluated performance via a model fit assessment and cross-model comparison.ResultsWe outlined the design and implementation of predictive models to support management decision making around areas impacted by COVID-19. The fit assessments indicated the models were most useful after 30 days from onset of local cases. We found our subreports were most accurate up to 7 days after model run.DiscusssionOur model allowed us to shape our health system’s executive policy response to implement a ‘hospital within a hospital’—one for patients with COVID-19 within a hospital able to care for the regular non-COVID-19 population. The surgical schedule is modified according to models that predict the number of new patients with Covid-19 who require admission. This enabled our hospital to coordinate resources to continue to support the community at large. Challenges included the need to frequently adjust or create new models to meet rapidly evolving requirements, communication, and adoption, and to coordinate the needs of multiple stakeholders. The model we created can be adapted to other health systems, provide a mechanism to predict local peaks in cases and inform hospital leadership regarding bed allocation, surgical volumes, staffing, and supplies one for COVID-19 patients within a hospital able to care for the regular non-COVID-19 population.ConclusionPredictive models are essential tools in supporting decision making when coordinating clinical operations during a pandemic.

ObjectivesUntil this point there was no national core competency framework for clinical informatics in the UK. We report on the final two iterations of work carried out in the formation of a national core competency framework. This follows an initial systematic literature review of existing skills and competencies and a job listing analysis.MethodsAn iterative approach was applied to framework development. Using a mixed-methods design we carried out semi-structured interviews with participants involved in informatics (n=15). The framework was updated based on the interview findings and was subsequently distributed as part of a bespoke online digital survey for wider participation (n=87). The final version of the framework is based on the findings of the survey.ResultsOver 102 people reviewed the framework as part of the interview or survey process. This led to a final core competency framework containing 6 primary domains with 36 subdomains containing 111 individual competencies.ConclusionsAn iterative mixed-methods approach for competency development involving the target community was appropriate for development of the competency framework. There is some contention around the depth of technical competencies required. Care is also needed to avoid professional burnout, as clinicians and healthcare practitioners already have clinical competencies to maintain. Therefore, how the framework is applied in practice and how practitioners meet the competencies requires careful consideration.

ObjectivesArgentina is a low and middle-income country (LMIC) with a highly fragmented healthcare system that conflicts with access to healthcare stated by the country’s Universal Health Coverage plan. A tele-mammography network could improve access to breast cancer screening decreasing its mortality. This research aims to conduct an economic evaluation of the implementation of a tele-mammography program to improve access to healthcare.MethodsA cost-utility analysis was performed to explore the incremental benefit of annual tele-mammography screening for at-risk Argentinian women over 40 years old. A Markov model was developed to simulate annual mammography or tele-mammography screening in two hypothetical population-based cohorts of asymptomatic women. Parameter uncertainty was evaluated through deterministic and probabilistic sensitivity analysis. Model structure uncertainty was also explored to test the robustness of the results.ResultsIt was estimated that 31 out of 100 new cases of breast cancer would be detected by mammography and 39/100 by tele-mammography. The model returned an incremental cost-effectiveness ratio (ICER) of £26 051/quality-adjusted life-year (QALY) which is lower than the WHO-recommended threshold of £26 288/QALY for Argentina. Deterministic sensitivity analysis showed the ICER is most sensitive to the uptake and sensitivity of the screening tests. Probabilistic sensitivity analysis showed tele-mammography is cost-effective in 59% of simulations.DiscussionTele-mammography should be considered for adoption as it could improve access to expertise in underserved areas where adherence to screening protocols is poor. Disaggregated data by province is needed for a better- informed policy decision. Telemedicine could also be beneficial in ensuring the continuity of care when health systems are under stress like in the current COVID-19 pandemic.ConclusionThere is a 59% chance that tele-mammography is cost-effective compared to mammography for at-risk Argentinian women over 40- years old, and should be adopted to improve access to healthcare in underserved areas of the country.

ObjectivesWith the unprecedented penetration of mobile devices in the developing world, mHealth applications are being leveraged for different health domains. Among the different factors that affect the use of mHealth interventions is the intention and preference of end-users to use the system. This study aimed to assess mother’s intention and preference to use text message reminders for vaccination in Ethiopia.MethodsA cross-sectional study was conducted among 460 mothers selected through a systematic random sampling technique. Initially, descriptive statistics were computed. Binary logistic regression analysis was also used to assess factors associated with the outcome variable.ResultsIn this study, of the 456 mothers included for analysis, 360 (78.9%) of mothers have intention to use text message reminders for vaccination. Of these, 270 (75%) wanted to receive the reminders a day before the vaccination due date. Mothers aged 35 years or more (AOR=0.35; 95% CI: 0.15 to 0.83), secondary education and above (AOR=4.43; 95% CI: 2.05 to 9.58), duration of mobile phone use (AOR=3.63; 95% CI: 1.66 to 7.94), perceived usefulness (AOR=6.37; 95% CI: 3.13 to 12.98) and perceived ease of use (AOR=3.85; 95% CI: 2.06 to 7.18) were predictors of intention to use text messages for vaccination.ConclusionIn conclusion, majority of mothers have the intention to use text message reminders for child vaccination. Mother’s age, education, duration of mobile phone use, perceived usefulness and perceived ease of use were associated with intention of mothers to use text messages for vaccination. Considering these predictors and user’s preferences before developing and testing text message reminder systems is recommended.

Prognostic risk prediction models aim to estimate the risk of a future outcome based on available clinical parameters.[1][1] There has been an increase in the development of such models given the move towards personalised and precision medicine, since they provide individualised risks for patients.[

Unscheduled admissions to hospital place great demands on the use of limited healthcare resources in health systems worldwide. A range of approaches exist to manage demand; however, interventions within hospitals have received less attention, and the evidence base on effectiveness is limited. This study aimed to assess the effectiveness of a novel intervention, implemented in National Health Service Lothian, to reduce the number of unscheduled attendances, and to estimate the impact on hospital admissions, length of hospital stay and overall total acute hospital costs.MethodsBefore and after observational study of an anticipatory care planning intervention targeted among people identified by a prediction algorithm (Scottish Patients at Risk of Readmission and Admission) as being at high risk of future unscheduled hospital admissions. The statistical significance of the difference in outcomes observed before and after implementation of the intervention between August 2014 and July 2015 was tested using difference-in-difference analysis.ResultsThe intervention was estimated to reduce the number of unscheduled hospital admissions and emergency department (ED) visits by approximately 0.36 (95% CI −0.905 to 0.191) per patient per year (based on 954 and 450 patients in the intervention and control groups, respectively). There was also non-significant reductions in length of hospital stay for unscheduled admissions and hospital costs for ED visits and inpatient care. The overall predicted effect of the intervention for the average participant was a saving of around £2912 (95% CI −7347.0 to 1523.9) per patient per year.ConclusionAn anticipatory care planning intervention focused among people judged to be at higher risk of future unscheduled hospital admissions can be effective in reducing the number of unscheduled admissions to hospital and ED visits, and may lead to an overall saving in use of hospital resources.

BackgroundThe recent outbreak of respiratory illness caused by COVID-19 in Wuhan, China, has received global attention as it has infected thousands of individuals there, and later it has also been reported from other countries internationally. This study aims at performing an exploratory study on Twitter to understand the information shared among the community regarding the COVID-19 outbreak.MethodsCOVID-19 related tweets were collected from Twitter using keywords from 18 January to 25 January 2020. Top-ranking tweets were taken as samples and then categorised based on the content. Expressions or opinion tweets were analysed qualitatively to understand the mindset of the people regarding the outbreak. Theme wise reachability evaluation of the messages was also performed.ResultsBased on the content of the tweets, five themes were evolved: (1) general information; (2) health information; (3) expressions; (4) humour and (5) others. 57.42% of messages are general information followed by expressive tweets (24.12%). Humorous messages were liked the most, whereas health information tweets were retweeted the maximum. Fear was the predominant emotion expressed in the messages.ConclusionThe results of the study would be useful to focus on the dissemination of the right information and effective communication on Twitter related to health and outbreak management.

Background COVID-19 presented significant challenges to healthcare organisations, which needed to rapidly remodel their services but were unable to allow staff to meet face to face to minimise infection risk. During this communication predicament, National Health Service (NHS) Digital announced the provision of Microsoft Teams, a digital communication and collaboration tool, which was implemented at Royal Free London NHS Foundation Trust within 2 weeks.Method Given the need to deploy at scale, rapidly and with minimal resource, an agile decentralised innovation management approach was used, empowering staff to be local implementors.Results Resulting use cases were highly original and varied, ranging from a COVID-19 Education Programme to coordination of oxygen demand. Analytics showed rapid and persistent adoption, surpassing 500 daily active users within 11 days. Usage continues to increase, consistent with a direct network effect.Conclusion These findings suggest a high demand for this format of communication and high willingness to adopt it. Further qualitative research into staff perceptions would be valuable to confirm this, and to assess the user experience.Overall, this has been a radical approach to digital implementation in healthcare, and has so far proved effective in delivering a cost minimal, rapid communication tool at scale in the midst of a global pandemic.

ObjectiveThe National Institute for Health Research (NIHR) Health Informatics Collaborative (HIC) is a programme of infrastructure development across NIHR Biomedical Research Centres. The aim of the NIHR HIC is to improve the quality and availability of routinely collected data for collaborative, cross-centre research. This is demonstrated through research collaborations in selected therapeutic areas, one of which is viral hepatitis.DesignThe collaboration in viral hepatitis identified a rich set of datapoints, including information on clinical assessment, antiviral treatment, laboratory test results and health outcomes. Clinical data from different centres were standardised and combined to produce a research-ready dataset; this was used to generate insights regarding disease prevalence and treatment response.ResultsA comprehensive database has been developed for potential viral hepatitis research interests, with a corresponding data dictionary for researchers across the centres. An initial cohort of 960 patients with chronic hepatitis B infections and 1404 patients with chronic hepatitis C infections has been collected.ConclusionFor the first time, large prospective cohorts are being formed within National Health Service (NHS) secondary care services that will allow research questions to be rapidly addressed using real-world data. Interactions with industry partners will help to shape future research and will inform patient-stratified clinical practice. An emphasis on NHS-wide systems interoperability, and the increased utilisation of structured data solutions for electronic patient records, is improving access to data for research, service improvement and the reduction of clinical data gaps.

IntroductionMany clinical education programmes have not incorporated the use of the electronic health record (EHR) into their curriculum. It is important to incorporate technologies that will be used in real-world settings to better prepare students for clinical practice.ObjectivesTo undertake a review of literature to identify a training evaluation framework; to conduct a self-completion survey, pretraining and post-training, to determine students’ perceptions on the benefit of using EHR training system.SettingNursing School, University, North West England, UK; University Ethic Committee Approval Received.ParticipantsRegistered nurses undertaking a validated return to practice course; 24 participants for the first cohort who completed pretraining questionnaire and 23 for the second post-training cohort.ResultsThe statistical results show that the students perceived that the training improved their capability in employing digital systems with statistically significant difference in the assessed preproficiency and post proficiency in the use of digital clinical systems (premedians and post medians are 2 and 5 on 10-point Likert scale, p=0.041). There was also an indication of an improvement in the knowledge of EHR systems although not statistically significant. Most students perceived it increased their knowledge on digital systems.ConclusionStudents perceived an increase in proficiency with the EHR. There was evidence of improvement in confidence in the use of the EHR, but this confidence would be enhanced by additional use of the system. Some desire to increase confidence further and to develop knowledge of digital systems was expressed.

When the UK government was first confronted with the very real threat of a COVID-19 pandemic, policy-makers turned quickly, and initially almost exclusively, to scientific data provided by epidemiological models. These models have had a direct and significant influence on the policies and decisions, such as social distancing and closure of schools, which aim to reduce the risk posed by COVID-19 to public health.1 The models suggested that depending on the strategies chosen, the number of deaths could vary by hundreds of thousands. From a safety engineering perspective, it is clear that the data generated by epidemiological models are safety critical, and that, therefore, the models themselves should be regarded as safety-critical systems.

Pandemics have left indelible marks on medicine. The 1918 influenza accelerated the emergence of American medicine as a scientific enterprise.[1][1] HIV ushered in universal precautions as the minimum requirement for the care of every patient.[2][2] One defining impact of COVID-19 might be the

There has been significant effort in linking different electronic datasets within healthcare and between health and social care,1–6 both for research purposes and for clinical care. These electronic data are referred to as ‘routinely collected’ as they are collected during usual clinical practice, in contrast to ‘research’ data which are collected in a bespoke manner within the context of a research project. Such datasets (here referred to as routinely collected electronic health and social care data) are already used extensively in epidemiological research, to assess the impact of healthcare interventions in real-world practice, and to provide outcome measures in clinical trials.7–9In wider society, the actions of large personal data processors have brought public scrutiny on the use and misuse of personal data. Previous attempts at large-scale use of electronic health and social care data (eg, the national Electronic Health Record (EHR) in England) have been controversial in concept and problematic in execution, with consequent damage to public confidence in the ability of organisations to act as trustworthy stewards of personal data.10–13 These negative experiences damage the relationship between individuals, their data and the research community.14 We argue for moving towards a model of consent and use based on respect for digital personhood, and review some of the technical and governance solutions that could enable this transition.

IntroductionElectronic health records (EHRs) can improve the quality and safety of care. However, the adoption and use of the EHR is influenced by several factors, including users’ perception.ObjectivesTo undertake a systematic review of the literature to understand healthcare professionals’ perceptions about the adoption and use of EHRs in Gulf Cooperation Council (GCC) countries in order to influence the implementation strategies, training programme and policy development in the GCC region.MethodA systematic literature search was undertaken on seven online databases to identify articles published between January 2006 and December 2017 examining healthcare professionals’ perception towards the adoption and use of EHR in the Gulf context.ResultsThe fourteen articles included in this review identified both positive and negative perceptions of the role of EHR in healthcare. The positive perceptions included EHR benefits, such as improvements to work efficiency, quality of care, communication and access to patient data. Conversely, the negative perceptions were associated with challenges or risks of adopting an EHR, such as disruption of provider–patient communication, privacy and security concerns and high initial costs. The perceptions were influenced by personal factors (eg, age, occupation and computer literacy) and system factors (perceived usefulness and perceived ease of use).ConclusionPositive perceptions of EHRs by the healthcare professionals could facilitate the adoption of this technology in the Gulf region, particularly when barriers are addressed early. Negative perceptions may inform change management strategies during adoption and implementation. The perceptions should be further evaluated from a technology acceptance perspective.

ObjectiveOpenClinical.net is a way of disseminating clinical guidelines to improve quality of care whose distinctive feature is to combine the benefits of clinical guidelines and other human-readable material with the power of artificial intelligence to give patient-specific recommendations. A key objective is to empower healthcare professionals to author, share, critique, trial and revise these ‘executable’ models of best practice.DesignOpenClinical.net Alpha (www.openclinical.net) is an operational publishing platform that uses a class of artificial intelligence techniques called knowledge engineering to capture human expertise in decision-making, care planning and other cognitive skills in an intuitive but formal language called PROforma.3 PROforma models can be executed by a computer to yield patient-specific recommendations, explain the reasons and provide supporting evidence on demand.ResultsPROforma has been validated in a wide range of applications in diverse clinical settings and specialties, with trials published in high impact peer-reviewed journals. Trials have included patient workup and risk assessment; decision support (eg, diagnosis, test and treatment selection, prescribing); adaptive care pathways and care planning. The OpenClinical software platform presently supports authoring, testing, sharing and maintenance. OpenClinical’s open-access, open-source repository Repertoire currently carries approximately 50+ diverse examples (https://openclinical.net/index.php?id=69).ConclusionOpenClinical.net is a showcase for a PROforma-based approach to improving care quality, safety, efficiency and better patient experience in many kinds of routine clinical practice. This human-centred approach to artificial intelligence will help to ensure that it is developed and used responsibly and in ways that are consistent with professional priorities and public expectations.

The COVID-19 pandemic has created many challenges for health and care services worldwide and has led to one of the largest societal crises in last century. It has also been a test for the maturity of digital health technologies, be it for frontline care, surveillance or discovery of new strategies.

BackgroundPerson-generated health data (PGHD) are produced by people when they use health information technologies. People who use PGHD may experience changes in their health and care process, such as engagement with their own healthcare, and their sense of social support and connectedness. Research into evaluating those reported effects has not kept up; thus, a method for measuring PGHD outcomes was previously designed and applied to the exemplar case of Kinect-based stroke rehabilitation systems. A key step of the method ensures that the patient’s voice is included. Allowing stroke survivors to participate in the development and evaluation of health services and treatment can inform healthcare providers on decisions about stroke care, and thereby improve health outcomes.ObjectiveThis paper presents the perspectives of stroke survivors and clinicians on the anticipated effects of stroke survivors’ use of PGHD from a poststroke simulated rehabilitation technology.MethodsThis study gathered the perspectives of stroke survivors and clinicians through three focus groups and three interviews, recruited for convenience. Participants were also asked questions intended to encourage them to comment on the initial items of the patient-reported outcome measure-PGHD. Deductive thematic analysis was performed.ResultsThis paper has further demonstrated that outcomes of using PGHD can be measured. For instance, stroke survivors described that using PGHD could result in positive, negative and nil effects on their health behaviours. Survivors and clinicians had varying perspectives in three of the six themes presented, and emphasise the importance of allowing stroke survivors to participate in the evaluation of digital health services.

IntroductionA mixed-method, co-design approach to studying the adoption of mobile health (mHealth) technology among African-American (AA) women has not been fully explored. Qualitative data may contextualise existing knowledge surrounding perceptions of mHealth among AA women as part of formative work for designing a physical activity application (app).MethodsA convenience sample of 16 AA women completed an informatics survey prior to participating in focus groups exploring their use of mobile technology and health apps. Survey responses provided frequency data, while iterative transcript analysis of focus groups identified themes.ResultsThe majority of participants (mean age=62.1 years, SD=6.6) felt comfortable using a tablet/smartphone (75.0%). Most (68.8%) reported using health-related apps, primarily focused on physical activity and nutrition. Focus groups revealed four overarching concepts, including (1) user attachment, (2) technology adoption, (3) potential facilitators and (4) potential barriers. Important features which may serve as facilitators or barriers to future adoption of a mobile app for an mHealth intervention include individual app tailoring and software concerns, respectively.DiscussionThematic analysis revealed high user attachment to smartphones and described participants’ process for adopting new mHealth technology.ConclusionEarly engagement of target end users as a part of a broader co-design and community-based participatory research process for developing mHealth technologies may be useful for sustained adoption of these tools in future mHealth behavioural interventions.

IntroductionWe present the integration of telemedicine into the healthcare system of West China Hospital of Sichuan University (WCH), one of the largest hospitals in the world with 4300 inpatient beds, as a means for maximising the efficiency of healthcare delivery during the COVID-19 pandemic.MethodsImplemented on 22 January 2020, the telemedicine technology allowed WCH providers to conduct teleconsultations, telerounds, teleradiology and tele-intensive care unit, which in culmination provided screening, triage and treatment for COVID-19 and other illnesses. To encourage its adoption, the government and the hospital publicised the platform on social media and waived fees.DiscussionFrom 1 February to 1 April 2020, 10557 online COVID-19 consultations were conducted for 6662 individuals; meanwhile, 32676 patients without COVID completed virtual follow-ups. We discuss that high-quality, secure, affordable and user-friendly telemedical platforms should be integrated into global healthcare systems to help decrease the transmission of the virus and protect healthcare providers from infection.

ObjectivesHigh rates of depression and suicide and a lack of trained psychiatrists have emerged as significant concerns in the low-income and middle-income countries (LMICs) such as the Pacific Island Countries (PICs). Readily available smartphones were leveraged with community health nurses (CHNs) in task-sharing for early identification of suicide and depression risks in Fiji Islands, the largest of PICs. This investigation examines how CHNs can efficiently and effectively process patient information about depression and suicide risk for making diagnostic and management decisions without compromising safety. The research is driven by the theoretical framework of text comprehension (knowledge representation and interpretation) and decision-making.MethodsMobile health (mHealth) Application for Suicide Risk and Depression Assessment (ASRaDA) was designed to include culturally useful clinical guidelines for these disorders. A representative sample of 48 CHNs was recruited and presented with two clinical cases (depression and suicide) in a simulated setting under three conditions: No support, paper-based and mobile-based culturally valid guideline support. Data were collected as the nurses read through the scenarios, ‘thinking aloud’, before summarising, diagnoses and follow-up recommendations. Transcribed audiotapes were analysed using formal qualitative discourse analysis methods for diagnostic accuracy, comprehension of clinical problems and reasoning patterns.ResultsUsing guidelines on ASRaDA, the CHNs took less time to process patient information with more accurate diagnostic and therapeutic decisions for depression and suicide risk than with paper-based or no guideline conditions. A change in reasoning pattern for nurses’ information processing was observed with decision support.DiscussionAlthough these results are shown in a mental health setting in Fiji, there are reasons to believe they are generalisable beyond mental health and other lower-to-middle income countries.ConclusionsCulturally appropriate clinical guidelines on mHealth supports efficient information processing for quick and accurate decisions and a positive shift in reasoning behaviour by the nurses. However, translating complex qualitative patient information into quantitative scores could generate conceptual errors. These results are valid in simulated conditions.

ObjectivesPredictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data.MethodsWe performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020.ResultsMost models failed validation when applied to our institution’s data. Included studies reported an average validation area under the receiver–operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies’ reported AUROC values.DiscussionPublished and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations.ConclusionsClinicians should employ caution when applying models for clinical prediction without careful validation on local data.

While relatively novel, patient peer review has the potential to change the healthcare publishing paradigm. It can do this by helping researchers enlarge the pool of people who are welcome to read, understand and participate in healthcare research. Academic journals who are early adopters of patient

High-quality research is essential in guiding evidence-based care, and should be reported in a way that is reproducible, transparent and where appropriate, provide sufficient detail for inclusion in future meta-analyses. Reporting guidelines for various study designs have been widely used for clinical (and preclinical) studies, consisting of checklists with a minimum set of points for inclusion. With the recent rise in volume of research using artificial intelligence (AI), additional factors need to be evaluated, which do not neatly conform to traditional reporting guidelines (eg, details relating to technical algorithm development). In this review, reporting guidelines are highlighted to promote awareness of essential content required for studies evaluating AI interventions in healthcare. These include published and in progress extensions to well-known reporting guidelines such as Standard Protocol Items: Recommendations for Interventional Trials-AI (study protocols), Consolidated Standards of Reporting Trials-AI (randomised controlled trials), Standards for Reporting of Diagnostic Accuracy Studies-AI (diagnostic accuracy studies) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis-AI (prediction model studies). Additionally there are a number of guidelines that consider AI for health interventions more generally (eg, Checklist for Artificial Intelligence in Medical Imaging (CLAIM), minimum information (MI)-CLAIM, MI for Medical AI Reporting) or address a specific element such as the ‘learning curve’ (Developmental and Exploratory Clinical Investigation of Decision-AI) . Economic evaluation of AI health interventions is not currently addressed, and may benefit from extension to an existing guideline. In the face of a rapid influx of studies of AI health interventions, reporting guidelines help ensure that investigators and those appraising studies consider both the well-recognised elements of good study design and reporting, while also adequately addressing new challenges posed by AI-specific elements.

ObjectivesPatient feedback is critical to identify and resolve patient safety and experience issues in healthcare systems. However, large volumes of unstructured text data can pose problems for manual (human) analysis. This study reports the results of using a semiautomated, computational topic-modelling approach to analyse a corpus of patient feedback.MethodsPatient concerns were received by Alberta Health Services between 2011 and 2018 (n=76 163), regarding 806 care facilities in 163 municipalities, including hospitals, clinics, community care centres and retirement homes, in a province of 4.4 million. Their existing framework requires manual labelling of pre-defined categories. We applied an automated latent Dirichlet allocation (LDA)-based topic modelling algorithm to identify the topics present in these concerns, and thereby produce a framework-free categorisation.ResultsThe LDA model produced 40 topics which, following manual interpretation by researchers, were reduced to 28 coherent topics. The most frequent topics identified were communication issues causing delays (frequency: 10.58%), community care for elderly patients (8.82%), interactions with nurses (8.80%) and emergency department care (7.52%). Many patient concerns were categorised into multiple topics. Some were more specific versions of categories from the existing framework (eg, communication issues causing delays), while others were novel (eg, smoking in inappropriate settings).DiscussionLDA-generated topics were more nuanced than the manually labelled categories. For example, LDA found that concerns with community care were related to concerns about nursing for seniors, providing opportunities for insight and action.ConclusionOur findings outline the range of concerns patients share in a large health system and demonstrate the usefulness of using LDA to identify categories of patient concerns.

BackgroundMonitoring and evaluations of digital health (DH) solutions for the management of chronic diseases are quite heterogeneous and evidences around evaluating frameworks are inconsistent. An evidenced-based framework is needed to inform the evaluation process and rationale of such interventions. We aimed to explore the nature, extent and components of existing DH frameworks for chronic diseases.MethodsThis review was conducted based on the five steps of Arksey and O’Malley’s scoping review methodology. Out of 172 studies identified from, PubMed, Embase and Web of Science, 11 met our inclusion criteria. The reviewed studies developed DH frameworks for chronic diseases and published between 2010 and 2018.ResultsAccording to WHO guidelines for monitoring and evaluation of DH interventions, we identified seven Conceptual frameworks, two Results frameworks, one Logical framework and one Theory of change. The frameworks developed for providing interventions such as self-management, achieving personal goals and reducing relapse for cardiovascular disease, diabetes, chronic obstructive pulmonary disease and severe mental health. A few studies reported evaluation of the frameworks using randomised clinical trials (n=3) and feasibility testing via Likert scale survey (n=2). A wide range of outcomes were reported including access to care, cost-effectiveness, behavioural outcomes, patient–provider communications, technology acceptance and user experience.ConclusionThere is a lack of evidence on the application of consistent DH frameworks. Future research should address the use of evidence-based frameworks into the research design, monitoring and evaluation process. This review explores the nature of DH frameworks for the management of chronic diseases and provides examples to guide monitoring and evaluation of interventions.

ProblemLearning health systems (LHS) are an underexplored concept. How LHS will operate in clinical practice is not well understood. This paper investigates the relationships between LHS, clinical care process specifications (CCPS) and the established levels of medical practice to enable LHS integration into daily healthcare practice.MethodsConcept analysis and thematic analysis were used to develop an LHS characterisation. Pathway theory was used to create a framework by relating LHS, CCPS, health information systems and the levels of medical practice. A case study approach evaluates the framework in an established health informatics project.ResultsFive concepts were identified and used to define the LHS learning cycle. A framework was developed with five pathways, each having three levels of practice specificity spanning population to precision medicine. The framework was evaluated through application to case studies not previously understood to be LHS.DiscussionClinicians show limited understanding of LHS, increasing resistance and limiting adoption and integration into care routine. Evaluation of the presented framework demonstrates that its use enables: (1) correct analysis and characterisation of LHS; (2) alignment and integration into the healthcare conceptual setting; (3) identification of the degree and level of patient application; and (4) impact on the overall healthcare system.ConclusionThis paper contributes a theoretical framework for analysis, characterisation and use of LHS. The framework allows clinicians and informaticians to correctly identify, characterise and integrate LHS within their daily routine. The overall contribution improves understanding, practice and evaluation of the LHS application in healthcare.

IntroductionTechnological change in healthcare demands new ways of working. Access to, and use of, digital technology by nurses in Australia lags behind other professions. Governance frameworks and professional standards guide scopes of practice; however, there is an urgent need for current registered nurses to master using digital technology and model digital professionalism to the next generation. Sustaining digital professionalism requires organisational readiness to accommodate changing technological environments.MethodsPrevious original research findings investigating the nature and scope of digital technology use by nurses were systematically analysed. With reference to current understandings of capability, a matrix for assessing organisational readiness of capability of digital technology use by nurses was developed.ResultsThe 4E3P digital professionalism model articulates the elements necessary for establishing organisational readiness and assessing the capability development of individuals and groups. When the physical and social environment is conducive and the 4E elements of equipment, electronic access, engagement and education are present, preparedness, proficiency and professional behaviours can be nurtured and supported.DiscussionThe model describes the physical and social attributes that enable capability development for sustaining digital professionalism to advance nursing practice. When elements of the matrix are lacking, both individuals and groups miss opportunities to develop and sustain digitally professional behaviour.ConclusionIt is imperative that healthcare environments in Australia support the development of digital professionalism. Deployment of the 4E3P digital professionalism model will enable identification and remediation of challenges, barriers or risks to promote sustainability found within physical and social healthcare environments.

BackgroundA clinical decision support system (CDSS) covers a broad spectrum of applications, for example, screening reminders, can reduce malpractice, improve preventive services and enable better management of chronic conditions. CDSSs have traditionally been used successfully in large hospitals. The availability (ie, whether the function is provided by the software) and usage (ie, actual use) of a CDSS in office-based primary care settings, however, are less well studied.ObjectiveTo establish a benchmark of CDSS availability and usage in office-based primary care settings, particularly given the large volume of visits in such settings.MethodsWe used the 2015 Centers for Disease Control and Prevention’s National Ambulatory Medical Care Survey to conduct secondary data analysis. We selected preventive services reminders and drug interaction alerts, along with several other variables as examples of a CDSS.ResultsCDSS usage rates ranged from 68.5% to 100% among solo or non-solo primary care practices owned by physicians or physician groups that have electronic medical records (EMRs)/electronic health records (EHRs) and 44.7% to 96.1%, regardless of EMR/EHR status. According to proportion tests, solo practices had significantly lower CDSS usage and availability rates on several measures if the practice is entirely EMR/EHR based and significantly lower (16.3%–28.9%) CDSS usage rates than did non-solo practices on each measure, regardless of EMR/EHR status.ConclusionIn the USA, a CDSS, especially under the categories of basic preventive reminders and drug interaction alerts, is used routinely between 68% and 100% in primary care if a practice is entirely EMR/EHR based. More work is needed, however, to determine the reasons for large usage gaps between solo and non-solo practices and to reduce such gaps.

IntroductionThe implementation of home-based cardiac rehabilitation has demonstrated potential to increase patient participation, but the content and the delivering of the programmes varies across countries. The objective of this study is to investigate whether an Australian-validated mobile health (mHealth) platform for cardiac rehabilitation will be accepted and adopted irrespectively from the existing organisational and contextual factors in five different European countries.Methods and analysisThis international multicentre feasibility study will use surveys, preliminary observations and analysis to evaluate the use and the user’s perceptions (satisfaction) of a validated mHealth platform in different contextual settings.Ethics and disseminationThis study protocol has been approved by the Australian research organisation CSIRO and the respective ethical committees of the European sites. The dissemination of this trial will serve as a ground for the further implementation of an international large randomised controlled trial which will contribute to an effective global introduction of mHealth into daily clinical practice.

Machine learning for healthcare (MLHC) is at the juncture of leaping from the pages of journals and conference proceedings to clinical implementation at the bedside. Succeeding in this endeavour requires the synthesis of insights from both the machine learning and healthcare domains, in order to

Although media coverage of the COVID-19 crisis may have gradually desensitised us to the daily updates on numbers of cases and deaths, broadcast interviews with distraught front-line health workers continue to evoke an empathic response as we hear about the traumatic conditions of their experiences

IntroductionTelehealth became the most practical option for general practice consultations in Aotearoa New Zealand (NZ) as a result of the national lockdowns in response to the COVID-19 pandemic. What is the consumer experience of access to telehealth and how do consumers and providers perceive this mode of care delivery going forward?Methods and analysisA national survey of general practice consumers and providers who used telehealth services since the national lockdowns in 2020 will be distributed. It is based on the Unified Theory of Acceptance and Use of Technology framework of technology acceptance and the access to care framework. The data will be statistically analysed to create a foundation for in-depth research on the use of telehealth services in NZ general practice services, with a specific focus on consumer experiences and health outcomes.Ethics and disseminationEthics approval was granted by the Auckland Health Research Ethics Committee on 13/11/2020, reference AH2539. The survey will be disseminated online.

ObjectivesIdentifying those individuals requiring medical care is a basic tenet of the pandemic response. Here, we examine the COVID-19 community triage pathways employed by four nations, specifically comparing the safety and efficacy of national online ‘symptom checkers’ used within the triage pathway.MethodsA simulation study was conducted on current, nationwide, patient-led symptom checkers from four countries (Singapore, Japan, USA and UK). 52 cases were simulated to approximate typical COVID-19 presentations (mild, moderate, severe and critical) and COVID-19 mimickers (eg, sepsis and bacterial pneumonia). The same simulations were applied to each of the four country’s symptom checkers, and the recommendations to refer on for medical care or to stay home were recorded and compared.ResultsThe symptom checkers from Singapore and Japan advised onward healthcare contact for the majority of simulations (88% and 77%, respectively). The USA and UK symptom checkers triaged 38% and 44% of cases to healthcare contact, respectively. Both the US and UK symptom checkers consistently failed to identify severe COVID-19, bacterial pneumonia and sepsis, triaging such cases to stay home.ConclusionOur results suggest that whilst ‘symptom checkers’ may be of use to the healthcare COVID-19 response, there is the potential for such patient-led assessment tools to worsen outcomes by delaying appropriate clinical assessment. The key features of the well-performing symptom checkers are discussed.

IntroductionNumerous scientific journal articles related to COVID-19 have been rapidly published, making navigation and understanding of relationships difficult.MethodsA graph network was constructed from the publicly available COVID-19 Open Research Dataset (CORD-19) of COVID-19-related publications using an engine leveraging medical knowledge bases to identify discrete medical concepts and an open-source tool (Gephi) to visualise the network.ResultsThe network shows connections between diseases, medications and procedures identified from the title and abstract of 195 958 COVID-19-related publications (CORD-19 Dataset). Connections between terms with few publications, those unconnected to the main network and those irrelevant were not displayed. Nodes were coloured by knowledge base and the size of the node related to the number of publications containing the term. The data set and visualisations were made publicly accessible via a webtool.ConclusionKnowledge management approaches (text mining and graph networks) can effectively allow rapid navigation and exploration of entity inter-relationships to improve understanding of diseases such as COVID-19.

BackgroundUp to half of all musculoskeletal injuries are investigated with plain radiographs. However, high rates of image interpretation error mean that novel solutions such as artificial intelligence (AI) are being explored.ObjectivesTo determine patient confidence in clinician-led radiograph interpretation, the perception of AI-assisted interpretation and management, and to identify factors which might influence these views.MethodsA novel questionnaire was distributed to patients attending fracture clinic in a large inner-city teaching hospital. Categorical and Likert scale questions were used to assess participant demographics, daily electronics use, pain score and perceptions towards AI used to assist in interpretation of their radiographs, and guide management.Results216 questionnaires were included (M=126, F=90). Significantly higher confidence in clinician rather than AI-assisted interpretation was observed (clinician=9.20, SD=1.27 vs AI=7.06, SD=2.13), 95.4% reported favouring clinician over AI-performed interpretation in the event of disagreement.Small positive correlations were observed between younger age/educational achievement and confidence in AI-assistance. Students demonstrated similarly increased confidence (8.43, SD 1.80), and were over-represented in the minority who indicated a preference for AI-assessment over their clinicians (50%).ConclusionsParticipant’s held the clinician’s assessment in the highest regard and expressed a clear preference for it over the hypothetical AI assessment. However, robust confidence scores for the role of AI-assistance in interpreting skeletal imaging suggest patients view the technology favourably.Findings indicate that younger, more educated patients are potentially more comfortable with a role for AI-assistance however further research is needed to overcome the small number of responses on which these observations are based.

IntroductionNon-adherence to antipsychotic medications for individuals with serious mental illness increases risk of relapse and hospitalisation. Real time monitoring of adherence would allow for early intervention. AI2 is a both a personal nudging system and a clinical decision support tool that applies machine learning on Medicare prescription and benefits data to raise alerts when patients have discontinued antipsychotic medications without supervision, or when essential routine health checks have not been performed.Methods and analysisWe outline two intervention models using AI2. In the first use-case, the personal nudging system, patients receive text messages when an alert of a missed medication or routine health check is detected by AI2. In the second use-case, as a clinical decision support tool, AI2 generated alerts are presented as flags through a dashboard to the community mental health professionals. Implementation protocols for different scenarios of AI2, along with a mixed-methods evaluation, are planned to identify pragmatic issues necessary to inform a larger randomised control trial, as well as improve the application.Ethics and disseminationThis study protocol has been approved by The Southern Adelaide Clinical Human Research Ethics Committee. The dissemination of this trial will serve to inform further implementation of the AI2 into daily personal and clinical practice.

BackgroundHome-based self-monitoring has failed to show intended savings to healthcare budgets yet it continues to emerge and gain popularity.ObjectiveWe set out to verify stakeholders’ perspectives of remote vital sign telemonitoring.DesignAn observational design was adopted by devising a survey for distribution to service users and their informal carers.SampleService users in South Eastern Health and Social Care Trust were included. A total of 274 questionnaires were issued. Data from 97 patients (35% response rate) and 49 carers were analysed. Of these, 81 patients and 48 of their carers experienced a monitoring service known as TF3 and 16 patients and 1 carer experienced a service known as U-Tell. The cohorts comprised people living with a number of long-term conditions: diabetes, hypertension after stroke, chronic heart failure, chronic obstructive pulmonary disorder, bronchiectasis and those requiring anticoagulation using warfarin.ResultsAnalysis showed that respondents were supportive of the technology with 90.7% of patients agreeing or strongly agreeing with the statement: the remote monitoring system assisted me in managing my health on a day-to-day basis. The patients liked the technology largely because it provided empowerment and control for self-management and allowed them to continue with their lives without major disruption. These views were independent of the technology used and not associated with the patient’s long-term conditions, gender or age. There were no reported adverse incidents.ConclusionAs self-monitoring becomes more relevant to healthcare delivery, the technology will be accepted by many in the population with long-term conditions.

BackgroundEmergency Medicine Telehealth (TeleEM) represents an opportunity to work directly with referral centres, rural facilities and underserved areas to mitigate unnecessary testing, optimise resource utilisation and facilitate patient transfers across health systems. To optimise the impact of a TeleEM programme, a tool is needed to remotely monitor patient activity in multiple emergency department facilities, concurrently.MethodsAfter identifying data sources for activation criteria put forth by the TeleEM operations group, rules were constructed within the electronic health record to facilitate data checks and ultimately produce a yes/no response if the category’s conditions were met. Responses were organised into a table, with functionality allowing end users to drill into the different sites to see patient-specific information for patients meeting activation criteria.ConclusionsThe TeleEM dashboard allows for proactive engagement by the TeleEM physician and strengthens the team-based approach of critically ill.

ObjectiveThis study evaluated the potential for electronic medical record (EMR) video tutorials to improve diabetes (type 1 and 2) care processes by primary care physicians (PCP) using OSCAR EMR.DesignA QUAN(qual) mixed methods approach with an embedded design was used for the overall research study. EMR video tutorials were developed based on the chronic care model (CCM), value-adding EMR use, best practice guidelines for designing software video tutorials and clinician-led EMR training.ResultsIn total, 18 PCPs from British Columbia, Canada, participated in the study. The video EMR intervention elicited a statistically significant increase in EMR advanced feature use for diabetes care, with a large effect size (ie, F(1,51)=6.808, p2=0.286).ConclusionThis small-scale efficacy study demonstrates the potential of CCM-based EMR video tutorials to improve EMR use for chronic diseases, such as diabetes. A larger-scale effectiveness study with a control group is needed to further validate the study findings and determine their generalisability. The demonstrated efficacy of the intervention suggests that EMR video tutorials may be a cost-effective, sustainable and scalable strategy for supporting EMR optimisation and the continuous learning and development of PCPs. Health informatics practitioners may develop video tutorials for their respective EMR/electronic health record software based on theory and best practices for video tutorial design. For patients, EMR video tutorials may lead to improved tracking of processes of care for diabetes, and potentially other chronic conditions.