Search

Objective To determine the effects of using unstructured clinical text in machine learning (ML) for prediction, early detection, and identification of sepsis. Materials and methods PubMed, Scopus, ACM DL, dblp, and IEEE Xplore databases were searched. Articles utilizing clinical text for ML or natural language processing (NLP) to detect, identify, recognize, diagnose, or predict the onset, development, progress, or prognosis of systemic inflammatory response syndrome, sepsis, severe sepsis, or septic shock were included. Sepsis definition, dataset, types of data, ML models, NLP techniques, and evaluation metrics were extracted. Results The clinical text used in models include narrative notes written by nurses, physicians, and specialists in varying situations. This is often combined with common structured data such as demographics, vital signs, laboratory data, and medications. Area under the receiver operating characteristic curve (AUC) comparison of ML methods showed that utilizing both text and structured data predicts sepsis earlier and more accurately than structured data alone. No meta-analysis was performed because of incomparable measurements among the 9 included studies. Discussion Studies focused on sepsis identification or early detection before onset; no studies used patient histories beyond the current episode of care to predict sepsis. Sepsis definition affects reporting methods, outcomes, and results. Many methods rely on continuous vital sign measurements in intensive care, making them not easily transferable to general ward units. Conclusions Approaches were heterogeneous, but studies showed that utilizing both unstructured text and structured data in ML can improve identification and early detection of sepsis.

Background With a motivation of quality assurance, machine learning techniques were trained to classify Norwegian radiology reports of paediatric CT examinations according to their description of abnormal findings. Methods 13.506 reports from CT-scans of children, 1000 reports from CT scan of adults and 1000 reports from X-ray examination of adults were classified as positive or negative by a radiologist, according to the presence of abnormal findings. Inter-rater reliability was evaluated by comparison with a clinician’s classifications of 500 reports. Test–retest reliability of the radiologist was performed on the same 500 reports. A convolutional neural network model (CNN), a bidirectional recurrent neural network model (bi-LSTM) and a support vector machine model (SVM) were trained on a random selection of the children’s data set. Models were evaluated on the remaining CT-children reports and the adult data sets. Results Test–retest reliability: Cohen’s Kappa = 0.86 and F1 = 0.919. Inter-rater reliability: Kappa = 0.80 and F1 = 0.885. Model performances on the Children-CT data were as follows. CNN: (AUC = 0.981, F1 = 0.930), bi-LSTM: (AUC = 0.978, F1 = 0.927), SVM: (AUC = 0.975, F1 = 0.912). On the adult data sets, the models had AUC around 0.95 and F1 around 0.91. Conclusions The models performed close to perfectly on its defined domain, and also performed convincingly on reports pertaining to a different patient group and a different modality. The models were deemed suitable for classifying radiology reports for future quality assurance purposes, where the fraction of the examinations with abnormal findings for different sub-groups of patients is a parameter of interest.

Objective: Electronic health records (EHRs) are used for both clinical practice and research. Because mental health service users’ views are underrepresented in perspectives on EHR use, the authors examined service users’ awareness, attitudes, and opinions about EHR data storage and sharing. Methods: A mixed-methods, cross-sectional design was used to examine attitudes of 253 Norwegian mental health service users who were recruited online to complete a quantitative and qualitative (free-text) survey about EHR utilization. Results: Most participants were aware that EHRs were stored (95%) and shared (58%). Most thought that patients benefited from EHR storage (84%), trusted authorities with EHR sharing (71%), were willing to share their EHRs to help others (75%), felt they benefited from EHR sharing (75%), and thought EHR sharing was ethical for health care and research (71%). Fewer were aware of EHR sharing for research (36%), and 62% were aware that shared data were anonymized. Of the participants, 69% recognized privacy risks associated with sharing. Lack of transparency and skepticism about anonymization and misuse of EHR data were concerns and perceived risks. Mental health service users thought that EHRs should be shared for policy development (81%), education and training (85%), improving care quality (89%), research (91%), and clinical decision support (81%). Conclusions: Participants were aware of and supported EHR sharing for research and clinical care. They supported sharing to help others and were willing to fully participate in clinical care and research, as well as to share EHR information for their own care, research, and the care of others.

This paper recounts the successful BUPdata, a discontinued electronic health record (EHR) system for Child and Adolescent Mental Health Services (CAMHS) in Norway. It was developed and owned by the national association for CAMHS and fulfilled needs for collaborative care, practice insight, and service management. It aimed to unify the requirements of government, administration, clinicians, patients, and researchers alike, with the goal of providing uniform quality of care nationally. When CAMHS became integrated with specialist healthcare, BUPdata was replaced with more a general EHR system offering far less functionality and insight into CAMHS practice. We have studied BUPdata, and interviewed stakeholders in order to develop decision aids based on practice data analysis and give clinicians and patients insight into successful local practice, collaboration patterns, and overview of local resources.

Background The Living Evidence Map Project at the Norwegian Institute of Public Health (NIPH) gives an updated overview of research results and publications. As part of NIPH’s mandate to inform evidence-based infection prevention, control and treatment, a large group of experts are continously monitoring, assessing, coding and summarising new COVID-19 publications. Screening tools, coding practice and workflow are incrementally improved, but remain largely manual. Results This paper describes how deep learning methods have been employed to learn classification and coding from the steadily growing NIPH COVID-19 dashboard data, so as to aid manual classification, screening and preprocessing of the rapidly growing influx of new papers on the subject. Our main objective is to make manual screening scalable through semi-automation, while ensuring high-quality Evidence Map content. Conclusions We report early results on classifying publication topic and type from titles and abstracts, showing that even simple neural network architectures and text representations can yield acceptable performance.

Background Nearly half of all mental health disorders develop prior to the age of 15. Early assessments, diagnosis, and treatment are critical to shortening single episodes of care, reducing possible comorbidity and long-term disability. In Norway, approximately 20% of all children and adolescents are experiencing mental health problems. To address this, health officials in Norway have called for the integration of innovative approaches. A clinical decision support system (CDSS) is an innovative, computer-based program that provides health professionals with clinical decision support as they care for patients. CDSS use standardized clinical guidelines and big data to provide guidance and recommendations to clinicians in real-time. IDDEAS (Individualised Digital DEcision Assist System) is a CDSS for diagnosis and treatment of child and adolescent mental health disorders. The aim of IDDEAS is to enhance quality, competency, and efficiency in child and adolescent mental health services (CAMHS). Methods/design IDDEAS is a mixed-methods innovation and research project, which consists of four stages: 1) Assessment of Needs and Preparation of IDDEAS; 2) The Development of IDDEAS CDSS Model; 3) The Evaluation of the IDDEAS CDSS; and, 4) Implementation & Dissemination. Both qualitative and quantitative methods will be used for the evaluation of IDDEAS CDSS model. Child and adolescent psychologists and psychiatrists (n = 30) will evaluate the IDDEAS` usability, acceptability and relevance for diagnosis and treatment of attention-deficit/hyperactivity disorder. Discussion The IDDEAS CDSS model is the first guidelines and data-driven CDSS to improve efficiency of diagnosis and treatment of child and adolescent mental health disorders in Norway. Ultimately, IDDEAS will help to improve patient health outcomes and prevent long-term adverse outcomes by providing each patient with evidence-based, customized clinical care. © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Loss of consciousness, so-called syncope, is a commonly occurring symptom associated with worse prognosis for a number of heart-related diseases. We present a comparison of methods for a diagnosis classification task in Norwegian clinical notes, targeting syncope, i.e. fainting cases. We find that an often neglected baseline with keyword matching constitutes a rather strong basis, but more advanced methods do offer some improvement in classification performance, especially a convolutional neural network model. The developed pipeline is planned to be used for quantifying unregistered syncope cases in Norway.

Software engineering courses include practical and theoretical elements that give many options for pedagogical combinations among them. In this paper, we report on two different pedagogical approaches for an undergraduate, introductory project-based software engineering course with more than 500 students working in collaborative scrum teams. We call one approach ‘Every Student is an Innovator’, and the other ‘No Student Left Behind’. This SE course has been long-running, with stable learning objectives and content. However, from one year to another, we radically changed the pedagogical approach of the course along several dimensions, among them the technical framework, software tools, project topic, mentor roles, assessment form and frequency, feedback and degree of student innovativeness. We report on the perceived challenges, detailed changes, the anticipated effects on the course learning outcomes. The results showed that innovativeness and fun need freedom and flexibility with processes and technology. However, strict design requirements and systematic guidance ensure fulfillment of learning objectives. Analyzing student and staff feedback, we find that both approaches lead to students using more time than intended and worrying about unknown assessment criteria. © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Infections caused by central venous catheter (CVC) use is a serious and under-reported problem in healthcare. The CVC is almost ubiquitous in critical care because it enables fast circulatory monitoring and central ad- ministration of medication and nutrition. However, the CVC exposes the patient to a risk of blood-stream infections (BSI). Explicit documentation of normal CVC usage and exposure is sparse and indirect in the health record. For a clinician, CVC presence is simple to infer from record statements about procedures, plans and results related to CVC. In order to capture evidence about CVC-related risk of infections and complications, it is important to develop computerized tools that can estimate individual patient days of CVC exposure retrospectively for large cohorts of patients. Towards that objec- tive, we have developed methods for learning classifiers for statements about CVC-related events occurring in the textual health record. This includes developing and testing an annotation ontology of events and indicators, an- notation guidelines, a gold standard of annotated clinical records selected from a corpus of complete health records for more 800 episodes of care and collecting alternate health register evidence for validation purposes. This paper describes the available data and gold standard, feature selection ap- proaches and our experiments with different classification algorithms. We find that even with limited data it is possible to build reasonably accurate sentence classifiers for the most important events. We also find that mak- ing use of document meta information helps improve classification quality by providing additional context to a sentence. Finally, we outline some strate- gies on using our results for future analysis and reasoning about CVC usage intervals and CVC exposure over individual patient trajectories.

Infections caused by central venous catheter (CVC) use is a serious and under-reported problem. In our research we explore methods of automatically detecting CVC use from clinical documentation for quality improvement and surveillance purposes. This paper describes our initial research on this topic, where we build CVC event classifiers based on an episodes of care corpus and an annotated gold standard. After describing the available data and gold standard we then experiment with different classification algorithms and feature selection approaches. We find that even with limited data it is possible to build reasonably accurate sentence classifiers, at least for the events that are most important to us. We also find that making use of document meta information may help improve classification quality by providing additional context to a sentence. Finally, we outline some strategies on using these preliminary clinical document-centric results as a tool for future analysis and elicitation of CVC usage intervals over full patient histories. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

In this article, we describe the development of annotation guidelines for family history information in Norwegian clinical text. We make use of incrementally developed synthetic clinical text describing patients’ family history relating to cases of cardiac disease and present a general methodology which integrates the synthetically produced clinical statements and guideline development. We analyze inter-annotator agreement based on the developed guidelines and present results from experiments aimed at evaluating the validity and applicability of the annotated corpus using machine learning techniques. The resulting annotated corpus contains 477 sentences and 6030 tokens. Both the annotation guidelines and the annotated corpus are made freely available and as such constitutes the first publicly available resource of Norwegian clinical text.

Advances in information systems have enabled guideline developer organisations to disseminate the guidelines on the web with accompanied search functionality. The aim of searching functions is enabling guideline users to find the most relevant guideline for their clinical questions. Hence evaluating the user experiences in finding answers to their clinical questions in the published guidelines on the web with search functionality is important in order to understand the factors that affect users’ satisfaction and performance. Thus, in this study we studied how users interact with clinical guidelines during the decision making process, and examined how the structure of the clinical guidelines accompanied with search function impacted finding the right answer. We conducted an empirical study with a total of 19 medical students, divided into two groups. The first group accessed the list of all available national stroke guidelines while the second group accessed the most relevant guidelines. First a scenario was given to the participants. Second, the participants were asked to follow a set of tasks and then fill out a questionnaire. Analysis of the answers revealed that the more relevant the presented guideline was to the patient scenario, the more the user was successful in finding the right answer, the easier it was for the user to find the answer and the more satisfied the user was with the guideline structure. Although the objective measure of finding the answer in the second group was higher, their perception of finding the right answer was lower than the first group. Hence, a more detailed study is required to investigate what factors impacted the participant's perception score. Our findings highlight that conducting a usability test on the national guidelines user interface and investigating the best position of the search box for capturing the users’ attention are necessary.

Researchers have proposed methods and tools in order to translate the guidelines to a computer interpretable format in order to generate patient-specific recommendations and highlight the most relevant recommendation to clinicians, known as a clinical decision support system (CDSS). Although there are many evaluation aspects that have been considered to assess the impact of CDSS, however investigating the users’ feedback with respect to recommendation ranking has been neglected. Hence, in this study we compared the search-based and content-based recommendation rankings for the guideline users. We first developed two prototypes and then conducted a case study. A survey, usability test and questionnaire were administered in order to achieve more accurate results. Our results showed that users were more successful in finding the right recommendation for treatment using a content-based prototype. 75% of participants preferred the content-based prototype. Furthermore, we concluded that the less experienced user preferred the content-based prototype. Although usability test scores in the searched-based prototype were higher compared to content-based module, our interview results revealed the opposite outcome regarding user satisfaction. Our results indicated that users are mostly concerned with credibility of recommendations; hence providing additional information with the graded recommendation would increase their confidence in choosing recommendations. In addition, participants stated that understanding the recommendation ranking increased their confidence when choosing recommendations. Regarding the search-based prototype, users mostly searched for an acronym rather than the full length term. Participants suggested a combination of search-based and content-based approaches when finding the guidelines and relevant treatments.

Background To improve consistency and streamline development and publication of clinical guidelines (GL), there is a need for appropriate software support. We have found few specific tools for the actual authoring and maintaining of GLs, and correspondingly few analyses or reviews of GL development tool functionality. In order to assist GL developers in selecting and evaluating tools, this study tries to address the perceived gap by pursuing four goals: 1) identifying available tools, 2) reviewing a representative group of tools and their supported functionalities, 3) uncovering themes of features that the studied tools support, and 4) compare the selected tools with respect to the themes. Methods We conducted a literature search using PubMed and Google Scholar in order to find GL development tools (GDT). We also explored tools and Content Management Systems (CMS) used in representative organisations and international communities that develop and maintain GLs. By reading a selected representative group of five GL tool manuals, exploring tools hands-on, we uncovered 8 themes of features. All found tools were compared according to these themes in order to identify the level of functionality they offer to support the GL development and publishing process. In order to limit the scope, tools for designing computer-interpretable/executable GL are excluded. Results After finding 1552 published papers, contacting 7 organizations and international communities, we identified a total of 19 unique tools, of which 5 tools were selected as representative in this paper. We uncovered a total of 8 themes of features according to the identified functionalities that each tool provides. Four features were common among tools: Collaborative authoring process support, user access control, GL repository management, electronic publishing. We found that the GRADE methodology was supported by three of the reviewed tools, while only two tools support annotating GL with MeSH terms. We also identified that monitoring progress, reference management, Managing versions (version control), and Change control (tracking) were often the missing features. Conclusion The results can promote sector discussion and eventual agreement on important tool functionality. It may aid tool and GL developers towards more efficient, and effective, GL authoring. © The Author(s). 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Great advances in molecular biology, genetics and imaging serve to enhance the desire to develop multi-level and multi-scale models for "personalized medicine" but they remain very challenging for high prevalence, high impact childhood onset neuropsychiatric disorders. We currently have the capacity to develop innovative, effective and efficient clinical decision support models, while also creating the opportunities for rapidly incorporating multi-scale, multi-level data as they become available in the very near future. Sensitive to these complex issues, this paper discusses how these existing resources can be used to develop state-of-the-art clinical decision support models that will improve patient care and reduce costs in primary care and specialist settings in the present while creating a mechanism for adding biomarker and other data as it emerges. The new clinical decision support system for child and adolescent mental disorders needs to integrate existing heterogeneous, geographically distinct, current and historical patient-specific and population-specific data in order to generate new information and models for clinic decision support at the level of the individual patient, using the already available informatics frameworks.

How can we use information visualization to support retrospective, exploratory analysis of collections of histories for patients admitted to acute care? This paper describes a novel design for visual cohort identification and exploration. We have developed a tool that integrates multiple, heterogeneous clinical data sources and allows alignment, querying and abstraction in a common workbench. This paper presents results from two projects and a review of related work in the field of information visualization including both presentation and interactive navigation of the information. We have developed an interactive prototype and present the visualization aspect of this prototype and a brief demonstration of its use in a research project with a large cohort of patients. The prototype represents and reasons with patient events in different OWL-formalizations according to the perspective and use: One for integration and alignment of patient records and observations; Another for visual presentation of individual or cohort trajectories. Health researchers have successfully analyzed large cohorts (over 100,000 individuals) using the tool. We have also used the tool to produce interactive personal health time-lines (for more than 10,000 individuals) on the web. Utility, usability and effect have been tested extensively and the results so far are promising. We envision that clinicians who want to learn more about groups of patients and their treatment processes will find the tool valuable. In addition, we believe that the visualization can be useful to researchers looking at data to be statistically evaluated, in order to discover new hypotheses or get ideas for the best analysis strategies. Our main conclusion is that the tool is usable, but it can be challenging to use for large data sets. Postprint version. © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.

We have developed a tool for annotation of electronic health record (ERR) data. Currently we are in the process of manually annotating a corpus of Norwegian general practitioners' ERRs with mainly linguistic information. The purpose of this project is to attain a linguistically annotated corpus of patient histories from general practice. This corpus will be put to future use in medicallanguage processing and information extraction applications. The paper outlines some of our practical experiences from developing such a corpus and, in particular, the effects of semi-automated annotation. We have also dene some preliminary experiments with part-of-speech tagging based on our corpus. The results indicate that relevant training data from the clinical domain gives better results for the tagging task in this domain than training the tagger on a corpus from a more general domain. We are planning to expand the corpus annotations with medical information at a later stage.

Background Free text is helpful for entering information into electronic health records, but reusing it is a challenge. The need for language technology for processing Finnish and Swedish healthcare text is therefore evident; however, Finnish and Swedish are linguistically very dissimilar. In this paper we present a comparison of characteristics in Finnish and Swedish free-text nursing narratives from intensive care. This creates a framework for characterising and comparing clinical text and lays the groundwork for developing clinical language technologies. Methods Our material included daily nursing narratives from one intensive care unit in Finland and one in Sweden. Inclusion criteria for patients were an inpatient period of least five days and an age of at least 16 years. We performed a comparative analysis as part of a collaborative effort between Finnish- and Swedish-speaking healthcare and language technology professionals that included both qualitative and quantitative aspects. The qualitative analysis addressed the content and structure of three average-sized health records from each country. In the quantitative analysis 514 Finnish and 379 Swedish health records were studied using various language technology tools. Results Although the two languages are not closely related, nursing narratives in Finland and Sweden had many properties in common. Both made use of specialised jargon and their content was very similar. However, many of these characteristics were challenging regarding development of language technology to support producing and using clinical documentation. Conclusions The way Finnish and Swedish intensive care nursing was documented, was not country or language dependent, but shared a common context, principles and structural features and even similar vocabulary elements. Technology solutions are therefore likely to be applicable to a wider range of natural languages, but they need linguistic tailoring. Availability The Finnish and Swedish data can be found at: http://www.dsv.su.se/hexanord/data/.

Many small and medium-sized companies that develop software experience the same problems repeatedly, and have few systems in place to learn from their own mistakes as well as their own successes. Here, we propose a lightweight method to collect experience from completed software projects, and compare the results of this method to more widely applied experience reports. We find that the new method captures more information about core processes related to software development in contrast to experience reports that focus more on management processes.