Search

Natural language processing (NLP) of clinical trial documents can be useful in new trial design. Here we identify entity types relevant to clinical trial design and propose a framework called CT-BERT for information extraction from clinical trial text. We trained named entity recognition (NER) models to extract eligibility criteria entities by fine-tuning a set of pre-trained BERT models. We then compared the performance of CT-BERT with recent baseline methods including attention-based BiLSTM and Criteria2Query. The results demonstrate the superiority of CT-BERT in clinical trial NLP., HealthNLP 2021, IEEE International Conference on Healthcare Informatics (ICHI 2021)

Background Finding specific scientific articles in a large collection is an important natural language processing challenge in the biomedical domain. Systematic reviews and interactive article search are the type of downstream applications that benefit from addressing this problem. The task often involves screening articles for a combination of selection criteria. While machine learning was previously used for this purpose, it is not known if different criteria should be modeled together or separately in an ensemble model. The performance impact of the modern contextual language models on the task is also not known. Methods We framed the problem as text classification and conducted experiments to compare ensemble architectures, where the selection criteria were mapped to the components of the ensemble. We proposed a novel cascade ensemble analogous to the step-wise screening process employed in developing the gold standard. We compared performance of the ensembles with a single integrated model, which we refer to as the individual task learner (ITL). We used SciBERT, a variant of BERT pre-trained on scientific articles, and conducted experiments using a manually annotated dataset of ~49 K MEDLINE abstracts, known as Clinical Hedges. Results The cascade ensemble had significantly higher precision (0.663 vs. 0.388 vs. 0.478 vs. 0.320) and F measure (0.753 vs. 0.553 vs. 0.628 vs. 0.477) than ITL and ensembles using Boolean logic and a feed-forward network. However, ITL had significantly higher recall than the other classifiers (0.965 vs. 0.872 vs. 0.917 vs. 0.944). In fixed high recall studies, ITL achieved 0.509 precision @ 0.970 recall and 0.381 precision @ 0.985 recall on a subset that was studied earlier, and 0.295 precision @ 0.985 recall on the full dataset, all of which were improvements over the previous studies. Conclusion Pre-trained neural contextual language models (e.g. SciBERT) performed well for screening scientific articles. Performance at high fixed recall makes the single integrated model (ITL) more suitable among the architectures considered here, for systematic reviews. However, high F measure of the cascade ensemble makes it a better approach for interactive search applications. The effectiveness of the cascade ensemble architecture suggests broader applicability beyond this task and the dataset, and the approach is analogous to query optimization in Information Retrieval and query optimization in databases.

Background: Identifying relationships between clinical events and temporal expressions is a key challenge in meaningfully analyzing clinical text for use in advanced AI applications. While previous studies exist, the state-of-the-art performance has significant room for improvement. Methods: We studied several variants of BERT (Bidirectional Encoder Representations using Transformers) some involving clinical domain customization and the others involving improved architecture and/or training strategies. We evaluated these methods using a direct temporal relations dataset which is a semantically focused subset of the 2012 i2b2 temporal relations challenge dataset. Results: Our results show that RoBERTa, which employs better pre-training strategies including using 10x larger corpus, has improved overall F measure by 0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by 24% relative to the previous state-of-the-art performance achieved with an SVM (support vector machine) model. Conclusion: Modern contextual language modeling neural networks, pre-trained on a large corpus, achieve impressive performance even on highly-nuanced clinical temporal relation tasks., Comment: 10 pages, 1 Figure, 7 Tables

Abbreviations sense disambiguation is a special case of word sense disambiguation. Machine learning methods based on neural networks showed promising results for word sense disambiguation (Festag and Spreckelsen, 2017) [1] and, here we assess their effectiveness for abbreviation sense disambiguation.Convolutional Neural Network (CNN) models were trained, one for each abbreviation, to disambiguate abbreviation senses. A reverse substitution (of long forms with short forms) method from a previous study was used on clinical narratives from Cleveland Clinic, USA, to auto-generate training data. Accuracy of the CNN and traditional Support Vector Machine (SVM) models were studied using: (a) 5-fold cross validation on the auto-generated training data; (b) a manually created, set-aside gold standard; and (c) 10-fold cross validation on a publicly available dataset from a previous study.CNN improved accuracy by 1-4 percentage points on all the three datasets compared to SVM, and the improvement was the most for the set-aside dataset. The improvement was statistically significant at p 0.05 on the auto-generated dataset. We found that for some common abbreviations, sense distributions mismatch between the test and auto generated training data, and mitigating the mismatch significantly improved the model accuracy.The neural network models work well in disambiguating abbreviations in clinical narratives, and they are robust across datasets. This avoids feature-engineering for each dataset. Coupled with an enhanced auto-training data generation, neural networks can simplify development of a practical abbreviation disambiguation system.

We present a new model of patient record search, called SemanticFind, which goes beyond traditional textual and medical synonym matches by locating patient data that a clinician would want to see rather than just what they ask for. The new model is implemented by making extensive use of the UMLS semantic network, distributional semantics, and NLP, to match query terms along several dimensions in a patient record with the returned matches organized accordingly. The new approach finds all clinically related concepts without the user having to ask for them. An evaluation of the accuracy of SemanticFind shows that it found twice as many relevant matches compared to those found by literal (traditional) search alone, along with very high precision and recall. These results suggest potential uses for SemanticFind in clinical practice, retrospective chart reviews, and in automated extraction of quality metrics.

Background The complicated nature of cloud computing encompassing internet-based technologies and service models for delivering IT applications, processing capability, storage, and memory space brings along challenging problems. Some issues such as information security, privacy, and legal aspects of cloud computing may become challenging while cross passing with another complex domain like healthcare. Objectives The present study was conducted to report the results of a systematic literature review on the legal aspects of health cloud. Method The original English papers published in Pub Med, Scopus, Web of Science, and IEEE Digital Library databases were extracted, among which1582 were related to the legal aspects of health cloud environment and were selected using predefined search strings. Conclusion Through the review process, effective factors in relation to a health cloud legal framework were identified and accordingly, a proper design was developed for this domain. Next, the identified factors were confirmed and adjusted by mapping the contents of the selected papers to different categories and subcategories under the proposed framework. Five Main categories like the issues related to the compliance, data protection, Identity Credential Access Management (ICAM), ownership, and quality of service were selected as the basic pillars in the proposed framework. Finally, 22 papers were selected, among which 19 were mapped to the compliance issues, 18 the issues related to “Data protection” were addressed, and 14 “Identity Credential Access Management (ICAM)” was discussed. Fifteen Papers were mapped to “Data ownership” and “Quality of service” categories. Some papers were found to present some solutions in all the mentioned areas; however, most of them have addressed only a few issues.

Objective An accurate, comprehensive and up-to-date problem list can help clinicians provide patient-centered care. Unfortunately, problem lists created and maintained in electronic health records by providers tend to be inaccurate, duplicative and out of date. With advances in machine learning and natural language processing, it is possible to automatically generate a problem list from the data in the EHR and keep it current. In this paper, we describe an automated problem list generation method and report on insights from a pilot study of physicians’ assessment of the generated problem lists compared to existing providers-curated problem lists in an institution’s EHR system. Materials and methods The natural language processing and machine learning-based Watson 1 method models clinical thinking in identifying a patient’s problem list using clinical notes and structured data. This pilot study assessed the Watson method and included 15 randomly selected, de-identified patient records from a large healthcare system that were each planned to be reviewed by at least two internal medicine physicians. The physicians created their own problem lists, and then evaluated the overall usefulness of their own problem lists (P), Watson generated problem lists (W), and the existing EHR problem lists (E) on a 10-point scale. The primary outcome was pairwise comparisons of P, W, and E. Results Six out of the 10 invited physicians completed 27 assessments of P, W, and E, and in process evaluated 732 Watson generated problems and 444 problems in the EHR system. As expected, physicians rated their own lists, P, highest. However, W was rated higher than E. Among 89% of assessments, Watson identified at least one important problem that physicians missed. Conclusion Cognitive computing systems like this Watson system hold the potential for accurate, problem-list-centered summarization of patient records, potentially leading to increased efficiency, better clinical decision support, and improved quality of patient care.

An enterprise typically operates multiple data center sites, each handling workloads according to an enterprise-level strategy. Sharing resources across multiple sites (or enterprises) brings up several important problems. Each site may have its own policies that govern its interactions with other remote sites. Different policies impact the system performance in different ways. The site administrators and system designers need to understand the effects of a given set of policies on different workloads. In this paper, we describe an analysis methodology that determines the impact of policies on the workloads, and we present results and validation for a prototypical multi-site resource sharing system. Our analytical tool is capable of evaluating complex policies on a large scale system and permits independent policies for each site, so that policy makers can quickly evaluate several alternatives and their effects on the workloads before deploying them.

Modern business information systems are typically multitiered distributed systems comprising Web services, application services, databases, enterprise information systems, file systems, storage controllers, and other storage systems. In such environments, data is stored in different forms at multiple tiers, with each tier associated with some level of data abstraction. An information entity owned by an application generally maps to several data entities, logically associated across tiers and related to the application. Discovery of such relationships in a distributed system is a challenging problem, complicated by the widespread adoption of virtualization technologies and by the traditional tendency to manage each tier as an independent domain. In this paper, we present a system and methodology for model-driven discovery of end-to-end application-data relationships spanning multiple tiers, from the applications to the lowest levels of the storage hierarchy. The key to our methodology involves modeling how data is used and transformed by distributed software components. An important benefit of our system, which we call Galapagos, is the ability to reflect business decisions expressed at the application level to the level of storage.

Exponential growth in storage requirements and an increasing number of heterogeneous devices and application policies are making enterprise storage management a nightmare for administrators. Back-of-the-envelope calculations, rules of thumb, and manual correlation of individual device data are too error prone for the day-to-day administrative tasks of resource provisioning, problem determination, performance management, and impact analysis. Storage management tools have evolved over the past several years from standardizing the data reported by storage subsystems to providing intelligent planners. In this paper, we describe that evolution in the context of the IBM Total Storage® Productivity Center (TPC)--a suite of tools to assist administrators in the day-to-day tasks of monitoring, configuring, provisioning, managing change, analyzing configuration, managing performance, and determining problems. We describe our ongoing research to develop ways to simplify and automate these tasks by applying advanced analytics on the performance statistics and raw configuration and event data collected by TPC using the popular Storage Management Initiative-Specification (SMI-S). In addition, we provide details of SMART (storage management analytics and reasoning technology) as a library that provides a collection of data-aggregation functions and optimization algorithms.

The JavaTM platform has the potential to revolutionize computing, with its promise of "write once, run anywhere"TM development. However, in order to realize this potential, Java applications must demonstrate satisfactory performance. Rapid progress has been made in addressing Java performance, although most of the initial efforts have targeted Java client applications. To make a significant impact in network computing, server applications written in the Java language, or those using Java extensions, frameworks, or components, must exhibit a competitive level of performance. One obstacle to obtaining this goal has been the lack of well-defined, server-specific, Java benchmarks. This paper helps address this shortcoming by defining representative Java server benchmarks. These benchmarks represent server application areas, including Web-based dynamic content delivery (servlets), business object frameworks, and multitier transactional data acquisition. Where applicable, we present benchmarks written using both the Java programming model (i.e., servlets) and the legacy model (i.e., the Common Gateway Interface) for direct comparisons of delivered performance. We also present performance measurements and analysis from multiple IBM server platforms, including both uniprocessor and multiprocessor systems.

Presents dynamic load-sharing heuristics that use predicted resource requirements of processes to manage workloads in a distributed system. A previously developed statistical pattern-recognition method is employed for resource prediction. While nonprediction-based heuristics depend on a rapidly changing system status, the new heuristics depend on slowly changing program resource usage patterns. Furthermore, prediction-based heuristics can be more effective since they use future requirements rather than just the current system state. Four prediction-based heuristics, two centralized and two distributed, are presented. Using trace driven simulations, they are compared against random scheduling and two effective nonprediction based heuristics. Results show that the prediction-based centralized heuristics achieve up to 30% better response times than the nonprediction centralized heuristic, and that the prediction-based distributed heuristics achieve up to 50% improvements relative to their nonpredictive counterpart. >

The goal of autonomic storage allocation is to achieve management of storage resources, including allocation, performance monitoring, and hotspot elimination, by specifying comparatively high-level goals, rather than by means of low-level manual steps. The process of automation should allow specification of policies as administrator specified constraints under which the resources are managed. This paper describes the system design and implementation experiences from a prototype autonomic storage manager being developed in IBM Research. The prototype is being developed for a storage network that includes a SAN switch, an IBM Enterprise Storage Subsystem, and AIX servers. Our early experience from this prototype implementation is that there are a large number of mundane manual steps in storage management and it is feasible to automate them such that the automation is driven by higher-level goals under policy control. However, to manage heterogeneous storage a standard ontology is needed for specification of goals and how to achieve them.

A statistical approach is developed for predicting the CPU time, the file I/O, and the memory requirements of a program at the beginning of its life, given the identity of the program. Initially, statistical clustering is used to identify high-density regions of process resource usage. The identified regions form the states for building a state-transition model to characterize the resource usage of each program in its past executions. The prediction scheme uses the knowledge of the program's resource usage in its last execution together with its state-transition model to predict the resource usage in its next execution. The prediction scheme is shown to work using process resource-usage data collected from a VAX 11/780 running 4.3 BSD Unix. The results show that the predicted values correlate strongly with the actual; the coefficient of correlation between the predicted and actual values for CPU time is 0.84. The errors in prediction are mostly small and are heavily skewed toward small values. >