Search

In this work, we formulated the named entity recognition (NER) task as a multi-answer knowledge guided question-answer task (KGQA) and showed that the knowledge guidance helps to achieve state-of-the-art results for 11 of 18 biomedical NER datasets. We prepended five different knowledge contexts—entity types, questions, definitions, and examples—to the input text and trained and tested BERT-based neural models on such input sequences from a combined dataset of the 18 different datasets. This novel formulation of the task (a) improved named entity recognition and illustrated the impact of different knowledge contexts, (b) reduced system confusion by limiting prediction to a single entity-class for each input token (i.e., B , I , O only) compared to multiple entity-classes in traditional NER (i.e., B entity 1, B entity 2, I entity 1, I , O ), (c) made detection of nested entities easier, and (d) enabled the models to jointly learn NER-specific features from a large number of datasets. We performed extensive experiments of this KGQA formulation on the biomedical datasets, and through the experiments, we showed when knowledge improved named entity recognition. We analyzed the effect of the task formulation, the impact of the different knowledge contexts, the multi-task aspect of the generic format, and the generalization ability of KGQA. We also probed the model to better understand the key contributors for these improvements.

Natural language processing (NLP) of clinical trial documents can be useful in new trial design. Here we identify entity types relevant to clinical trial design and propose a framework called CT-BERT for information extraction from clinical trial text. We trained named entity recognition (NER) models to extract eligibility criteria entities by fine-tuning a set of pre-trained BERT models. We then compared the performance of CT-BERT with recent baseline methods including attention-based BiLSTM and Criteria2Query. The results demonstrate the superiority of CT-BERT in clinical trial NLP., HealthNLP 2021, IEEE International Conference on Healthcare Informatics (ICHI 2021)

Special purpose lexicons are invaluable in biomedical natural language processing. They are especially crucial for a task such as the 13-criteria based cohort identification from clinical notes, process in N2C2 2018 Track 1 Challenge. While manually developed lexicons helped us achieve high performance, the process was ad hoc and nonreproducible. This paper presents a semi-automated lexicon induction method, using Logistic Regression (LR) and word embeddings, which brings rigor to the process. The key idea was to use n-grams in the training corpus as features of LR and identify those features (n-grams) with the most impact on the outcome as the lexicon. The semi-automatically generated lexicons achieved overall F measure of 0.9166 versus 0.9003 with manually generated lexicons. Therefore, this study shows that lexicons generated using a rigorous, semi-automated approach can retain performance while bringing rigor to the process.

Adverse drug events (ADEs) occur in approximately 2–5% of hospitalized patients, often resulting in poor outcomes or even death. Extraction of ADEs from clinical narratives can accelerate and automate pharmacovigilance. Using state-of-the-art deep-learning neural networks to jointly model concept and relation extraction, we achieved the highest integrated task score in the 2018 Medication and Adverse Drug Event (MADE) 1.0 challenge. We used a combined bidirectional long short-term memory (BiLSTM) and conditional random fields (CRF) neural network to detect medical entities relevant to ADEs and a combined BiLSTM and attention network to determine relations, including the adverse drug reaction relation between medication and sign or symptom entities. Using these models, we conducted three experiments: (1) separate and sequential modeling of entities and relations; (2) joint modeling where relations between medications and sign or symptoms determined ADE and indication entities; (3) use of information from external resources such as the US FDA’s adverse event database as additional input to the second method. Joint modeling improved the overall task accuracy from 0.62 to 0.65 F measure, and the additional use of external resources improved the accuracy to 0.66 F measure. Given the gold-standard medical entity labels, the joint model plus external resources method achieved F measures of 0.83 for ADE-relevant medical entity detection and 0.87 for relation detection. It is important to use joint modeling techniques and external resources for effectively detecting ADEs from clinical narratives in electronic health record (EHR) systems. While the extraction of entities and relations individually achieved high accuracy, the integrated task still has room for further improvement.

Background Finding specific scientific articles in a large collection is an important natural language processing challenge in the biomedical domain. Systematic reviews and interactive article search are the type of downstream applications that benefit from addressing this problem. The task often involves screening articles for a combination of selection criteria. While machine learning was previously used for this purpose, it is not known if different criteria should be modeled together or separately in an ensemble model. The performance impact of the modern contextual language models on the task is also not known. Methods We framed the problem as text classification and conducted experiments to compare ensemble architectures, where the selection criteria were mapped to the components of the ensemble. We proposed a novel cascade ensemble analogous to the step-wise screening process employed in developing the gold standard. We compared performance of the ensembles with a single integrated model, which we refer to as the individual task learner (ITL). We used SciBERT, a variant of BERT pre-trained on scientific articles, and conducted experiments using a manually annotated dataset of ~49 K MEDLINE abstracts, known as Clinical Hedges. Results The cascade ensemble had significantly higher precision (0.663 vs. 0.388 vs. 0.478 vs. 0.320) and F measure (0.753 vs. 0.553 vs. 0.628 vs. 0.477) than ITL and ensembles using Boolean logic and a feed-forward network. However, ITL had significantly higher recall than the other classifiers (0.965 vs. 0.872 vs. 0.917 vs. 0.944). In fixed high recall studies, ITL achieved 0.509 precision @ 0.970 recall and 0.381 precision @ 0.985 recall on a subset that was studied earlier, and 0.295 precision @ 0.985 recall on the full dataset, all of which were improvements over the previous studies. Conclusion Pre-trained neural contextual language models (e.g. SciBERT) performed well for screening scientific articles. Performance at high fixed recall makes the single integrated model (ITL) more suitable among the architectures considered here, for systematic reviews. However, high F measure of the cascade ensemble makes it a better approach for interactive search applications. The effectiveness of the cascade ensemble architecture suggests broader applicability beyond this task and the dataset, and the approach is analogous to query optimization in Information Retrieval and query optimization in databases.

Background: Identifying relationships between clinical events and temporal expressions is a key challenge in meaningfully analyzing clinical text for use in advanced AI applications. While previous studies exist, the state-of-the-art performance has significant room for improvement. Methods: We studied several variants of BERT (Bidirectional Encoder Representations using Transformers) some involving clinical domain customization and the others involving improved architecture and/or training strategies. We evaluated these methods using a direct temporal relations dataset which is a semantically focused subset of the 2012 i2b2 temporal relations challenge dataset. Results: Our results show that RoBERTa, which employs better pre-training strategies including using 10x larger corpus, has improved overall F measure by 0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by 24% relative to the previous state-of-the-art performance achieved with an SVM (support vector machine) model. Conclusion: Modern contextual language modeling neural networks, pre-trained on a large corpus, achieve impressive performance even on highly-nuanced clinical temporal relation tasks., Comment: 10 pages, 1 Figure, 7 Tables

Social media posts may yield clues to the subject’s (usually, the writer’s) suicide risk and intent, which can be used for timely intervention. This research, motivated by the CLPsych 2019 shared task, developed neural network-based methods for analyzing posts in one or more Reddit forums to assess the subject’s suicide risk. One of the technical challenges this task poses is the large amount of text from multiple posts of a single user. Our neural network models use the advanced multi-headed Attention-based autoencoder architecture, called Bidirectional Encoder Representations from Transformers (BERT). Our system achieved the 2nd best performance of 0.477 macro averaged F measure on Task A of the challenge. Among the three different alternatives we developed for the challenge, the single BERT model that processed all of a user’s posts performed the best on all three Tasks.

Abbreviations sense disambiguation is a special case of word sense disambiguation. Machine learning methods based on neural networks showed promising results for word sense disambiguation (Festag and Spreckelsen, 2017) [1] and, here we assess their effectiveness for abbreviation sense disambiguation.Convolutional Neural Network (CNN) models were trained, one for each abbreviation, to disambiguate abbreviation senses. A reverse substitution (of long forms with short forms) method from a previous study was used on clinical narratives from Cleveland Clinic, USA, to auto-generate training data. Accuracy of the CNN and traditional Support Vector Machine (SVM) models were studied using: (a) 5-fold cross validation on the auto-generated training data; (b) a manually created, set-aside gold standard; and (c) 10-fold cross validation on a publicly available dataset from a previous study.CNN improved accuracy by 1-4 percentage points on all the three datasets compared to SVM, and the improvement was the most for the set-aside dataset. The improvement was statistically significant at p 0.05 on the auto-generated dataset. We found that for some common abbreviations, sense distributions mismatch between the test and auto generated training data, and mitigating the mismatch significantly improved the model accuracy.The neural network models work well in disambiguating abbreviations in clinical narratives, and they are robust across datasets. This avoids feature-engineering for each dataset. Coupled with an enhanced auto-training data generation, neural networks can simplify development of a practical abbreviation disambiguation system.

Background The complicated nature of cloud computing encompassing internet-based technologies and service models for delivering IT applications, processing capability, storage, and memory space brings along challenging problems. Some issues such as information security, privacy, and legal aspects of cloud computing may become challenging while cross passing with another complex domain like healthcare. Objectives The present study was conducted to report the results of a systematic literature review on the legal aspects of health cloud. Method The original English papers published in Pub Med, Scopus, Web of Science, and IEEE Digital Library databases were extracted, among which1582 were related to the legal aspects of health cloud environment and were selected using predefined search strings. Conclusion Through the review process, effective factors in relation to a health cloud legal framework were identified and accordingly, a proper design was developed for this domain. Next, the identified factors were confirmed and adjusted by mapping the contents of the selected papers to different categories and subcategories under the proposed framework. Five Main categories like the issues related to the compliance, data protection, Identity Credential Access Management (ICAM), ownership, and quality of service were selected as the basic pillars in the proposed framework. Finally, 22 papers were selected, among which 19 were mapped to the compliance issues, 18 the issues related to “Data protection” were addressed, and 14 “Identity Credential Access Management (ICAM)” was discussed. Fifteen Papers were mapped to “Data ownership” and “Quality of service” categories. Some papers were found to present some solutions in all the mentioned areas; however, most of them have addressed only a few issues.

This paper presents a natural language processing (NLP) based cognitive decision support system that automatically identifies the status of a disease from the clinical notes of a patient record. The system relies on IBM Watson Patient Record NLP analytics and supervised or semi-supervised learning techniques. It uses unstructured text in clinical notes, data from the structured part of a patient record, and disease control targets from the clinical guidelines. We evaluated the system using de-identified patient records of 414 hypertensive patients from a multi-specialty hospital system in the U.S. The experimental results show that, using supervised learning methods, our system can achieve an average 0.86 F1-score in identifying disease status passages and average accuracy of 0.77 in classifying the status as controlled or not. To the best of our knowledge, this is the first system to automatically identify disease control status from clinical notes.

An enterprise typically operates multiple data center sites, each handling workloads according to an enterprise-level strategy. Sharing resources across multiple sites (or enterprises) brings up several important problems. Each site may have its own policies that govern its interactions with other remote sites. Different policies impact the system performance in different ways. The site administrators and system designers need to understand the effects of a given set of policies on different workloads. In this paper, we describe an analysis methodology that determines the impact of policies on the workloads, and we present results and validation for a prototypical multi-site resource sharing system. Our analytical tool is capable of evaluating complex policies on a large scale system and permits independent policies for each site, so that policy makers can quickly evaluate several alternatives and their effects on the workloads before deploying them.

Exponential growth in storage requirements and an increasing number of heterogeneous devices and application policies are making enterprise storage management a nightmare for administrators. Back-of-the-envelope calculations, rules of thumb, and manual correlation of individual device data are too error prone for the day-to-day administrative tasks of resource provisioning, problem determination, performance management, and impact analysis. Storage management tools have evolved over the past several years from standardizing the data reported by storage subsystems to providing intelligent planners. In this paper, we describe that evolution in the context of the IBM Total Storage® Productivity Center (TPC)--a suite of tools to assist administrators in the day-to-day tasks of monitoring, configuring, provisioning, managing change, analyzing configuration, managing performance, and determining problems. We describe our ongoing research to develop ways to simplify and automate these tasks by applying advanced analytics on the performance statistics and raw configuration and event data collected by TPC using the popular Storage Management Initiative-Specification (SMI-S). In addition, we provide details of SMART (storage management analytics and reasoning technology) as a library that provides a collection of data-aggregation functions and optimization algorithms.

The explosive growth of data has led to a situation where the human brain is overloaded with more information than it can process. It is particularly dire in healthcare where critical information may be buried in the mountains of data in the Electronic Medical Record systems (EMR systems) and healthcare workers struggle to make sense of this information to provide the best care for their patients. Cognitive computing, exemplified by Watson, offers the promise of transforming EMR systems from mere data storage to intelligent systems that help physicians in providing improved patient care. When seeing a patient, a physician needs to quickly grasp the summary of the patient’s medical history from the EMR to prepare for the visit and to put the patient’s complaints in context. During the visit, there may be a need to supplement, confirm, and investigate the information that the patient provides with information from the EMR. These information needs can be fulfilled by a cognitive system using advanced analytics on the patient record data. Some of the ways this can happen are a problem-oriented summary of a patient record, precisely answering natural language questions about the patient record content, automatically identifying urgent abnormalities, and by providing precise causes for such abnormalities. In this cognitive computing view, an EMR is an active entity that leverages the vast knowledge of the medical sciences, drug information, and medical ontologies in the context of the patient medical records to meet the information needs of the healthcare provider.

As the use of Electronic Medical Records (EMRs) becomes widespread, the amount of data in an EMR becomes a challenge for its comprehension. We developed problem-oriented EMR summarization to address this issue, as a part of a larger effort of adapting IBM Watson to the medical domain. The problem-orientation refers to the central role of a patient's medical problems in the summary. The summarization uses a generated problem list, relates these generated medical problems to relevant clinical data, and organizes the clinical data in a medically meaningful manner. Watson analytics are used for creating the summarization. This is a step in building the next generation EMR, one that is based not on just keeping record but instead on a conceptual understanding of medicine, thereby crossing the threshold from record storage to an intelligent entity for clinical decision making.

This paper presents a new, compelling argument for a larger potential to reduce labor cost by adopting cloud computing in enterprise IT. The additional cost reduction is possible in the labor intensive activities of customer-specific customization of cloud services, tools support, and customer-facing service management. These essential service delivery processes are excellent candidates for standardization and automation using the cloud computing principles. We present examples of such processes and potential for optimization in these processes.

Presents dynamic load-sharing heuristics that use predicted resource requirements of processes to manage workloads in a distributed system. A previously developed statistical pattern-recognition method is employed for resource prediction. While nonprediction-based heuristics depend on a rapidly changing system status, the new heuristics depend on slowly changing program resource usage patterns. Furthermore, prediction-based heuristics can be more effective since they use future requirements rather than just the current system state. Four prediction-based heuristics, two centralized and two distributed, are presented. Using trace driven simulations, they are compared against random scheduling and two effective nonprediction based heuristics. Results show that the prediction-based centralized heuristics achieve up to 30% better response times than the nonprediction centralized heuristic, and that the prediction-based distributed heuristics achieve up to 50% improvements relative to their nonpredictive counterpart. >

Today, enterprise IT environments are complex as never before with individual applications, tiers, or technologies segregated into individual management domains. Typically, the value of business applications and the dependencies between business and IT objects and IT objects among each other is completely unknown or at least not up to date. Thus, ultimately, the business value of individual IT tasks is unknown. Hence it is very hard to perform global management services such as performance optimization in resource-constrained environments. This deficiency is even more deeply felt by an internal or external services provider called in to set up a new optimization or IT management framework.We propose a framework ITBVM for business-value driven IT optimization with particular emphasis on such enterprise environments. A key part is the use of discovery technologies to provide the link between business value and IT objects. As one instance of the framework, we show how discovery can improve a performance-optimization problem in an otherwise blackbox scenario. We validate these improvements through experiments in a controlled setup and through statistical interpretation of fine-grained dependency discovery in a large real enterprise environment.

IT out sourcing service providers are increasingly being challenged to reduce costs while improving the quality of service implementation and delivery. A key contributor to those goals can be an improved methodology for services solution engineering - which involves analysis of the client's requirements and environment, mapping the information to the capabilities of the service provider, documenting the design and implementation of each service, and defining transformations of the customer's environment to the target IT infrastructure defined by the overall design that facilitates more efficient service delivery. One such improved methodology leverages reuse, in the form of a taxonomy of standardized service offerings and a repository of standardized service designs they can be mapped to. In this paper, we describe a prototype system that provides automation for such a methodology by encoding design policies that are used to assist the services solution architect gather relevant information from the customer and cross check design decisions. The approach demonstrates how the notion of computer-aided design can be applied to the world of IT services and introduced in an incremental manner. This paper presents the system architecture, discusses the sources and models of knowledge, and illustrates how this knowledge can be used with specific examples from the services field.

Enterprise IT environments are complex: business applications rely on distributed middleware running on diverse hardware with components depending on each other in many unexpected ways. Discovery of applications' dependency on IT is a critical step in managing application and IT infrastructure together. Many tools and practices have emerged to discover and report IT assets and applications' dependency on the IT assets. However, our experience in the field shows that there are significant challenges in effectively deploying the tools. There is a critical need to research and develop flexible processes, methods, and practices, and architecture-level support for them in the tools to enable successful discovery using a "built-to-order" approach. In this paper we discuss our experiences with an advanced application-data relationship discovery tools in large scale enterprise environments and based on these experiences we identify three main challenges of effective discovery. They are: deployment process and related security issues; unavailability of software and administration-related information; and tool integration. To address these challenges, here we demonstrate a holistic approach that includes flexible processes, methods, and practices in the tools for achieving the necessary built-to order capability.

This paper addresses the important problem of finding relevant information in the context of a business process. It presents an information access solution called EIL (enterprise information leverage) which combines information extraction and semantic search to support information needs of professionals selling IT services. EIL leverages structured and unstructured data using novel architecture and special purpose algorithms. Our approach is to organize information around business activities (e.g. a sale), and the system supports semantic concept based information retrieval by utilizing both database query and document search where the relevant business activities act as a contextual constraint. We experimentally show that this approach is promising for reducing noise in search results. EIL is currently under pilot deployment in one of the IBM services sales units.

In outsourcing, service obligations broadly refer to the duties and responsibilities of the service provider related to the provision of services. It is critical that the provider clearly understands and properly implements its complete service obligations to the client. The BlueMoon project in IBM develops a novel approach to service obligations management in the context of IT outsourcing. BlueMoon conceptualizes different kinds of service obligations through a formal obligations model. It provides an overarching obligations configurator tool for gathering and validating obligation requirements throughout the outsourcing lifecycle. It further facilitates the implementation of service obligations by automatically binding them to the IT architecture planning and provisioning process and to their uses in various service documents. Work is under way to monitor service delivery operations and verify obligations compliance. Addressing a key challenge in IT outsourcing, bluemoon promises to significantly improve the client's satisfaction, the health of the outsourcing account, and the provider's bottom line.

Services business generates significant amount of human and machine created data. Search and discovery of relevant information from this data is a critical factor in enhancing workforce efficiency and operational excellence in such a business. The challenge is how to leverage the interplay among information retrieval techniques, role of human experts, and business processes. This paper presents a case for using "business activity" (e.g. a sale) as a contextual basis to meet the challenge. We developed a tool based on this approach to address the needs of services professionals. The tool is designed to support role-specific concept based queries that return the most relevant business activities first and subsequently enable retrieval of the most relevant documents and information for the business activity. Any relevant knowledge privy to an expert can be obtained by first finding the most relevant sale, followed by corresponding participant information, and finally by communicating with the participant expert. Business activity as a critical context enhances the search quality while bringing in the social network and business process. This approach suggests a new methodology for search and information management that delivers better business results for the services business.

Large enterprises operate two or more data centers for scaling and fault tolerance. Each data center maintains a certain degree of autonomy while sharing resources with other data centers. In this paper we describe how policy-based resource management can be used in existing commercial products such that each data center can control when, what, and how to share resources with other data centers.

Administrators of the modem IT infrastructure are faced with the problem of escalating complexities and increasing costs associated with distributed resource management. This position paper presents an architectural framework for policy-based management to help mitigate this cost and complexity for the storage environment. In this framework the storage environment is managed by policy at application, data, and storage levels. We use collections of logical attributes to describe classes at each level and use connectors to provide mapping between the collections at different levels.

The goal of autonomic storage allocation is to achieve management of storage resources, including allocation, performance monitoring, and hotspot elimination, by specifying comparatively high-level goals, rather than by means of low-level manual steps. The process of automation should allow specification of policies as administrator specified constraints under which the resources are managed. This paper describes the system design and implementation experiences from a prototype autonomic storage manager being developed in IBM Research. The prototype is being developed for a storage network that includes a SAN switch, an IBM Enterprise Storage Subsystem, and AIX servers. Our early experience from this prototype implementation is that there are a large number of mundane manual steps in storage management and it is feasible to automate them such that the automation is driven by higher-level goals under policy control. However, to manage heterogeneous storage a standard ontology is needed for specification of goals and how to achieve them.

This paper describes design and preliminary measurements of a file server recovery scheme that uses naturally replicated state among clients. This scheme, implemented in the Calypso file system, is truly transparent to the user and avoids the overhead of explicit replication. A three-phase protocol reconstructs the server state either on a backup node (if disks are multi-ported) or on the rebooted server node. Measurements show that the recovery time is about 21 seconds for a busy 10-node cluster. However, the time to rebuild the distributed state is only about 1.5 seconds, and most of the recovery time is spent in replaying the write-ahead log of the underlying file system. Fortunately, the log redo time is bounded by the log size.

The World-Wide Web is the 'killer application' of the mid-1990's, and as a result it serves as the market place for various services. This position document discusses the problem of creating complex Web services from simpler ones, under client control. For example, one might want an annotated forecast of a company's stock price from three simple services, viz., stock price data, forecasting and annotation. We propose meta-scripts as a solution. A client accesses a meta-script, which contains procedures for accessing and composing simpler services. Meta-identifiers are used in the meta-script instead of static links. Developments in type-consistency mechanisms and various enabling services of the Web help in realization of the meta-script based approach. This model of service composition separates control from data manipulation, and gives this control to a client. As clients are empowered with the selection and coordination of service providers, this can lead to a free market for services.