1. Functional evaluation of out-of-the-box text-mining tools for data-mining tasks
- Author
-
Jung, Kenneth, LePendu, Paea, Iyer, Srinivasan, Bauer-Mehren, Anna, Percha, Bethany, and Shah, Nigam H
- Subjects
Networking and Information Technology R&D (NITRD) ,Patient Safety ,Artificial Intelligence ,Data Mining ,Databases as Topic ,Drug Interactions ,Drug-Related Side Effects and Adverse Reactions ,Electronic Health Records ,Humans ,Natural Language Processing ,Obesity ,electronic health records ,natural language processing ,text mining ,Information and Computing Sciences ,Engineering ,Medical and Health Sciences ,Medical Informatics - Abstract
ObjectiveThe trade-off between the speed and simplicity of dictionary-based term recognition and the richer linguistic information provided by more advanced natural language processing (NLP) is an area of active discussion in clinical informatics. In this paper, we quantify this trade-off among text processing systems that make different trade-offs between speed and linguistic understanding. We tested both types of systems in three clinical research tasks: phase IV safety profiling of a drug, learning adverse drug-drug interactions, and learning used-to-treat relationships between drugs and indications.MaterialsWe first benchmarked the accuracy of the NCBO Annotator and REVEAL in a manually annotated, publically available dataset from the 2008 i2b2 Obesity Challenge. We then applied the NCBO Annotator and REVEAL to 9 million clinical notes from the Stanford Translational Research Integrated Database Environment (STRIDE) and used the resulting data for three research tasks.ResultsThere is no significant difference between using the NCBO Annotator and REVEAL in the results of the three research tasks when using large datasets. In one subtask, REVEAL achieved higher sensitivity with smaller datasets.ConclusionsFor a variety of tasks, employing simple term recognition methods instead of advanced NLP methods results in little or no impact on accuracy when using large datasets. Simpler dictionary-based methods have the advantage of scaling well to very large datasets. Promoting the use of simple, dictionary-based methods for population level analyses can advance adoption of NLP in practice.
- Published
- 2015