Search

Health and life sciences’ research fields like personalized medicine, drug discovery, pharmacovigilance and systems biology make an intensive use of graphical information to represent knowledge in the form of domain-specific diagrams, such as molecular pathway‘s. The aim is to provide added value to written text in scientific literature and related documents. Enabling access to all the existing literature for further research requires enabling access to the information contained in these diagrams. Molecular pathways are very different from more conventional diagrams (e.g. flowcharts), and therefore interpretation of molecular pathway diagrams requires domain-specific knowledge to remove ambiguity. In this paper, we propose a method that automatically extracts information from molecular pathways using computer vision techniques. To the best of our knowledge this is the first attempt to retrieve information from images depicting molecular pathway diagrams. The lack of a significant, publicly available dataset with annotated ground truth has led to experimental evaluation on synthetic data. Results show high precision and recall values for the detection of entities and relations. We compare and describe the substantial differences between the proposed method and prior art on the closest diagram type using CLEF-IP flowchart summarization task.

Densely-packed but structured scientific data are typically presented in the form of tables, which often appear in raster image form. To interpret data from scanned tables, understanding their hierarchical structure is vital. To further address the vast variability of table representations, we propose a fully automatic methodology that uses a bottom-up reasoning that is independent on the existence of representation features, such as lines. We evaluate our approach on the ICDAR 2013 dataset and demonstrate its effectiveness on detecting tables cells and their content and classifying header and data cells. For detecting the cell hierarchy, we demonstrate results on synthetic data due to lack of ground truth.

Content-based medical image retrieval has been proposed as a technique that allows not only for easy access to images from the relevant literature and electronic health records but also for training physicians, for research and clinical decision support. The bag-of-visual-words approach is a widely used technique that tries to shorten the semantic gap by learning meaningful features from the dataset and describing documents and images in terms of the histogram of these features. Visual vocabularies are often redundant, over-complete and noisy. Larger than required vocabularies lead to high-dimensional feature spaces, which present important disadvantages with the curse of dimensionality and computational cost being the most obvious ones. In this article a visual vocabulary pruning and descriptor transformation technique is presented. It enormously reduces the amount of required words to describe a medical image dataset with no significant effect on the accuracy. Results show that a reduction of up to 90 % can be achieved without impact on the system performance. Obtaining a more compact representation of a document enables multimodal description as well as using classifiers requiring low-dimensional representations.

Information retrieval algorithms have changed the way we manage and use various data sources, such as images, music or multimedia collections. First, free text information of documents from varying sources became accessible in addition to structured data in databases, initially for exact search and then for more probabilistic models. Novel approaches enable content-based visual search of images using computerized image analysis making visual image content searchable without requiring high quality manual annotations. Other multimedia data followed such as video and music retrieval, sometimes based on techniques such as extracting objects and classifying genre. 3D (surface) objects and solid textures have also been produced in quickly increasing quantities, for example in medical tomographic imaging. For these two types of 3D information sources, systems have become available to characterize the objects or textures and search for similar visual content in large databases. With 3D moving sequences (i.e., 4D), in particular medical imaging, even higher-dimensional data have become available for analysis and retrieval and currently present many multimedia retrieval challenges. This article systematically reviews current techniques in various fields of 3D and 4D visual information retrieval and analyses the currently dominating application areas. The employed techniques are analysed and regrouped to highlight similarities and complementarities among them in order to guide the choice of optimal approaches for new 3D and 4D retrieval problems. Opportunities for future applications conclude the article. 3D or higher-dimensional visual information retrieval is expected to grow quickly in the coming years and in this respect this article can serve as a basis for designing new applications.

Volumetric medical images contain an enormous amount of visual information that can discourage the exhaustive use of local descriptors for image analysis, comparison and retrieval. Distinctive features and patterns that need to be analyzed for finding diseases are most often local or regional, often in only very small parts of the image. Separating the large amount of image data that might contain little important information is an important task as it could reduce the current information overload of physicians and make clinical work more efficient. In this paper a novel method for detecting key-regions is introduced as a way of extending the concept of keypoints often used in 2D image analysis. In this way also computation is reduced as important visual features are only extracted from the detected key regions. The region detection method is integrated into a platform-independent, web-based graphical interface for medical image visualization and retrieval in three dimensions. This web-based interface makes it easy to deploy on existing infrastructures in both small and large-scale clinical environments. By including the region detection method into the interface, manual annotation is reduced and time is saved, making it possible to integrate the presented interface and methods into clinical routine and workflows, analyzing image data at a large scale.

When physicians are searching for articles in the medical literature, images of the articles can help determining relevance of the article content for a specific information need. The visual image representation can be an advantage in effectiveness (quality of found articles) and also in efficiency (speed of determining relevance or irrelevance) as many articles can likely be excluded much quicker by looking at a few representative images. In domains such as medical information retrieval, allowing to determine relevance quickly and accurately is an important criterion. This becomes even more important when small interfaces are used as it is frequently the case on mobile phones and tablets to access scientific data whenever information needs arise. In scientific articles many figures are used and particularly in the biomedical literature only a subset may be relevant for determining the relevance of a specific article to an information need. In many cases clinical images can be seen as more important for visual appearance than graphs or histograms that require looking at the context for interpretation. To get a clearer idea of image relevance in articles, a user test with a physician was performed who classified images of biomedical research articles into categories of importance that can subsequently be used to evaluate algorithms that automatically select images as representative examples. The manual sorting of images of 50 journal articles of BioMedCentral with each containing more than 8 figures by importance also allows to derive several rules that determine how to choose images and how to develop algorithms for choosing the most representative images of specific texts. This article describes the user tests and can be a first important step to evaluate automatic tools to select representative images for representing articles and potentially also images in other contexts, for example when representing patient records or other medical concepts when selecting images to represent RadLex terms in tutorials or interactive interfaces for example. This can help to make the image retrieval process more efficient and effective for physicians.

Journal images represent an important part of the knowledge stored in the medical literature. Figure classification has received much attention as the information of the image types can be used in a variety of contexts to focus image search and filter out unwanted information or ”noise”, for example non–clinical images. A major problem in figure classification is the fact that many figures in the biomedical literature are compound figures and do often contain more than a single figure type. Some journals do separate compound figures into several parts but many do not, thus requiring currently manual separation. In this work, a technique of compound figure separation is proposed and implemented based on systematic detection and analysis of uniform space gaps. The method discussed in this article is evaluated on a dataset of journal figures of the open access literature that was created for the ImageCLEF 2012 benchmark and contains about 3000 compound figures. Automatic tools can easily reach a relatively high accuracy in separating compound figures. To further increase accuracy efforts are needed to improve the detection process as well as to avoid over–separation with powerful analysis strategies. The tools of this article have also been tested on a database of approximately 150’000 compound figures from the biomedical literature, making these images available as separate figures for further image analysis and allowing to filter important information from them.

We propose rotation covariant texture analysis of 4D dual energy computed tomography (DECT) as a diagnosis aid tool for acute pulmonary embolism in emergency radiology. The cornerstone of the proposed approach is to align 3D Riesz directional filters along bronchovascular structures to enable rotation covariant comparisons of the parenchymal texture. The latter is enabled using the steerability property of Riesz filterbanks. Blockwise classification of the parenchyma from all pulmonary lobes allowed a mean area under the receiver operating characteristic (ROC) curve of 0.85 which suggests that the proposed approach can be successfully used to assist radiologists in DECT data interpretation. © 2013 IEEE.

Interstitial lung diseases (ILDs) are regrouping over 150 heterogeneous disorders of the lung parenchyma. High---Resolution Computed Tomography (HRCT) plays an important role in diagnosis, as standard chest x---rays are often non---specific for ILDs. Assessment of ILDs is considerd hard for clinicians because the diseases are rare, patterns often look visually similar and various clinical data need to be integrated. An image retrieval system to support interpretation of HRCT images by retrieving similar images is presented in this paper. The system uses a wavelet transform based on Difference of Gaussians (DoG) in order to extract texture descriptors from a set of 90 image series containing 1679 manually annotated regions corresponding to various ILDs. Visual words are used for feature aggregation and to describe tissue patterns. The optimal scale---progression scheme, number of visual words, as well as distance measure for clustering to generate visual words are investigated. A sufficiently high number of visual words is required to accurately describe patterns with high intra---class variations such as healthy tissue. Scale progression has less influence and the Euclidean distance performs better than other distances. The results show that the system is able to learn the wide intra---class variations of healthy tissue and the characteristics of abnormal lung tissue to provide reliable assistance to clinicians.

Texture-based computerized analysis of high-resolution computed tomography images from patients with interstitial lung diseases is introduced to assist radiologists in image interpretation. The cornerstone of our approach is to learn lung texture signatures using a linear combination of N-th order Riesz templates at multiple scales. The weights of the linear combination are derived from one-versus-all support vector machines. Steerability and multiscale properties of Riesz wavelets allow for scale and rotation covariance of the texture descriptors with infinitesimal precision. Orientations are normalized among texture instances by locally aligning the Riesz templates, which is carried out analytically. The proposed approach is compared with state-of-the-art texture attributes and shows significant improvement in classification performance with an average area under receiver operating characteristic curves of 0.94 for five lung tissue classes. The derived lung texture signatures illustrate optimal class wise discriminative properties.

We develop a texture analysis framework to assist radiologists in interpreting high-resolution computed tomography (HRCT) images of the lungs of patients affected with interstitial lung diseases (ILD). Novel texture descriptors based on the Riesz transform are proposed to analyze lung texture without any assumption on prevailing scales and orientations. A global classification accuracy of 78.3% among five lung tissue types is achieved using locally-oriented Riesz components. Comparative performance analysis with features derived from optimized grey-level co-occurrence matrices showed an absolute gain of 6.1% in classification accuracy. The adaptability of the Riesz features is demonstrated by reconstructing templates according to the first principal components of the lung textures. The balanced performance achieved among the various lung textures suggest that the proposed methods can complement human observers in HRCT interpretation, and opens interesting perspectives for future research.