Search

The human visual system is assumed to transform low level visual features to object and scene representations via features of intermediate complexity. How the brain computationally represents intermediate features is still unclear. To further elucidate this, we compared the biologically plausible HMAX model and Bag of Words (BoW) model from computer vision. Both these computational models use visual dictionaries, candidate features of intermediate complexity, to represent visual scenes, and the models have been proven effective in automatic object and scene recognition. These models however differ in the computation of visual dictionaries and pooling techniques. We investigated where in the brain and to what extent human fMRI responses to short video can be accounted for by multiple hierarchical levels of the HMAX and BoW models. Brain activity of 20 subjects obtained while viewing a short video clip was analyzed voxel-wise using a distance-based variation partitioning method. Results revealed that both HMAX and BoW explain a significant amount of brain activity in early visual regions V1, V2, and V3. However, BoW exhibits more consistency across subjects in accounting for brain activity compared to HMAX. Furthermore, visual dictionary representations by HMAX and BoW explain significantly some brain activity in higher areas which are believed to process intermediate features. Overall our results indicate that, although both HMAX and BoW account for activity in the human visual system, the BoW seems to more faithfully represent neural responses in low and intermediate level visual areas of the brain.

There is a large variety of trackers, which have been proposed in the literature during the last two decades with some mixed success. Object tracking in realistic scenarios is a difficult problem, therefore, it remains a most active area of research in computer vision. A good tracker should perform well in a large number of videos involving illumination changes, occlusion, clutter, camera motion, low contrast, specularities, and at least six more aspects. However, the performance of proposed trackers have been evaluated typically on less than ten videos, or on the special purpose datasets. In this paper, we aim to evaluate trackers systematically and experimentally on 315 video fragments covering above aspects. We selected a set of nineteen trackers to include a wide variety of algorithms often cited in literature, supplemented with trackers appearing in 2010 and 2011 for which the code was publicly available. We demonstrate that trackers can be evaluated objectively by survival curves, Kaplan Meier statistics, and Grubs testing. We find that in the evaluation practice the F-score is as effective as the object tracking accuracy (OTA) score. The analysis under a large variety of circumstances provides objective insight into the strengths and weaknesses of trackers.

Reconstruction of 3D scene geometry is an important element for scene understanding, autonomous vehicle and robot navigation, image retrieval, and 3D television. We propose accounting for the inherent structure of the visual world when trying to solve the scene reconstruction problem. Consequently, we identify geometric scene categorization as the first step toward robust and efficient depth estimation from single images. We introduce 15 typical 3D scene geometries called stages, each with a unique depth profile, which roughly correspond to a large majority of broadcast video frames. Stage information serves as a first approximation of global depth, narrowing down the search space in depth estimation and object localization. We propose different sets of low-level features for depth estimation, and perform stage classification on two diverse data sets of television broadcasts. Classification results demonstrate that stages can often be efficiently learned from low-dimensional image representations.

This paper studies automatic image classification by modeling soft assignment in the popular codebook model. The codebook model describes an image as a bag of discrete visual words selected from a vocabulary, where the frequency distributions of visual words in an image allow classification. One inherent component of the codebook model is the assignment of discrete visual words to continuous image features. Despite the clear mismatch of this hard assignment with the nature of continuous features, the approach has been successfully applied for some years. In this paper, we investigate four types of soft assignment of visual words to image features. We demonstrate that explicitly modeling visual word assignment ambiguity improves classification performance compared to the hard assignment of the traditional codebook model. The traditional codebook model is compared against our method for five well-known data sets: 15 natural scenes, Caltech-101, Caltech-256, and Pascal VOC 2007/2008. We demonstrate that large codebook vocabulary sizes completely deteriorate the performance of the traditional model, whereas the proposed model performs consistently. Moreover, we show that our method profits in high-dimensional feature spaces and reaps higher benefits when increasing the number of image categories.

The visual appearance of natural scenes is governed by a surprisingly simple hidden structure. The distributions of contrast values in natural images generally follow a Weibull distribution, with beta and gamma as free parameters. Beta and gamma seem to structure the space of natural images in an ecologically meaningful way, in particular with respect to the fragmentation and texture similarity within an image. Since it is often assumed that the brain exploits structural regularities in natural image statistics to efficiently encode and analyze visual input, we here ask ourselves whether the brain approximates the beta and gamma values underlying the contrast distributions of natural images. We present a model that shows that beta and gamma can be easily estimated from the outputs of X-cells and Y-cells. In addition, we covaried the EEG responses of subjects viewing natural images with the beta and gamma values of those images. We show that beta and gamma explain up to 71% of the variance of the early ERP signal, substantially outperforming other tested contrast measurements. This suggests that the brain is strongly tuned to the image's beta and gamma values, potentially providing the visual system with an efficient way to rapidly classify incoming images on the basis of omnipresent low-level natural image statistics.

In multitarget tracking, the main challenge is to maintain the correct identity of targets even under occlusions or when differences between the targets are small. The paper proposes a new approach to this problem by incorporating the context information. The context of a target in an image sequence has two components: the spatial context including the local background and nearby targets and the temporal context including all appearances of the targets that have been seen previously. The paper considers both aspects. We propose a new model for multitarget tracking based on the classification of each target against its spatial context. The tracker searches a region similar to the target while avoiding nearby targets. The temporal context is included by integrating the entire history of target appearance based on probabilistic principal component analysis (PPCA). We have developed a new incremental scheme that can learn the full set of PPCA parameters accurately online. The experiments show robust tracking performance under the condition of severe clutter, occlusions, and pose changes.

This paper presents the semantic pathfinder architecture for generic indexing of multimedia archives. The semantic pathfinder extracts semantic concepts from video by exploring different paths through three consecutive analysis steps, which we derive from the observation that produced video is the result of an authoring-driven process. We exploit this authoring metaphor for machine-driven understanding. The pathfinder starts with the content analysis step. In this analysis step, we follow a data-driven approach of indexing semantics. The style analysis step is the second analysis step. Here, we tackle the indexing problem by viewing a video from the perspective of production. Finally, in the context analysis step, we view semantics in context. The virtue of the semantic pathfinder is its ability to learn the best path of analysis steps on a per-concept basis. To show the generality of this novel indexing approach, we develop detectors for a lexicon of 32 concepts and we evaluate the semantic pathfinder against the 2004 NIST TRECVID video retrieval benchmark, using a news archive of 64 hours. Top ranking performance in the semantic concept detection task indicates the merit of the semantic pathfinder for generic indexing of multimedia archives.

This paper offers a sparse, multiscale representation of objects. It captures the object appearance by selection from a very large dictionary of Gaussian differential basis functions. The learning procedure results from the matching pursuit algorithm, while the recognition is based on polynomial approximation to the bases, turning image matching into a problem of polynomial evaluation. The method is suited for coarse recognition between objects and, by adding more bases, also for fine recognition of the object pose. The advantages over the common representation using PCA include storing sampled points for recognition is not required, adding new objects to an existing data set is trivial because retraining other object models is not needed, and significantly in the important case where one has to scan an image over multiple locations in search for an object, the new representation is readily available as opposed to PCA projection at each location. The experimental result on the COIL-100 data set demonstrates high recognition accuracy with real-time performance.

Luminance-based features are widely used as low-level input for computer vision applications, even when color data is available. The extension of feature detection to the color domain prevents information loss due to isoluminance and allows us to exploit the photometric information. To fully exploit the extra information in the color data, the vector nature of color data has to be taken into account and a sound framework is needed to combine feature and photometric invariance theory. In this paper, we focus on the structure tensor, or color tensor, which adequately handles the vector nature of color images. Further, we combine the features based on the color tensor with photometric invariant derivatives to arrive at photometric invariant features. We circumvent the drawback of unstable photometric invariants by deriving an uncertainty measure to accompany the photometric invariant derivatives. The uncertainty is incorporated in the color tensor, hereby allowing the computation of robust photometric invariant features. The combination of the photometric invariance theory and tensor-based features allows for detection of a variety of features such as photometric invariant edges, corners, optical flow, and curvature. The proposed features are tested for noise characteristics and robustness to photometric changes. Experiments show that the proposed features are robust to scene incidental events and that the proposed uncertainty measure improves the applicability of full invariants.

Segmentation of the spine directly from three-dimensional (3-D) image data is desirable to accurately capture its morphological properties. We describe a method that allows true 3-D spinal image segmentation using a deformable integral spine model. The method learns the appearance of vertebrae from multiple continuous features recorded along vertebra boundaries in a given training set of images. Important summarizing statistics are encoded into a necklace model on which landmarks are differentiated on their free dimensions. The landmarks are used within a priority segmentation scheme to reduce the complexity of the segmentation problem. Necklace models are coupled by string models. The string models describe in detail the biological variability in the appearance of spinal curvatures from multiple continuous features recorded in the training set. In the segmentation phase, the necklace and string models are used to interactively detect vertebral structures in new image data via elastic deformation reminiscent of a marionette with strings allowing for movement between interrelated structures. Strings constrain the deformation of the spine model within feasible solutions. The driving application in this work is analysis of computed tomography scans of the human lumbar spine. An illustration of the segmentation process shows that the method is promising for segmentation of the spine and for assessment of its morphological properties.

We propose a new method for object tracking in image sequences using template matching. To update the template, appearance features are smoothed temporally by robust Kalman filters, one to each pixel. The resistance of the resulting template to partial occlusions enables the accurate detection and handling of more severe occlusions. Abrupt changes of lighting conditions can also be handled, especially when photometric invariant color features are used. The method has only a few parameters and is computationally fast enough to track objects in real time.

We propose a method for concept-based medical image retrieval that is a superset of existing semantic-based image retrieval methods. We conceive of a concept as an incremental and interactive formalization of the user's conception of an object in an image. The premise is that such a concept is closely related to a user's specific preferences and subjectivity and, thus, allows to deal with the complexity and content-dependency of medical image content. We describe an object in terms of multiple continuous boundary features and represent an object concept by the stochastic characteristics of an object population. A population-based incrementally learning technique, in combination with relevance feedback, is then used for concept customization. The user determines the speed and direction of concept customization using a single parameter that defines the degree of exploration and exploitation of the search space. Images are retrieved from a database in a limited number of steps based upon the customized concept. To demonstrate our method we have performed concept-based image retrieval on a database of 292 digitized X-ray images of cervical vertebrae with a variety of abnormalities. The results show that our method produces precise and accurate results when doing a direct search. In an open-ended search our method efficiently and effectively explores the search space.

We derive the decomposition of the anisotropic Gaussian in a one-dimensional (1-D) Gauss filter in the x-direction followed by a 1-D filter in a nonorthogonal direction phi. So also the anisotropic Gaussian can be decomposed by dimension. This appears to be extremely efficient from a computing perspective. An implementation scheme for normal convolution and for recursive filtering is proposed. Also directed derivative filters are demonstrated. For the recursive implementation, filtering an 512 x 512 image is performed within 40 msec on a current state of the art PC, gaining over 3 times in performance for a typical filter, independent of the standard deviations and orientation of the filter. Accuracy of the filters is still reasonable when compared to truncation error or recursive approximation error. The anisotropic Gaussian filtering method allows fast calculation of edge and ridge maps, with high spatial and angular accuracy. For tracking applications, the normal anisotropic convolution scheme is more advantageous, with applications in the detection of dashed lines in engineering drawings. The recursive implementation is more attractive in feature detection applications, for instance in affine invariant edge and ridge detection in computer vision. The proposed computational filtering method enables the practical applicability of orientation scale-space analysis.

We propose a new method for contour tracking in video. The inverted distance transform of the edge map is used as an edge indicator function for contour detection. Using the concept of topographical distance, the watershed segmentation can be formulated as a minimization. This new viewpoint gives a way to combine the results of the watershed algorithm on different surfaces. In particular, our algorithm determines the contour as a combination of the current edge map and the contour, predicted from the tracking result in the previous frame. We also show that the problem of background clutter can be relaxed by taking the object motion into account. The compensation with object motion allows to detect and remove spurious edges in background. The experimental results confirm the expected advantages of the proposed method over the existing approaches.

Segmentation of the object of interest is a difficult step in the analysis of digital images. Fully automatic methods sometimes fail, producing incorrect results and requiring the intervention of a human operator. This is often true in medical applications, where image segmentation is particularly difficult due to restrictions imposed by image acquisition, pathology and biological variation. In this paper we present an early review of the largely unknown territory of human-computer interaction in image segmentation. The purpose is to identify patterns in the use of interaction and to develop qualitative criteria to evaluate interactive segmentation methods. We discuss existing interactive methods with respect to the following aspects: the type of information provided by the user, how this information affects the computational part, and the purpose of interaction in the segmentation process. The discussion is based on the potential impact of each strategy on the accuracy, repeatability and interaction efficiency. Among others, these are important aspects to characterise and understand the implications of interaction to the results generated by an interactive segmentation method. This survey is focused on medical imaging, however similar patterns are expected to hold for other applications as well.

Background: The analysis of three-dimensional (3D) motion is becoming increasingly important in life cell imaging. A simple description of sometimes complex patterns of movement in living cells gives insight in the underlying mechanisms governing these movements., Methods: We evaluate a velocity estimation method based on intensity derivatives in spatial and temporal domain from 3D confocal images of living cells. Cells of the sample contain intense spots throughout the cell nucleus. In simulations, we model these spots as Gaussian intensity profiles which are constant in intensity and shape. To quantify the quality of the estimated velocity, we introduce a reliability measure., Results: For constant linear velocity, the velocity estimation is unbiased. For accelerated motion paths or when a neighboring spot disturbs the intensity profile, the method results are biased. The influence of the point-spread function on the velocity estimation can be compensated for by introducing anisotropic derivative kernels. The insight gained in the simulations is confirmed by the results of the method applied on an image sequence of a living cell with fluorescently labeled chromatin., Conclusions: With the velocity estimation method, a tool for estimating 3D velocity fields is described which is successfully applied to a living cell sequence. With the estimated velocity fields, motion patterns can be observed, which are a useful starting point for the analysis of dynamic processes in living cells., (Copyright 2001 Wiley-Liss, Inc.)

The shape of mitotic prophase chromosomes has been studied in root tip nuclei by confocal microscopy and 3D-image analysis. Crepis capillaris chromosome no. 1 was used as a test object. Chromosome conformation was studied in early, mid- and in late prophase. In mid- and late prophase, individual chromosomes could be distinguished on the basis of their length. Early prophase chromosomes could not be distinguished as individuals. The central axes of prophase chromosomes were traced with an automated computer procedure and then represented as a string of 3D coordinates. This representation facilitated measurement along the chromosome axis of shape parameters such as curvature (amount of bending), torsion (helical winding) and torsion sign (helical handedness). Stretches of early prophase chromosomes showed full helical turns, which could be left- or right-handed. In the later prophase stages curvature and torsion were statistically analysed. Our data on 40 midprophase chromosomes no. 1 show that they are still highly curved, but full helical turns were no longer found. Instead, an overall meandering pattern was observed. In late prophase, one central loop persisted, flanked by two preferential regions of high curvature.

We aim at combining color and shape invariants for indexing and retrieving images. To this end, color models are proposed independent of the object geometry, object pose, and illumination. From these color models, color invariant edges are derived from which shape invariant features are computed. Computational methods are described to combine the color and shape invariants into a unified high-dimensional invariant feature set for discriminatory object retrieval. Experiments have been conducted on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. From the theoretical and experimental results it is concluded that object retrieval based on composite color and shape invariant features provides excellent retrieval accuracy. Object retrieval based on color invariants provides very high retrieval accuracy whereas object retrieval based entirely on shape invariants yields poor discriminative power. Furthermore, the image retrieval scheme is highly robust to partial occlusion, object clutter and a change in the object's pose. Finally, the image retrieval scheme is integrated into the PicToSeek system on-line at http://www.wins.uva.nl/research/isis/PicToSeek/ for searching images on the World Wide Web.

Background: A critical step in automatic microscopy is focusing. This report describes a robust and fast autofocus approach useful for a wide range of microscopic modalities and preparations., Methods: The focus curve is measured over the complete focal range, reducing the chance that the best focus position is determined by dust or optical artifacts. Convolution with the derivative of a Gaussian smoothing function reduces the effect of noise on the focus curve. The influence of mechanical tolerance is accounted for., Results: The method is shown to be robust in fluorescence, bright-field and phase contrast microscopy, in fixed and living cells, as well as in fixed tissue. The algorithm was able to focus accurately within 2 or 3 s, even under extremely noisy and low contrast imaging conditions., Conclusions: The proposed method is generally applicable in light microscopy, whenever the image information content is sufficient. The reliability of the autofocus method allows for unattended operation on a large scale., (Copyright 2000 Wiley-Liss, Inc.)

Background: Characterization of tissues can be based on the topographical relationship between the cells. Such characterization should be insensitive to distortions intrinsic to the acquisition of biological preparation. In this paper, a method for the robust segmentation of tissues based on the spatial distribution of cells is proposed., Materials and Methods: The neighborhood of each cell in the tissue is modeled by the distances to the surrounding cells. Comparison with an example or prototype neighborhood reveals topographical similarity between tissue and prototype. Processing of all cells in the tissue extracts the regions with tissue architecture similar to the given example., Results: Comparison with other topographical-segmentation methods shows that the proposed method is better suited for partitioning tissue architecture. As an example, the quantification of the structural integrity in rat hippocampi after ischemia is demonstrated. In contrast to other methods, the algorithm correlates well with expert evaluation., Conclusions: The present method reduces the nonbiological variation in the analysis of tissue sections and thus improves confidence in the result. The method can be applied to any field where regular patterns have to be detected, as long as the directional distribution of neighbors may be neglected.

Randomness and relation between point positions play an important role in archeology, cosmology, geography, and biology. An often-discarded effect is the edge effect, the effect that points are bound to a certain region. Without an appropriate correction, the outcome will be wrong. We studied the problem of randomness by comparing the distribution of interpoint distances with what can be expected for randomly distributed points (pair correlation function in statistics), and applied this to two sets of nuclear proteins inside the cell nucleus. The technique comprised labelling the proteins with a fluorescent dye, recording the fluorescent distribution with a 3D confocal microscope, and detecting the positions of the individual fluorescent spots. Results showed that, apart from studying randomness, the method is well equipped to quantitatively analyze a spot detection procedure, as the resolving power and the subvoxel accuracy were clearly visible. Given the results of assessing the randomness of the general transcription factor BRG1 and the RNA synthesizing protein RNA polymerase II (polII) in the cell nucleus, we concluded that the high intensity spots of the BRG1 protein are regularly spaced. The low intensity spots of the BRG1 protein and the low- and high-intensity spots of the polII protein showed more random behavior. The BRG1 and polII proteins showed correlation; unexpectedly, the relation was also found for the low-intensity spots, which were expected to have a more random behavior.

A study on the chromatin organisation of synchronised G1 and G2 populations of maize root cell nuclei is reported using 3-D images acquired with a confocal fluorescence microscope. The analysis is based on the concept of accessibility. Accessibility of a position x is the effort to arrive at x, when choosing the minimum effort path to arrive at x from the nuclear border. The effort is then taken to be proportional to the amount of all mass encountered on the path, and computed by a technique called the grey valued distance transform. The approach relies heavily on quantitative analysis of the intensity information. Hence, considerable attention was paid to the quantitative modification of the confocal intensity values by diffraction, absorption and scatter corrections. Three texture features are extracted from the accessibility maps: the global object inaccessibility, the relative object accessibility, and the object homogeneity. On the basis of individual texture features, no distinction between the G1 and G2 populations could be established. However, the three features combined did show a clear difference with a high significance.

When studying the three-dimensional shape of prophase chromosomes (or any other tubular structure), it is useful to represent these structures as a string of three-dimensional Cartesian coordinates along the medial axis. This procedure was automated in order to limit the number of human interactions and to improve reproducibility. In this paper the design, implementation, and validation of the automated method is presented. From the data presented it can be concluded that the cursor algorithm provides an objective and therefore reproducible method to trace the medial axes of prophase chromosomes automatically. This method could allow a more extensive understanding of the (changes in) chromosome organisation throughout the cell cycle, its relation to cell function, and the complex process of chromosome condensation.

This study describes an image processing method for the assessment of the mitotic count in Feulgen-stained breast cancer sections. The segmentation procedure was optimized to eliminate 95-98% of the nonmitoses, whereas 11% of the mitoses did not survive the segmentation procedure. Contour features and optical density measurements of the remaining objects were computed to allow for classification. Twelve specimens were analyzed, nine used to serve as a training set, and three put aside for later use as independent test set. The fully automatic image processing method correctly classified 81% of the mitoses at the specimen level while inserting 30% false positives. The automatic procedure strongly correlated with the interactive counting procedure (r = 0.98). Although the fully automatic method provided satisfactory results, it is not yet suited for clinical practice. The automated method with an interactive evaluation step gave an accurate reflection of the mitotic count showing an almost perfect correlation with the results of the interactive morphometry (r = 0.998). Therefore this semiautomated method may be useful as prescreening device.

The paper describes an image analysis technique for automated selection of the epithelium-rich areas in standard paraffin tissue sections of ovarian and endometrial premalignancies and malignancies. Two staining procedures were evaluated, Feulgen (pararosanilin) and CAM 5.2, demonstrating the presence of cytokeratin 8 and 18; both were counterstained with naphthol yellow. The technique is based on the corresponding image processing method of automated estimation of the percentage of epithelium in interactively selected microscope fields. With the technique, one image is recorded with a filter to demonstrate where epithelium and stroma lie. This filter is chosen according to the type of staining: it is yellow (lambda = 552 nm) for Feulgen and blue (lambda = 470 nm) for anticytokeratin CAM 5.2. When stroma cannot be distinguished from lumina with the green filter or from epithelium with the blue filter, a second image is recorded from the same microscope field, with a blue filter (lambda = 420 nm) for Feulgen and a yellow filter (lambda = 576 nm) for anticytokeratin CAM 5.2. Discrimination between epithelium and stroma is based on the image contrast range and the packing of nuclei in the yellow image and on the automated classification of the gray value histogram peaks in the blue image. For Feulgen stain the method was evaluated on 30 ovarian tumors of the common epithelial types (8 borderline tumors and 22 carcinomas with various degrees of differentiation) and 30 endometrial carcinomas of different grades.(ABSTRACT TRUNCATED AT 250 WORDS)

In reasoning systems, uncertainty plays a crucial part, especially for those fields in which judgements are essential, as in pathology. Uncertainty has several aspects, such as prevalence of diseases, occurrence of findings and the sensitivity and predictive value of findings. For the functioning of a reasoning system, two aspects are crucial: (1) the internal representation of the uncertainty and (2) the way in which the uncertainty is propagated in the reasoning process when combining formal statements. Five well-known reasoning strategies (Bayes' probability theory, MYCIN's certainty factor model, fuzzy set theory, the theory of Dempster-Shafer and Pathfinder's scoring mechanism) are compared, with particular attention to: (1) Under what conditions will the model function? In particular, what information is to be specified a priori to the system? (2) Can the different aspects of uncertainty be dealt with as separate entities? (3) How are unknown uncertainties dealt with? (4) How is evidence in favor of a hypothesis combined with evidence against it? (5) How does the model treat the simultaneous occurrence of more than one disorder, that is, how does the model support reasoning with compound hypotheses? It is preliminarily concluded that the different aspects of uncertainty are expressed as separate entities only in Pathfinder and probability theory. Hence, the other models do not accurately represent uncertain knowledge. Also, such theoretically attractive models as the Bayes, MYCIN and Dempster-Shafer theory can only function properly under the tight condition of mutual exclusiveness of hypotheses, which is not always suited for broader areas of pathology. They may, however, be suited for smaller areas, with a limited number of defined diseases and a limited number of features. All models but the Bayes model lack a predictable performance since there is no (or only a partial) underlying theory to guarantee minimization of the overall error.

The influence of various DNA measurement errors using a commercially available scanning microdensitometer was evaluated on Feulgen-stained cytologic and histologic samples prepared from paraffin blocks containing invasive ductal breast cancers. The overall average total measurement error was 5.5% for the cytologic specimens and 10.9% for the 4 micron histologic sections. Components of the error included microscopic adjustment variation and focussing errors (3.5% and 1.1%, respectively, for both cytologic and histologic samples) and background intensity estimation errors (3.0% for the cytologic samples and 10.0% for the histologic samples). Measurements of the integrated optical density had a minimal error of 0.5% and an average error of 1.0%. Limitations due to the histologic architecture and/or heterogeneous cell population gave rise to large differences in the selection of nuclei when differently sized scanning masks were used. To improve the reproducibility, masks used should be based on the individual cell size, and background intensity values should be carefully estimated in the vicinity of the selected cells. Overall, the cytologic tumor samples were preferable to the histologic samples for static DNA measurements. It was easier to select cells suitable for measurement in the cytologic samples, and the cytologic measurements were less time consuming and produced a smaller measurement error.

The paper describes an image analysis technique for automated estimation of the epithelial percentage in standard paraffin tissue sections of invasive ductal breast cancers. Two staining procedures are evaluated: Feulgen (pararosanilin) and CAM 5.2-demonstrating the presence of cytokeratin 8 and 18-, both counterstained with naphthol yellow. In the technique, one image is recorded with a filter to visualize where the epithelium lies. This filter is chosen corresponding to the type of staining: it is yellow for Feulgen and blue for anti-cytokeratin CAM 5.2. To visualize where the stroma lies, the same image can be used for anti-cytokeratin CAM 5.2, whereas for Feulgen, a second image has to be recorded from the same microscope field with a blue filter. The image processing steps to determine the total tissue area comprise correction for shading, segmentation of the tissue area, and restoration of the segmented image by removal of small artefacts and closure of small tears in the tissue. The method for determination of the epithelial area consists of the following steps: correction for shading, gaussian blurring, segmentation of nuclei or epithelial cells, and editing of the segmented image by removal of small objects and closure of small spaces between the epithelial nuclei or cells. These image processing steps are compared to those for quantification of the epithelial percentage in gynecologic tumors of epithelial origin. For the Feulgen stain, the method is evaluated on 30 breast cancers of the ductal type (4 grade I, 12 grade II, and 14 grade III).(ABSTRACT TRUNCATED AT 250 WORDS)

The Diagnostic Encyclopedia Workstation (DEW) is a computer system that provides completely integrated pictorial and textual information as reference knowledge in the field of ovarian pathology. The textual component comprises information per diagnosis such as descriptions of macroscopic and microscopic images, clinical signs, and prognosis. In addition, the system offers lists of differential diagnoses and criteria to differentiate among lists of differential diagnoses and criteria to differentiate among them. The present study evaluates to what extent the system influences the diagnostic process in efficiency and outcome. Therefore, two groups of six pathologists each, covering a wide spectrum of experience in ovarian pathology, participated in the evaluation of the DEW. The quality of the resulting diagnoses was statistically analyzed with the Wilcoxon rank sum test with respect to five different viewpoints: classification, morphology, clinical consequences, duration of diagnostic process, and consensus among the participants. The results are discussed and it is concluded that classification and morphology showed better results when books were used. The evaluation experiment was, however, very rigid and negatively biased with respect to the DEW system. Positive aspects of the encyclopedia are the easy access to diagnostic and differential diagnostic information and the large set of illustrations. Insight is acquired with respect to existing bottlenecks and how they may be overcome.

A tool is introduced for the acquisition of pathology knowledge in a formalized form, directly by the expert. Formalization of the knowledge is intended to make descriptive pathology knowledge more suitable for computerized diagnostic support since a formal representation of knowledge allows more extensive indexing, hence more flexible access. The knowledge acquisition (KA) tool also provides a useful research instrument to investigate to what extent pathology knowledge can be made explicit, to what degree ambiguity is present, in what way experts differ when formalizing knowledge, and whether it is feasible to incrementally acquire decision criteria on the basis of the formalized descriptive knowledge. Crucial in the design of the KA tool is the incorporated meta-knowledge, which is reflected by the knowledge-base structure and is used to elicit knowledge from the expert. Knowledge is acquired from the expert via a menu-driven user interface, which follows the general steps of the pathologist when describing a case. The paper discusses the considerations underlying the design, the implementation of the KA tool, and the research goals.

The Diagnostic Encyclopedia Workstation (DEW) contains reference knowledge for diagnostic support in pathology. Illustrations are accessible via a video disc device. DEW can hold more knowledge, pictures and case histories than books, and its information is accessible via several entries. Software for data entry has been written in MUMPS with use of the relational database toolkit AIDA, which is particularly suited for manipulation of free text. The graphical mouse-driven user interface is written in C using MetaWindows. The DEW contains 85 diagnoses in ovarian pathology, covering all frequent cases and many rarities, illustrated by approximately 3000 pictures, divided among 158 cases.

A description is given of a combination of three programs developed for computer-assisted stage scanning cytophotometry and cytofluorometry of isolated, close-lying, or touching objects in different types of microscopical preparations. The advantages and limitations of the individual programs are discussed, as well as the local specimen conditions determining the optimal application range of each of the programs. The applicability of these programs was investigated by determination of the integrated absorbance values of Feulgen or gallocyanin-chrome alum stained chicken erythrocytes, human leucocytes, and skin biopsy cells in imprint preparations, as well as of guinea pig peritoneal granulocytes which had been submitted to a simultaneous coupling azo dye incubation for alkaline phosphatase activity.

Diagnostic Encyclopaedia Workstation (DEW) is the name of a digital encyclopaedia constructed to contain reference knowledge with respect to the pathology of the ovary. Comparing DEW with the common sources of reference knowledge (i.e. books) leads to the following advantages of DEW: it contains more verbal knowledge, pictures and case histories, and it offers information adjusted to the needs of the user. Based on an analysis of the structure of this reference knowledge we have chosen AIDA to develop a relational database and we use a video-disc player to contain the pictorial part of the database. The system consists of a database input version and a read-only run version. The design of the database input version is discussed. Reference knowledge for ovary pathology requires 1-3 Mbytes of memory. At present 15% of this amount is available. The design of the run version is based on an analysis of which information must necessarily be specified to the system by the user to access a desired item of information. Finally, the use of AIDA in constructing DEW is evaluated.

Because of the visual complexity of medical images and the intensive tutorial experience required to develop image recognition expertise, professional training programs have concentrated on educating limited numbers of experts. Videodisk and CD-ROM (compact disk read only memory) image storage media now make it possible for a microcomputer workstation to provide a learning environment substantially equivalent to that of conventional time-consuming tutorial methods. A demonstration hypermedia program on echocardiography was constructed that provides a user-controlled learning environment with instant access to 54,000 video frames encompassing 1,200 clinical items. The instructional module is controlled by a microcomputer, which provides electronic linkage to relevant graphics, animations, text, categorized data bases, and digitized sound. The system has been successfully used in a residency program as the primary instructional tool for achieving an intermediate level of clinical expertise. Hypermedia offer substantial advantages over conventional books as a clinical reference source.

If a continuous straight line segment is digitized on a regular grid, obviously a loss of information occurs. As a result, the discrete representation obtained (e.g., a chaincode string) can be coded more conveniently than the continuous line segment, but measurements of properties (such as line length) performed on the representation have an intrinsic inaccuracy due to the digitization process. In this paper, two fundamental properties of the quantization of straight line segments are treated. 1) It is proved that every ``straight'' chaincode string can be represented by a set of four unique integer parameters. Definitions of these parameters are given. 2) A mathematical expression is derived for the set of all continuous line segments which could have generated a given chaincode string. The relation with the chord property is briefly discussed.

This paper describes an image analysis technique for automated estimation of the percentages of epithelium and stroma in (tumor) tissue. The program is evaluated on ovarian tumors of the serous, mucinous, and endometrioid type. From standard paraffin sections, stained with pararosanilin Feulgen and naphthol yellow, a blue-yellow image pair was recorded. The blue image was used for the determination of the total tissue area and the yellow image for the epithelial area. For the latter the image processing method is based on the fact that epithelial nuclei are generally more tightly packed than stromal nuclei. A structural approach was applied, in which the segmentation of the nuclei was based on the image contrast range in the density domain. The method has been tested with 78 image pairs from 19 ovarian tumors with varying degrees of malignancy. The area percentages, as assessed with image processing, were strongly correlated to control percentages, established by interactive morphometry (r = 0.98).

This paper partly addresses the question "What artificial intelligence (AI) tools are appropriate for which parts of pathology?" by analyzing the structure and components of knowledge in pathology (e.g., observations plus archival and reference data) and which aspects of that knowledge should be expressible in an AI consulting system. The different aspects of uncertainty (observational, prevalence and validity) play an important role in both human and computer-based decision-making processes, as do relationships between the components of knowledge. The design of an AI consultant system is discussed in terms of the way uncertainty is expressed and in how many parameters, the way uncertainty is propagated (Bayes, certainty factors, Dempster-Schafer, logic or Pathfinder heuristic methods), whether the system reasons from data to a conclusion or vice versa and what the aim of the system is. The suitability of an AI tool is determined by the knowable facts of the pathology subfield, by the match with its knowledge structure and by its requirements. While the success of an AI tool will partly depend on an appropriate definition of its scope, the appropriate combinatoric also depends on the expertise of the user.

The combination of a personal computer and a laser-vision disc player is an adroit tool for storing and retrieving textual information in combination with pictorial information. This paper describes such a system (a "diagnostic encyclopedia workstation"), which provides information to the pathologist engaged in daily diagnostic practice. The system contains a considerable amount of descriptive textual information that might be useful in making diagnoses in histopathology, along with many illustrations; the text is presented in a natural language (English). The descriptive information is divided into 18 categories, including such topics as histology, macroscopy, immunopathology and clinical data; each topic has a separate display in the system. Also present are two types of "decision rules" for making diagnoses (confirmative criteria for a diagnosis under consideration and exclusive criteria for many other diagnoses) and a classifying structure. The present version of this system contains information on about 100 diagnoses in ovarian pathology, illustrated with about 3,000 color slides from about 140 cases. The information and pictures are immediately available. Further characteristics of this system are its flexibility, accessibility and user friendliness; it has been structured so that components of artificial intelligence may be added later.

The paper describes an improved segmentation method to measure the percentages of epithelium and stroma in ovarian (tumor) tissue with automated image analysis and evaluates its prognostic value. In the image processing method, a blue-yellow image pair is recorded from standard paraffin sections and stained with pararosanilin Feulgen and naphthol yellow. The blue image is used for automated determination of the total tissue area and the yellow image for the epithelial area. Results are obtained with 114 ovarian tumors of the common epithelial types (14 borderline tumors and 100 invasive carcinomas with varying degrees of differentiation). The fraction of epithelium in the total tissue shows a strong correlation with the epithelial percentage resulting from interactive morphometry (r = 0.991) for 15 tumors of varying histological grades. The prognostic value is evaluated on the 100 invasive carcinomas. Survival analysis implies that the epithelial percentage is of prognostic importance (Mantel-Cox 7.4, p = 0.0064). Multivariate analysis shows that the estimated fraction of epithelium is the strongest factor and that the FIGO stage has additional prognostic value (Mantel-Cox 12.5, p = 0.0004). It can be concluded that epithelial volume, as automatically estimated by image processing, seems useful in predicting the prognosis of patients with ovarian cancers.

In endometrial carcinomas, the epithelial area measured by interactive morphometry is an important feature in the classification of tumors of varying histologic grades. This report describes an image analysis technique for the fully automated estimation of the area percentages of epithelium and stroma in tissue sections of the endometrium obtained from hysterectomy specimens. The method is evaluated using endometrial carcinomas with varying degrees of malignancy. From standard paraffin sections stained with pararosanilin Feulgen and naphthol yellow, a blue-yellow image pair was recorded. The blue image was used to determine the total tissue area and the yellow image was used to determine the epithelial area. Image processing of the blue image was comprised of correction for shading, segmentation of the tissue area, and restoration of the segmented image by removing small artefacts and closing small tears in the tissue. Image processing of the yellow image was based on the fact that epithelial nuclei are generally more tightly packed than stromal nuclei and consists of the following steps: correction for shading, gaussian blurring, segmentation of nuclei, and editing the segmented image by removing small objects and closing small spaces between the epithelial nuclei. These image processing steps are compared with those used for quantification of the epithelial area in ovarian tumors. The performance of the method was evaluated using 120 image pairs from 30 endometrial carcinomas of varying histologic grades. The epithelial area percentages, as assessed by digital image processing, strongly correlate to control percentages that were established by interactive morphometry (r = .987).

A computer program is described for delineation and measurement of microscopical objects, such as cells and chromosomes, which may have been scanned using absorbance, fluorescence or reflectance microscopy. The quality of the object delineation is optimized through the controur ratio, which is simply computed from the object contour. Geometrical features, like the perimeter and area are computed. A new definition is introduced for the background region, especially suited for the analysis of closely packed objects. This definition is based upon a comparison between total staining material content values resulting from a variety of methods and circumstances. The program is principally intended for use in the on-line real-time environment of a small laboratory computer, but may be used off-line as well. It has facilities for displaying the results and the process by which they are produced. A program module is described for displaying scan data on a bilevel display using an adaptation of the sigma-delta method.

Measurement is the instrument of morphometry. Several recurring issues of measurement are reviewed: types of measurement scales and their statistical consequences; types of measurement errors and their relationship to measurement scales; the differences between image resolution, digitization resolution and measurement resolution; and image measurement methods. The importance of formalized measurement protocols is illustrated with an example from daily morphometric practice.

In a group of 335 patients with primary breast carcinoma the presence of immunoreactive carcinoembryonic antigen (CEA) and the binding of the lectin peanut agglutinin (PNA) in the primary carcinoma and in axillary lymph node metastases were investigated. The correlation between these results and a variety of established clinical, histopathologic, morphometric and biochemical prognosticators was studied. These features included lymph node status, tumour diameter, tumour type, nuclear grade, histologic grade, oestrogen receptor status, mitotic activity index and a number of nuclear measurements. The results indicate that CEA immunoreactivity of and PNA binding to tumour cells in primary breast carcinomas or lymph node metastases do not correlate with established prognostic factors in breast cancer.

This paper describes an image analysis technique for the counting of nuclei in mitosis in tissue sections. Five experienced pathologists scored mitoses in photographs of preselected areas of tissue sections of the breast. Objects consistently labelled as mitotic cells by all five pathologists were considered "mitoses" in the analysis. In total, there were 45 mitotic nuclei, 68 possible mitotic nuclei and 1,172 nonmitotic nuclei. The image analysis procedure was designed to give priority to a low false negative rate, i.e., misclassification of mitoses. The procedure consists of three steps: 1. Segmentation of the image. 2. Reduction of the number of nonmitotic nuclei by using feature values based on the brightness histogram of the objects. 3. Fully automatic classification of the remaining objects using contour features. The objects remaining after the first two steps were visualized in a composite display for interactive evaluation: 10% of the mitotic nuclei were missed, and 85% of the nonmitotic nuclei were eliminated. The result of the fully automatic procedure described in this paper is rather disappointing and gave a loss of 37% of the mitoses while 5% of the nonmitotic nuclei remained.

A novel approach to the problem of automated white blood cell classification is described. Whereas in most earlier attempts the segmentation of the cells has been recognized as the most difficult and most critical step in the sequence of operations resulting in the classification, the method described here eliminates the necessity of the detection of the contour of the nucleus and of the cytoplasm, and is therefore less sensitive to such disturbing factors as the presence of granules, of other cells touching the cell of interest, etc. The multiple sequential threshold method to be described here in two slightly different variants yields a correct classification rate of 94.7% for a 4 class problem (90 cells in the test set), and 91.8% for an 8 class problem (279 cells in the test set). Both experiments include immature cell types.

Counting immunoglobulin (Ig)-containing cells in colonic mucosal biopsies can help to objectively support the differential diagnosis of ulcerative colitis and Crohn's disease. Before a method for counting Ig-containing cells can be applied in a clinical setting, however, its reproducibility must be determined. This study investigated the reproducibility of two different methods for counting such cells. The use of a light microscope with an ocular grid resulted in a slightly better reproducibility than did the use of a projection microscope with a graphics tablet. Moreover, the ocular grid method had a higher efficiency. The counting of IgM- and IgG-containing cells had a considerably higher reproducibility than did the counting of IgA-containing cells. To determine the minimal number of cells to be counted in order to ascertain a stabilized mean number of Ig-containing cells, the running means of counts of Ig-containing cells were calculated for two observers. When at least 600 Ig-containing cells (i.e., two to four fields) were counted, the interobserver variation of the running means was less than 10% for IgA and IgG counts and less than 5% for IgM counts. Since earlier studies showed differences in the counts of IgA-, IgG- and IgM-containing cells between ulcerative colitis and Crohn's disease to be, respectively, 3%, 25% and 28%, the results of the present study suggest that the proposed counting method can be useful in the differentiation between these entities.

The morphometric analysis of nuclear characteristics by means of a graphic tablet is, in principle, objective and highly reproducible. However, a recent study found considerable variation in the morphometric assessments, which was in contrast to the findings of others. The way in which measurements were performed differed in these studies. Therefore, measuring system factors that can potentially influence the quantitative results were analyzed systematically. One observer, experienced in microscopic analysis and working with a commercially available graphic tablet, conducted all the measurements, thus excluding interobserver variation. The tracing speed, localization (on the graphic tablet), magnification, pen and cursor usage, shape, and orientation on the graphic tablet were analyzed. A nomogram was developed for cursor application that indicates the relation between "projected" particle size, tracing speed, and required coefficient of variation (CV). When the influence of these factors is taken into account, a measuring system can be tuned optimally. With such a regimen, the CV can be kept below 1.5%. Our results show that in the assessment of morphometric features with the use of a graphic tablet, errors due to the measuring system can be virtually eliminated.