Search

This paper presents an algorithm for automated extraction of interest points (IPs)in multispectral and hyperspectral images. Interest points are features of the image that capture information from its neighbours and they are distinctive and stable under transformations such as translation and rotation. Interest-point operators for monochromatic images were proposed more than a decade ago and have since been studied extensively. IPs have been applied to diverse problems in computer vision, including image matching, recognition, registration, 3D reconstruction, change detection, and content-based image retrieval. Interest points are helpful in data reduction, and reduce the computational burden of various algorithms (like registration, object detection, 3D reconstruction etc) by replacing an exhaustive search over the entire image domain by a probe into a concise set of highly informative points. An interest operator seeks out points in an image that are structurally distinct, invariant to imaging conditions, stable under geometric transformation, and interpretable which are good candidates for interest points. Our approach extends ideas from Lowe's keypoint operator that uses local extrema of Difference of Gaussian (DoG) operator at multiple scales to detect interest point in gray level images. The proposed approach extends Lowe's method by direct conversion of scalar operations such as scale-space generation, and extreme point detection into operations that take the vector nature of the image into consideration. Experimental results with RGB and hyperspectral images which demonstrate the potential of the method for this application and the potential improvements of a fully vectorial approach over band-by-band approaches described in the literature.

Multispectral imaging is extensively used to multiplex the labeling of tissue features for diagnostic purposes. This technique enables multiple biomarkers with overlapping spectra to be distinguished using unmixing algorithms. However, spectral information alone is sometimes not sufficient enough to obtain perfect fluorochrome separation. By collecting time-resolved multispectral data containing a fluorescence decay time-series at each wavelength, the added temporal information can be used along with the spectral measurements as an additional distinguishing factor between the different dyes. A monoexponential decay model is used to segment the image into pure and mixed pixels based on the fit residuals. Using this information, the lifetimes and spectra of the dyes are retrieved using a global parameter estimation model within the segmented regions. The image is unmixed using the retrieved spectra and lifetimes of the individual components.

Change detection is the process of automatically identifying and analyzing regions that have undergone spatial or spectral changes from multi temporal images. Detecting and representing change provides valuable information of the possible transformations a given scene has suffered over time. Change detection in sequences of hyperspectral images is complicated by the fact that change can occur in the temporal and/or spectral domains. This work studies the use of Temporal Principal Component Analysis (TPCA) for change detection in multi/hyperspectral images. Two additional methods were implemented in order to compare its results with TPCA. These were: Image Differencing and Conventional Principal Component Analysis. Experimental results using phantom hyperspectral imagery taken with Surface Optics SOC-700 hyperspectral camera are presented. The algorithms were implemented using Matlab, and their performance is compared in terms of false alarms, missed changes and overall error. Results show that the performance of TPCA was the best, obtaining the smallest percentages of error, missed changes, and false alarms using global or local threshold. TPCA with local threshold gave the best performance.

The number of different subtypes of neurons, which form the basic component of the mammalian brain, has not been determined. Histological study is typically limited to the simultaneous detection of very few markers, in part because of the spectral overlap and quenching properties of organic fluorophores. The photostability and narrow emission spectra of non-organic colloidal quantum-dot fluorophores (QDs) make them desirable candidates for multiplex immunohistochemistry (IHC) and for fluorescent in situ hybridization (FISH). IHC is used to study specific protein epitopes and FISH to study the expression of specific mRNA transcripts. In order to investigate the patterns of coexpression of multiple specific protein and nucleic acid targets within cells in complex tissues, such as brain, we have developed protocols for the multiplex use of different QDs and organic fluorophores for combined IHC and FISH. We developed a method for direct QD labeling of modified oligonucleotide probes through streptavidin and biotin interactions and validated this technique in mouse brainstem sections. The reproducible histological results obtained with this protocol allow the use of high throughput computer image analysis to quantify the cellular and subcellular spatial pattern of expression of all markers studied. This approach is being utilized to generate a multiplex co-expression map of neuronal subtypes in mouse brain regions.

Tumor vasculature has a high degree of irregularity as compared to normal vasculature. The quantification of the morphometric complexity in tumor images can be useful in diagnosis. Also, it is desirable in several other medical applications to have an automated complexity analysis to aid in diagnosis and prognosis under treatment. e.g. in diabetic retinopathy and in arteriosclerosis. In addition, prior efforts at segmentation of the tumor vasculature using matched filtering, template matching and splines have been hampered by the irregularity of these vessels. We try to solve both problems by introducing a novel technique for vessel detection, followed by a tracing-independent complexity analysis based on a combination of ideas. First, the vessel cross-sectional profile is modeled using a continuous and everywhere differentiable family of super-Gaussian curves. This family generates rectangular profiles that can accurately localize the vessel boundaries in microvasculature images. Second, a robust non-linear regression algorithm based on M-estimators is used to estimate the parameters that optimally characterize the vessel’s shape. A framework for the quantitative analysis of the complexity of the vasculature based on the vessel detection is presented. A set of measures that quantify the complexity are proposed viz. Squared Error, Entropy-based and Minimum Description Length-based Shape Complexities. They are completely automatic and can deal with complexities of the entire vessel unlike existing tortuousity measures which deal only with vessel centerlines. The results are validated using carefully constructed phantom and real image data with ground truth information from an expert observer.

Automated methods are described for in vivo quantitation of changes in tumor vasculature. The tumor subsurface is imaged non-invasively over time with two-photon confocal microscopy aided by a variety of chronic animal window preparations. This results in time series of three-dimensional (3-D) image stacks for each specimen at high resolution (768x512x32 voxels, 8 bits/voxel, every 24 hours for 7 days), imaging depth and signal-to-background ratio. Next, automated image analysis allows detection and quantitation of vascular changes in a rapid and objective manner without manual tedium. We describe a fast new algorithm for fully automated 3-D tracing (50 seconds to trace a 10 MB stack on a Dell 1 GHz Pentium III personal computer). A variety of measurements including tortuosity, length, thickness, and branching order are generated and analyzed. Quantitative validation of the performance of the tracing algorithm against manual tracing resulted in 81% concordance. This enables a broader set of change analysis studies including testing the efficacy of anti-angiogenic therapies and deriving vessel growth parameters that may be correlated with physiological and gene expression profiles in tumor.

Blind deconvolution algorithms are being developed for reconstructing (deblurring) 2D and 3D optically sectioned light micrographs, including widefield fluorescence, transmitted light brightfield and confocal fluorescence micrographs. The blind deconvolution concurrently reconstructs the point spread function (PSF) with the image data. This is important because it obviates the necessity to measure the PSF. The PSF measurement is esoteric and sometimes impossible and it thereby will hinder wide routine biological and clinical usage. The iterative algorithms are primarily based on a stochastic model of the physics of fluorescence quantum photons and the optimization criterion of maximum likelihood estimation (MLE), as extended from precursory nuclear medicine MLE algorithms. The algorithm design is mostly model based, although it contains some non- model-based components which have important practical benefits.

Fast algorithms are presented for effective removal of the noise artifact in 3-D confocal fluorescence microscopy images of extended spatial objects such as neurons. The algorithms are unsupervised in the sense that they automatically estimate and adapt to the spatially and temporally varying noise level in the microscopy data. An important feature of the algorithms is the fact that a 3-D segmentation of the field emerges jointly with the intensity estimate. The role of the segmentation is to limit any smoothing to the interiors of regions and hence avoid the blurring that is associated with conventional noise removal algorithms. Fast computation is achieved by parallel computation methods, rather than by algorithmic or modelling compromises. The noise-removal proceeds iteratively, starting from a set of approximate user- supplied, or default initial guesses of the underlying random process parameters. An expectation maximization algorithm is used to obtain a more precise characterization of these parameters, that are then input to a hierarchical estimation algorithm. This algorithm computes a joint solution of the related problems corresponding to intensity estimation, segmentation, and boundary-surface estimation subject to a combination of stochastic priors and syntactic pattern constraints. Three-dimensional stereoscopic renderings of processed 3-D images of murine hippocampal neurons are presented to demonstrate the effectiveness of the method. The processed images exhibit increased contrast and significant smoothing and reduction of the background intensity while avoiding any blurring of the neuronal structures.

This paper presents recent results of our reconstructions of 3-D data from Drosophila chromosomes as well as our simulations with a refined version of the algorithm used in the former. It is well known that the calibration of the point spread function (PSF) of a fluorescence microscope is a tedious process and involves esoteric techniques in most cases. This problem is further compounded in the case of confocal microscopy where the measured intensities are usually low. A number of techniques have been developed to solve this problem, all of which are methods in blind deconvolution. These are so called because the measured PSF is not required in the deconvolution of degraded images from any optical system. Our own efforts in this area involved the maximum likelihood (ML) method, the numerical solution to which is obtained by the expectation maximization (EM) algorithm. Based on the reasonable early results obtained during our simulations with 2-D phantoms, we carried out experiments with real 3-D data. We found that the blind deconvolution method using the ML approach gave reasonable reconstructions. Next we tried to perform the reconstructions using some 2-D data, but we found that the results were not encouraging. We surmised that the poor reconstructions were primarily due to the large values of dark current in the input data. This, coupled with the fact that we are likely to have similar data with considerable dark current from a confocal microscope prompted us to look into ways of constraining the solution of the PSF. We observed that in the 2-D case, the reconstructed PSF has a tendency to retain values larger than those of the theoretical PSF in regions away from the center (outside of those we considered to be its region of support). This observation motivated us to apply an upper bound constraint on the PSF in these regions. Furthermore, we constrain the solution of the PSF to be a bandlimited function, as in the case in the true situation. We have derived two separate approaches for implementing the constraint. One approach involves the mathematical rigors of Lagrange multipliers. This approach is discussed in another paper. The second approach involves an adaptation of the Gershberg Saxton algorithm, which ensures bandlimitedness and non-negativity of the PSF. Although the latter approach is mathematically less rigorous than the former, we currently favor it because it has a simpler implementation on a computer and has smaller memory requirements. The next section describes briefly the theory and derivation of these constraint equations using Lagrange multipliers.

We have developed image reconstruction algorithms for generating 3-D renderings of biological specimens from brightfield micrographs. The algorithm presented here is founded on the maximum likelihood estimation theory where steepest ascent and conjugate gradient techniques are used to optimize the solution to the multidimensional equation. The estimation problem posed is that of reconstructing the optical density, or linear attenuation coefficients, similar to that of computed tomography, under the simplifying assumption of geometric optics. We assume white Gaussian noise corrupts the signal generating a Gaussian distributed signal according to the modeling of the system impulse response. One of the challenges of the algorithms presented here is in restoring the values within the missing cone region of the system optical transfer function. The algorithm and programming are straightforward and incorporate standard Fourier techniques. The theoretical development of the algorithms is outlined. Simulations of reconstructions using this technique are currently being performed.

Essential to the solution of ill posed problems in vision and image processing is the need to use object constraints in the reconstruction. While Bayesian methods have shown the greatest promise, a fundamental difficulty has persisted in that many of the available constraints are in the form of deterministic rules rather than as probability distributions and are thus not readily incorporated as Bayesian priors. In this paper, we propose a general method for mapping a large class of rule-based constraints to their equivalent stochastic Gibbs' distribution representation. This mapping allows us to solve stochastic estimation problems over rule-generated constraint spaces within a Bayesian framework. As part of this approach we derive a method based on Langevin's stochastic differential equation and a regularization technique based on the classical autologistic transfer function that allows us to update every site simultaneously regardless of the neighbourhood structure. This allows us to implement a completely parallel method for generating the constraint sets corresponding to the regular grammar languages on massively parallel networks. We illustrate these ideas by formulating the image reconstruction problem based on a hierarchy of rule-based and stochastic constraints, and derive a fully parallelestimator structure. We also present results computed on the AMT DAP500 massively parallel digital computer, a mesh-connected 32x32 array of processing elements which are configured in a Single-Instruction, Multiple Data stream architecture.