Search

In this paper, the quickest change detection problem is studied in two-state hidden Markov models (HMM), where the vector parameter \theta of the HMM changes from \theta0 to \theta1 at some unknown time, and one wants to detect the true change as quickly as possible while controlling the false alarm rate. It turns out that the generalized likelihood ratio (GLR) scheme, while theoretically straightforward, is generally computationally infeasible for the HMM. To develop efficient but computationally simple schemes for the HMM, we first discuss a subtlety in the recursive form of the generalized likelihood ratio (GLR) scheme for the HMM. Then we show that the recursive CUSUM scheme proposed in Fuh (Ann. Statist., 2003) can be regarded as a quasi-GLR scheme for pseudo post-change hypotheses with certain dependence structure between pre- and postchange observations. Next, we extend the quasi-GLR idea to propose recursive score schemes in the scenario when the postchange parameter \theta1 of the HMM involves a real-valued nuisance parameter. Finally, the Kullback-Leibler (KL) divergence plays an essential role in the quickest change detection problem and many other fields, however it is rather challenging to numerically compute it in HMMs. Here we develop a non-Monte Carlo method that computes the KL divergence of two-state HMMs via the underlying invariant probability measure, which is characterized by the Fredholm integral equation. Numerical study demonstrates an unusual property of the KL divergence for HMM that implies the severe effects of misspecifying the postchange parameter for the HMM. [ABSTRACT FROM AUTHOR]

We concentrate on the paradigm question how much stochasticity and how much chaos are present in the methane emission model. In particular we analyze the residua from the process of methane emissions from wetlands in the sedge-grass marsh, in South Bohemia, Czech Republic. Relation to entropy and a specific version of Kullback–Leibler divergence will be given. A graphical tool to assess the amount of entropy in the system is developed and illustrated on real data from the sedge-grass marsh methane emission. [ABSTRACT FROM AUTHOR]

Four problems related to information divergence measures defined on finite alphabets are considered. In three of the cases we consider, we illustrate a contrast that arises between the binary-alphabet and larger alphabet settings. This is surprising in some instances, since characterizations for the larger alphabet settings do not generalize their binary-alphabet counterparts. In particular, we show that $f $ -divergences are not the unique decomposable divergences on binary alphabets that satisfy the data processing inequality, thereby clarifying claims that have previously appeared in the literature. We also show that Kullback–Leibler (KL) divergence is the unique Bregman divergence, which is also an $f $ -divergence for any alphabet size. We show that KL divergence is the unique Bregman divergence, which is invariant to statistically sufficient transformations of the data, even when nondecomposable divergences are considered. Like some of the problems we consider, this result holds only when the alphabet size is at least three. [ABSTRACT FROM AUTHOR]

The paper provides a spatio-temporal change detection framework for the analysis of image time series. In this framework, the detection of changes in time is addressed at the image level by using a matrix of cross-dissimilarities computed upon wavelet and curvelet image features. This makes possible identifying the acquisitions of interest: the acquisitions that exhibit singular behavior with respect to their neighborhood in the time series, and those that are representatives of some stationary behavior. These acquisitions of interest are compared at the pixel level to detect spatial changes characterizing the evolution of the time series. Experiments carried out over European Remote Sensing (ERS) and TerraSAR-X time series highlight the relevancy of the approach for analyzing synthetic aperture radar image time series. [ABSTRACT FROM PUBLISHER]

In view-based 3D object retrieval and recognition, each object is described by multiple views. A central problem is how to estimate the distance between two objects. Most conventional methods integrate the distances of view pairs across two objects as an estimation of their distance. In this paper, we propose a discriminative probabilistic object modeling approach. It builds probabilistic models for each object based on the distribution of its views, and the distance between two objects is defined as the upper bound of the Kullback–Leibler divergence of the corresponding probabilistic models. 3D object retrieval and recognition is accomplished based on the distance measures. We first learn models for each object by the adaptation from a set of global models with a maximum likelihood principle. A further adaption step is then performed to enhance the discriminative ability of the models. We conduct experiments on the ETH 3D object dataset, the National Taiwan University 3D model dataset, and the Princeton Shape Benchmark. We compare our approach with different methods, and experimental results demonstrate the superiority of our approach. [ABSTRACT FROM PUBLISHER]

In hyperspectral imagery, low-dimensional representations are sought in order to explain well the nonlinear characteristics that are hidden in high-dimensional spectral channels. While many algorithms have been proposed for dimension reduction and manifold learning in Euclidean spaces, very few attempts have focused on non-Euclidean spaces. Here, we propose a novel approach that embeds hyperspectral data, transformed into bilateral probability similarities, onto a nonlinear unit norm coordinate system. By seeking a unit \ell2-norm nonlinear manifold, we encode similarity representations onto a space in which important regularities in data are easily captured. In its general application, the technique addresses problems related to dimension reduction and visualization of hyperspectral images. Unlike methods such as multidimensional scaling and spherical embeddings, which are based on the notion of pairwise distance computations, our approach is based on a stochastic objective function of spherical coordinates. This allows the use of an Exit probability distribution to discover the nonlinear characteristics that are inherent in hyperspectral data. In addition, the method directly learns the probability distribution over neighboring pixel maps while computing for the optimal embedding coordinates. As part of evaluation, classification experiments were conducted on the manifold spaces for hyperspectral data acquired by multiple sensors at various spatial resolutions over different types of land cover. Various visualization and classification comparisons to five existing techniques demonstrated the strength of the proposed approach while its algorithmic nature is guaranteed to converge to meaningful factors underlying the data. [ABSTRACT FROM PUBLISHER]

This paper studies the problem of 3-D rigid-motion-invariant texture discrimination for discrete 3-D textures that are spatially homogeneous by modeling them as stationary Gaussian random fields. The latter property and our formulation of a 3-D rigid motion of a texture reduce the problem to the study of 3-D rotations of discrete textures. We formally develop the concept of 3-D texture rotations in the 3-D digital domain. We use this novel concept to define a “distance” between 3-D textures that remains invariant under all 3-D rigid motions of the texture. This concept of “distance” can be used for a monoscale or a multiscale 3-D rigid-motion-invariant testing of the statistical similarity of the 3-D textures. To compute the “distance” between any two rotations R1 and R2 of two given 3-D textures, we use the Kullback–Leibler divergence between 3-D Gaussian Markov random fields fitted to the rotated texture data. Then, the 3-D rigid-motion-invariant texture distance is the integral average, with respect to the Haar measure of the group SO(3), of all of these divergences when rotations R1 and R2 vary throughout SO(3). We also present an algorithm enabling the computation of the proposed 3-D rigid-motion-invariant texture distance as well as rules for 3-D rigid-motion-invariant texture discrimination/classification and experimental results demonstrating the capabilities of the proposed 3-D rigid-motion texture discrimination rules when applied in a multiscale setting, even on very general 3-D texture models. [ABSTRACT FROM PUBLISHER]

We develop and analyze M-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a nonasymptotic variational characterization of f-divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve optimal minimax rates for the likelihood ratio and the divergence functionals in certain regimes. We derive an efficient optimization algorithm for computing our estimates, and illustrate their convergence behavior and practical viability by simulations. [ABSTRACT FROM PUBLISHER]

Particle filters have revolutionized object tracking in video sequences. The conventional particle filter, also called the CONDENSATION filter, uses the state transition distribution as the proposal distribution, from which the particles are drawn at each iteration. However, the transition distribution does not take into account the current observations, and thus many particles can be wasted in low likelihood regions. One of the most popular methods to improve the performance of particle filters relied on the motion-based proposal density. Although the motivation for motion-based particle filters could be explained on an intuitive level, up until now a mathematical rationale for the improved performance of motion-based particle filters has not been presented. In this letter, we investigate the performance of motion-based particle filters and provide an analytical justification of their superiority over the classical CONDENSATION filter. We rely on the characterization of the optimal proposal density, which minimizes the variance of the particles' weights. However, this density does not admit an analytical expression, making direct sampling from this optimal distribution impossible. We use the Kullback-Leibler (KL) divergence as a similarity measure between density functions and denote a particle filter as superior if the KL divergence between its proposal and the optimal proposal function is lower. We subsequently prove that under mild conditions on the estimated motion vector, the motion-based particle filter outperforms the CONDENSATION filter, in terms of the KL performance measure. Simulation results are presented to support the theoretical analysis. [ABSTRACT FROM AUTHOR]

Given a correlated Gaussian signal, may a chi-squared law of probability always be used to describe a spectrogram coefficient distribution? If not, would a "chi-squared description" lead to an acceptable amount of error when detection problems are to be faced in the time-frequency domain? These two questions prompted the study reported in this paper. After deriving the probability distribution of spectrogram coefficients in the context of a noncentered Gaussian correlated signal, the Kullback-Leibler divergence is first used to evaluate to what extent the nonwhiteness of the signal and the Fourier analysis window impact the probability distribution of the spectrogram. To complete the analysis, a detection task formulated as a binary hypothesis test is considered. We evaluate the error committed on the probability of false alarm when the likelihood ratio test is expressed with chi-squared laws. From these results, a chi-squared description of the spectrogram distribution appears accurate when the analysis window used to construct the spectrogram decreases to zero at its boundaries, regardless of the level of correlation contained in the signal. When other analysis windows are used, the length of the window and the correlation contained in the analyzed signal impact the validity of the chi-squared description. [ABSTRACT FROM AUTHOR]

In this paper, we present a new similarity measure for automatic change detection in multitemporal synthetic aperture radar images. This measure is based on the evolution of the local statistics of the image between two dates. The local statistics are estimated by using a cumulant-based series expansion, which approximates probability density functions in the neighborhood of each pixel in the image. The degree of evolution of the local statistics is measured using the Kullback-Leibler divergence. An analytical expression for this detector is given, allowing a simple computation which depends on the four first statistical moments of the pixels inside the analysis window only. The proposed change indicator is compared to the classical mean ratio detector and also to other model-based approaches. Tests on the simulated and real data show that our detector outperforms all the others. The fast computation of the proposed detector allows a multiscale approach in the change detection for operational use. The so-called multiscale change profile (MCP) is introduced to yield change information on a wide range of scales and to better characterize the appropriate scale. Two simple yet useful examples of applications show that the MCP allows the design of change indicators, which provide better results than a monoscale analysis. [ABSTRACT FROM AUTHOR]

This letter shows an innovative voice activity detector (VAD) based on the Kullback-Leibler (KL) divergence measure. The algorithm is evaluated in the context of the recently approved ETSI standard for distributed speech recognition (DSR). The VAD uses long-term information of the noisy speech signal in order to define a more robust decision rule yielding high accuracy. The Mel-scaled filter bank log-energies (FBE) are modeled by means of Gaussian distributions, and a symmetric KL divergence is used for the estimation of the distance between speech and noise distributions. The decision rule is formulated in terms of the average subband KL divergence that is compared to a noise-adaptable threshold. An exhaustive analysis using the AURORA databases is conducted in order to assess the performance of the proposed method and to compare it to existing standard VAD methods. [ABSTRACT FROM AUTHOR]

Given a nominal statistical model, we consider the minimax estimation problem consisting of finding the best least-squares estimator for the least favorable statistical model within a neighborhood of the nominal model. The neighborhood is formed by placing a bound on the Kullback-Leibler (KL) divergence between the actual and nominal models. For a Gaussian nominal model and a finite observations interval, or for a stationary Gaussian process over an infinite interval, the usual noncausal Wiener filter remains optimal. However, the worst case performance of the filter is affected by the size of the neighborhood representing the model uncertainty. On the other hand, standard causal least-squares estimators are not optimal, and a characterization is provided for the causal estimator and the corresponding least favorable model. The causal estimator takes the form of a risk-sensitive estimator with an appropriately selected risk sensitivity coefficient. [ABSTRACT FROM AUTHOR]

Detection of faults at the incipient stage is critical to improving the availability and continuity of satellite services. The application of a local optimum projection vector and the Kullback–Leibler (KL) divergence can improve the detection rate of incipient faults. However, this suffers from the problem of high time complexity. We propose decomposing the KL divergence in the original optimization model and applying the property of the generalized Rayleigh quotient to reduce time complexity. Additionally, we establish two distribution models for subfunctions F 1 (w) and F 3 (w) to detect the slight anomalous behavior of the mean and covariance. The effectiveness of the proposed method was verified through a numerical simulation case and a real satellite fault case. The results demonstrate the advantages of low computational complexity and high sensitivity to incipient faults. [ABSTRACT FROM AUTHOR]

For the existing Closed Set Recognition (CSR) methods mistakenly identify unknown jamming signals as a known class, a Conditional Gaussian Encoder (CG-Encoder) for 1-dimensional signal Open Set Recognition (OSR) is designed. The network retains the original form of the signal as much as possible and deep neural network is used to extract useful information. CG-Encoder adopts residual network structure and a new Kullback-Leibler (KL) divergence is defined. In the training phase, the known classes are approximated to different Gaussian distributions in the latent space and the discrimination between classes is increased to improve the recognition performance of the known classes. In the testing phase, a specific and effective OSR algorithm flow is designed. Simulation experiments are carried out on 9 jamming types. The results show that the CSR and OSR performance of CG-Encoder is better than that of the other three kinds of network structures. When the openness is the maximum, the open set average accuracy of CG-Encoder is more than 70%, which is about 30% higher than the worst algorithm, and about 20% higher than the better one. When the openness is the minimum, the average accuracy of OSR is more than 95%.

Background: Ultrasound imaging is safer than other imaging modalities, because it is noninvasive and nonradiative. Speckle noise degrades the quality of ultrasound images and has negative effects on visual perception and diagnostic operations., Methods: In this paper, a nonlocal total variation (NLTV) method for ultrasonic speckle reduction is proposed. A spatiogram similarity measurement is introduced for the similarity calculation between image patches. It is based on symmetric Kullback-Leibler (KL) divergence and signal-dependent speckle model for log-compressed ultrasound images. Each patch is regarded as a spatiogram, and the spatial distribution of each bin of the spatiogram is regarded as a weighted Gamma distribution. The similarity between the corresponding bins of the two spatiograms is computed by the symmetric KL divergence. The Split-Bregman fast algorithm is then used to solve the adapted NLTV object function. Kolmogorov-Smirnov (KS) test is performed on synthetic noisy images and real ultrasound images., Results: We validate our method on synthetic noisy images and clinical ultrasound images. Three measures are adopted for the quantitative evaluation of the despeckling performance: the signal-to-noise ratio (SNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). For synthetic noisy images, when the noise level increases, the proposed algorithm achieves slightly higher SNRS than that of the other two algorithms, and the SSIMS yielded by the proposed algorithm is obviously higher than that of the other two algorithms. For liver, IVUS and 3DUS images, the NIQE values are 8.25, 6.42 and 9.01, all of which are higher than that of the other two algorithms., Conclusions: The results of the experiments over synthetic and real ultrasound images demonstrate that the proposed method outperforms current state-of-the-art despeckling methods with respect to speckle reduction and tissue texture preservation.