Search

A new information criterion is proposed for nonlinear dimension reduction (NLDR) based on probability density function (PDF) estimation. As the PDF is estimated, the transformation to the latent space is learned and information transfer is maximized. The method (a) maximizes information at the output, (b) makes no assumptions about the data structure, (b) is invariant to invertible transformations, (c) can produce any desired output distribution (such as independent uniform or Gaussian latent variables) and (d) is completely general. In addition to performing dimension reduction, the approach results in a complete statistical model of the data including tractable likelihood function and ability to generate synthetic data. The method specializes to principal component analysis (PCA) for the linear/Gaussian case. When the transformation has limited approximation power, the trade-off between information transfer and approximating the desired output distribution can be controlled using a constant $\beta$ , which is analogous to $\beta$ -VAE. For efficiency, the method can be implemented with a neural network architecture using a variation of PDF projection, called projected belief network (PBN). In experiments with high-dimensional non-Gaussian input data, the superiority of PBN is shown relative to PCA, restricted Boltzmann machine (RBM), and a $\beta$ -variational auto-encoder ($\beta$ -VAE). [ABSTRACT FROM AUTHOR]

Maximum entropy PDF projection (MEPP) is a way to construct generative models from feature transformations. Corresponding to each dimension-reducing feature mapping, such as a feed-forward neural network or an algorithm to calculate linear-prediction coefficients from time series, and given a prior distribution for the features, there exists a unique generative model for the input data, which subject to mild requirements, is maximum entropy (MaxEnt) among all probability density functions (PDFs) that are consistent with the given feature prior. In this paper, we consider the problem of sampling from these MaxEnt projected PDFs. The sampling process consists of drawing a sample from the given feature prior distribution, then drawing samples uniformly distributed on the inversion set (set of input samples consistent with the drawn feature value, usually a manifold). The process is called uniform manifold sampling (UMS). We describe UMS for simple nonlinear and iterative feature transformations, then focus on linear transformations with input data constraints ($x_i > 0$ or $0\leq x_i \leq 1$ ), which require MCMC-based sampling. We discover that the manifold centroid (sample mean for a fixed feature value) is useful as a deterministic MaxEnt feature inversion solution. We show how to predict the centroid efficiently without sampling and demonstrate its usefulness in speeding up MCMC by an order of magnitude, and in spectral estimation and image reconstruction. Finally, we provide an example of UMS in a classification experiment in which we use Monte Carlo integration to create true generative models from arbitrary classifiers. [ABSTRACT FROM PUBLISHER]

We present a new classifier for acoustic time series that involves a mixture of generative models. The models use a variety of segmentation sizes and feature extraction methods, yet can be combined at a higher level using a mixture probability density function (PDF) thanks to the PDF projection theorem (PPT) that converts the feature PDF to raw time series PDFs. The effectiveness of the method is compared with the leading methods and is shown to be superior using three data sets. [ABSTRACT FROM PUBLISHER]

This paper revisits an existing method of constructing high-dimensional probability density functions (PDFs) based on the PDF at the output of a dimension-reducing feature transformation. We show how to modify the method so that it can provide the PDF with the highest entropy among all PDFs that generate the given low-dimensional PDF. The method is completely general and applies to arbitrary feature transformations. The chain-rule is described for multi-stage feature calculations typically used in signal processing. Examples are given including MFCC and auto-regressive features. Experimental verification of the results using simulated data is provided including a comparison with competing generative methods. [ABSTRACT FROM PUBLISHER]

This paper describes a tracker specially designed to track clicking beaked whales using widely spaced bottom-mounted hydrophones, although it can be adapted to different species and sensors. The input to the tracker is a sequence of static localization solutions obtained using time difference of arrival information at widely spaced hydrophones. To effectively handle input localizations with high ambiguity, the tracker is based on multi-hypothesis tracker concepts, so it considers all potential association hypotheses and keeps a large number of potential tracks in memory. The method is demonstrated on actual data and shown to successfully track multiple beaked whales at depth. [ABSTRACT FROM AUTHOR]

The recently introduced shift method is applied to detect and characterize burst-pulse vocalizations produced by marine mammals. To this end, burst pulses are modeled as sequences of click-like events that are repeated after a certain inter-click interval (ICI). The shift method is used to first emphasize events that repeat within an input signal. Afterwards, the ICI can be estimated. A qualitative comparison of the method is made against classical cepstrum using real data. The detection performance is measured using random trials of simulated data with impulsive noise. It is shown that although the cepstrum outperforms in Gaussian noise at low signal-to-noise ratio, the shift method performs significantly better in impulsive noise. [ABSTRACT FROM AUTHOR]

The problem of click-train separation is cast as a linear assignment problem to obtain a faster solution guaranteed to achieve the global minimum error. It is shown how the problem can be cast in a compact matrix form that is solvable by an off-the-shelf algorithm, the Jonker-Volgenant algorithm. [ABSTRACT FROM AUTHOR]

We present a fast implementation of the optimal linear processor for “chirp” signals with O() computational load for data length. The method is technically an approximation since it has an interpolation step but we show that the errors are negligible. In simulations it performs up to 20 times faster than the existing method using radix-2 fast Fourier transform (FFT) and has significantly lower threshold signal-to-noise ratio (SNR) against noise and interference compared with a leading fast method. [ABSTRACT FROM PUBLISHER]

We describe a new active sonar processor that obtains simultaneously fine range and Doppler resolution by fusing data from sequentially transmitted linear frequency-modulated (LFM) and CW waveforms. We model the received data as returns from a small number of point-reflectors and fuse returns from the two waveforms using a novel formulation that assumes common amplitude and random phase (CARP). We demonstrate the method using simulated data. [ABSTRACT FROM AUTHOR]

This paper is concerned with the localization of clicking Blainville's beaked whales (Mesoplodon densirostris) using an array of widely spaced bottom-mounted hydrophones. A set of signal and data processing advances are presented that together make reliable tracking a possibility. These advances include a species-specific detector, elimination of spurious time-difference-of-arrival (TDOA) estimates, improved tracking of TDOA estimates, positive association of TDOA estimates using different hydrophone pairs, and joint localization of multiple whales. A key innovation in three of these advances is the principle of click-matching. The methods are demonstrated using real data. [ABSTRACT FROM AUTHOR]

In this paper an algorithm is described for the localization of individual sperm whales in situations where several near-by animals are simultaneously vocalizing. The algorithm operates on time-difference of arrival (TDOA) measurements observed at sensor pairs and assumes no prior knowledge of the TDOA-whale associations. In other words, it solves the problem of associating TDOAs to whales. The algorithm is able to resolve association disputes where a given TDOA measurement may fit to more than one position estimate and can handle spurious TDOAs. The algorithm also provides estimates of Cramer-Rao lower bound for the position estimates. The algorithm was tested with real data using TDOA estimates obtained by cross-correlating click-trains. The click-trains were generated by a separate algorithm that operated independently on each sensor to produce click-trains corresponding to a given whale and to reject click-trains from reflected propagation paths. [ABSTRACT FROM AUTHOR]

In this paper an algorithm is described for grouping sperm whale clicks received at a single hydrophone into click-trains associated with a given whale and propagation path. To accomplish this, features are extracted from pairs of clicks in order to derive a statistical measure of click similarity. Then, the algorithm maximizes a global measure of similarity between the associated clicks. In addition to grouping clicks according to click-trains, it classifies click-trains into a propagation path (direct, surface, bottom reverberation). The propagation path is identified using a combination of (1) a statistical classifier operating on features extracted from individual clicks and (2) the cross-correlation of click-trains to identify the relative time delays. The algorithm is demonstrated on at-sea recorded data, showing the elimination of time-delay ambiguities associated with the reverberation paths observed at sensor pairs. [ABSTRACT FROM AUTHOR]

This paper describes an efficient model to describe an autoregressive (AR) signal with slowly-varying amplitude in additive white Gaussian noise (WGN). Even a simple low-order AR model becomes complicated by varying amplitude and additive white noise. However, by approximating the signal amplitude as piecewise-constant, an efficient filtering approach can be applied in order to compute the maximum likelihood (ML) estimate for the entire data record. The model is efficient both in terms of having a compact set of parameters and in the computational sense. Simulation results are provided. The algorithm has applications in signal modeling for underwater acoustic signals, particularly active wideband signals such as explosive sources. [ABSTRACT FROM AUTHOR]

The class-specific (CS) method of signal classification operates by computing low-dimensional feature sets defined for each signal class of interest. By computing separate feature sets tailored to each class, i.e., CS features, the CS method avoids estimating probability distributions in a high-dimension feature space common to all classes. Building a CS classifier amounts to designing feature extraction modules for each class of interest. In this paper we present the design of three CS modules used to form a CS classifier for narrowband signals of finite duration. A general module for narrowband signals based on a narrowband tracker is described. The only assumptions this module makes regarding the time evolution of the signal spectrum are: 1) one or more narrowband lines are present, and 2) the lines wandered either not at all, e.g., CW signal, or with a purpose, e.g., swept FM signal. The other two modules are suited for specific classes of waveforms and assume some a priori knowledge of the signal is available from training data. For in situ training, the tracker-based module can be used to detect as yet unobserved waveforms and classify them into general categories, for example short CW, long CW, fast FM, slow FM, etc. Waveform-specific class-models can then be designed using these waveforms for training. Classification results are presented comparing the performance of a probabilistic conventional classifier with that of a CS classifier built from general modules and a CS classifier built from waveform-specific modules. Results are also presented for hybrid discriminative/generative versions of the classifiers to illustrate the performance gains attainable in using a hybrid over a generative classifier alone. [ABSTRACT FROM AUTHOR]

In this paper, we derive a technique for analysis of local distortions which affect data in real-world applications. In the paper, we focus on image data, specifically handwritten characters. Given a reference image and a distorted copy of it, the method is able to efficiently determine the rotations, translations, scaling, and any other distortions that have been applied. Because the method is robust, it is also able to estimate distortions for two unrelated images, thus determining the distortions that would be required to cause the two images to resemble each other. The approach is based on a polynomial series expansion using matrix powers of linear transformation matrices. The technique has applications in pattern recognition in the presence of distortions. [ABSTRACT FROM AUTHOR]

In this paper, we present the theoretical foundation for optimal classification using class-specific features and provide examples of its use. A new probability density function (PDF) projection theorem makes it possible to project probability density functions from a low-dimensional feature space back to the raw data space. An M-ary classifier is constructed by estimating the PDFs of class-specific features, then transforming each PDF back to the raw data space where they can be fairly compared. Although statistical sufficiency is not a requirement, the classifier thus constructed will become equivalent to the optimal Bayes classifier if the features meet sufficiency requirements individually for each class. This classifier is completely modular and avoids the dimensionality curse associated with large complex problems. By recursire application of the projection theorem, it is possible to analyze complex signal processing chains. We apply the method to feature sets including linear functions of independent random variables, cepstrum, and MEL cepstrum. In addition, we demonstrate how it is possible to automate the feature and model selection process by direct comparison of log-likelihood values on the common raw data domain. [ABSTRACT FROM AUTHOR]

Highlights a study which presented an approach to the design of probabilistic classifiers that circumvents the dimensionality problem. Main theorem of the study; Illustration of the application of the class-specific method in a controlled experiment using synthetic signals; Conclusions.

The maximum entropy principle introduced by Jaynes proposes that a data distribution should maximize the entropy subject to constraints imposed by the available knowledge. Jaynes provided a solution for the case when constraints were imposed on the expected value of a set of scalar functions of the data. These expected values are typically moments of the distribution. This paper describes how the method of maximum entropy PDF projection can be used to generalize the maximum entropy principle to constraints on the joint distribution of this set of functions. [ABSTRACT FROM AUTHOR]

In this paper we present a statistical model with a nonsymmetric half-plane (NSHP) region of support for two-dimensional continuous-valued vector fields. It has the simplicity, efficiency, and ease of use of the well-known hidden Markov model (HMM) and associated Baum-Welch algorithms for time-series and other one-dimensional problems. At the same time it is able to learn textures on a two-dimensional field. We describe a fast approximate forward procedure for computation of the joint probability density function (pdf) of the vector field as well as an approximate Baum-Welch algorithm for parameter reestimation. Radar and sonar applications include classification of two-dimensional fields such as range versus azimuth or range versus aspect angle data wherein each data point in the field consists of a multi-dimensional feature vector. We test the method using synthetic textures. [ABSTRACT FROM PUBLISHER]

In this paper, we present a Bayesian classification approach for automatic text categorization using class-specific features. Unlike conventional text categorization approaches, our proposed method selects a specific feature subset for each class. To apply these class-specific features for classification, we follow Baggenstoss's PDF Projection Theorem (PPT) to reconstruct the PDFs in raw data space from the class-specific PDFs in low-dimensional feature subspace, and build a Bayesian classification rule. One noticeable significance of our approach is that most feature selection criteria, such as Information Gain (IG) and Maximum Discrimination (MD), can be easily incorporated into our approach. We evaluate our method's classification performance on several real-world benchmarks, compared with the state-of-the-art feature selection approaches. The superior results demonstrate the effectiveness of the proposed approach and further indicate its wide potential applications in data mining. [ABSTRACT FROM PUBLISHER]

Presents a study which addressed the problem of calculating multidimensional probability density functions of statistics derived from the standard normal distribution. Description of the saddlepoint approximation theory; Applications of the saddlepoint approximation; Application to autoregressive model-order selection.