Search

Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension $p$ is high, while the data aspect ratio $n/p$ is low. We propose an estimator called Tabasco (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptive) regularization parameters that are designed to minimize the mean squared error (MSE) between the proposed shrinkage estimator and the true covariance matrix. These parameters are derived under the general assumption that the data is sampled from an unspecified elliptically symmetric distribution with finite 4th order moments (both real- and complex-valued cases are addressed). Simulation studies show that the proposed Tabasco outperforms all competing tapering covariance matrix estimators in diverse setups. An application to space-time adaptive processing (STAP) also illustrates the benefit of the proposed estimator in a practical signal processing setup. [ABSTRACT FROM AUTHOR]

A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, Tyler's, or t weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from an unspecified heavy-tailed elliptically symmetric distribution. Also, real-world and synthetic stock market data validate the performance of the proposed method in practical applications. [ABSTRACT FROM AUTHOR]

This paper considers the problem of estimating a high-dimensional covariance matrix in a low sample support situation where the sample size is smaller, or not much larger, than the dimensionality of the data, which could potentially be very large. We develop a regularized sample covariance matrix (RSCM) estimator which can be applied in commonly occurring high-dimensional data problems. The proposed RSCM estimator is based on estimators of the unknown optimal (oracle) shrinkage parameters that yield the minimum mean squared error between the RSCM and the true covariance matrix when the data is sampled from an unspecified elliptically symmetric distribution. We propose two variants of the RSCM estimator which differ in the approach in which they estimate the underlying sphericity parameter involved in the theoretical optimal shrinkage parameter. The performance of the proposed RSCM estimators are evaluated with numerical simulation studies. In particular, when the sample sizes are low, the proposed RSCM estimators often show a significant improvement over the conventional RSCM estimator by Ledoit and Wolf (2004). We further evaluate the performance of the proposed estimators in a portfolio optimization problem with real data wherein the proposed methods are able to outperform the benchmark methods. [ABSTRACT FROM AUTHOR]

This paper proposes efficient algorithms for accurate recovery of direction-of-arrivals (DoAs) of sources from single-snapshot measurements using compressed beamforming (CBF). In CBF, the conventional sensor array signal model is cast as an underdetermined complex-valued linear regression model and sparse signal recovery methods are used for solving the DoA finding problem. A complex-valued pathwise weighted elastic net (c-PW-WEN) algorithm is developed that finds solutions at the knots of penalty parameter values over a path (or grid) of elastic net (EN) tuning parameter values. c-PW-WEN also computes least absolute shrinkage and selection operator (LASSO) or weighted LASSO in its path. A sequential adaptive EN (SAEN) method is then proposed that is based on c-PW-WEN algorithm with adaptive weights that depend on previous solution. Extensive simulation studies illustrate that SAEN improves the probability of exact recovery of true support compared to conventional sparse signal recovery approaches such as LASSO, EN, or orthogonal matching pursuit in several challenging multiple target scenarios. The effectiveness of SAEN is more pronounced in the presence of high mutual coherence. [ABSTRACT FROM AUTHOR]

In this paper, we develop low complexity and stable bootstrap procedures for FastICA estimators. Our bootstrapping techniques allow for performing cost efficient and reliable bootstrap-based statistical inference in the ICA model. Performing statistical inference is needed to quantitatively assess the quality of the estimators and testing hypotheses on mixing coefficients in the ICA model. The developed bootstrap procedures stem from the fast and robust bootstrap (FRB) method [1], which is applicable for estimators that may be found as solutions to fixed-point (FP) equations. We first establish analytical results on the structure of the weighted covariance matrix involved in the FRB formulation. Then, we exploit our analytical results to compute the FRB replicas at drastically reduced cost. The developed enhanced FRB method (EFRB) for FastICA permits using bootstrap-based statistical inference in a variety of applications (e.g., EEG, fMRI) in which ICA is commonly applied. Such an approach has not been possible earlier due to incurred substantial computational efforts of the conventional bootstrap. Our simulation studies compare the complexity and numerical stability of the proposed methods with the conventional bootstrap method. We also provide an example of utilizing the developed bootstrapping techniques in identifying equipotential lines of the brain dipoles from electroencephalogram (EEG) recordings. [ABSTRACT FROM AUTHOR]

In this paper we address the problem of performing statistical inference for large scale data sets i.e., Big Data. The volume and dimensionality of the data may be so high that it cannot be processed or stored in a single computing node. We propose a scalable, statistically robust and computationally efficient bootstrap method, compatible with distributed processing and storage systems. Bootstrap resamples are constructed with smaller number of distinct data points on multiple disjoint subsets of data, similarly to the bag of little bootstrap method (BLB) [A. Kleiner, A. Talwalkar, P. Sarkar, and M. I. Jordan, “A scalable bootstrap for massive data,” J. Roy. Statist. Soc.: Ser. B (Statist. Methodol.), vol. 76, no. 4, pp. 795–816, 2014]. The disjoint subsets are significantly smaller than the original full data set and they may be processed in different storage and computing units in parallel. Then significant savings in computation is achieved by avoiding the recomputation of the estimator for each bootstrap sample. Instead, a computationally efficient fixed-point estimation equation is analytically solved via a smart approximation following the Fast and Robust Bootstrap method (FRB) [M. Salibián-Barrera, S. Van Aelst, and G. Willems, “Fast and robust bootstrap,” Statist. Methods Appl., vol. 17, no. 1, pp. 41–71, 2008]. Our proposed bootstrap method facilitates the use of highly robust statistical methods in analyzing large scale data sets. The favorable statistical properties of the method are established analytically. Numerical examples demonstrate scalability, low complexity and robust statistical performance of the method in analyzing large data sets. [ABSTRACT FROM AUTHOR]

Complex elliptically symmetric (CES) distributions have been widely used in various engineering applications for which non-Gaussian models are needed. In this overview, circular CES distributions are surveyed, some new results are derived and their applications e.g., in radar and array signal processing are discussed and illustrated with theoretical examples, simulations and analysis of real radar data. The maximum likelihood (ML) estimator of the scatter matrix parameter is derived and general conditions for its existence and uniqueness, and for convergence of the iterative fixed point algorithm are established. Specific ML-estimators for several CES distributions that are widely used in the signal processing literature are discussed in depth, including the complex t-distribution, K-distribution, the generalized Gaussian distribution and the closely related angular central Gaussian distribution. A generalization of ML-estimators, the M-estimators of the scatter matrix, are also discussed and asymptotic analysis is provided. Applications of CES distributions and the adaptive signal processors based on ML- and M-estimators of the scatter matrix are illustrated in radar detection problems and in array signal processing applications for Direction-of-Arrival (DOA) estimation and beamforming. Furthermore, experimental validation of the usefulness of CES distributions for modelling real radar data is given. [ABSTRACT FROM PUBLISHER]

In this correspondence, we provide a multiple hypothesis test to detect the number of latent noncircular signals in a complex Gaussian random vector. Our method sequentially tests the results of individual generalized likelihood ratio test (GLRT) statistics with known asymptotic distributions to form the multiple hypothesis detector. Specifically, we are able to set a threshold yielding a precise probability of error. This test can be used to statistically determine if a given complex observation is circular Gaussian, and if not, how many latent signals in the observation are noncircular. Simulations are used to quantify the performance of the detector as compared to a detector based on the minimum description length (MDL) criterion. The utility of the detector is shown by applying it to a beamforming application using independent component analysis (ICA). [ABSTRACT FROM AUTHOR]

Complex elliptically symmetric (CES) distributions constitute a flexible and broad class of distributions for many engineering applications and include the widely used complex Gaussian, complex t-, complex generalized Gaussian and symmetric \alpha-stable distributions for example. Their careful statistical characterization is needed. Moreover, the mechanisms for generating random variables from these distributions are not well defined in literature. In this paper we provide such treatment in order to provide a better insight on their statistical properties and simplifying proofs and derivations. For example, insightful expressions for complex kurtosis coefficients of CES distributions are derived providing an interpretation on what complex kurtosis really measures. Also derived are asymptotic distributions of circularity measures, the sample circularity quotient and coefficient, assuming i.i.d. samples from an unspecified CES distribution. Also new Wald's type detectors of circularity are proposed that are valid within CES distributions with finite fourth-order moments. These results are accompanied with examples that are of interest in developing signal processing algorithms for complex-valued signals. [ABSTRACT FROM PUBLISHER]

This paper provides a rigorous statistical analysis of the deflation-based FastICA estimator, where the independent components (ICs) are extracted sequentially. The focus is on two aspects of the estimator: robustness against outliers as measured by the influence function (IF) and on its asymptotic relative efficiency (ARE) as measured by the ratio of the asymptotic variance of the FastICA w.r.t. the optimal maximum likelihood estimator (MLE). The derived compact closed-form expression of the IF reveals the vulnerability of the FastICA estimator to outliers regardless of the used nonlinearity. A cautionary finding is that even a moderate observation towards certain directions can render the estimator deficient in the sense that its separation performance degrades worse than a plain guess. The IF allows the derivation of a compact closed-form expression for the asymptotic covariance matrix of the FastICA estimator and subsequently its asymptotic relative efficiencies (AREs). The ARE figures calculated for some selected source distributions illustrate the fact that the order which the ICs are found is crucial as the accuracy of the previously extracted components can dominantly affect the accuracy of the successive deflation stages. [ABSTRACT FROM AUTHOR]

Abstract: An extension of the whitening transformation for complex random vectors, called the generalized uncorrelating transformation (GUT), is introduced. GUT is a generalization of the strong-uncorrelating transform [J. Eriksson, V. Koivunen, Complex-valued ICA using 2nd-order statistics, in: Proceedings of the IEEE Workshop on Machine Learning for Signal Processing (MLSP’04), Sao Luis, Brazil, 2004] based upon generalized estimators of the covariance and pseudo-covariance matrix, called the scatter matrix and spatial pseudo-scatter matrix, respectively. Depending on the selected scatter and spatial pseudo-scatter matrix, GUT estimators can have largely different statistical properties. Special emphasis is put on robust GUT estimators. We show that GUT is a separating matrix estimator for complex-valued independent component analysis (ICA) when at most one source random variable possess circularly symmetric distribution and sources do not have identical distribution. In the context of ICA, our approach is computationally attractive as it is based on straightforward matrix computations. Simulations and examples are used to confirm reliable performance of our method. [Copyright &y& Elsevier]

In this paper, we consider array processors that are scale-invariant functions of the array covariance matrix. The emphasis is on Capon's MVDR beamformer. We call such an array processor as scatter matrix based (SMB) array processor since the covariance matrix is required only up to a constant scalar and thus a scatter matrix (proportional to covariance under finite co- variance assumption) provides sufficient information. In order to establish interesting statistical robustness and large sample properties, we derive a general expression for the influence function and the asymptotic covariance structure of SMB-MVDR beam- former weights. Our results apply under the class of complex elliptically symmetric distributions, which includes the commonly used complex normal distribution as a special case. We illustrate the theory by deriving the IF and asymptotic relative efficiencies of the conventional SMB-MVDR beamformer that employs the sample covariance matrix and beamformers that employ robust M-estimators of scatter. Theoretical findings are confirmed by simulations. Our findings favor beamformers based upon M-estimators of scatter, since they combine a high efficiency with appealing robustness properties. [ABSTRACT FROM AUTHOR]

Despite of the increased interest in independent component analysis (ICA) during the past two decades, a simple closed form expression of the Cramér-Rao bound (CRB) for the demixing matrix estimation has not been established in the open literature. In the present paper we fill this gap by deriving a simple closed-form expression for the CRB of the demixing matrix directly from its definition. A simulation study comparing ICA estimators with the CRB is given. [ABSTRACT FROM AUTHOR]

In this paper, we derive a stochastic maximum-likelihood (ML) method for estimating spatio-temporal parameters for multiple-input multiple-output (M1MO) channels. Such estimators are needed in propagation studies where extensive channel measurements and sounding are required. These are seminal tasks in the process of developing advanced channel models. The proposed method employs an angular von Mises distribution model which is appropriate for angular data observed in channel measurement campaigns. The signal model is stochastic, and consequentially the method is particularly useful for estimation of the diffuse scattering component. This approach leads to lower complexity and faster convergence in comparison to deterministic models. These benefits are due to lower dimensionality of the model, leading to a simpler optimization problem. The statistical performance of the estimator is studied by establishing the Cramér-Rao lower bound (CRLB) and comparing the variances. The simulations show that the variance of the proposed estimation technique reaches the CRLB for relatively small sample size. The estimator is robust in the sense that meaningful results are obtained when applied to data generated by channel models other than the one used in its derivation. [ABSTRACT FROM AUTHOR]

We consider the affine equivariant sign covariance matrix (SCM) introduced by Visuri et al. (J. Statist. Plann. Inference 91 (2000) 557). The population SCM is shown to be proportional to the inverse of the regular covariance matrix. The eigenvectors and standardized eigenvalues of the covariance matrix can thus be derived from the SCM. We also construct an estimate of the covariance and correlation matrix based on the SCM. The influence functions and limiting distributions of the SCM and its eigenvectors and eigenvalues are found. Limiting efficiencies are given in multivariate normal and t-distribution cases. The estimates are highly efficient in the multivariate normal case and perform better than estimates based on the sample covariance matrix for heavy-tailed distributions. Simulations confirmed these findings for finite-sample efficiencies. [Copyright &y& Elsevier]

We introduce a new equivariant estimation method of the parameters of the multivariate regression model with q responses and p regressors. The estimate matrix is derived from the lift rank covariance matrix (LRCM) where the lift rank vectors are based on the Oja criterion function. The k = p + q variate ranks and k + 1 variate lift ranks are constructed using hyperplanes (or fits) going through k observations. The new LRCM regression estimate and the least squares (LS) estimate are shown to be weighted sums of the elemental estimates based on these hyperplanes. The LRCM regression estimate is equivariant and convergent, has a limiting multinormal distribution, and is highly efficient in the multivariate normal case. For heavy-tailed distributions, it performs better than the standard LS estimate. Estimation of the variance-covariance matrix of the LRCM estimate is briefly discussed. The theory is illustrated by simulations and a real data example. [ABSTRACT FROM AUTHOR]

A robust and sparse Direction of Arrival (DOA) estimator is derived for array data that follows a Complex Elliptically Symmetric (CES) distribution with zero-mean and finite second-order moments. The derivation allows to choose the loss function and four loss functions are discussed in detail: the Gauss loss which is the Maximum-Likelihood (ML) loss for the circularly symmetric complex Gaussian distribution, the ML-loss for the complex multivariate t -distribution (MVT) with ν degrees of freedom, as well as Huber and Tyler loss functions. For Gauss loss, the method reduces to Sparse Bayesian Learning (SBL). The root mean square DOA error of the derived estimators is discussed for Gaussian, MVT, and ϵ -contaminated data. The robust SBL estimators perform well for all cases and nearly identical with classical SBL for Gaussian array data. • SBL algorithm with general loss function for DOA M-estimation. • The CES data model includes Gaussian data as special case. • Robust and sparse DOA M-estimator is insensitive to heavy tails, outliers, and unknown source correlations. [ABSTRACT FROM AUTHOR]

We consider the problem of estimating high-dimensional covariance matrices of $K$ -populations or classes in the setting where the sample sizes are comparable to the data dimension. We propose estimating each class covariance matrix as a distinct linear combination of all class sample covariance matrices. This approach is shown to reduce the estimation error when the sample sizes are limited, and the true class covariance matrices share a somewhat similar structure. We develop an effective method for estimating the coefficients in the linear combination that minimize the mean squared error under the general assumption that the samples are drawn from (unspecified) elliptically symmetric distributions possessing finite fourth-order moments. To this end, we utilize the spatial sign covariance matrix, which we show (under rather general conditions) to be an asymptotically unbiased estimator of the normalized covariance matrix as the dimension grows to infinity. We also show how the proposed method can be used in choosing the regularization parameters for multiple target matrices in a single class covariance matrix estimation problem. We assess the proposed method via numerical simulation studies including an application in global minimum variance portfolio optimization using real stock data. [ABSTRACT FROM AUTHOR]

Abstract: Subject-level resting-state fMRI (RS-fMRI) spatial independent component analysis (sICA) may provide new ways to analyze the data when performed in the sliding time window. However, whether principal component analysis (PCA) and voxel-wise variance normalization (VN) are applicable pre-processing procedures in the sliding-window context, as they are for regular sICA, has not been addressed so far. Also model order selection requires further studies concerning sliding-window sICA. In this paper we have addressed these concerns. First, we compared PCA-retained subspaces concerning overlapping parts of consecutive temporal windows to answer whether in-window PCA and VN can confound comparisons between sICA analyses in consecutive windows. Second, we compared the PCA subspaces between windowed and full data to assess expected comparability between windowed and full-data sICA results. Third, temporal evolution of dimensionality estimates in RS-fMRI data sets was monitored to identify potential challenges in model order selection in a sliding-window sICA context. Our results illustrate that in-window VN can be safely used, in-window PCA is applicable with most window widths and that comparisons between windowed and full data should not be performed from a subspace similarity point of view. In addition, our studies on dimensionality estimates demonstrated that there are sustained, periodic and very case-specific changes in signal-to-noise ratio within RS-fMRI data sets. Consequently, dimensionality estimation is needed for well-founded model order determination in the sliding-window case. The observed periodic changes correspond to a frequency band of ≤0.1 Hz, which is commonly associated with brain activity in RS-fMRI and become on average most pronounced at window widths of 80 and 60 time points (144 and 108 s, respectively). Wider windows provided only slightly better comparability between consecutive windows, and 60 time point or shorter windows also provided the best comparability with full-data results. Further studies are needed to determine the cause for dimensionality variations. [Copyright &y& Elsevier]

• We formulate graph error models for the adjacency matrix, which help to quantify the deviation from the true matrix using a few parameters. • We study the structural effects of the proposed error models on the adjacency matrix. • The effects of different type of errors in adjacency matrix specification are illustrated in filtering of graph signal and ICA of graph signals. The first step for any graph signal processing (GSP) procedure is to learn the graph signal representation, i.e., to capture the dependence structure of the data into an adjacency matrix. Indeed, the adjacency matrix is typically not known a priori and has to be learned. However, it is learned with errors. A little attention has been paid to modelling such errors in the adjacency matrix, and studying their effects on GSP methods. However, modelling errors in the adjacency matrix will enable both to study the graph error effects in GSP and to develop robust GSP algorithms. In this paper, we therefore introduce practically justifiable graph error models. We also study, both analytically when possible and numerically, the graph error effect on the performance of GSP methods in different types of problems such as filtering of graph signals and independent component analysis of graph signals (graph decorrelation). [ABSTRACT FROM AUTHOR]

• A general learning-based decomposition model suitable for fusing images from various imaging modalities is proposed. • The multimodal images are decomposed into correlated and uncorrelated components. • A CDL method based on simultaneous sparse approximation is proposed for estimating the correlated features. • The uncorrelated components are estimated using a Pearson correlation-based constraint. This paper presents a multimodal image fusion method using a novel decomposition model based on coupled dictionary learning. The proposed method is general and can be used for a variety of imaging modalities. In particular, the images to be fused are decomposed into correlated and uncorrelated components using sparse representations with identical supports and a Pearson correlation constraint, respectively. The resulting optimization problem is solved by an alternating minimization algorithm. Contrary to other learning-based fusion methods, the proposed approach does not require any training data, and the correlated features are extracted online from the data itself. By preserving the uncorrelated components in the fused images, the proposed fusion method significantly improves on current fusion approaches in terms of maintaining the texture details and modality-specific information. The maximum-absolute-value rule is used for the fusion of correlated components only. This leads to an enhanced contrast-resolution without causing intensity attenuation or loss of important information. Experimental results show that the proposed method achieves superior performance in terms of both visual and objective evaluations compared to state-of-the-art image fusion methods. [ABSTRACT FROM AUTHOR]

Abstract: Spatial independent components analysis (sICA) has become a widely applied data-driven method for fMRI data, especially for resting-state studies. These sICA approaches are often based on iterative estimation algorithms and there are concerns about accuracy due to noise. Repeatability measures such as ICASSO, RAICAR and ARABICA have been introduced as remedies but information on their effects on estimates is limited. The contribution of this study was to provide more of such information and test if the repeatability analyses are necessary. We compared FastICA-based ordinary and repeatability approaches concerning mixing vector estimates. Comparisons included original FastICA, FSL4 Melodic FastICA and original and modified ICASSO. The effects of bootstrapping and convergence threshold were evaluated. The results show that there is only moderate improvement due to repeatability measures and only in the bootstrapping case. Bootstrapping attenuated power from time courses of resting-state network related ICs at frequencies higher than 0.1Hz and made subsets of low frequency oscillations more emphasized IC-wise. The convergence threshold did not have a significant role concerning the accuracy of estimates. The performance results suggest that repeatability measures or strict converge criteria might not be needed in sICA analyses of fMRI data. Consequently, the results in existing sICA fMRI literature are probably valid in this sense. A decreased accuracy of original bootstrapping ICASSO was observed and corrected by using centrotype mixing estimates but the results warrant for thorough evaluations of data-driven methods in general. Also, given the fMRI-specific considerations, further development of sICA methods is strongly encouraged. [Copyright &y& Elsevier]

Abstract: In this paper, the influence functions and limiting distributions of the canonical correlations and coefficients based on affine equivariant scatter matrices are developed for elliptically symmetric distributions. General formulas for limiting variances and covariances of the canonical correlations and canonical vectors based on scatter matrices are obtained. Also the use of the so-called shape matrices in canonical analysis is investigated. The scatter and shape matrices based on the affine equivariant Sign Covariance Matrix as well as the Tyler''s shape matrix serve as examples. Their finite sample and limiting efficiencies are compared to those of the Minimum Covariance Determinant estimators and S-estimator through theoretical and simulation studies. The theory is illustrated by an example. [Copyright &y& Elsevier]