Robustness and performance analysis of subspace-based DOA estimation for rectilinear correlated sources in CES data model.

• This paper has shown that all the NC subspace-based algorithms built from the SCM designed for uncorrelated rectilinear sources embedded in spatially white CCG noise can be also applied for correlated rectilinear sources in the contexts of SCM estimate with C-CES noise and M-estimate with NC-CES observations. • A perturbation analysis has been performed to derive closed-form expressions for the asymptotic covariance matrices of DOA estimates for three NC MUSIC-like algorithms in two CES data models. • Interpretable closed-form expressions of the asymptotic variance of the estimated DOA of two equi-power correlated sources has been derived for the first time. • A number of properties that highlight how the asymptotic variances of NC MUSIClike DOA estimation algorithms depend on key parameters such as SNR, DOA and phase separations, correlation factor and C-CES noise parameters were derived. • Analytical robustness results were illustrated proving that the use of robust Mestimators enhances the robustness of the subspace-based DOA estimation algorithms against heavy-tailed C-CES observations model deviations, with negligibleloss in performance for C-CG distributed observations. This paper focuses on a theoretical performance analysis of subspace-based algorithms for the localization of spatially correlated rectilinear sources embedded in circular complex elliptically symmetric (C-CES) distributed noise model and also when the observations are non-circular CES (NC-CES) distributed with dependent scatter matrices on the direction of arrival (DOA) parameters. A perturbation analysis has been performed to derive closed-form expressions for the asymptotic covariance matrices of DOA estimates for non-circular subspace-based algorithms in two CES data models. Robustness of subspace-based algorithms is theoretical evaluated using robust covariance matrix estimators (instead of the sample covariance matrix (SCM)). We prove, for the first time, interpretable closed-form expressions of the asymptotic variance of the estimated DOA of two equi-power correlated sources, which allows us to derive a number of properties describing the DOA variance's dependence on signals parameters and non-Gaussian distribution of the noise. Different robustness properties are theoretically analyzed. In particular, we prove in the framework of NC-CES distributed observations, that Tyler's M -estimator enhances the performance for heavy-tailed distributions w.r.t. the SCM, with negligible loss in performance for circular Gaussian distributed observations. Finally, some Monte Carlo illustrations are given for quantifying this robustness and specifying the domain of validity of our theoretical asymptotic results. [ABSTRACT FROM AUTHOR]