Search

In this paper, we extend the bilinear generalized approximate message passing (BiG-AMP) approach, originally proposed for high-dimensional generalized bilinear regression, to the multi-layer case for the handling of cascaded problem such as matrix-factorization problem arising in relay communication among others. Assuming statistically independent matrix entries with known priors, the new algorithm called ML-BiGAMP could approximate the general sum-product loopy belief propagation (LBP) in the high-dimensional limit enjoying a substantial reduction in computational complexity. We demonstrate that, in large system limit, the asymptotic MSE performance of ML-BiGAMP could be fully characterized via a set of simple one-dimensional equations termed state evolution (SE). We establish that the asymptotic MSE predicted by ML-BiGAMP’ SE matches perfectly the exact MMSE predicted by the replica method, which is well-known to be Bayes-optimal but infeasible in practice. This consistency indicates that the ML-BiGAMP may still retain the same Bayes-optimal performance as the MMSE estimator in high-dimensional applications, although ML-BiGAMP's computational burden is far lower. As an illustrative example of the general ML-BiGAMP, we provide a detector design that could estimate the channel fading and the data symbols jointly with high precision for the two-hop amplify-and-forward relay communication systems. [ABSTRACT FROM AUTHOR]

The first moment and second central moments of the portfolio return, a.k.a. mean and variance, have been widely employed to assess the expected profit and risk of the portfolio. Investors pursue higher mean and lower variance when designing the portfolios. The two moments can well describe the distribution of the portfolio return when it follows the Gaussian distribution. However, the real world distribution of assets return is usually asymmetric and heavy-tailed, which is far from being a Gaussian distribution. The asymmetry and the heavy-tailedness are characterized by the third and fourth central moments, i.e., skewness and kurtosis, respectively. Higher skewness and lower kurtosis are preferred to reduce the probability of extreme losses. However, incorporating high-order moments in the portfolio design is very difficult due to their non-convexity and rapidly increasing computational cost with the dimension. In this paper, we propose a very efficient and convergence-provable algorithm framework based on the successive convex approximation (SCA) algorithm to solve high-order portfolios. The efficiency of the proposed algorithm framework is demonstrated by the numerical experiments. [ABSTRACT FROM AUTHOR]

Orthogonal greedy algorithms are popular sparse signal reconstruction algorithms. Their principle is to select atoms one by one. A series of unconstrained least-square subproblems of gradually increasing size is solved to compute the approximation coefficients, which is efficiently performed using a fast recursive update scheme. When dealing with non-negative sparse signal reconstruction, a series of non-negative least-squares subproblems have to be solved. Fast implementation becomes tricky since each subproblem does not have a closed-form solution anymore. Recently, non-negative extensions of the classical orthogonal matching pursuit and orthogonal least squares algorithms were proposed, using slow (i.e., non-recursive) or recursive but inexact implementations. In this paper, we revisit these algorithms in a unified way. We define a class of non-negative orthogonal greedy algorithms and exhibit their structural properties. We propose a fast and exact implementation based on the active-set resolution of non-negative least-squares and exploiting warm start initializations. The algorithms are assessed in terms of accuracy and computational complexity for a sparse spike deconvolution problem. We also present an application to near-infrared spectra decomposition. [ABSTRACT FROM AUTHOR]

Many resource allocation tasks are challenging global (i.e., non-convex) optimization problems. The main issue is that the computational complexity of these problems grows exponentially in the number of variables instead of polynomially as for many convex optimization problems. However, often the non-convexity stems only from a subset of variables. Conventional global optimization frameworks like monotonic optimization or DC programming treat all variables as global variables and require complicated, problem specific decomposition approaches to exploit the convexity in some variables. To overcome this challenge, we develop an easy-to-use algorithm that inherently differentiates between convex and non-convex variables, preserving the low computational complexity in the number of convex variables. Another issue with these widely used frameworks is that they may suffer from severe numerical problems. We discuss this issue in detail and provide a clear motivating example. The solution to this problem is to replace the traditional approach of finding an $\varepsilon $ -approximate solution by the novel concept of $\varepsilon $ -essential feasibility. The underlying algorithmic approach is called successive incumbent transcending (SIT) algorithm and builds the foundation of our developed algorithm. A further highlight of this algorithm is that it inherently treats fractional objectives making the use of Dinkelbach's iterative algorithm obsolete. Numerical experiments show a speed-up of four orders of magnitude over state-of-the-art algorithms and almost three orders of magnitude of additional speed-up over Dinkelbach's algorithm for fractional programs. [ABSTRACT FROM AUTHOR]

In sketched clustering, a dataset of $T$ samples is first sketched down to a vector of modest size, from which the centroids are subsequently extracted. Its advantages include 1) reduced storage complexity and 2) centroid extraction complexity independent of $T$. For the sketching methodology recently proposed by Keriven et al., which can be interpreted as a random sampling of the empirical characteristic function, we propose a sketched clustering algorithm based on approximate message passing. Numerical experiments suggest that our approach is more efficient than the state-of-the-art sketched clustering algorithm “CL-OMPR” (in both computational and sample complexity) and more efficient than k-means++ when $T$ is large. [ABSTRACT FROM AUTHOR]

In this paper, a computationally efficient algorithm for Bayesian joint change point detection (CPD) in multiple time series is presented. The data generation model includes a number of change configurations (CC), each affecting a unique subset of the time series, which introduces correlation between the positions of change points (CPs) in the monitored time series. The inference objective is to identify joint changes and the associated CC. The algorithm consists of two stages: First, a univariate CPD algorithm is applied separately to each of the involved time series. The outcomes of this step are maximum a posteriori (MAP) detected CPs and posterior distributions of CPs conditioned on the MAP CPs. These outcomes are used in combination to approximate the posterior for the CCs. In the second algorithm stage, dynamic programming is used to find the maxima of this approximate CC posterior. The algorithm is applied to synthetic data, and it is shown to be both significantly faster and more accurate compared to a previously proposed algorithm designed to solve similar problems. Also, the initial algorithm is extended with steps from the maximization–maximization algorithm, which allows the hyperparameters of the data generation model to be estimated jointly with the CCs, and we show that these estimates coincide with estimates obtained from a Markov chain Monte Carlo algorithm. [ABSTRACT FROM AUTHOR]

This paper considers the reconstruction of a discrete-valued random vector from possibly underdetermined linear measurements using sum-of-absolute-value (SOAV) optimization. The proposed algorithm, referred to as discreteness-aware approximate message passing (DAMP), is based on the idea of approximate message passing (AMP), which has been originally proposed for compressed sensing. The DAMP algorithm has low computational complexity and its performance in the large system limit can be predicted analytically via state evolution framework, where we provide a condition for the exact reconstruction with DAMP in the noise-free case. From the analysis, we also propose a method to determine the parameters of the SOAV optimization. Moreover, based on the state evolution, we provide Bayes optimal DAMP, which has the minimum mean-square-error at each iteration of the algorithm. Simulation results show that the DAMP algorithms can reconstruct the discrete-valued vector from underdetermined linear measurements and the empirical performance agrees with our theoretical results in large-scale systems. When the problem size is not large enough, the SOAV optimization with the proposed parameters can achieve better performance than the DAMP algorithms for high signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

An algorithm for the estimation of multiple targets from partial and corrupted observations is introduced based on the concept of a partially distinguishable multitarget system. It combines the advantages of engineering solutions like multiple hypothesis tracking with the rigor of point-process-based methods. It is demonstrated that under intuitive assumptions and approximations, the complexity of the proposed multitarget estimation algorithm can be made linear in terms of the number of tracks and the number of observations, while naturally preserving distinct tracks for detected targets, unlike point-process-based methods. [ABSTRACT FROM AUTHOR]

This paper presents an approximate minimum mean squared error (MMSE) adaptive beam-pattern design (ABD) method for MIMO active sensing systems. The proposed approximate MMSE ABD method leverages the physics of the MIMO arrays to provide a linear-Gaussian approximation that is specific to MIMO active sensing systems, and yields a computationally simple optimization problem. Computational complexity analysis confirms this theoretical reduction in the number of floating-point operations required, most notably that evaluation of the proposed approximate optimization cost function grows polynomially with the number of targets being tracked, whereas for evaluation of the exact cost the growth is exponential. Additionally, numerical results indicate that, even for a simple scenario with a single target being tracked, the proposed approximate MMSE ABD method does indeed reduce the mean squared error of target parameter estimation compared to the nonadaptive case, with a reduction in computation time of four orders of magnitude compared to exact MMSE ABD. [ABSTRACT FROM AUTHOR]

BigBand is a technique for wideband spectrum sensing (WSS) that can capture gigahertz of sparse spectrum in real time by using shifted multiple channels without sampling the signal at GS/s. In this paper, we focus on enforcing BigBand with the proposed improved algorithm, namely, AD-BigBand, for efficient and robust computation. AD-BigBand converts the nonlinear process (locating frequency and computing the spectrum value) of sparse WSS into two steps of solving linear equations. In particular, we define an r -degree frequency locator polynomial to rapidly locate nonzero frequencies. The computing complexity of the proposed method decreases from O(K{\log ^2}K ({{{n\log K} \over K}})^{\log K}) of shifted sampling based BigBand to O(K{\log ^2}K)$, where the $K$-sparse signals of bandwitdth $n$ are subsampled with arithmetic time-shifted sensing channels. We implement AD-BigBand on a commercial four-channel acquisition card. Experimental results show that AD-BigBand improves the WSS capabilities and exhibits higher processing speed and lower error spectrum construction rate than BigBand, especially in the case of relative low signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

In this paper, we propose a strategy that is based on expectation maximization for tracking multiple point targets. The algorithm is similar to probabilistic multi-hypothesis tracking (PMHT) but does not relax the point target model assumptions. According to the point target models, a target can generate at most one measurement, and a measurement is generated by at most one target. With this model assumption, we show that the proposed algorithm can be implemented as iterations of Rauch-Tung-Striebel (RTS) smoothing for state estimation, and the loopy belief propagation method for marginal data association probabilities calculation. Using example illustrations with tracks, we compare the proposed algorithm with PMHT and joint probabilistic data association (JPDA) and show that PMHT and JPDA exhibit coalescence when there are closely moving targets whereas the proposed algorithm does not. Furthermore, extensive simulations c comparing the mean optimal subpattern assignment (MOSPA) performance of the algorithm for different scenarios averaged over several Monte Carlo iterations show that the proposed algorithm performs better than JPDA and PMHT. We also compare it to benchmarking algorithm: $N$-scan pruning based track-oriented multiple hypothesis tracking (TOMHT). The proposed algorithm shows a good tradeoff between computational complexity and the MOSPA performance. [ABSTRACT FROM PUBLISHER]

We present a novel method for the design of finite impulse response (FIR) filters with discrete coefficients that belong in the sum of powers-of-two (POT) space. The importance of this class of filters cannot be overstated, given the ever-increasing number of applications for which a specific hardware implementation is needed. Filters that have coefficients that belong to such a class are also referred to as multiplierless filters, given that the operations performed by the filter can all be implemented by using appropriately designed shifts of the input data, making them a perfect choice whenever implementation simplicity and processing speed are the ultimate goal. To produce such a design, we employ a vector successive approximation technique successfully used in data compression that has a very low computational complexity, the Matching Pursuits Generalized BitPlanes algorithm (MPGBP). We derive optimality conditions for the approximation dictionary. We compare filters obtained with the proposed method with those derived in previous works. Based on this comparative analysis, we show that this new and powerful way of producing the filters' coefficients is also among the simplest available in the literature. [ABSTRACT FROM PUBLISHER]

A linearly-constrained recursive least-squares adaptive filtering algorithm based on the method of weighting and the dichotomous coordinate descent (DCD) iterations is proposed. The method of weighting is employed to incorporate the linear constraints into the least-squares problem. The normal equations of the resultant unconstrained least-squares problem are then solved using the DCD iterations. The proposed algorithm has a significantly smaller computational complexity than the previously proposed constrained recursive least square (CRLS) algorithm while delivering convergence performance on par with CRLS. The effectiveness of the proposed algorithm is demonstrated by simulation examples. [ABSTRACT FROM PUBLISHER]

This paper presents two new iterative multiuser detection algorithms for spectrally efficient frequency division multiple access continuous phase modulation in additive white Gaussian noise. The detectors are derived from the sum-product algorithm and the factor graph framework, performing approximate maximum a posteriori (MAP) bit detection. A factor graph of the actual multiuser detection problem is considered, rather than only factor graphs of the individual single user detection problems combined with ad-hoc inter-user interference cancellation. The practical implementation of the detectors is based on the assumption that a suitable set of sum-product messages can be approximated by a Gaussian distribution. This methodology allows to considerably reduce the computational complexity and memory size requirements as compared to a straightforward application of the sum-product rules. The proposed algorithms further succeed in achieving a good error performance. [ABSTRACT FROM PUBLISHER]

While loopy belief propagation (LBP) performs reasonably well for inference in some Gaussian graphical models with cycles, its performance is unsatisfactory for many others. In particular for some models LBP does not converge, and in general when it does converge, the computed variances are incorrect (except for cycle-free graphs for which belief propagation (BP) is non-iterative and exact). In this paper we propose feedback message passing (FMP), a message-passing algorithm that makes use of a special set of vertices (called a feedback vertex set or FVS) whose removal results in a cycle-free graph. In FMP, standard BP is employed several times on the cycle-free subgraph excluding the FVS while a special message-passing scheme is used for the nodes in the FVS. The computational complexity of exact inference is O(k2n), where is the number of feedback nodes, and is the total number of nodes. When the size of the FVS is very large, FMP is computationally costly. Hence we propose approximate FMP, where a pseudo-FVS is used instead of an FVS, and where inference in the non-cycle-free graph obtained by removing the pseudo-FVS is carried out approximately using LBP. We show that, when approximate FMP converges, it yields exact means and variances on the pseudo-FVS and exact means throughout the remainder of the graph. We also provide theoretical results on the convergence and accuracy of approximate FMP. In particular, we prove error bounds on variance computation. Based on these theoretical results, we design efficient algorithms to select a pseudo-FVS of bounded size. The choice of the pseudo-FVS allows us to explicitly trade off between efficiency and accuracy. Experimental results show that using a pseudo-FVS of size no larger than log (n), this procedure converges much more often, more quickly, and provides more accurate results than LBP on the entire graph. [ABSTRACT FROM PUBLISHER]

This paper is concerned with the challenging and timely problem of joint channel estimation and equalization for orthogonal frequency division multiplexing (OFDM) systems in the presence of frequency selective and very rapidly time varying channels. The resulting algorithm is based on the space alternating generalized expectation maximization-maximum a posteriori probability (SAGE-MAP) technique which is particularly well suited to multicarrier signal formats. The algorithm is implemented in the time-domain which enables one to use the Gaussian approximation of the transmitted OFDM samples. Consequently, the averaging process of the nonpilot data symbols becomes analytically possible resulting in a feasible and computationally efficient channel estimation algorithm leading to a receiver structure that yields also an equalized output from which the data symbols are detected with excellent symbol error rate (SER) performance. Based on this Gaussian approximation the exact Bayesian Cramér Rao lower bound (CRLB) as well as the convergence rate of the algorithm are derived analytically. To reduce the computational complexity of the algorithm, discrete Legendre orthogonal basis functions are employed to represent the rapidly time-varying fading channel. It is shown that, depending on the normalized Doppler frequency, only a small number of expansion coefficients is sufficient to approximate the channel very well and there is no need to know the correlation function of the input signal. The computational complexity of the algorithm is shown to be ~O(NL) per detected data symbol and per SAGE-MAP algorithm cycle where is the number of OFDM subcarriers and is the number of multipath components. [ABSTRACT FROM PUBLISHER]

High-resolution spectral Doppler is an important and powerful noninvasive tool for estimation of velocities in blood vessels using medical ultrasound scanners. Such estimates are typically formed using an averaged periodogram technique, resulting in well-known limitations in the resulting spectral resolution. Recently, we have proposed techniques to instead form high-resolution data-adaptive estimates exploiting measurements along both depth and emission. The resulting estimates gives noticeably superior velocity estimates as compared to the standard technique, but suffers from a high computational complexity, making it interesting to formulate computationally efficient implementations of the estimators. In this work, by exploiting the rich structure of the iterative adaptive approach (IAA) based estimator, we examine how these estimates can be efficiently implemented in a time-recursive manner using both exact and approximate formulations of the method. The resulting algorithms are shown to reduce the necessary computational load with several orders of magnitude without noticeable loss of performance. [ABSTRACT FROM PUBLISHER]

Intercell interference is the major limiting factor in wireless multicell networks. Recently, it has been shown that significant performance gains can be achieved by cooperation among base stations. Different degrees of cooperation are possible. In this paper, cooperation in the form of intercell interference management is considered. The base stations are equipped with multiple antennas, while the mobile terminals only have a single antenna. The base stations jointly coordinate beamforming and scheduling, and thus perform a weaker form of cooperation among base stations than suggested in recently proposed joint transmission techniques. The terminals treat intercell interference as noise. The corresponding resource allocation problem is cast as a utility maximization problem, which includes common performance objectives such as the arithmetic mean, the geometric mean, and the max-min operation of achievable user rates. The resulting utility maximization problem is a nonconvex optimization problem. After a suitable reformulation, the problem can be solved to global optimality using the framework of monotonic optimization. Although, the numerical complexity of the proposed method is exponential, the reformulation step leads to an optimization problem which only scales in the number of mobile terminals instead of the entire set of physical layer parameters. In essence, the proposed framework represents a powerful tool for computing benchmarks for certain scenarios and utility functions under jointly optimal beamforming and scheduling. [ABSTRACT FROM PUBLISHER]

In this correspondence, we develop superfast approximative one-dimensional algorithms for the computationally efficient implementation of the recent iterative adaptive approach (IAA) spectral estimate. The proposed methods are based on rewriting the IAA algorithm with suitable Gohberg–Semencul representations, solving the resulting linear systems of equations using the preconditioned conjugate gradient method, where a novel preconditioning is applied using an incomplete factorization of the Toeplitz matrix. Numerical simulations illustrate the efficiency of both the proposed preconditioning as well as the overall algorithm, offering a computational reduction of up to two orders of magnitude as compared to our recently proposed efficient and exact IAA implementation. [ABSTRACT FROM PUBLISHER]

Widely distributed multiple radar architectures offer parameter estimation improvement for target localization. For a large number of radars, with full resource allocation, the achievable localization minimum estimation mean-square error (MSE) may extend beyond the system predetermined performance goals. In this paper, performance driven resource allocation schemes for multiple radar systems are proposed. Two operational policies are considered. In the first, the number of transmit and receive antennas employed in the estimation process is minimized by effectively selecting a subset of active antennas such that the required MSE performance threshold is attained. In the second, an optimal subset of active antennas of predetermined size is selected such that the localization MSE is minimized. These problems are formulated in a combinatorial optimization framework as a knapsack problem (KP), where the goal is to obtain a performance level with the lowest cost, in terms of active system elements. The Cramer-Rao bound (CRB) is used as a performance metric. Cost parameters, representing operational cost or any other utilization constraints on the antennas, are associated with each of the radars. These are incorporated in the KP formulation, integrating decision making factors in the selection process. Antenna subset selection is implemented through a heuristic algorithm, by successively selecting antennas so as to minimize the performance gap between the temporal CRB and a given MSE goal or a given subset size. The proposed approximate algorithms offer considerable reduction in computational complexity when compared with an exhaustive search. By minimizing the number of operational antennas needed to complete the task, this concept introduces savings in both communication link needs and central processing load, in addition to the operational ones. [ABSTRACT FROM PUBLISHER]

We develop a novel sampling theorem on the sphere and corresponding fast algorithms by associating the sphere with the torus through a periodic extension. The fundamental property of any sampling theorem is the number of samples required to represent a band-limited signal. To represent exactly a signal on the sphere band-limited at L, all sampling theorems on the sphere require \cal O(L^2) samples. However, our sampling theorem requires less than half the number of samples of other equiangular sampling theorems on the sphere and an asymptotically identical, but smaller, number of samples than the Gauss-Legendre sampling theorem. The complexity of our algorithms scale as \cal O(L^3), however, the continual use of fast Fourier transforms reduces the constant prefactor associated with the asymptotic scaling considerably, resulting in algorithms that are fast. Furthermore, we do not require any precomputation and our algorithms apply to both scalar and spin functions on the sphere without any change in computational complexity or computation time. We make our implementation of these algorithms available publicly and perform numerical experiments demonstrating their speed and accuracy up to very high band-limits. Finally, we highlight the advantages of our sampling theorem in the context of potential applications, notably in the field of compressive sampling. [ABSTRACT FROM AUTHOR]

In the past, ad hoc methods have been used to choose gains in proportionate-type normalized least mean-square algorithms without strong theoretical under-pinnings. In this correspondence, a theoretical framework and motivation for adaptively choosing gains is presented, such that the mean-square error will be minimized at any given time. As a result of this approach, a new optimal proportionate-type normalized least mean-square algorithm is introduced. A computationally simplified version of the theoretical optimal algorithm is derived as well. Both of these algorithms require knowledge of the mean-square weight deviations. Feasible implementations, which estimate the mean-square weight deviations, are presented. The performance of these new feasible algorithms are compared to the performance of standard adaptive algorithms when operating with sparse, non-sparse, and time-varying impulse responses, when the input signal is white. Specifically, we consider the transient and steady-state mean-square errors as well as the overall computational complexity of each algorithm. [ABSTRACT FROM AUTHOR]