Search

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

This paper studies the convergence properties of the well-known message-passing algorithm for convex optimisation. Under the assumption of pairwise separability and scaled diagonal dominance, asymptotic convergence is established and a simple bound for the convergence rate is provided for message-passing. In comparison with previous results, our results do not require the given convex program to have known convex pairwise components and that our bound for the convergence rate is tighter and simpler. When specialised to quadratic optimisation, we generalise known results by providing a very simple bound for the convergence rate. [ABSTRACT FROM AUTHOR]

The emergence of big data has caused a dramatic shift in the operating regime for optimization algorithms. The performance bottleneck, which used to be computations, is now often communications. Several gradient compression techniques have been proposed to reduce the communication load at the price of a loss in solution accuracy. Recently, it has been shown how compression errors can be compensated for in the optimization algorithm to improve the solution accuracy. Even though convergence guarantees for error-compensated algorithms have been established, there is very limited theoretical support for quantifying the observed improvements in solution accuracy. In this paper, we show that Hessian-aided error compensation, unlike other existing schemes, avoids accumulation of compression errors on quadratic problems. We also present strong convergence guarantees of Hessian-based error compensation for stochastic gradient descent. Our numerical experiments highlight the benefits of Hessian-based error compensation, and demonstrate that similar convergence improvements are attained when only a diagonal Hessian approximation is used. [ABSTRACT FROM AUTHOR]

In this paper we consider maximum-likelihood (ML) MIMO detection under one-bit quantized observations and binary symbol constellations. This problem is motivated by the recent interest in adopting coarse quantization in massive MIMO systems—as an effective way to scale down the hardware complexity and energy consumption. Classical MIMO detection techniques consider unquantized observations, and many of them are not applicable to the one-bit MIMO case. We develop a new non-convex optimization algorithm for the one-bit ML MIMO detection problem, using a strategy called homotopy optimization. The idea is to transform the ML problem into a sequence of approximate problems, from easy (convex) to hard (close to ML), and with each problem being a gradual modification of its previous. Then, our attempt is to iteratively trace the solution path of these approximate problems. This homotopy algorithm is well suited to the application of deep unfolding, a recently popular approach for turning certain model-based algorithms into data-driven, and performance enhanced, ones. While our initial focus is on one-bit MIMO detection, the proposed technique also applies naturally to the classical unquantized MIMO detection. We performed extensive simulations and show that the proposed homotopy algorithms, both non-deep and deep, have satisfactory bit-error probability performance compared to many state-of-the-art algorithms. Also, the deep homotopy algorithm has attractively low computational complexity. [ABSTRACT FROM AUTHOR]

Phase retrieval is an inverse problem which consists in recovering an unknown signal from a set of absolute squared projections. Recently, gradient descent algorithms have been developed to solve this problem. However, their optimization cost functions are non-convex and non-smooth. To address the non-smoothness of the cost function, some of these methods use truncation thresholds to calculate a truncated step gradient direction. But, the truncation requires designing parameters to obtain a desired performance in the phase recovery, which drastically modifies the search direction update, increasing the sampling complexity. Therefore, this paper develops the Phase Retrieval Smoothing Conjugate Gradient method (PR-SCG) which uses a smoothing function to retrieve the signal. PR-SCG is based on the smooth-ing projected gradient method which is useful for non-convex optimization problems. PR-SCG uses a nonlinear conjugate gradient of the smoothing function as the search direction to accelerate the convergence. Furthermore, the incremental Stochastic Smoothing Phase Retrieval algorithm (SSPR) is developed. SSPR involves a single equation per iteration which results in a simple, scalable, and fast approach useful when the size of the signal is large. Also, it is shown that SSPR converges linearly to the true signal, up to a global unimodular constant. Additionally, the proposed methods do not require truncation parameters. Simulation results are provided to validate the efficiency of PR-SCG and SSPR compared to existing phase retrieval algorithms. It is shown that PR-SCG and SSPR are able to reduce the number of measurements and iterations to recover the phase, compared with recently developed algorithms. [ABSTRACT FROM AUTHOR]

This paper considers unimodular sequence synthesis under similarity constraint for both the continuous and discrete phase cases. A computationally efficient iterative algorithm for the continuous phase case (IA-CPC) is proposed to sequentially optimize the quadratic objective function. The quadratic optimization problem is turned into multiple one-dimensional optimization problems with closed-form solutions. For the discrete phase case, we present an iterative block optimization algorithm. Specifically, we partition the design variables into $K$ blocks, and then, we sequentially optimize each block via exhaustive search while fixing the remaining $K-1$ blocks. Finally, we evaluate the computational costs and performance gains of the proposed algorithms in comparison with power method-like and semidefinite relaxation related techniques. [ABSTRACT FROM PUBLISHER]

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g., a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both nonadaptive and adaptive filter identification problems. [ABSTRACT FROM PUBLISHER]

Performance evaluation of multi-target tracking algorithms is of great practical importance in the design, parameter optimization and comparison of tracking systems. The goal of performance evaluation is to measure the distance between two sets of tracks: the ground truth tracks and the set of estimated tracks. This paper proposes a mathematically rigorous metric for this purpose. The basis of the proposed distance measure is the recently formulated consistent metric for performance evaluation of multi-target filters, referred to as the OSPA metric. Multi-target filters sequentially estimate the number of targets and their position in the state space. The OSPA metric is therefore defined on the space of finite sets of vectors. The distinction between filtering and tracking is that tracking algorithms output tracks and a track represents a labeled temporal sequence of state estimates, associated with the same target. The metric proposed in this paper is therefore defined on the space of finite sets of tracks and incorporates the labeling error. Numerical examples demonstrate that the proposed metric behaves in a manner consistent with our expectations. [ABSTRACT FROM PUBLISHER]

In this paper, we consider a robust lattice alignment design for K-user quasi-static multiple-input multiple-output (MIMO) interference channels with imperfect channel knowledge. With random Gaussian inputs, the conventional interference alignment (IA) method has the feasibility problem when the channel is quasi-static. On the other hand, structured lattices can create structured interference as opposed to the random interference caused by random Gaussian symbols. The structured interference space can be exploited to transmit the desired signals over the gaps. However, the existing alignment methods on the lattice codes for quasi-static channels either require infinite signal-to-noise ratio (SNR) or symmetric interference channel coefficients. Furthermore, perfect channel state information (CSI) is required for these alignment methods, which is difficult to achieve in practice. In this paper, we propose a robust lattice alignment method for quasi-static MIMO interference channels with imperfect CSI at all SNR regimes, and a two-stage decoding algorithm to decode the desired signal from the structured interference space. We derive the achievable data rate based on the proposed robust lattice alignment method, where the design of the precoders, decorrelators, scaling coefficients and interference quantization coefficients is jointly formulated as a mixed integer and continuous optimization problem. The effect of imperfect CSI is also accommodated in the optimization formulation, and hence the derived solution is robust to imperfect CSI. We also design a low complex iterative optimization algorithm for our robust lattice alignment method by using the existing iterative IA algorithm that was designed for the conventional IA method. Numerical results verify the advantages of the proposed robust lattice alignment method compared with the time-division multiple-access (TDMA), two-stage maximum-likelihood (ML) decoding, generalized Han–Kobayashi (HK), distributive IA and conventional IA methods in the literature. [ABSTRACT FROM PUBLISHER]

Achieving the boundary of the capacity region in the multiple-input multiple-output (MIMO) broadcast channel requires the use of dirty paper coding (DPC). As practical nearly optimum implementations of DPC are computationally complex, purely linear approaches are often used instead. However, in this case, the problem of maximizing a weighted sum rate constitutes a nonconvex and, in most cases, also a combinatorial optimization problem. In this paper, we present two heuristic nearly optimum algorithms with reduced computational complexity. For this purpose, a lower bound for the weighted sum rate under linear zero-forcing constraints is used. Based on this bound, both greedy algorithms successively allocate data streams to users. In each step, the user is determined that is given an additional data stream such that the increase in weighted sum rate becomes maximum. Thereby, the data stream allocations and filters obtained in the previous steps are kept fixed and only the filter corresponding to the additional data stream is optimized. The first algorithm determines the receive and transmit filters directly in the downlink. The other algorithm operates in the dual uplink, from which the downlink transmit and receive filters can be obtained via the general rate duality leading to nonzero-forcing in the downlink. Simulation results reveal marginal performance losses compared to more complex algorithms. [ABSTRACT FROM AUTHOR]

This paper considers the problem of recovering a sparse signal representation according to a signal dictionary. This problem could be formalized as a penalized least-squares problem in which sparsity is usually induced by a ℓ1-norm penalty on the coefficients. Such an approach known as the Lasso or Basis Pursuit Denoising has been shown to perform reasonably well in some situations. However, it was also proved that nonconvex penalties like the pseudo ℓq-norm with q < 1 or smoothly clipped absolute deviation (SCAD) penalty are able to recover sparsity in a more efficient way than the Lasso. Several algorithms have been proposed for solving the resulting nonconvex least-squares problem. This paper proposes a generic algorithm to address such a sparsity recovery problem for some class of nonconvex penalties. Our main contribution is that the proposed methodology is based on an iterative algorithm which solves at each iteration a convex weighted Lasso problem. It relies on the family of nonconvex penalties which can be decomposed as a difference of convex functions (DC). This allows us to apply DC programming which is a generic and principled way for solving nonsmooth and nonconvex optimization problem. We also show that several algorithms in the literature dealing with nonconvex penalties are particular instances of our algorithm. Experimental results demonstrate the effectiveness of the proposed generic framework compared to existing algorithms, including iterative reweighted least-squares methods. [ABSTRACT FROM AUTHOR]

Recently, the duality between multiple-input multiple-output (MIMO) multiple-access channels (MAC) and MIMO broadcast channels (BCs) has been established under a total power constraint. The same set of rates for MAC can be achieved in BC exploiting the MAC-BC duality formulas while preserving the total power constraint. In this paper, we describe the BC optimal power allocation applying this duality in a downstream x-digital subscriber lines (xDSLs) context under a total power constraint for all modems over all tones. Then, a new algorithm called BC-optimal spectrum balancing (BC-OSB) is devised for a more realistic power allocation under per-modem total power constraints. The capacity region of the primal BC problem under per-modem total power constraints is found by the dual optimization problem for the BC under per-modem total power constraints which can be rewritten as a dual optimization problem in the MAC by means of a precoder matrix based on the Lagrange multipliers. We show that the duality gap between the two problems is zero. The multiuser power allocation problem has been solved for interference channels and MAC using the OSB algorithm. In this paper we solve the problem of multiuser power allocation for the BC case using the OSB algorithm as well and we derive a computational efficient algorithm that will be referred to as BC-OSB. Simulation results are provided for two VDSL2 scenarios: the first one with differential-mode (DM) transmission only and the second one with both DM and phantom-mode (PM) transmissions. [ABSTRACT FROM AUTHOR]

This paper considers localization of a source or a sensor from distance measurements. We argue that linear algorithms proposed for this purpose are susceptible to poor noise performance. Instead given a set of sensors/anchors of known positions and measured distances of the source/sensor to be localized from them we propose a potentially nonconvex weighted cost function whose global minimum estimates the location of the source/sensor one seeks. The contribution of this paper is to provide nontrivial ellipsoidal and polytopic regions surrounding these sensors/anchors of known positions, such that if the object to be localized is in this region, localization occurs by globally exponentially convergent gradient descent in the noise free case. Exponential convergence in the noise free case represents practical convergence as it ensures graceful performance degradation in the presence of noise. These results guide the deployment of sensors/anchors so that small subsets can be made responsible for practical localization in geographical areas determined by our approach. [ABSTRACT FROM AUTHOR]

We study the problem of power efficient multiuser beamforming transmission for both uplink and downlink. The base station is equipped with multiple antennas, whereas the mobile units have single antennas. In the uplink, interference is canced by successive decoding. in the downlink, ideal "dirty paper" precoding is assumed. The design goal is to minimize the total transmit power while maintaining individual SINR constraints. In the uplink, the optimization problem is solved by a recursive formula with low computational complexity. The downlink problem is solved by exploiting the duality between uplink and downlink; thus, the uplink solution carries over to the downlink. In the second part of the paper, we show how the solution can be applied to the problem of rate balancing in Gaussian multiuser channels. We propose a strategy for throughput-wise optimal transmission for broadcast and multiple access channels under a sum power constraint. Finally, we show that single-user transmission achieves the sum capacity in the low-SNR regime. We completely characterize the SNR-range where single-user transmission is optimal. [ABSTRACT FROM AUTHOR]

This paper considers the problem of robustly estimating a structured covariance matrix with an elliptical underlying distribution with a known mean. In applications where the covariance matrix naturally possesses a certain structure, taking the prior structure information into account in the estimation procedure is beneficial to improving the estimation accuracy. We propose incorporating the prior structure information into Tyler's M-estimator and formulating the problem as minimizing the cost function of Tyler's estimator under the prior structural constraint. First, the estimation under a general convex structural constraint is introduced with an efficient algorithm for finding the estimator derived based on the majorization-minimization (MM) algorithm framework. Then, the algorithm is tailored to several special structures that enjoy a wide range of applications in signal processing related fields, namely, sum of rank-one matrices, Toeplitz, and banded Toeplitz structure. In addition, two types of non-convex structures, i.e., the Kronecker structure and the spiked covariance structure, are also discussed, where it is shown that simple algorithms can be derived under the guidelines of MM. The algorithms are guaranteed to converge to a stationary point of the problems. Furthermore, if the constraint set is geodesically convex, such as the Kronecker structure set, then the algorithm converges to a global minimum. Numerical results show that the proposed estimator achieves a smaller estimation error than the benchmark estimators at a lower computational cost. [ABSTRACT FROM AUTHOR]

Waveform design is a pivotal component of the fully adaptive radar construct. In this paper, we consider waveform design for radar space time adaptive processing (STAP), accounting for the waveform dependence of the clutter correlation matrix. Due to this dependence, in general, the joint problem of receiver filter optimization and radar waveform design becomes an intractable, nonconvex optimization problem, Nevertheless, it is, however, shown to be individually convex either in the filter or in the waveform variables. We derive constrained versions of a) the alternating minimization algorithm, b) proximal alternating minimization, and c) the constant modulus alternating minimization, which, at each step, iteratively optimizes either the STAP filter or the waveform independently. A fast and slow time model permits waveform design in radar STAP, but the primary bottleneck is the computational complexity of the algorithms. [ABSTRACT FROM PUBLISHER]

In Part II of this paper, also in this issue, we carried out a detailed mean-square-error analysis of the performance of asynchronous adaptation and learning over networks under a fairly general model for asynchronous events including random topologies, random link failures, random data arrival times, and agents turning on and off randomly. In this Part III, we compare the performance of synchronous and asynchronous networks. We also compare the performance of decentralized adaptation against centralized stochastic-gradient (batch) solutions. Two interesting conclusions stand out. First, the results establish that the performance of adaptive networks is largely immune to the effect of asynchronous events: the mean and mean-square convergence rates and the asymptotic bias values are not degraded relative to synchronous or centralized implementations. Only the steady-state mean-square-deviation suffers a degradation in the order of \nu, which represents the small step-size parameters used for adaptation. Second, the results show that the adaptive distributed network matches the performance of the centralized solution. These conclusions highlight another critical benefit of cooperation by networked agents: cooperation does not only enhance performance in comparison to stand-alone single-agent processing, but it also endows the network with remarkable resilience to various forms of random failure events and is able to deliver performance that is as powerful as batch solutions. [ABSTRACT FROM PUBLISHER]

In this paper, the problem of matrix rank minimization under affine constraints is addressed. The state-of-the-art algorithms can recover matrices with a rank much less than what is sufficient for the uniqueness of the solution of this optimization problem. We propose an algorithm based on a smooth approximation of the rank function, which practically improves recovery limits on the rank of the solution. This approximation leads to a non-convex program; thus, to avoid getting trapped in local solutions, we use the following scheme. Initially, a rough approximation of the rank function subject to the affine constraints is optimized. As the algorithm proceeds, finer approximations of the rank are optimized and the solver is initialized with the solution of the previous approximation until reaching the desired accuracy. On the theoretical side, benefiting from the spherical section property, we will show that the sequence of the solutions of the approximating programs converges to the minimum rank solution. On the experimental side, it will be shown that the proposed algorithm, termed SRF standing for smoothed rank function, can recover matrices, which are unique solutions of the rank minimization problem and yet not recoverable by nuclear norm minimization. Furthermore, it will be demonstrated that, in completing partially observed matrices, the accuracy of SRF is considerably and consistently better than some famous algorithms when the number of revealed entries is close to the minimum number of parameters that uniquely represent a low-rank matrix. [ABSTRACT FROM PUBLISHER]

Frequency-Response Masking (FRM) technique has been widely used in all kinds of applications where FIR filters with extremely narrow transition band are needed. Thus, researchers have invested much effort to find ways to increase the efficiency of the FRM algorithm. In this paper, a novel 2-stage FRM structure was proposed. The three interpolation factors of the subfilters in the second stage can be chosen independently. The band-edge shaping filter synthesized by the second stage has a non-periodical frequency response, which is quite different from conventional FRM structures. Various filters were designed to test the performance of the proposed method. Experiments show that the proposed structure achieves lower complexity, in terms of the number of multipliers, compared to the FRM algorithms such as 1-stage FRM, 2-stage FRM, IFIR-FRM, SFFM-FRM, serial-masking FRM and some current FRM structures. [ABSTRACT FROM PUBLISHER]

This paper studies optimal transmission over wireless channels with imperfect channel state information available at the transmitter side in the context of point-to-point channels, multiuser orthogonal frequency division multiplexing, and random access. Terminals adapt transmitted power and coding mode to channel estimates in order to maximize expected throughput subject to average power constraints. To reduce the likelihood of packet losses due to the mismatch between channel estimates and actual channel values, a backoff function is further introduced to enforce the selection of more conservative coding modes. Joint determination of optimal power allocations and backoff functions is a nonconvex stochastic optimization problem with infinitely many variables that despite its lack of convexity is part of a class of problems with null duality gap. Exploiting the resulting equivalence between primal and dual problems, we show that optimal power allocations and channel backoff functions are uniquely determined by optimal dual variables. This affords considerable simplification because the dual problem is convex and finite dimensional. We further exploit this reduction in computational complexity to develop iterative algorithms to find optimal operating points. These algorithms implement stochastic subgradient descent in the dual domain and operate without knowledge of the probability distribution of the fading channels. Numerical results corroborate theoretical findings. [ABSTRACT FROM PUBLISHER]

The linearly constrained tensor n-rank minimization problem is an extension of matrix rank minimization. It is applicable in many fields which use the multi-way data, such as data mining, machine learning and computer vision. In this paper, we adapt operator splitting technique and convex relaxation technique to transform the original problem into a convex, unconstrained optimization problem and propose a fixed point iterative method to solve it. We also prove the convergence of the method under some assumptions. By using a continuation technique, we propose a fast and robust algorithm for solving the tensor completion problem, which is called FP-LRTC (Fixed Point for Low n-Rank Tensor Completion). Our numerical results on randomly generated and real tensor completion problems demonstrate that this algorithm is effective, especially for “easy” problems. [ABSTRACT FROM PUBLISHER]

Recently, the filter bank multicarrier with offset quadrature amplitude modulation (FBMC-OQAM) has attracted increasing attention. However, most peak-to-average power ratio (PAPR) reduction schemes developed for orthogonal frequency division multiplexing (OFDM) signals are not effective for FBMC-OQAM signals, due to the overlapping structure of FBMC-OQAM signals. In this paper, we propose an improved partial transmit sequence (PTS) scheme by employing multi-block joint optimization (MBJO) for the PAPR reduction of FBMC-OQAM signals, called as MBJO-PTS scheme. In PTS scheme, one data block is divided into several subblocks and each subblock is multiplied by a phase rotation factor for the subblock. The PTS scheme searches over all combinations of allowed phase factors to lower the PAPR. Unlike existing PAPR reduction schemes of independently optimizing the data blocks, the MBJO-based scheme exploits the overlapping structure of the FBMC-OQAM signal and jointly optimizes multiple data blocks. Moreover, we develop two algorithms for the optimization problem in the MBJO-PTS scheme, including a dynamic programming (DP) algorithm to guarantee the optimal solution and avoid exhaustive search. Theoretical analysis and simulations show that the proposed MBJO-PTS scheme could provide a significant PAPR reduction in the FBMC-OQAM system, by exploiting the overlapping structure of the FBMC-OQAM signal. Employing the proposed DP algorithm, the FBMC-OQAM system with the proposed MBJO-PTS scheme even outperforms the OFDM system with the conventional PTS scheme by 0.9 dB at CCDF of 10^-3 in PAPR reduction, under the same number of subcarriers, modulation type and PTS parameters given in refid="sec5"Section V . [ABSTRACT FROM PUBLISHER]

Bounded Component Analysis (BCA) has recently been introduced as an alternative linear decomposition scheme. In this approach the boundedness property of sources is exploited to replace the usual independence assumption with a weaker assumption, which enables development of methods to separate both independent and dependent components from their mixtures. This paper positions total output range minimization based blind source separation approach as a BCA method for the separation of uncorrelated sources. It is shown that the global minimizers of the corresponding optimization problem are the perfect separators. Furthermore, a stationary point analysis for the corresponding algorithms based on symmetrical orthogonalization is provided. The main result of this analysis is that the range minimization based parallel BCA algorithm and the kurtosis maximization based Independent Component Analysis algorithm have related set of identified stationary points. [ABSTRACT FROM PUBLISHER]

Recent work has demonstrated that using a carefully designed dictionary instead of a predefined one, can improve the sparsity in jointly representing a class of signals. This has motivated the derivation of learning methods for designing a dictionary which leads to the sparsest representation for a given set of signals. In some applications, the signals of interest can have further structure, so that they can be well approximated by a union of a small number of subspaces (e.g., face recognition and motion segmentation). This implies the existence of a dictionary which enables block-sparse representations of the input signals once its atoms are properly sorted into blocks. In this paper, we propose an algorithm for learning a block-sparsifying dictionary of a given set of signals. We do not require prior knowledge on the association of signals into groups (subspaces). Instead, we develop a method that automatically detects the underlying block structure given the maximal size of those groups. This is achieved by iteratively alternating between updating the block structure of the dictionary and updating the dictionary atoms to better fit the data. Our experiments show that for block-sparse data the proposed algorithm significantly improves the dictionary recovery ability and lowers the representation error compared to dictionary learning methods that do not employ block structure. [ABSTRACT FROM AUTHOR]

Intercell interference is the major limitation to the performance of future cellular systems. Despite the joint detection and joint transmission techniques aiming at interference cancelation which suffer from the limited possible cooperation among the nodes, power allocation is a promising approach for optimizing the system performance. If the interference is treated as noise, the power allocation optimization problem aiming at maximizing the sum rate with a total power constraint is nonconvex and up to now an open problem. In the present paper, the solution is found by reformulating the nonconvex objective function of the sum rate as a difference of two concave functions. A globally optimum power allocation is found by applying a branch-and-bound algorithm to the new formulation. In principle, the algorithm partitions the feasible region recursively into subregions where for every subregion the objective function is upper and lower bounded. For each subregion, a linear program is applied for estimating the upper bound of the sum rate which is derived from a convex maximization formulation of the problem with piecewise linearly approximated constraints. The performance is investigated by system-level simulations. The results show that the proposed algorithm outperforms the known conventional suboptimum schemes. Furthermore, it is shown that the algorithm asymptotically converges to a globally optimum power allocation. [ABSTRACT FROM PUBLISHER]

The present paper is concerned with a sensor network, where each sensor is modeled by either a linear or nonlinear sensing system. These sensors team up in observing either static or dynamic random targets and transmit their observations through noisy communication channels to a fusion center (FC) for locating/tracking the targets. Physically, the network is limited by energy resource. According to the available sum power budget, we develop a novel technique for power allocation to the sensor nodes that enables the FC produce the best linear estimate in terms of the mean square error (MSE). Regardless of whether the sensor measurements are linear or nonlinear, the targets are scalar or vectors, static or dynamic, the corresponding optimization problems are shown to be semidefinite programs (SDPs) of tractable optimization and thus are globally and efficiently solved by any existing SDP solver. In other words, new tractably computational algorithms of distributed Bayes filtering are derived with full multisensor diversity achieved. Intensive simulation shows that these algorithms clearly outperform previously known algorithms. [ABSTRACT FROM PUBLISHER]

Parallel multiple-input multiple-output (MIMO) broadcast channels have been shown to be separable from an information theoretic point of view, i.e., capacity can be achieved with a strategy that performs separate encoding and decoding on each subchannel subject to a power allocation across subchannels. In this paper, we show that separability does not necessarily hold if the transmit strategy is restricted to linear transceivers. The proof will be done by identifying a rate tuple that is achievable in a certain set of channels with joint treatment of the subchannels, but lies outside the achievable rate region of separate linear precoding. The implications of this result are that any algorithm to optimize transmit strategies in parallel MIMO broadcast channels that is based on both linear transceivers and separate encoding on each subchannel does in fact introduce two suboptimalities and not only one. [ABSTRACT FROM AUTHOR]

In signal processing, data analysis and scientific computing, one often encounters the problem of decomposing a tensor into a sum of contributions. To solve such problems, both the search direction and the step size are two crucial elements in numerical algorithms, such as alternating least squares algorithm (ALS). Owing to the nonlinearity of the problem, the often used linear search direction is not always powerful enough. In this paper, we propose two higher-order search directions. The first one, geometric search direction, is constructed via a combination of two successive linear directions. The second one, algebraic search direction, is constructed via a quadratic approximation of three successive iterates. Then, in an enhanced line search along these directions, the optimal complex step size contains two arguments: modulus and phase. A current strategy is ELSCS that finds these two arguments alternately. So it may suffer from a local optimum. We broach a direct method, which determines these two arguments simultaneously, so as to obtain the global optimum. Finally, numerical comparisons on various search direction and step size schemes are reported in the context of blind separation-equalization of convolutive DS-CDMA mixtures. The results show that the new search directions have greatly improve the efficiency of ALS and the new step size strategy is competitive. [ABSTRACT FROM AUTHOR]

The alternating direction multipliers method (ADMM) has been recently proposed as a practical and efficient algorithm for distributed computing. We discuss its applicability to the average consensus problem in this paper. By carefully relaxing ADMM augmentation coefficients we are able to analytically investigate its properties, and to propose simple and strict analytical bounds. These provide a clear indication on how to choose system parameters for optimized performance. We prove both analytically and via simulations that the proposed approach exhibits convergence speed between the best in the literature (classical and optimized solutions), while providing the most powerful resilience to noise. [ABSTRACT FROM AUTHOR]

The recently emerged compressive sensing (CS) framework aims to acquire signals at reduced sample rates compared to the classical Shannon-Nyquist rate. To date, the CS theory has assumed primarily real-valued measurements; it has recently been demonstrated that accurate and stable signal acquisition is still possible even when each measurement is quantized to just a single bit. This property enables the design of simplified CS acquisition hardware based around a simple sign comparator rather than a more complex analog-to-digital converter; moreover, it ensures robustness to gross nonlinearities applied to the measurements. In this paper we introduce a new algorithm—restricted-step shrinkage (RSS)—to recover sparse signals from 1-bit CS measurements. In contrast to previous algorithms for 1-bit CS, RSS has provable convergence guarantees, is about an order of magnitude faster, and achieves higher average recovery signal-to-noise ratio. RSS is similar in spirit to trust-region methods for nonconvex optimization on the unit sphere, which are relatively unexplored in signal processing and hence of independent interest. [ABSTRACT FROM AUTHOR]

This paper introduces a tensor framework to solve the problem of nonunitary joint block diagonalization (JBD) of a set of real or complex valued matrices. We show that JBD can be seen as a particular case of the block-component-decomposition (BCD) of a third-order tensor. The resulting tensor model fitting problem does not require the block-diagonalizer to be a square matrix: the over- and underdetermined cases can be handled. To compute the tensor decomposition, we build an efficient nonlinear conjugate gradient (NCG) algorithm. In the over- and exactly determined cases, we show that exact JBD can be computed by a closed-form solution based on eigenvalue analysis. In approximate JBD problems, this solution can be used to efficiently initialize any iterative JBD algorithm such as NCG. Finally, we illustrate the performance of our technique in the context of independent subspace analysis (ISA) based on second-order statistics (SOS). [ABSTRACT FROM PUBLISHER]

In this paper, we consider a channel estimation scheme for a two-hop nonregenerative MIMO relay system without the direct link between source and destination. This scheme has two phases. In the first phase, the source does not transmit while the relay transmits and the destination receives. In the second phase, the source transmits, the relay amplifies and forwards, and the destination receives. At the destination, the data received in the first phase are used to estimate the relay-to-destination channel, and the data received in the second phase are used to estimate the source-to-relay channel. The linear minimum mean-square error estimation (LMMSE) is used for channel estimation, which allows the use of prior knowledge of channel correlations. For phase 1, an algorithm is developed to compute the optimal source pilot matrix for use at the relay. For phase 2, an algorithm is developed to compute the optimal source pilot matrix for use at the source and the optimal relay pilot matrix for use at the relay. [ABSTRACT FROM AUTHOR]

This paper proposes a novel framework for delay-tolerant particle filtering that is computationally efficient and has limited memory requirements. Within this framework the informativeness of a delayed (out-of-sequence) measurement (OOSM) is estimated using a lightweight procedure and uninformative measurements are immediately discarded. The framework requires the identification of a threshold that separates informative from uninformative; this threshold selection task is formulated as a constrained optimization problem, where the goal is to minimize state estimation error whilst controlling the computational requirements. We develop an algorithm that provides an approximate solution for the optimization problem. Simulation experiments provide an example where the proposed framework processes less than 40% of all OOSMs with only a small reduction in state estimation accuracy. [ABSTRACT FROM PUBLISHER]

In this paper, low-complexity joint power assignment algorithms are developed for multi-source multi-destination relay networks where multiple sources share a common relay that forwards all received signals simultaneously to destinations. In particular, we consider the following power optimization strategies: (i) Minimization of the total transmission power of the sources and the relay under the constraint that the signal-to-interference-plus-noise ratio (SINR) requirement of each source-destination pair is satisfied, and (ii) Maximization of the minimum SINR among all source-destination pairs subject to any given total power budget. Both optimization problems involve K power variables, where K is the number of source-destination pairs in the network, and an exhaustive search is prohibitive for large K. In this work, we develop a methodology that allows us to obtain an asymptotically tight approximation of the SINR and reformulate the original optimization problems to single-variable optimization problems, which can be easily solved by numerical search of the single variable. Then, the corresponding optimal transmission power at each source and relay can be calculated directly. The proposed optimization schemes are scalable and lead to power assignment algorithms that exhibit the same optimization complexity for any number (K) of source-destination pairs in the network. Moreover, we apply the methodology that we developed to solve a related max-min SINR based optimization problem in which we determine power assignment for the sources and the relay to maximize the minimum SINR among all source-destination pairs subject to any given total power budget. Extensive numerical studies illustrate and validate our theoretical developments. [ABSTRACT FROM AUTHOR]

In a cellular wireless system, users located at cell edges often suffer significant out-of-cell interference. Assuming each base station is equipped with multiple antennas, we can model this scenario as a multiple-input single-output (MISO) interference channel. In this paper we consider a coordinated beamforming approach whereby multiple base stations jointly optimize their downlink beamforming vectors in order to simultaneously improve the data rates of a given group of cell edge users. Assuming perfect channel knowledge, we formulate this problem as the maximization of a system utility (which balances user fairness and average user rates), subject to individual power constraints at each base station. We show that, for the single-carrier case and when the number of antennas at each base station is at least two, the optimal coordinated beamforming problem is NP-hard for both the harmonic mean utility and the proportional fairness utility. For general utilities, we propose a cyclic coordinate descent algorithm, which enables each transmitter to update its beamformer locally with limited information exchange and establish its global convergence to a stationary point. We illustrate its effectiveness in computer simulations by using the space matched beamformer as the benchmark. [ABSTRACT FROM AUTHOR]

Distributed power control over interference limited network has received an increasing intensity of interest over the past few years. Distributed solutions (like the iterative water-filling, gradient projection, etc.) have been intensively investigated under quasi-static channels. However, as such distributed solutions involve iterative updating and explicit message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. Unfortunately, the behavior of those distributed solutions under time-varying channels is in general unknown. In this paper, we shall investigate the distributed scaled gradient projection algorithm (DSGPA) in a K pairs multicarrier interference network under a finite-state Markov channel (FSMC) model. We shall analyze the convergence property as well as tracking performance of the proposed DSGPA. Our analysis shows that the proposed DSGPA converges to a limit region rather than a single point under the FSMC model. We also show that the order of growth of the tracking errors is given by \cal O(1 / \barN), where \barN is the average sojourn time of the FSMC. Based on the analysis, we shall derive the tracking error optimal scaling matrices via Markov decision process modeling. We shall show that the tracking error optimal scaling matrices can be implemented distributively at each transmitter. The numerical results show the superior performance of the proposed DSGPA over three baseline schemes, such as the gradient projection algorithm with a constant stepsize. [ABSTRACT FROM PUBLISHER]

This paper considers the downlink beamforming optimization problem that minimizes the total transmission power subject to global shaping constraints and individual shaping constraints, in addition to the constraints of quality of service (QoS) measured by signal-to-interference-plus-noise ratio (SINR). This beamforming problem is a separable homogeneous quadratically constrained quadratic program (QCQP), which is difficult to solve in general. Herein we propose efficient algorithms for the problem consisting of two main steps: 1) solving the semidefinite programming (SDP) relaxed problem, and 2) formulating a linear program (LP) and solving the LP (with closed-form solution) to find a rank-one optimal solution of the SDP relaxation. Accordingly, the corresponding optimal beamforming problem (OBP) is proven to be "hidden" convex, namely, strong duality holds true under certain mild conditions. In contrast to the existing algorithms based on either the rank reduction steps (the purification process) or the Perron-Frobenius theorem, the proposed algorithms are based on the linear program strong duality theorem. [ABSTRACT FROM AUTHOR]

This paper develops an optimal decentralized algorithm for sparse signal recovery and demonstrates its application in monitoring localized phenomena using energy-constrained large-scale wireless sensor networks. Capitalizing on the spatial sparsity of localized phenomena, compressive data collection is enforced by turning off a fraction of sensors using a simple random node sleeping strategy, which conserves sensing energy and prolongs network lifetime. In the absence of a fusion center, sparse signal recovery via decentralized in-network processing is developed, based on a consensus optimization formulation and the alternating direction method of multipliers. In the proposed algorithm, each active sensor monitors and recovers its local region only, collaborates with its neighboring active sensors through low-power one-hop communication, and iteratively improves the local estimates until reaching the global optimum. Because each sensor monitors the local region rather than the entire large field, the iterative algorithm converges fast, in addition to being scalable in terms of transmission and computation costs. Further, through collaboration, the sensing performance is globally optimal and attains a high spatial resolution commensurate with the node density of the original network containing both active and inactive sensors. Simulations demonstrate the performance of the proposed approach. [ABSTRACT FROM AUTHOR]

Node localization is essential to most applications of wireless sensor networks (WSNs). In this paper, we consider both range-based node localization and range-free node localization with uncertainties in range measurements, radio range, and anchor positions. First, a greedy optimization algorithm, named sequential greedy optimization (SGO) algorithm, is presented, which is more suitable for distributed optimization in networks than the classical nonlinear Gauss-Seidel algorithm. Then a unified optimization framework is proposed for both range-based localization and range-free localization, and two convex localization formulations are obtained based on semidefinite programming (SDP) relaxation techniques. By applying the SGO algorithm to the edge-based SDP relaxation formulation, we propose a second-order cone programming (SOCP)-based distributed node localization algorithm. Two distributed refinement algorithms are also proposed by using the SGO algorithm to nonconvex localization formulations. The proposed localization algorithms all can be implemented partially asynchronously in networks. Finally, extensive simulations are conducted to demonstrate the efficiency and accuracy of the proposed distributed localization algorithms. [ABSTRACT FROM AUTHOR]

In recent years an increasing effort was made to reduce the energy consumption in digital subscriber line equipment. Dynamic spectrum management (DSM) has been identified as one promising method to achieve energy-efficiency in discrete multitone based systems. An open research question is how to ensure system robustness when applying highly optimized energy-efficient spectrum management. In this paper, we study the problem of uncertainty in crosstalk noise and parameters, the knowledge of which is indispensable for many DSM algorithms. We introduce robust optimization for spectrum balancing as a technique to achieve feasibility of the optimal power-allocation under a deterministic parameter uncertainty model. This can be seen as an extension of current schemes for spectrum balancing. As a special case we consider the simple strategy of scaling the crosstalk parameters to theirworst-case values, which corresponds to a specific uncertainty model and entails no changes to current DSM algorithms. Finally, we quantify the benefit in worst-case performance and the price in terms of energy by simulations. [ABSTRACT FROM AUTHOR]

In this paper, we introduce the iterative subspace identification (ISI) algorithm for learning subspaces in which the data may live. Our subspace identification method differs from currently available method in its ability to infer the dimension of the subspaces from the data without prior knowledge. The learned subspaces can be combined to produce a data driven overcomplete dictionary with good sparseness and generalizability qualities, or can be directly exploited in applications where block sparseness is needed. We describe the ISI algorithm and a complementary optimization method. We demonstrate the ability of the proposed method to produce sparse representations comparable to those achieved with the K-SVD algorithm, but with less than one eighth the training time. Furthermore, the computation savings allows us to develop a shift-tolerant training procedure. We also illustrate its benefits in underdetermined blind source separation of audio, where performance is directly impacted by the sparseness of the representation. [ABSTRACT FROM AUTHOR]

Dynamic spectrum management (DSM) has been recognized as a key technology to significantly improve the performance of digital subscriber line (DSL) broadband access networks. The basic concept of DSM is to coordinate transmission over multiple DSL lines so as to mitigate the impact of crosstalk interference amongst them. Many algorithms have been proposed to tackle the nonconvex optimization problems appearing in DSM, many of them relying on a standard subgradient based dual decomposition approach. In practice however, this approach is often found to lead to extremely slowconvergence or even no convergence at all, one of the reasons being the very difficult tuning of the stepsize parameters. In this paper we propose a novel improved dual decomposition approach inspired by recent advances in mathematical programming. It uses a smoothing technique for the Lagrangian combined with an optimal gradient based scheme for updating the Lagrange multipliers. The stepsize parameters are furthermore selected optimally removing the need for a tuning strategy. With this approach we show how the convergence of current state-of-the-art DSM algorithms based on iterative convex approximations (SCALE and CA-DSB)can be improvedby one order of magnitude. Furthermore we apply the improved dual decomposition approach to otherDSM algorithms (OSB, ISB, ASB, (MS)-DSB, and MIW) and propose further improvements to obtain fast and robust DSM algorithms. Finally, we demonstrate the effectiveness of the improved dual decomposition approach for a number of realistic multiuser DSL scenarios. [ABSTRACT FROM AUTHOR]

ℓ0 norm based algorithms have numerous potential applications where a sparse signal is recovered from a small number of measurements. The direct ℓ0 norm optimization problem is NP-hard. In this paper we work with the the smoothed ℓ0 (SL0) approximation algorithm for sparse representation. We give an upper bound on the run-time estimation error. This upper bound is tighter than the previously known bound. Subsequently, we develop a reliable stopping criterion. This criterion is helpful in avoiding the problems due to the underlying discontinuities of the ℓ0 cost function. Furthermore, we propose an alternative optimization strategy, which results in a Newton like algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we propose a general cross-layer optimization framework for delay-sensitive applications over single wireless links in which we explicitly consider both the heterogeneous and dynamically changing characteristics (e.g., delay deadlines, dependencies, distortion impacts, etc.) of delay-sensitive applications and the underlying time-varying channel conditions. We first formulate this problem as a nonlinear constrained optimization by assuming complete knowledge of the application characteristics and the underlying channel conditions. This constrained cross-layer optimization is then decomposed into several subproblems, each corresponding to the cross-layer optimization for one DU. The proposed decomposition method explicitly considers how the cross-layer strategies selected for one DU will impact its neighboring DUs as well as the DUs that depend on it through the resource price (associated with the resource constraint) and neighboring impact factors (associated with the scheduling constraints). However, the attributes (e.g., distortion impact, delay deadline, etc.) of future DUs as well as the channel conditions are often unknown in the considered real-time applications. In this case, the cross-layer optimization is formulated as a constrained Markov decision process (MDP) in which the impact of current cross-layer actions on the future DUs can be characterized by a state-value function. We then develop a low-complexity cross-layer optimization algorithm using online learning for each DU transmission. This online optimization utilizes information only about the previous transmitted DUs and past experienced channel conditions, which can be easily implemented in real-time in order to cope with unknown source characteristics, channel dynamics and resource constraints. Our numerical results demonstrate the efficiency of the proposed online algorithm. [ABSTRACT FROM AUTHOR]

Distributed coding of correlated sources with memory poses a number of considerable challenges that threaten its practical application, particularly (but not only) in the context of sensor networks. This problem is strongly motivated by the obvious observation that most common sources exhibit temporal correlations that may be at least as important as spatial or intersource correlations. This paper presents an analysis of the underlying tradeoffs, paradigms for coding systems, and approaches for distributed predictive coder design optimization. Motivated by practical limitations on both complexity and delay (especially for dense sensor networks) the focus here is on predictive coding. From the source coding perspective, the most basic tradeoff (and difficulty) is due to conflicts that arise between distributed coding and prediction, wherein "standard" distributed quantization of the prediction errors, if coupled with imposition of zero decoder drift, would drastically compromise the predictor performance and hence the ability to exploit temporal correlations. Another challenge arises from instabilities in the design of closed-loop predictors, whose impact has been observed in the `past, but is greatly exacerbated in the case of distributed coding. In the distributed predictive coder design, we highlight the fundamental tradeoffs encountered within a more general paradigm where decoder drift is allowable or unavoidable, and must be effectively accounted for and controlled. We derive an overall design optimization method for distributed predictive coding that avoids the pitfalls of naive distributed predictive quantization and produces an optimized low complexity and low delay coding system. The proposed iterative algorithms for distributed predictive coding subsume traditional single-source predictive coding and memoryless distributed coding as extreme special cases. [ABSTRACT FROM AUTHOR]

The problem of minimizing the sum of a number of component functions is of great importance in the real world. In .this paper, a new incremental optimization algorithm, named normalized incremental subgradient (NIS) algorithm, is proposed for a class of such problems where the component functions have common local minima. The NIS algorithm is performed incrementally just as the general incremental subgradient (IS) algorithm and thus can be implemented in a distributed way. In the NIS algorithm, the update of each subiteration is based on a search direction obtained by individually normalizing each component of subgradients of component functions, resulting in much better convergence performance as compared to the IS algorithm and other traditional optimization methods (e.g., Gauss-Newton method). The convergence of the NIS algorithm with both diminishing stepsizes and constant stepsizes is proved and analyzed theoretically. Two important applications are presented. One is to solve a class of convex feasibility problems in a distributed way and the other is distributed maximum likelihood estimation. Numerical examples, arising from two important topics in the area of wireless sensor networks-source localization and node localization-demonstrate the effectiveness and efficiency of the NIS algorithm. [ABSTRACT FROM AUTHOR]

Recently, the problem of optimal spectral balancing (OSB) for digital subscriber lines (DSL) with constrained transmit power has been solved using Lagrange's dual optimization technique and a weighted sum rate maximization (WSRM) approach. In many cases, the total power constraint is not binding. Although, this means a huge computational complexity reduction, the algorithm fails to reach certain points on the rate region (RR). In this paper, an in-depth analytical view of the WSRM approach is provided, and it is shown that when the RR is not strictly convex, the WSRM approach fails to reach certain points on the RR. Moreover, using N-dimensional geometry, a novel iterative facet dividing algorithm (IFDA) capable of reaching any point on the RR is proposed. Analytical and simulation results show that our technique is much more reliable and considerably faster than current algorithms. Moreover, it can be used for a wide range of problems which use WSRM approach, including OSB in the general case. [ABSTRACT FROM AUTHOR]

Iteratively reweighted least-squares (IRLS) algorithms have been successfully used in compressive sensing to reconstruct sparse signals from incomplete linear measurements taken in nonsparse domains. The underlying optimization problem corresponds to finding the vector that solves the ℓp, minimization while explaining the measurements, and IRLS allows to easily control the used value of p, with effect on the number of required measurements. In this paper, we propose a weighting strategy in the reconstruction method based on IRLS in order to add prior information on the support of the sparse domain. Our simulation results show that the use of prior knowledge about positions of at least some of the nonzero coefficients in the sparse domain leads to a reduction in the number of linear measurements required for unambiguous reconstruction. This reduction occurs for all values of p, so that a further reduction can be achieved by decreasing p and using prior information. The proposed weighting scheme also reduces the computational complexity with respect to the IRLS with no prior information, both in terms of number of iterations and computation time. [ABSTRACT FROM AUTHOR]

One of the main issues of the orthogonal frequency-division multiplexing (OFDM) modulation is the high peak-to-average power ratio (PAPR) of the transmitted signal, which adversely affects the complexity of power amplifiers. In this paper, we consider transmitters that reduce the PAPR by slightly disturbing the symbols in carriers used to transmit information and by sending dummy symbols-i.e., symbols not conveying information-in unused carriers. The optimal choice of the data and dummy symbols is determined by the solution of a convex optimization problem. To reduce the PAPR with low complexity, we apply a modified version of the adaptive projected subgradient method to a sequence of convex cost functions closely related to the original optimization problem. The resulting algorithm achieves near-optimal PAPR in practical scenarios, generalizes existing algorithms based on Polyak's method, and can easily handle multiple constraints. [ABSTRACT FROM AUTHOR]