Search

The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables. At each iteration of this method, a single block of variables is optimized, while the remaining variables are held fixed. To ensure the convergence of the BCD method, the subproblem of each block variable needs to be solved to its unique global optimal. Unfortunately, this requirement is often too restrictive for many practical scenarios. In this dissertation, we first study an alternative inexact BCD approach which updates the variable blocks by successively minimizing a sequence of approximations of f which are either locally tight upper bounds of f or strictly convex local approximations of f. Different block selection rules are considered such as cyclic (Gauss-Seidel), greedy (Gauss-Southwell), randomized, or even multiple (Parallel) simultaneous blocks. We characterize the convergence conditions and iteration complexity bounds for a fairly wide class of such methods, especially for the cases where the objective functions are either non-differentiable or non-convex. Also the case of existence of a linear constraint is studied briefly using the alternating direction method of multipliers (ADMM) idea. In addition to the deterministic case, the problem of minimizing the expected value of a cost function parameterized by a random variable is also investigated. An inexact sample average approximation (SAA) method, which is developed based on the successive convex approximation idea, is proposed and its convergence is studied. Our analysis unifies and extends the existing convergence results for many classical algorithms such as the BCD method, the difference of convex functions (DC) method, the expectation maximization (EM) algorithm, as well as the classical stochastic (sub-)gradient (SG) method for the nonsmooth nonconvex optimization, all of which are popular for large scale optimization problems involving big data. In the second part of this dissertation, we apply our proposed framework to two practical problems: interference management in wireless networks and the dictionary learning problem for sparse representation. First, the computational complexity of these problems are studied. Then using the successive convex approximation framework, we propose novel algorithms for these practical problems. The proposed algorithms are evaluated through extensive numerical experiments on real data.

This thesis aims to theoretically study a modern linear transceiver design strategy, namely interference alignment, in wireless networks. We consider an interference channel whereby each transmitter and receiver are equipped with multiple antennas. The basic problem is to design optimal linear transceivers (or beamformers) that can maximize the system throughput. The recent work [1] suggests that optimal beamformers should maximize the total degrees of freedom through the interference alignment equations. In this thesis, we first state the interference alignment equations and study the computational complexity of solving these equations. In particular, we prove that the problem of maximizing the total degrees of freedom for a given interference channel is NP-hard. Moreover, it is shown that even checking the achievability of a given tuple of degrees of freedom is NP-hard when each receiver is equipped with at least three antennas. Interestingly, the same problem becomes polynomial time solvable when each transmit/receive node is equipped with no more than two antennas.The second part of this thesis answers an open theoretical question about interference alignment on generic channels: What degrees of freedom tuples (d1, d2, ..., dK) are achievable through linear interference alignment for generic channels? We partially answer this question by establishing a general condition that must be satisfied by any degrees of freedom tuple (d1, d2, ..., dK) achievable through linear interference alignment. For a symmetric system with dk = d for all k, this condition implies that the total achievable DoF cannot grow linearly with K, and is in fact no more than K(M + N)/(K + 1), where M and N are the number of transmit and receive antennas, respectively. We also show that this bound is tight when the number of antennas at each transceiver is divisible by the number of data streams.