Search

We investigate multitask learning in the context of multivariate linear regression with high dimensional covariates and heavy‐tailed noise, while under the constraint of limited memory. To tackle the computational complexity arising from the non‐smoothness of the quantile loss, we reformulate it as an equivalent least squares loss, which yields robust solutions even in the presence of heavy‐tailed noise. We incorporate a group lasso penalty into the quantile loss to produce sparse solutions, and an accelerated proximal sub‐gradient descent algorithm to speed up the computation while ensuring explicit forms for penalized solutions at each iteration. The proposed algorithm is general and can be applied to similar optimization problems. Moreover, we introduce a communication‐efficient distributed algorithm that guarantees optimal convergence rates after finite communication rounds in cases where computing resources such as memory are insufficient. We also study the theoretical properties of the resultant estimate and relax the widely used model selection consistency assumption on the initial estimate. We demonstrate the effectiveness of our proposal through extensive numerical studies. [ABSTRACT FROM AUTHOR]

AbstractDecentralized low-rank learning is an active research domain with extensive practical applications. A common approach to producing low-rank and robust estimations is to employ a combination of the nonsmooth quantile regression loss and nuclear-norm regularizer. Nevertheless, directly applying existing techniques may result in slow convergence rates due to the doubly nonsmooth objective. To expedite the computation process, a decentralized surrogate matrix quantile regression method is proposed in this article. The proposed algorithm has a simple implementation and can provably converge at a linear rate. Additionally, we provide a statistical guarantee that our estimate can achieve an almost optimal convergence rate, regardless of the number of nodes. Numerical simulations confirm the efficacy of our approach. [ABSTRACT FROM AUTHOR]

Sparse quantile regression is a useful tool for variable selection, robust estimation and heteroscedasticity treatment in high-dimensional data analysis. Due to the non-smooth of quantile loss, the computation is heavier than the least square models. In the literature, there are various numerical methods for linear quantile regression, such as ADMM developed in Gu et al. [Technometrics. 2017;60(3):319–331]. However, the computation of multivariate quantile regression has not yet been fully resolved, especially when the dimension is high. Motivated by this, we will focus on the design of fast numerical algorithms for row-sparse multivariate quantile regression model. By virtue of proximal operator and Majorize–Minimization, four smoothed algorithms are designed. For all the obtained algorithms, we analyse their convergence and the parameter selection. We conduct plenty of simulations and four real data sets analysis. Finally, we conclude that the smoothed method is faster than the non-smooth method, especially when the number of predictors is large. [ABSTRACT FROM AUTHOR]

We develop a communication-efficient distributed estimation for the 1-bit compressive sensing where unknown sparse signals are coded into binary measurements with noises and sign flips. We allow for distinctive sign-flipped probabilities and intensities of noises for measurements collected at different nodes, which raises a heterogeneity issue. We suggest a distributed algorithm through penalized least squares to recover sparse signals. This algorithm is computationally very efficient with only gradient information communicated. The resulting distributed estimate converges after a single iteration even when a lousy initial estimate is provided, and achieves a nearly oracle rate after a constant number of iterations. We prove that, under some mild conditions, with high probability, the distributed estimate approximates the underlying true sparse signal with precision δ after a finite number of iterations, as long as the total sample size N satisfies (s log p) / (δ 2 N) = O (1) , where p is the dimension and s is the number of nonzero elements of the underlying true sparse signals. We establish statistical guarantee for support recovery. Extensive experiments are provided to illustrate the effectiveness of our proposed distributed algorithm. for this article are available online. [ABSTRACT FROM AUTHOR]