A highly efficient ADMM-based algorithm for outlier-robust regression with Huber loss.

Huber robust regression (HRR) has attracted much attention in machine learning due to its greater robustness to outliers compared to least-squares regression. However, existing algorithms for HRR are computationally much less efficient than those for least-squares regression. Based on a maximally split alternating direction method of multipliers (MS-ADMM) for model fitting, a highly computationally efficient algorithm referred to as the modified MS-ADMM is derived in this article for HRR. After analyzing the convergence of the modified MS-ADMM, a parameter selection scheme is presented for the algorithm. With the parameter values calculated via this scheme, the modified MS-ADMM converges very rapidly, much faster than several typical HRR algorithms. Through applications in the training of stochastic neural networks and comparisons with existing algorithms, the modified MS-ADMM is shown to be computationally much more efficient than the convex quadratic programming method, the Newton method, the iterative reweighted least-squares method, and Nesterov's accelerated gradient method. Implementation of the proposed algorithm on a GPU-based parallel computing platform demonstrates its higher GPU acceleration ratio compared to the competing methods and, thus, its greater superiority in computational efficiency over the competing methods. [ABSTRACT FROM AUTHOR]