Estimating the Penalty Level of $\ell_{1}$-minimization via Two Gaussian Approximation Methods

In this paper, we aim to give a theoretical approximation for the penalty level of $\ell_{1}$-regularization problems. This can save much time in practice compared with the traditional methods, such as cross-validation. To achieve this goal, we develop two Gaussian approximation methods, which are based on a moderate deviation theorem and Stein's method respectively. Both of them give efficient approximations and have good performances in simulations. We apply the two Gaussian approximation methods into three types of ultra-high dimensional $\ell_{1}$ penalized regressions: lasso, square-root lasso, and weighted $\ell_{1}$ penalized Poisson regression. The numerical results indicate that our two ways to estimate the penalty levels achieve high computational efficiency. Besides, our prediction errors outperform that based on the 10-fold cross-validation.
Comment: 24 pages, 3 figures