Search

We consider sparse Bayesian estimation in the classical multivariate linear regression model with p regressors and q response variables. In univariate Bayesian linear regression with a single response y , shrinkage priors which can be expressed as scale mixtures of normal densities are popular for obtaining sparse estimates of the coefficients. In this paper, we extend the use of these priors to the multivariate case to estimate a p × q coefficients matrix B . We derive sufficient conditions for posterior consistency under the Bayesian multivariate linear regression framework and prove that our method achieves posterior consistency even when p > n and even when p grows at nearly exponential rate with the sample size. We derive an efficient Gibbs sampling algorithm and provide the implementation in a comprehensive R package called MBSP . Finally, we demonstrate through simulations and data analysis that our model has excellent finite sample performance. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel variable selection approach in the framework of multivariate linear models taking into account the dependence that may exist between the responses. It consists in estimating beforehand the covariance matrix Σ of the responses and to plug this estimator in a Lasso criterion, in order to obtain a sparse estimator of the coefficient matrix. The properties of our approach are investigated both from a theoretical and a numerical point of view. More precisely, we give general conditions that the estimators of the covariance matrix and its inverse have to satisfy in order to recover the positions of the null and non null entries of the coefficient matrix when the size of Σ is not fixed and can tend to infinity. We prove that these conditions are satisfied in the particular case of some Toeplitz matrices. Our approach is implemented in the R package MultiVarSel available from the Comprehensive R Archive Network (CRAN) and is very attractive since it benefits from a low computational load. We also assess the performance of our methodology using synthetic data and compare it with alternative approaches. Our numerical experiments show that including the estimation of the covariance matrix in the Lasso criterion dramatically improves the variable selection performance in many cases. [ABSTRACT FROM AUTHOR]

We consider quasi-admissibility/inadmissibility of Stein-type shrinkage estimators of the mean of a multivariate normal distribution with covariance matrix an unknown multiple of the identity. Quasi-admissibility/inadmissibility is defined in terms of non-existence/existence of a solution to a differential inequality based on Stein’s unbiased risk estimate (SURE). We find a sharp boundary between quasi-admissible and quasi-inadmissible estimators related to the optimal James–Stein estimator. We also find a class of priors related to the Strawderman class in the known variance case where the boundary between quasi-admissibility and quasi-inadmissibility corresponds to the boundary between admissibility and inadmissibility in the known variance case. Additionally, we also briefly consider generalization to the case of general spherically symmetric distributions with a residual vector. [ABSTRACT FROM AUTHOR]

Many modern statistical problems can be cast in the framework of multivariate regression, where the main task is to make statistical inference for a possibly sparse and low-rank coefficient matrix. The low-rank structure in the coefficient matrix is of intrinsic multivariate nature, which, when combined with sparsity, can further lift dimension reduction, conduct variable selection, and facilitate model interpretation. Using a Bayesian approach, we develop a unified sparse and low-rank multivariate regression method to both estimate the coefficient matrix and obtain its credible region for making inference. The newly developed sparse and low-rank prior for the coefficient matrix enables rank reduction, predictor selection and response selection simultaneously. We utilize the marginal likelihood to determine the regularization hyperparameter, so our method maximizes its posterior probability given the data. For theoretical aspect, the posterior consistency is established to discuss an asymptotic behavior of the proposed method. The efficacy of the proposed approach is demonstrated via simulation studies and a real application on yeast cell cycle data. [ABSTRACT FROM AUTHOR]

We propose a regularized partially functional quantile regression model where the response variable is scalar while the explanatory variables involve both infinite-dimensional predictor processes viewed as functional data, and high-dimensional scalar covariates. Despite extensive work focusing on functional linear models, little effort has been devoted to the development of robust methodologies that tackle the scenarios of non-normal errors. This motivates our proposal of functional quantile regression that seeks an alternative and robust solution to least squares type procedures within the partially functional regression framework. We focus on estimating and selecting the important variables in the high-dimensional covariates, which is complicated by the infinite-dimensional functional predictor. We establish the asymptotic properties of the resulting shrinkage estimator, and empirical illustrations are given by simulation and an application to a brain imaging dataset. [ABSTRACT FROM AUTHOR]

We propose a smooth hypothesis-testing type method for model fitting in regression and develop its theoretical properties in a moderately high-dimensional setting. We derive the asymptotic behavior of the L 2 prediction risk and the optimal choice of the threshold-determining parameter under the Gaussian regression framework with orthogonal covariates. When the covariates are not orthogonal, this method can be used in conjunction with some reasonable procedures for orthogonalizing the covariates, such as a stepwise regression coupled with the Gram–Schmidt procedure. Simulation results show that our proposed method often outperforms its competitors. Asymptotic approximation of the mean square error of the proposed estimator is obtained and it is shown that the optimal rate of convergence depends on the behavior of the smoothing function at zero. [ABSTRACT FROM AUTHOR]

Latent variables that should be examined using multiple observed variables are common in substantive research. The structural equation model (SEM) is widely recognized as the most important statistical tool for assessing interrelationships among latent variables. As a recent advancement, Bayesian quantile SEM provides a comprehensive assessment of the conditional quantile of the response latent variables given the explanatory covariates and latent variables. In this study, we develop Bayesian least absolute shrinkage and selection operator (Lasso) and Bayesian adaptive Lasso procedures to conduct simultaneous estimation and variable selection in the context of quantile SEM. We propose the use of the Markov chain Monte Carlo method to conduct Bayesian inference. Various features, including the finite sample performance of the proposed procedures, are validated through simulation studies. The proposed method is applied to investigate the determinants of the capital structure of Chinese-listed companies. [ABSTRACT FROM AUTHOR]

In this paper, we consider multivariate response regression models with high dimensional predictor variables. One way to estimate the coefficient matrix is through its decomposition. Among various decomposition of the coefficient matrix, we focus on the decomposition which leads to the best approximation to the signal part in the response vector given any rank. Finding this decomposition is equivalent to performing a principal component analysis for the signal. Given any rank, this decomposition has nearly the smallest expected prediction error among all decompositions of the coefficient matrix with the same rank. To estimate the decomposition, we solve a penalized generalized eigenvalue problem followed by a least squares procedure. In the high-dimensional setting, allowing a general covariance structure for the noise vector, we establish the oracle inequalities for the estimates. Simulation studies and application to real data show that the proposed method has good prediction performance and is efficient in dimension reduction for various models. [ABSTRACT FROM AUTHOR]

The problem of estimating a large covariance matrix using a factor model is addressed when both the sample size and the dimension of the covariance matrix tend to infinity. We consider a general class of weighted estimators which includes (i) linear combinations of the sample covariance matrix and the model-based estimator under the factor model, and (ii) linear shrinkage estimators without factors as special cases. The optimal weights in the class are derived, and plug-in weighted estimators are proposed, given that the optimal weights depend on unknown parameters. Numerical results show that our method performs well. Finally, we provide an application to portfolio management. [ABSTRACT FROM AUTHOR]

We study partially linear single-index models where both model parts may contain high-dimensional variables. While the single-index part is of fixed dimension, the dimension of the linear part is allowed to grow with the sample size. Due to the addition of penalty terms to the loss function in order to provide sparse estimators, such as obtained by lasso or smoothly clipped absolute deviation, the construction of confidence intervals for the model parameters is not as straightforward as in the classical low-dimensional data framework. By adding a correction term to the penalized estimator a desparsified estimator is obtained for which asymptotic normality is proven. We study the construction of confidence intervals and hypothesis tests for such models. The simulation results show that the method performs well for high-dimensional single-index models. [ABSTRACT FROM AUTHOR]

We examine the rate of convergence of the Lasso estimator of lower dimensional components of the high-dimensional parameter. Under bounds on the ℓ 1 -norm on the worst possible sub-direction these rates are of order | J | log p / n where p is the total number of parameters, n is the number of observations and J ⊂ { 1 , … , p } represents a subset of the parameters. We also derive rates in sup-norm in terms of the rate of convergence in ℓ 1 -norm. The irrepresentable condition on a set J requires that the ℓ 1 -norm of the worst possible sub-direction is sufficiently smaller than one. In that case sharp oracle results can be obtained. Moreover, if the coefficients in J are small enough the Lasso will put these coefficients to zero. By de-sparsifying one obtains fast rates in supremum norm without conditions on the worst possible sub-direction. The results are extended to M-estimation with ℓ 1 -penalty for generalized linear models and exponential families. For the graphical Lasso this leads to an extension of known results to the case where the precision matrix is only approximately sparse. The bounds we provide are non-asymptotic but we also present asymptotic formulations for ease of interpretation. [ABSTRACT FROM AUTHOR]

The paper considers spatial econometric model and presents a family of shrinkage estimators for the regression coefficients vector. The asymptotic distribution of the proposed family of estimators has been derived under the assumption that sample size is large. The risk properties of least squares estimator and proposed improved family of estimators have been investigated under quadratic loss function and dominance conditions have been obtained. For investigating the finite sample behavior of various estimators belonging to proposed family of shrinkage estimators, a simulation study has been carried out and results have been presented. [ABSTRACT FROM AUTHOR]

Standard formulations of prediction problems in high dimension regression models assume the availability of fully observed covariates and sub-Gaussian and homogeneous model errors. This makes these methods inapplicable to measurement errors models where covariates are unobservable and observations are possibly non sub-Gaussian and heterogeneous. We propose a weighted penalized corrected quantile estimator for regression parameters in linear regression models with additive measurement errors, where unobservable covariate is nonrandom. The proposed estimators forgo the need for the above mentioned model assumptions. We study these estimators in a high dimensional sparse setup where the dimensionality can grow exponentially with the sample size. We provide bounds for the statistical error associated with the estimation, that hold with asymptotic probability 1, thereby providing the ℓ 1 -consistency of the proposed estimator. We also establish the model selection consistency in terms of the correctly estimated zero components of the parameter vector. A simulation study that investigates the finite sample accuracy of the proposed estimator is also included in the paper. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of fitting nonlinear regression models in high-dimensional situations, where the number of predictors, p , is large relative to the number of observations, n . Most of the research in this area has been conducted under the assumption that the regression function has a simple additive structure. This paper focuses instead on single index models, which are becoming increasingly popular in many scientific fields including biostatistics, economics and financial econometrics. Novel methodology is presented for estimating high-dimensional single index models and simultaneously performing variable selection. A computationally efficient algorithm is provided for constructing a solution path. Asymptotic theory is developed for the proposed estimates of the regression function and the index coefficients in the high-dimensional setting. An investigation of the empirical performance on both simulated and real data demonstrates strong performance of the proposed approach. [ABSTRACT FROM AUTHOR]

In this article, the two-step method for prediction, which was proposed by Li et al. (2012), is extended for modelling grouped data, which besides having near-collinear explanatory variables, also having different mean structure, i.e. the mean structure of some part of the data is more complex than other parts. In the first step, inspired by partial least squares regression (PLS), the information for explanatory variables is summarized by a multilinear model with Krylov structured design matrices, which for different groups have different size. The multilinear model is similar to the classical growth curve model except that the design matrices are unknown and are functions of the dispersion matrix. Under such a multilinear model, natural estimators for mean and dispersion matrices are proposed. In the second step, the response is predicted through a conditional predictor where the estimators obtained in the first step are utilized. [ABSTRACT FROM AUTHOR]

This paper develops a new minimum distance quantile regression (MD-QR) estimator for panel data models with fixed effects. The proposed estimator is efficient in the class of minimum distance estimators. In addition, the MD-QR estimator is computationally fast, especially for large cross-sections. We establish consistency and explicitly derive the limiting distribution of the MD-QR estimator for panels with large number of cross-sections and time-series. The limit theory allows for both sequential and joint limits. Monte Carlo simulations are conducted to evaluate the finite sample performance of the estimator. The simulation results confirm that the MD-QR approach produces approximately unbiased estimators with small variances, and is computationally advantageous. Finally, we illustrate the use of the estimator with a simple application to the investment equation model. [ABSTRACT FROM AUTHOR]

In this paper, we describe and study a class of linear shrinkage estimators of the covariance matrix that is well-suited for high dimensional matrices, has a rather wide domain of applicability, and is rooted into the Gaussian conjugate framework of Chen (1979). We propose here a new look at this framework. The linear shrinkage estimator is thereby obtained as the posterior mean of the covariance, using a Bayesian Gaussian model with conjugate inverse Wishart prior, and deriving the shrinkage intensity and target matrix by marginal likelihood maximization. We introduce some extensions to the seminal approach by deriving a closed-form expression of the marginal likelihood as well as computationally light schemes for its maximization. Further, these developments are implemented in a variety of situations and include a simulation-based performance comparison with a recent, widely used class of linear shrinkage estimators. The Gaussian conjugate estimators are found to outperform these estimators in every tested situation where the latter are available and to be more widely and directly applicable. [ABSTRACT FROM AUTHOR]

Abstract: We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided. [Copyright &y& Elsevier]

Abstract: We consider the problem of estimation of the parameters in Generalized Linear Models (GLM) with binary data when it is suspected that the parameter vector obeys some exact linear restrictions which are linearly independent with some degree of uncertainty. Based on minimum -divergence estimation , we consider some estimators for the parameters of the GLM: Unrestricted , restricted , Preliminary , Shrinkage , Shrinkage preliminary , James–Stein , Positive-part of Stein-Rule and Modified preliminary . Asymptotic bias as well as risk with a quadratic loss function are studied under contiguous alternative hypotheses. Some discussion about dominance among the estimators studied is presented. Finally, a simulation study is carried out. [Copyright &y& Elsevier]