Search

We address the problem of Bayesian variable selection for high-dimensional linear regression. We consider a generative model that uses a spike-and-slab-like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. The originality of the work is to consider inference through relaxing the model and using a type-II log-likelihood maximization based on an EM algorithm. Model selection is performed afterwards relying on Occam’s razor and on a path of models found by the EM algorithm. Numerical comparisons between our method, called spinyReg, and state-of-the-art high-dimensional variable selection algorithms (such as lasso, adaptive lasso, stability selection or spike-and-slab procedures) are reported. Competitive variable selection results and predictive performances are achieved on both simulated and real benchmark data sets. An original regression data set involving the prediction of the number of visitors of the Orsay museum in Paris using bike-sharing system data is also introduced, illustrating the efficiency of the proposed approach. The R package spinyReg implementing the method proposed in this paper is available on CRAN. [ABSTRACT FROM AUTHOR]

Abstract: Analyses of non-normal data and longitudinal data to study changes in variables measured repeatedly over time have received considerable attention in social and psychological research. This paper proposes a dynamic two-level nonlinear structural equation model with covariates for analyzing multivariate longitudinal responses that are mixed continuous and ordered categorical variables. To cope with the non-normal continuous data, the corresponding residual errors at both first-level and second-level models are modeled through a Bayesian semiparametric modeling on the basis of a truncated and centered Dirichlet process with stick-breaking priors. The first-level model is defined for measures taken at each time point nested within individuals for investigating their characteristics that vary with time; while the second level is defined for individuals to assess their characteristics that are invariant with time. An algorithm based on the blocked Gibbs sampler is implemented for estimation of parameters. An efficient model comparison statistic, namely the -measure, is also introduced. Results of a simulation study indicate that the performance of the Bayesian semiparametric estimation is satisfactory. The proposed methodologies are applied to a real longitudinal study concerning cocaine use. [Copyright &y& Elsevier]

Abstract: Bayesian inference methods are used extensively in the analysis of Generalized Linear Mixed Models (GLMMs), but it may be difficult to handle the posterior distributions analytically. In this paper, we establish the asymptotic normality of the joint posterior distribution of the parameters and the random effects in a GLMM by using Stein’s Identity. We also show that while incorrect assumptions on the random effects can lead to substantial bias in the estimates of the parameters, the assumed model for the random effects, under some regularity conditions, does not affect the asymptotic normality of the joint posterior distribution. This motivates the use of the approximate normal distributions for sensitivity analysis of the random effects distribution. We additionally illustrate that the approximate normal distribution performs reasonably using both real and simulated data. This creates a primary alternative to Markov Chain Monte Carlo (MCMC) sampling and avoids a wide range of problems for MCMC algorithms in terms of convergence and computational time. [Copyright &y& Elsevier]

Abstract: Statistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large small problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik’s -insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models. [Copyright &y& Elsevier]

This paper derives and evaluates an algorithm for estimating normal covariances. A particular concern is the performance of the estimator when the dimension of the space exceeds the number of observations. The algorithm is simple, tolerably well founded, and seems to be more accurate for its purpose than the alternatives. Other topics discussed are the joint estimation of variances in one and many dimensions; the loss function appropriate to a variance estimator; and its connection with a certain Bayesian prescription. [Copyright &y& Elsevier]