Search

In this paper we develop the randomized Sharded Bayesian Additive Regression Trees (SBT) model. We introduce a randomization auxiliary variable and a sharding tree to decide partitioning of data, and fit each partition component to a sub-model using Bayesian Additive Regression Tree (BART). By observing that the optimal design of a sharding tree can determine optimal sharding for sub-models on a product space, we introduce an intersection tree structure to completely specify both the sharding and modeling using only tree structures. In addition to experiments, we also derive the theoretical optimal weights for minimizing posterior contractions and prove the worst-case complexity of SBT., 46 pages, 10 figures (Appendix included)

In projection pursuit regression (PPR), an unknown response function is approximated by the sum of M "ridge functions," which are flexible functions of one-dimensional projections of a multivariate input space. Traditionally, optimization routines are used to estimate the projection directions and ridge functions via a sequential algorithm, and M is typically chosen via cross-validation. We introduce the first Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To learn the projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case (M=1), called the Single Index Model, for which Bayesian implementations do exist; then use reversible jump MCMC to learn the number of ridge functions M. We evaluate the predictive ability of our model in 20 simulation scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox., 30 pages, 14 figures, supplemental material