Search

We explain how effective automatic probability density function estimates can be constructed using contemporary Bayesian inference engines such as those based on no-U-turn sampling and expectation propagation. Extensive simulation studies demonstrate that the proposed density estimates have excellent comparative performance and scale well to very large sample sizes due a binning strategy. Moreover, the approach is fully Bayesian and all estimates are accompanied by pointwise credible intervals. An accompanying package in the R language facilitates easy use of the new density estimates.

Message passing on a factor graph is a powerful paradigm for the coding of approximate inference algorithms for arbitrarily graphical large models. The notion of a factor graph fragment allows for compartmentalization of algebra and computer code. We show that the Inverse G-Wishart family of distributions enables fundamental variational message passing factor graph fragments to be expressed elegantly and succinctly. Such fragments arise in models for which approximate inference concerning covariance matrix or variance parameters is made, and are ubiquitous in contemporary statistics and machine learning.

Summary Collecting information on multiple longitudinal outcomes is increasingly common in many clinical settings. In many cases, it is desirable to model these outcomes jointly. However, in large data sets, with many outcomes, computational burden often prevents the simultaneous modeling of multiple outcomes within a single model. We develop a mean field variational Bayes algorithm, to jointly model multiple Gaussian, Poisson, or binary longitudinal markers within a multivariate generalized linear mixed model. Through simulation studies and clinical applications (in the fields of sight threatening diabetic retinopathy and primary biliary cirrhosis), we demonstrate substantial computational savings of our approximate approach when compared to a standard Markov Chain Monte Carlo, while maintaining good levels of accuracy of model parameters.

A two-level group-specific curve model is such that the mean response of each member of a group is a separate smooth function of a predictor of interest. The three-level extension is such that one grouping variable is nested within another one, and higher level extensions are analogous. Streamlined variational inference for higher level group-specific curve models is a challenging problem. We confront it by systematically working through two-level and then three-level cases and making use of the higher level sparse matrix infrastructure laid down in ( Nolan and Wand (2020) , ANZIAM Journal, doi: 10.1017/S1446181120000061). A motivation is analysis of data from ultrasound technology for which three-level group-specific curve models are appropriate. Whilst extension to the number of levels exceeding three is not covered explicitly, the pattern established by our systematic approach sheds light on what is required for even higher level group-specific curve models.

We derive precise asymptotic results that are directly usable for confidence intervals and Wald hypothesis tests for likelihood-based generalized linear mixed model analysis. The essence of our approach is to derive the exact leading term behaviour of the Fisher information matrix when both the number of groups and number of observations within each group diverge. This leads to asymptotic normality results with simple studentizable forms. Similar analyses result in tractable leading term forms for the determination of approximate locally D-optimal designs.

Linear mixed models are a versatile statistical tool to study data by accounting for fixed effects and random effects from multiple sources of variability. In many situations, a large number of candidate fixed effects is available and it is of interest to select a parsimonious subset of those being effectively relevant for predicting the response variable. Variational approximations facilitate fast approximate Bayesian inference for the parameters of a variety of statistical models, including linear mixed models. However, for models having a high number of fixed or random effects, simple application of standard variational inference principles does not lead to fast approximate inference algorithms, due to the size of model design matrices and inefficient treatment of sparse matrix problems arising from the required approximating density parameters updates. We illustrate how recently developed streamlined variational inference procedures can be generalized to make fast and accurate inference for the parameters of linear mixed models with nested random effects and global-local priors for Bayesian fixed effects selection. Our variational inference algorithms achieve convergence to the same optima of their standard imple-mentations, although with significantly lower computational effort, mem-ory usage and time, especially for large numbers of random effects. Using simulated and real data examples, we assess the quality of automated procedures for fixed effects selection that are free from hyperparameters tuning and only rely upon variational posterior approximations. Moreover, we show high accuracy of variational approximations against model fitting via Markov Chain Monte Carlo sampling.

Expectation propagation is a general prescription for approximation of integrals in statistical inference problems. Its literature is mainly concerned with Bayesian inference scenarios. However, expectation propagation can also be used to approximate integrals arising in frequentist statistical inference. We focus on likelihood-based inference for binary response mixed models and show that fast and accurate quadrature-free inference can be realized for the probit link case with multivariate random effects and higher levels of nesting. The approach is supported by asymptotic theory in which expectation propagation is seen to provide consistent estimation of the exact likelihood surface. Numerical studies reveal the availability of fast, highly accurate and scalable methodology for binary mixed model analysis., Comment: 35 pages, 5 figures

© 2019 International Society for Bayesian Analysis. We build on recent work concerning message passing approaches to approximate fitting and inference for arbitrarily large regression models. The focus is on regression models where the response variable is modeled to have an elaborate distribution, which is loosely defined to mean a distribution that is more complicated than common distributions such as those in the Bernoulli, Poisson and Normal families. Examples of elaborate response families considered here are the Negative Binomial and t families. Variational message passing is more challenging due to some of the conjugate exponential families being non-standard and numerical integration being needed. Nevertheless, a factor graph fragment approach means the requisite calculations only need to be done once for a particular elaborate response distribution family. Computer code can be compartmentalized, including that involving numerical integration. A major finding of this work is that the modularity of variational message passing extends to elaborate response regression models.

© 2017 American Statistical Association. We show how the notion of message passing can be used to streamline the algebra and computer coding for fast approximate inference in large Bayesian semiparametric regression models. In particular, this approach is amenable to handling arbitrarily large models of particular types once a set of primitive operations is established. The approach is founded upon a message passing formulation of mean field variational Bayes that utilizes factor graph representations of statistical models. The underlying principles apply to general Bayesian hierarchical models although we focus on semiparametric regression. The notion of factor graph fragments is introduced and is shown to facilitate compartmentalization of the required algebra and coding. The resultant algorithms have ready-to-implement closed form expressions and allow a broad class of arbitrarily large semiparametric regression models to be handled. Ongoing software projects such as Infer.NET and Stan support variational-type inference for particular model classes. This article is not concerned with software packages per se and focuses on the underlying tenets of scalable variational inference algorithms. Supplementary materials for this article are available online.

© 2018, American Statistical Association. All rights reserved. We provide several examples of Bayesian semiparametric regression analysis via the Infer.NET package for approximate deterministic inference in Bayesian models. The examples are chosen to encompass a wide range of semiparametric regression situations. Infer.NET is shown to produce accurate inference in comparison with Markov chain Monte Carlo via the BUGS package, but to be considerably faster. Potentially, this contribution represents the start of a new era for semiparametric regression, where large and complex analyses are performed via fast Bayesian inference methodology and software, mainly being developed within Machine Learning.

Copyright © 2018 John Wiley & Sons, Ltd. There is substantial interest in assessing how exposure to environmental mixtures, such as chemical mixtures, affects child health. Researchers are also interested in identifying critical time windows of susceptibility to these complex mixtures. A recently developed method, called lagged kernel machine regression (LKMR), simultaneously accounts for these research questions by estimating the effects of time-varying mixture exposures and by identifying their critical exposure windows. However, LKMR inference using Markov chain Monte Carlo (MCMC) methods (MCMC-LKMR) is computationally burdensome and time intensive for large data sets, limiting its applicability. Therefore, we develop a mean field variational approximation method for Bayesian inference (MFVB) procedure for LKMR (MFVB-LKMR). The procedure achieves computational efficiency and reasonable accuracy as compared with the corresponding MCMC estimation method. Updating parameters using MFVB may only take minutes, whereas the equivalent MCMC method may take many hours or several days. We apply MFVB-LKMR to Programming Research in Obesity, Growth, Environment and Social Stressors (PROGRESS), a prospective cohort study in Mexico City. Results from a subset of PROGRESS using MFVB-LKMR provide evidence of significant and positive association between second trimester cobalt levels and z-scored birth weight. This positive association is heightened by cesium exposure. MFVB-LKMR is a promising approach for computationally efficient analysis of environmental health data sets, to identify critical windows of exposure to complex mixtures.

We consider approximate inference methods for Bayesian inference to longitudinal and multilevel data within the context of health science studies. The complexity of these grouped data often necessitates the use of sophisticated statistical models. However, the large size of these data can pose significant challenges for model fitting in terms of computational speed and memory storage. Our methodology is motivated by a study that examines trends in cesarean section rates in the largest state of Australia, New South Wales, between 1994 and 2010. We propose a group-specific curve model that encapsulates the complex nonlinear features of the overall and hospital-specific trends in cesarean section rates while taking into account hospital variability over time. We use penalized spline-based smooth functions that represent trends and implement a fully mean field variational Bayes approach to model fitting. Our mean field variational Bayes algorithms allow a fast (up to the order of thousands) and streamlined analytical approximate inference for complex mixed effects models, with minor degradation in accuracy compared with the standard Markov chain Monte Carlo methods.

Expectation propagation is a general approach to fast approximate inference for graphical models. The existing literature treats models separately when it comes to deriving and coding expectation propagation inference algorithms. This comes at the cost of similar, long-winded algebraic steps being repeated and slowing down algorithmic development. We demonstrate how factor graph fragmentization can overcome this impediment. This involves adoption of the message passing on a factor graph approach to expectation propagation and identification of factor graph sub-graphs, which we call fragments, that are common to wide classes of models. Key fragments and their corresponding messages are catalogued which means that their algebra does not need to be repeated. This allows compartmentalization of coding and efficient software development.

Summary We develop fast mean field variational methodology for Bayesian heteroscedastic semiparametric regression, in which both the mean and variance are smooth, but otherwise arbitrary, functions of the predictors. Our resulting algorithms are purely algebraic, devoid of numerical integration and Monte Carlo sampling. The locality property of mean field variational Bayes implies that the methodology also applies to larger models possessing variance function components. Simulation studies indicate good to excellent accuracy, and considerable time savings compared with Markov chain Monte Carlo. We also provide some illustrations from applications.

© 2014 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America. We develop algorithms for performing semiparametric regression analysis in real time, with data processed as it is collected and made immediately available via modern telecommunications technologies. Our definition of semiparametric regression is quite broad and includes, as special cases, generalized linear mixed models, generalized additive models, geostatistical models, wavelet nonparametric regression models and their various combinations. Fast updating of regression fits is achieved by couching semiparametric regression into a Bayesian hierarchical model or, equivalently, graphical model framework and employing online mean field variational ideas. An Internet site attached to this article, realtime-semiparametric-regression.net, illustrates the methodology for continually arriving stock market, real estate, and airline data. Flexible real-time analyses based on increasingly ubiquitous streaming data sources stand to benefit. This article has online supplementary material.

This article proposes a new method for estimation of the hazard function from a set of censored failure time data, with a view to extending the general approach to more complicated models. The approach is based on a mixed model representation of penalized spline hazard estimators. One payoff is the automation of the smoothing parameter choice through restricted maximum likelihood. Another is the option to use standard mixed model software for automatic hazard estimation.

We derive a variational inference procedure for approximate Bayesian inference in marginal longitudinal semiparametric regression. Fitting and inference is much faster than existing Markov chain Monte Carlo approaches. Numerical studies indicate that the new methodology is very accurate for the class of models under consideration. Copyright © 2013 John Wiley & Sons Ltd

Making effective public policy decisions is challenging at the best of times, but especially in the context of environmental regulation, which typically requires managing opposing interests and strong opinions from industry and private citizens. In this case study, Louise Ryan, Matt Wand and Alan Malecki show how statistical analysis can help resolve conflict and inform effective decision-making under uncertainty

Peter Gavin Hall was born in Sydney, Australia, on 20th November, 1951, and grew up in the Sydney suburb of Oatley. He received a Bachelor of Science with first class honours from the University of Sydney in 1974, a Master of Science from the Australian National University in 1976 and a Doctor of Philosophy from the University of Oxford in 1976. In 1976– 1978, he was Lecturer in Statistics at the University of Melbourne. During 1978–2006, he served as Lecturer, Senior Lecturer, Reader and Professor of Statistics at the Australian National University. He became Professor of Statistics at the University of Melbourne in 2006, Australian Laureate Fellow in 2012 and was Director of the Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers in 2014–2015. In 2005, he commenced a one-quarter professorial position at the University of California, Davis. Peter has published 4 books and more than 600 research articles. He has made seminal contributions concerning the bootstrap, rates of convergence, functional data analysis, martingale theory, measurement error models, nonparametric function estimation and smoothing parameter selection. Honours include the following: Officer of the Order of Australia; Fellow of the Australian Academy of Science, Fellow of the Academy of the Social Sciences in Australia, Fellow of the Royal Society of London, Fellow of the Institute of Mathematical Statistics and Fellow of the American Statistical Association; Corresponding Fellow of the Royal Society of Edinburgh, and Foreign Associate of the U.S. National Academy of Sciences; honorary doctorates from the Universite catholique de Louvain, the Universidad de Cantabria, the University of Glasgow and the University of Sydney; the Committee of Presidents of Statistical Societies Award and the Guy Medal in Silver from the Royal Statistical Society. He married Jeannie Jean Chien Lo on 15th April, 1977. His mother, Ruby Payne-Scott, had a distinguished scientific career as the first woman in radio astronomy and her story is published as Under the Radar. The First Woman in Radio Astronomy: Ruby Payne-Scott (2009) Springer. In 2015, Peter was diagnosed with acute leukaemia. He passed away on 9 January, 2016, in Melbourne, Australia. This conversation took place in Melbourne, Australia, on 11 March and 1 April, 2015.

The aged number theoretic concept of continued fractions can enhance certain Bayesian computations. The crux of this claim is due to continued fraction representations of numerically challenging special function ratios that arise in Bayesian computing. Continued fraction approximation via Lentz's Algorithm often leads to efficient and stable computation of such quantities. Copyright © 2012 John Wiley & Sons, Ltd.

Bayesian hierarchical models are attractive structures for conducting regression analyses when the data are subject to missingness. However, the requisite probability calculus is challenging andMonte Carlo methods typically are employed.We develop an alternative approach based on deterministic variational Bayes approximations. Both parametric and nonparametric regression are considered. Attention is restricted to the more challenging case of missing predictor data.We demonstrate that variational Bayes can achieve good accuracy, but with considerably less computational overhead. The main ramification is fast approximate Bayesian inference in parametric and nonparametric regression models with missing data. Supplemental materials accompany the online version of this article. © 2011 American Statistical Association.

Summary We develop Mean Field Variational Bayes methodology for fast approximate inference in Bayesian Generalized Extreme Value additive model analysis. Such models are useful for flexibly assessing the impact of continuous predictor variables on sample extremes. The new methodology allows large Bayesian models to be fitted and assessed without the significant computing costs of Markov Chain Monte Carlo methods. We illustrate our new methodology with maximum rainfall data from the Sydney, Australia, hinterland. Comparisons are made between the Mean Field Variational Bayes and Markov Chain Monte Carlo approaches.

We demonstrate and critique the new Bayesian inference package Infer.NET in terms of its capacity for statistical analyses. Infer.NET differs from the well-known BUGS Bayesian inference packages in that its main engine is the variational Bayes family of deterministic approximation algorithms rather than Markov chain Monte Carlo. The underlying rationale is that such deterministic algorithms can handle bigger problems due to their increased speed, despite some loss of accuracy. We find that Infer.NET is a well-designed computational framework and offers significant speed advantages over BUGS. Nevertheless, the current release is limited in terms of the breadth of models it can handle, and its inference is sometimes inaccurate. Supplemental materials accompany the online version of this article.

The t-distribution allows the incorporation of outlier robustness into statistical models while retaining the elegance of likelihood-based inference. In this paper, we develop and implement a linear mixed model for the general design of the linear mixed model using the univariate t-distribution. This general design allows a considerably richer class of models to be fit than is possible with existing methods. Included in this class are semi-parametric regression and smoothing and spatial models.

© 2014, Springer Science+Business Media New York. A new data science tool named wavelet-based gradient boosting is proposed and tested. The approach is special case of componentwise linear least squares gradient boosting, and involves wavelet functions of the original predictors. Wavelet-based gradient boosting takes advantages of the approximate $$\ell _1$$ℓ1 penalization induced by gradient boosting to give appropriate penalized additive fits. The method is readily implemented in R and produces parsimonious and interpretable regression fits and classifiers.

We derive the explicit form of expectation propagation for approximate deterministic Bayesian inference in a simple statistical model. The model corresponds to a random sample from the Normal distribution. The explicit forms, and their derivation, allow a deeper understanding of the issues and challenges involved in practical implementation of expectation propagation for statistical analyses. No auxiliary approximations are used: we follow the expectation propagation prescription exactly. A simulation study shows expectation propagation to be more accurate than mean field variational Bayes for larger sample sizes, but at the cost of considerably more algebraic and computational effort.

Highly structured generalised response models, such as generalised linear mixed models and generalised linear models for time series regression, have become an indispensable vehicle for data analysis and inference in many areas of application. However, their use in practice is hindered by high-dimensional intractable integrals. Quasi-Monte Carlo (QMC) is a dynamic research area in the general problem of high-dimensional numerical integration, although its potential for statistical applications is yet to be fully explored. We survey recent research in QMC, particularly lattice rules, and report on its application to highly structured generalised response models. New challenges for QMC are identified and new methodologies are developed. QMC methods are seen to provide significant improvements compared with ordinary Monte Carlo methods. © Springer Science+Business Media, LLC 2007.

The Fisher information for the canonical link exponential family generalised linear mixed model is derived. The contribution from the fixed effects parameters is shown to have a particularly simple form. © 2007 Elsevier Inc. All rights reserved.

This paper develops inference for the significance of features such as peaks and valleys observed in additive modeling through an extension of the SiZer-type methodology of Chaudhuri and Marron (1999) and Godtliebsen et al. (2002, 2004) to the case where the outcome is discrete. We consider the problem of determining the significance of features such as peaks or valleys in observed covariate effects both for the case of additive modeling where the main predictor of interest is univariate as well as the problem of studying the significance of features such as peaks, inclines, ridges and valleys when the main predictor of interest is geographical location. We work with low rank radial spline smoothers to allow to the handling of sparse designs and large sample sizes. Reducing the problem to a Generalised Linear Mixed Model (GLMM) framework enables derivation of simulation-based critical value approximations and guards against the problem of multiple inferences over a range of predictor values. Such a reduction also allows for easy adjustment for confounders including those which have an unknown or complex effect on the outcome. A simulation study indicates that our method has satisfactory power. Finally, we illustrate our methodology on several data sets. © Springer Science+Business Media, LLC 2007.

Fast variational approximate algorithms are developed for Bayesian semiparametric regression when the response variable is a count, i.e. a non-negative integer. We treat both the Poisson and Negative Binomial families as models for the response variable. Our approach utilizes recently developed methodology known as non-conjugate variational message passing. For concreteness, we focus on generalized additive mixed models, although our variational approximation approach extends to a wide class of semiparametric regression models such as those containing interactions and elaborate random effect structure., Comment: 19 pages, 7 figures

This paper develops a likelihood-based method for fitting additive models in the presence of measurement error. It formulates the additive model using the linear mixed model representation of penalized splines. In the presence of a structural measurement error model, the resulting likelihood involves intractable integrals, and a Monte Carlo expectation maximization strategy is developed for obtaining estimates. The method's performance is illustrated with a simulation study. © 2005 Australian Statistical Publishing Association Inc.

Penalised-spline-based additive models allow a simple mixed model representation where the variance components control departures from linear models. The smoothing parameter is the ratio of the random-coefficient and error variances and tests for linear regression reduce to tests for zero random-coefficient variances. We propose exact likelihood and restricted likelihood ratio tests for testing polynomial regression versus a general alternative modelled by penalised splines. Their spectral decompositions are used as the basis of fast simulation algorithms. We derive the asymptotic local power properties of the tests under weak conditions. In particular we characterise the local alternatives that are detected with asymptotic probability one. Confidence intervals for the smoothing parameter are obtained by inverting the tests for a fixed smoothing parameter versus a general alternative. We discuss F and R tests and show that ignoring the variability in the smoothing parameter estimator can have a dramatic effect on their null distributions. The powers of several known tests are investigated and a small set of tests with good power properties is identified. The restricted likelihood ratio test is among the best in terms of power. © 2005 Biometrika Trust.

Geographically referenced data are routinely smoothed using kriging or spline methodology. Features in the resulting surface such as peaks, inclines, ridges, and valleys are often of interest. This article develops inference for the significance of such features through extension of methodology for univariate features known as SiZer. We work with low rank radial spline smoothers. These allow the handling of sparse designs, large sample sizes, and simulation-based critical value approximation. We illustrate the methodology on two geostatistical datasets.

SUMMARY Maps depicting cancer incidence rates have become useful tools in public health research, giving valuable information about the spatial variation in rates of disease. Typically, these maps are generated using count data aggregated over areas such as counties or census blocks. However, with the proliferation of geographic information systems and related databases, it is becoming easier to obtain exact spatial locations for the cancer cases and suitable control subjects. The use of such point data allows us to adjust for individual-level covariates, such as age and smoking status, when estimating the spatial variation in disease risk. Unfortunately, such covariate information is often subject to missingness. We propose a method for mapping cancer risk when covariates are not completely observed. We model these data using a logistic generalized additive model. Estimates of the linear and non-linear effects are obtained using a mixed effects model representation. We develop an EM algorithm to account for missing data and the random effects. Since the expectation step involves an intractable integral, we estimate the E-step with a Laplace approximation. This framework provides a general method for handling missing covariate values when fitting generalized additive models. We illustrate our method through an analysis of cancer incidence data from Cape Cod, Massachusetts. These analyses demonstrate that standard complete-case methods can yield biased estimates of the spatial variation of cancer risk.

Smoothing methods that use. basis functions with penalisation can be formulated as maximum likelihood estimators and best predictors in a mixed model framework. Such connections are at least a quarter of a century old but, perhaps with the advent of mixed model software, have led to a paradigm shift in the field of smoothing. The reason is that most, perhaps all, models involving smoothing can be expressed as a mixed model and hence enjoy the benefit of the growing body of methodology and software for general mixed model analysis. The handling of other complications such as clustering, missing data and measurement error is generally quite straightforward with mixed model representations of smoothing.

Streamlined mean field variational Bayes algorithms for efficient fitting and inference in large models for longitudinal and multilevel data analysis are obtained. The number of operations is linear in the number of groups at each level, which represents a two orders of magnitude improvement over the naive approach. Storage requirements are also lessened considerably. We treat models for the Gaussian and binary response situations. Our algorithms allow the fastest ever approximate Bayesian analyses of arbitrarily large longitudinal and multilevel datasets, with little degradation in accuracy compared with Markov chain Monte Carlo. The modularity of mean field variational Bayes allows relatively simple extension to more complicated scenarios.

Generalized additive models have become one of the most widely used modern statistical tools. Traditionally, they are fit through scatterplot smoothing and the backfitting algorithm. However, a more recent development is the direct fitting through the use of low-rank smoothers (Hastie, J. Roy. Statist. Soc. Ser. B 58 (1996) 379). A particularly attractive example of this is through use of penalized splines (Marx and Eilers, Comput. Statist. Data Anal. 28 (1998) 193). Such an approach has a number of advantages, particularly regarding computation. In this paper, we exploit the explicitness of penalized spline additive models to derive some useful and revealing theoretical approximations.

Many statistical operations benefit from differential calculus. Examples include optimization of likelihood functions and calculation of information matrices. For multiparameter models differential calculus suited to vector argument functions is usually the most efficient means of performing the required calculations. We present a primer on vector differential calculus and demonstrate its application to statistics through several worked examples.

Regression spline smoothing involves modelling a regression function as a piecewise polynomial with a high number of pieces relative to the sample size. Because the number of possible models is so large, efficient strategies for choosing among them are required. In this paper we review approaches to this problem and compare them through a simulation study. For simplicity and conciseness we restrict attention to the univariate smoothing setting with Gaussian noise and the truncated polynomial regression spline basis.

SUMMARY The optimal amount of smoothing in penalised spline regression is investigated. In particular, a simple closed form approximation to the optimal smoothing parameter is derived. Comparisons with its exact counterpart show it to be a useful starting point for measuring the optimal amount of smoothing in penalised spline regression. It also lends itself to the development of quick and simple rules for automatic smoothing parameter selection.

Additive models based on backfitting estimators are among the most important recent contributions to modern statistical modelling. However, the statistical properties of backfitting estimators have received relatively little attention. Recently, J.-D. Opsomer and D. Ruppert (1997,Ann. Statist.25, 186-211; 1998,J. Amer. Statist. Assoc.93, 605–619) and J.-D. Opsomer (1997, preprint 96-12, Department of statistics, Iowa State University) derived their mean squared error properties in the case of local polynomial smoothers. In this paper the asymptotic distributional behaviour of backfitting estimators is investigated.

Summary Exponential smoothing is the most common model-free means of forecasting a future realization of a time series. It requires the specification of a smoothing factor which is usually chosen from the data to minimize the average squared residual of previous one-step-ahead forecasts. In this paper we show that exponential smoothing can be put into a nonparametric regression framework and gain some interesting insights into its performance through this interpretation. We also use theoretical developments from the kernel regression field to derive, for the first time, asymptotic properties of exponential smoothing forecasters.

Recent studies have shown that the asymptotic performance of nonparametric curve estimators in the presence of measurement error will often be very much inferior to that when the observations are error-free. For example, deconvolution of Gaussian measurement error worsens the usual algebraic convergence rates of kernel estimators to very slow logarithmic rates. However, the slow convergence rates mean that very large sample sizes may be required for the asymptotics to take effect, so the finite sample properties of the estimator may not be very well described by the asymptotics. In this article finite sample calculations are performed for the important cases of Gaussian and Laplacian measurement error which provide insight into the feasibility of deconvolving density estimators for practical sample sizes. Our results indicate that for lower levels of measurement error deconvolving density estimators can perform well for reasonable sample sizes.

© 2014, Institute of Mathematical Statistics. All rights received. We investigate mean field variational approximate Bayesian inference for models that use continuous distributions, Horseshoe, Negative-Exponential-Gamma and Generalized Double Pareto, for sparse signal shrinkage. Our principal finding is that the most natural, and simplest, mean field variational Bayes algorithm can perform quite poorly due to posterior dependence among auxiliary variables. More sophisticated algorithms, based on special functions, are shown to be superior. Continued fraction approximations via Lentz's Algorithm are developed to make the algorithms practical.

The most important parameter of a histogram is the bin width because it controls the tradeoff between presenting a picture with too much detail (“undersmoothing”) or too little detail (“oversmoothing”) with respect to the true distribution. Despite this importance there has been surprisingly little research into estimation of the “optimal” bin width. Default bin widths in most common statistical packages are, at least for large samples, quite far from the optimal bin width. Rules proposed by, for example, Scott lead to better large sample performance of the histogram, but are not consistent themselves. In this paper we extend the bin width rules of Scott to those that achieve root-n rates of convergence to the L 2-optimal bin width, thereby providing firm scientific justification for their use. Moreover, the proposed rules are simple, easy and fast to compute, and perform well in simulations.