Search

It is widely recognized that the generalization ability of neural networks can be greatly enhanced through carefully designing the training procedure. The current state-of-the-art training approach involves utilizing stochastic gradient descent (SGD) or Adam optimization algorithms along with a combination of additional regularization techniques such as weight decay, dropout, or noise injection. Optimal generalization can only be achieved by tuning a multitude of hyperparameters through grid search, which can be time-consuming and necessitates additional validation datasets. To address this issue, we introduce a practical PAC-Bayes training framework that is nearly tuning-free and requires no additional regularization while achieving comparable testing performance to that of SGD/Adam after a complete grid search and with extra regularizations. Our proposed algorithm demonstrates the remarkable potential of PAC training to achieve state-of-the-art performance on deep neural networks with enhanced robustness and interpretability., Comment: 30 pages, 15 figures, 7 tables

This note is an invited discussion of the article "Confidence Intervals for Nonparametric Empirical Bayes Analysis" by Ignatiadis and Wager. In this discussion, I review some goals of empirical Bayes data analysis and the contribution of Ignatiadis and Wager. Differences between across-group inference and group-specific inference are discussed. Standard empirical Bayes interval procedures focus on controlling the across-group average coverage rate. However, if group-specific inferences are of primary interest, confidence intervals with group-specific coverage control may be preferable., Comment: Invited comments for the JASA discussion article "Confidence Intervals for Nonparametric Empirical Bayes Analysis" by Ignatiadis and Wager

Against the backdrop of a social media reckoning, this paper seeks to demonstrate the potential of social tools to build virtuous behaviours online. We must assume that human behaviour is flawed, the truth can be elusive, and as communities we must commit to mechanisms to encourage virtuous social digital behaviours. Societies that use social platforms should be inclusive, responsive to evidence, limit punitive actions and allow productive discord and respectful disagreement. Social media success, we argue, is in the hypothesis. Documents are valuable to the degree that they are evidence in service of, or to challenge an idea for a purpose. We outline how a Bayesian social platform can facilitate virtuous behaviours to build evidence-based collective rationality. The chapter outlines the epistemic architecture of the platform's algorithms and user interface in conjunction with explicit community management to ensure psychological safety. The BetterBeliefs platform rewards users who demonstrate epistemically virtuous behaviours and exports evidence-based propositions for decision-making. A Bayesian social network can make virtuous ideas powerful., Comment: 38 pages, 3 tables, 13 figures submitted for peer review for inclusion in M. Alfano, C. Klein and J de Ridder (Eds.) Social Virtue Epistemology. Routledge [forthcoming]

Confidence intervals for the means of multiple normal populations are often based on a hierarchical normal model. While commonly used interval procedures based on such a model have the nominal coverage rate on average across a population of groups, their actual coverage rate for a given group will be above or below the nominal rate, depending on the value of the group mean. Alternatively, a coverage rate that is constant as a function of a group's mean can be simply achieved by using a standard $t$-interval, based on data only from that group. The standard $t$-interval, however, fails to share information across the groups and is therefore not adaptive to easily obtained information about the distribution of group-specific means. In this article we construct confidence intervals that have a constant frequentist coverage rate and that make use of information about across-group heterogeneity, resulting in constant-coverage intervals that are narrower than standard $t$-intervals on average across groups. Such intervals are constructed by inverting biased tests for the mean of a normal population. Given a prior distribution on the mean, Bayes-optimal biased tests can be inverted to form Bayes-optimal confidence intervals with frequentist coverage that is constant as a function of the mean. In the context of multiple groups, the prior distribution is replaced by a model of across-group heterogeneity. The parameters for this model can be estimated using data from all of the groups, and used to obtain confidence intervals with constant group-specific coverage that adapt to information about the distribution of group means.