Search

We use Stein characterizations to obtain new moment-type estimators for the parameters of three classical spherical distributions (namely the Fisher-Bingham, the von Mises-Fisher, and the Watson distributions) in the i.i.d. case. This leads to explicit estimators which have good asymptotic properties (close to efficiency) and therefore lead to interesting alternatives to classical maximum likelihood methods or more recent score matching estimators. We perform competitive simulation studies to assess the quality of the new estimators. Finally, the practical relevance of our estimators is illustrated on a real data application in spherical latent representations of handwritten numbers.

We use Stein characterisations to derive new moment-type estimators for the parameters of several truncated multivariate distributions in the i.i.d. case; we also derive the asymptotic properties of these estimators. Our examples include the truncated multivariate normal distribution and truncated products of independent univariate distributions. The estimators are explicit and therefore provide an interesting alternative to the maximum-likelihood estimator (MLE). The quality of these estimators is assessed through competitive simulation studies, in which we compare their behaviour to the performance of the MLE and the score matching approach., Comment: The original preprint ("Stein estimation in a multivariate setting") is now split into two preprints: "Stein's method of moments for truncated multivariate distributions" and "Stein's Method of Moments on the Sphere"

We investigate the entropy $H(\mu,t)$ of a probability measure $\mu$ along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that $\mu$ admits moments of any order, it is indeed proved in [Guo et al., 2010] that $t\mapsto H(\mu,t)$ is smooth, and in [Ledoux, 2016] that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of $\mu$. In the seminal contribution \cite{Led}, these algebraic expressions are derived through $\Gamma$-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called "MMSE conjecture" from [Guo et al., 2010] which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in [Ledoux, 2016]. As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable., Comment: Comments most welcome!

We introduce a version of Stein's method of comparison of operators specifically tailored to the problem of bounding the Wasserstein-1 distance between continuous and discrete distributions on the real line. Our approach rests on a new family of weighted discrete derivative operators, which we call bespoke derivatives. We also propose new bounds on the derivatives of the solutions of Stein equations for Integrated Pearson random variables; this is a crucial step in Stein's method. We apply our result to several examples, including the Central Limit Theorem, Polya-Eggenberger urn models, the empirical distribution of the ground state of a many-interacting-worlds harmonic oscillator, the stationary distribution for the number of genes in the Moran model, and the stationary distribution of the Erlang-C system. Whenever our bounds can be compared with bounds from the literature, our constants are sharper., Comment: Corrected some minor typos. Comments most welcome!

Stein operators allow to characterise probability distributions via differential operators. Based on these characterisations, we develop a new method of point estimation for marginal parameters of strictly stationary and ergodic processes, which we call \emph{Stein's Method of Moments} (SMOM). These SMOM estimators satisfy the desirable classical properties such as consistency and asymptotic normality. As a consequence of the usually simple form of the operator, we obtain explicit estimators in cases where standard methods such as (pseudo-) maximum likelihood estimation require a numerical procedure to calculate the estimate. In addition, with our approach, one can choose from a large class of test functions which allows to improve significantly on the moment estimator. Moreover, for i.i.d.\ observations, we retrieve data-dependent functions that result in asymptotically efficient estimators and give a sequence of explicit SMOM estimators that converge to the maximum likelihood estimator. Our simulation study demonstrates that for a number of important univariate continuous probability distributions our SMOM estimators possess excellent small sample behaviour, often outperforming the maximum likelihood estimator and other widely-used methods in terms of lower bias and mean squared error. We also illustrate the pertinence of our approach on a real data set related to rainfall modelisation.

We use a Stein identity to define a new class of parametric distributions which we call ``independent additive weighted bias distributions.'' We investigate related $L^2$-type discrepancy measures, empirical versions of which not only encompass traditional ODE-based procedures but also offer novel methods for conducting goodness-of-fit tests in composite hypothesis testing problems. We determine critical values for these new procedures using a parametric bootstrap approach and evaluate their power through Monte Carlo simulations. As an illustration, we apply these procedures to examine the compatibility of two real data sets with a compound Poisson Gamma distribution.

In this paper, we obtain quantitative, non-asymptotic, and data-dependent \textit{Bernstein-von Mises type} bounds on the normal approximation of the posterior distribution in exponential family models with arbitrary centring and scaling. Our bounds, stated in the total variation and Wasserstein distances, are valid for univariate and multivariate posteriors alike, and do not require a conjugate prior setting. They are obtained through a refined version of Stein's method of comparison of operators that allows for improved dimensional dependence in high-dimensional settings and may also be of interest in other problems. Our approach is rather flexible and, in certain settings, allows for the derivation of bounds with rates of convergence faster than the usual \( O(n^{-1/2}) \) rate (when \( n \) is the sample size). We illustrate our findings on a variety of exponential family distributions, including the Weibull, multinomial, and linear regression with unknown variance. The resulting bounds have an explicit dependence on the prior distribution and on sufficient statistics of the data from the sample, and thus provide insight into how these factors affect the quality of the normal approximation. Insights from our examples include identification of conditions under which faster \( O(n^{-1}) \) convergence rates occur for Bernoulli data, illustrations of how the quality of the normal approximation is influenced by the choice of standardisation, and dimensional dependence in high-dimensional settings.

We study the weighted Poincar\'e constant $C(p,w)$ of a probability density $p$ with weight function $w$ using integration methods inspired by Stein's method. We obtain a new version of the Chen-Wang variational formula which, as a byproduct, yields simple upper and lower bounds on $C(p,w)$ in terms of the so-called Stein kernel of $p$. We also iterate these variational formulas so as to build sequences of nested intervals containing the Poincar\'e constant, sequences of functions converging to said constant, as well as sequences of functions converging to the solutions of the corresponding spectral problem. Our results rely on the properties of a pseudo inverse operator of the classical Sturm-Liouville operator. We illustrate our methods on a variety of examples: Gaussian functionals, weighted Gaussian, beta, gamma, Subbotin, and Weibull distributions., Comment: 24 pages, 1 figure, supplementary material available on second author's website. Comments most welcome

Stein's method is used to study discrete representations of multidimensional distributions that arise as approximations of states of quantum harmonic oscillators. These representations model how quantum effects result from the interaction of finitely many classical "worlds," with the role of sample size played by the number of worlds. Each approximation arises as the ground state of a Hamiltonian involving a particular interworld potential function. Such approximations have previously been studied for one-dimensional quantum harmonic oscillators, but the multidimensional case has remained unresolved. Our approach, framed in terms of spherical coordinates, provides the rate of convergence of the discrete approximation to the ground state in terms of Wasserstein distance. The fastest rate of convergence to the ground state is found to occur in three dimensions. This result is obtained using a discrete density approach to Stein's method applied to the radial component of the ground state solution., Comment: Main text is 22 pages; the rest of the paper consists of (i) supplementary material (including some additional proofs) of 17 pages; (ii) Mathematica and R code

Stein's method compares probability distributions through the study of a class of linear operators called Stein operators. While mainly studied in probability and used to underpin theoretical statistics, Stein's method has led to significant advances in computational statistics in recent years. The goal of this survey is to bring together some of these recent developments and, in doing so, to stimulate further research into the successful field of Stein's method and statistics. The topics we discuss include tools to benchmark and compare sampling methods such as approximate Markov chain Monte Carlo, deterministic alternatives to sampling methods, control variate techniques, parameter estimation and goodness-of-fit testing., Comment: Accepted for publication by "Statistical Science"

This paper provides a general framework for Stein's density method for multivariate continuous distributions. The approach associates to any probability density function a canonical operator and Stein class, as well as an infinite collection of operators and classes which we call standardizations. These in turn spawn an entire family of Stein identities and characterizations for any continuous distribution on $\mathbb{R}^d$, among which we highlight those based on the score function and the Stein kernel. A feature of these operators is that they do not depend on normalizing constants. A new definition of Stein kernel is introduced and examined; integral formulas are obtained through a connection with mass transport, as well as ready-to-use explicit formulas for elliptical distributions. The flexibility of the kernels is used to compare in Stein discrepancy (and therefore 2-Wasserstein distance) between two normal distributions, Student and normal distributions, as well as two normal-gamma distributions. Upper and lower bounds on the 1-Wasserstein distance between continuous distributions are provided, and computed for a variety of examples: comparison between different normal distributions (improving on existing bounds in some regimes), posterior distributions with different priors in a Bayesian setting (including logistic regression), centred Azzalini--Dalla Valle distributions. Finally the notion of weak Stein equation and weak Stein factors is introduced. Bounds for solutions of the weak Stein equation are obtained for Lipschitz test functions if the distribution admits a Poincar\'e constant. We use these bounds to compare different copulas on the unit square in 1-Wasserstein distance., Comment: This is an updated version of our paper arXiv:1806.03478 with some material left out. We leave paper arXiv:1806.03478 up on arXiv for future reference

Using coupling techniques based on Stein's method for probability approximation, we revisit classical variance bounding inequalities of Chernoff, Cacoullos, Chen and Klaassen. Taking advantage of modern coupling techniques allows us to establish novel variance bounds in settings where the underlying density function is unknown or intractable. Applications include bounds for asymptotically Gaussian random variables using zero-biased couplings, bounds for random variables which are New Better (Worse) than Used in Expectation, and analysis of the posterior in Bayesian statistics.

We build on the formalism developed in [arXiv:1906.08372v1] to propose new representations of solutions to Stein equations. We provide new uniform and non uniform bounds on these solutions (a.k.a.\ Stein factors). We use these representations to obtain representations for differences between expectations in terms of solutions to the Stein equations. We apply these to compute abstract Stein-type bounds on Kolmogorov, Total Variation and Wasserstein distances between arbitrary distributions. We apply our results to several illustrative examples, and compare our results with current literature on the same topic, whenever possible. In all occurrences our results are competitive., Comment: Minor modifications, and updated references

In this paper we provide a probabilistic representation of Lagrange's identity which we use to obtain Papathanasiou-type variance expansions of arbitrary order. Our expansions lead to generalized sequences of weights which depend on an arbitrarily chosen sequence of (non-decreasing) test functions. The expansions hold for arbitrary univariate target distribution under weak assumptions, in particular they hold for continuous and discrete distributions alike. The weights are studied under different sets of assumptions either on the test functions or on the underlying distributions. Many concrete illustrations for standard probability distributions are provided (including Pearson, Ord, Laplace, Rayleigh, Cauchy, and Levy distributions)., Comment: 23 pages. This is an updated version of the second part of our previous arXiv submission on the same topic (arXiv:1812.10344) which we leave on the arXiv as a separate submission because it contains some material which may still be of interest

We propose probabilistic representations for inverse Stein operators (i.e. solutions to Stein equations) under general conditions; in particular we deduce new simple expressions for the Stein kernel. These representations allow to deduce uniform and non-uniform Stein factors (i.e. bounds on solutions to Stein equations) and lead to new covariance identities expressing the covariance between arbitrary functionals of an arbitrary {univariate} target in terms of a weighted covariance of the derivatives of the functionals. Our weights are explicit, easily computable in most cases, and expressed in terms of objects familiar within the context of Stein's method. Applications of the Cauchy-Schwarz inequality to these weighted covariance identities lead to sharp upper and lower covariance bounds and, in particular, weighted Poincar\'e inequalities. Many examples are given and, in particular, classical variance bounds due to Klaassen, Brascamp and Lieb or Otto and Menz are corollaries. Connections with more recent literature are also detailed., Comment: 32 pages, 3 tables, 2 figures. This is an updated version of the first part of our previous arXiv submission on the same topic (arXiv:1812.10344) which we leave on the arXiv as a separate submission because it contains some material which may still be of interest

We provide a general result for finding Stein operators for the product of two independent random variables whose Stein operators satisfy a certain assumption, extending a recent result of Gaunt, Mijoule and Swan \cite{gms18}. This framework applies to non-centered normal and non-centered gamma random variables, as well as a general sub-family of the variance-gamma distributions. Curiously, there is an increase in complexity in the Stein operators for products of independent normals as one moves, for example, from centered to non-centered normals. As applications, we give a simple derivation of the characteristic function of the product of independent normals, and provide insight into why the probability density function of this distribution is much more complicated in the non-centered case than the centered case., Comment: 18 pages. To appear in Brazilian Journal of Probability and Statistics, 2019+

In this paper, we present a minimal formalism for Stein operators which leads to different probabilistic representations of solutions to Stein equations. These in turn provide a wide family of Stein-Covariance identities which we put to use for revisiting the very classical topic of bounding the variance of functionals of random variables. Applying the Cauchy-Schwarz inequality yields first order upper and lower Klaassen-type variance bounds. A probabilistic representation of Lagrange's identity (i.e. Cauchy-Schwarz with remainder) leads to Papathanasiou-type variance expansions of arbitrary order. A matrix Cauchy-Schwarz inequality leads to Olkin-Shepp type covariance bounds. All results hold for univariate target distribution under very weak assumptions (in particular they hold for continuous and discrete distributions alike). Many concrete illustrations are provided.

In this paper we present a general framework for Stein's method for multivariate continuous distributions. The approach gives a collection of Stein characterisations, among which we highlight score-Stein operators and kernel Stein operators. Applications include copulas and distance between posterior distributions. We give a general construction for Stein kernels for elliptical distributions and discuss Stein kernels in generality, highlighting connections with Fisher information and mass transport. Finally, a goodness-of-fit test based on Stein discrepancies is given.

The problem of estimating the number $n$ of distinct keys of a large collection of $N$ data is well known in computer science. A classical algorithm is the adaptive sampling (AS). $n$ can be estimated by $R.2^D$, where $R$ is the final bucket (cache) size and $D$ is the final depth at the end of the process. Several new interesting questions can be asked about AS (some of them were suggested by P.Flajolet and popularized by J.Lumbroso). The distribution of $W=\log (R2^D/n)$ is known, we rederive this distribution in a simpler way. We provide new results on the moments of $D$ and $W$. We also analyze the final cache size $R$ distribution. We consider colored keys: assume that among the $n$ distinct keys, $n_C$ do have color $C$. We show how to estimate $p=\frac{n_C}{n}$. We also study colored keys with some multiplicity given by some distribution function. We want to estimate mean an variance of this distribution. Finally, we consider the case where neither colors nor multiplicities are known. There we want to estimate the related parameters. An appendix is devoted to the case where the hashing function provides bits with probability different from $1/2$.

Consider the multivariate Stein equation $\Delta f - x\cdot \nabla f = h(x) - E h(Z)$, where $Z$ is a standard $d$-dimensional Gaussian random vector, and let $f\_h$ be the solution given by Barbour's generator approach. We prove that, when $h$ is $\alpha$-H\"older ($0<\alpha\leq1$), all derivatives of order $2$ of $f\_h$ are $\alpha$-H\"older {\it up to a $\log$ factor}; in particular they are $\beta$-H\"older for all $\beta \in (0, \alpha)$, hereby improving existing regularity results on the solution of the multivariate Gaussian Stein equation. For $\alpha=1$, the regularity we obtain is optimal, as shown by an example given by Rai\v{c} \cite{raivc2004multivariate}. As an application, we prove a near-optimal Berry-Esseen bound of the order $\log n/\sqrt n$ in the classical multivariate CLT in $1$-Wasserstein distance, as long as the underlying random variables have finite moment of order $3$. When only a finite moment of order $2+\delta$ is assumed ($0<\delta<1$), we obtain the optimal rate in $\mathcal O(n^{-\frac{\delta}{2}})$. All constants are explicit and their dependence on the dimension $d$ is studied when $d$ is large.

In this paper we propose a new, simple and explicit mechanism allowing to derive Stein operators for random variables whose characteristic function satisfies a simple ODE. We apply this to study random variables which can be represented as linear combinations of (non necessarily independent) gamma distributed random variables. The connection with Malliavin calculus for random variables in the second Wiener chaos is detailed. An application to McKay Type I random variables is also outlined., Comment: This corresponds to the second section of arXiv:1601.03301 . New results added and applications provided

We provide a bound on a natural distance between finitely and infinitely supported elements of the unit sphere of $\ell^2(\mathbb{N}^*)$, the space of real valued sequences with finite $\ell^2$ norm. We use this bound to estimate the 2-Wasserstein distance between random variables which can be represented as linear combinations of independent random variables. Our results are expressed in terms of a discrepancy measure which is related to Nourdin and Peccati's Malliavin-Stein method. The main area of application of our results is towards the computation of quantitative rates of convergence towards elements of the second Wiener chaos. After particularizing our bounds to this setting and comparing them with the available literature on the subject (particularly the Malliavin-Stein method for Variance-gamma random variables), we illustrate their versatility by tackling three examples: chi-squared approximation for second order $U$-statistics, asymptotics for sequences of quadratic forms and the behavior of the generalized Rosenblatt process at extreme critical exponent., Comment: New section containing lower bounds

In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the derivative of the input-output mutual information of this non-linear channel with respect to the channel quality parameter. This relation is reminiscent of the well-known link between the derivative of the input-output mutual information of additive Gaussian noise channel with respect to the signal-to-noise ratio and the minimum mean-square error. The proof relies on a rescaled version of De Bruijn identity for gamma target distribution together with a stochastic representation for the gamma specific Fisher information. Finally, we are able to derive precise bounds and asymptotics for the input-output mutual information of the non-linear channel with gamma inputs., Comment: submitted for publication in the IEEE Transactions on Information Theory

Hall, Deckert and Wiseman (2014) recently proposed that quantum theory can be understood as the continuum limit of a deterministic theory in which there is a large, but finite, number of classical "worlds." A resulting Gaussian limit theorem for particle positions in the ground state, agreeing with quantum theory, was conjectured in Hall, Deckert and Wiseman (2014) and proven by McKeague and Levin (2016) using Stein's method. In this article we propose new connections between Stein's method and Many Interacting Worlds (MIW) theory. In particular, we show that quantum position probability densities for higher energy levels beyond the ground state arise as distributional fixed points in a new generalization of Stein's method. These are then used to obtain a rate of distributional convergence for conjectured particle positions in the first energy level above the ground state to the (two-sided) Maxwell distribution; new techniques must be developed for this setting where the usual "density approach" Stein solution (see Chatterjee and Shao (2011)) has a singularity., Comment: 17 pages

In this paper, we propose a general means of estimating the rate at which convergences in law occur. Our approach, which is an extension of the classical Stein-Tikhomirov method, rests on a new pair of linear operators acting on characteristic functions. In principle, this method is admissible for any approximating sequence and any target, although obviously the conjunction of several favorable factors is necessary in order for the resulting bounds to be of interest. As we briefly discuss, our approach is particularly promising whenever some version of Stein's method applies. We apply our approach to two examples. The first application concerns convergence in law towards targets $F_\infty$ which belong to the second Wiener chaos (i.e. $F_{\infty}$ is a linear combination of independent centered chi-squared rvs). We detail an application to $U$-statistics. The second application concerns convergence towards targets belonging to the generalized Dickman family of distributions. We detail an application to a theorem from number theory. In both cases our method produces bounds of the correct order (up to a logarithmic loss) in terms of quantities which occur naturally in Stein's method., Comment: 45 pages. Complete re-writing of the previous version. Former applications detailed and new applications provided. Comments are welcome

We build upon recent advances on the distributional aspect of Stein's method to propose a novel and flexible technique for computing Stein operators for random variables that can be written as products of independent random variables. We show that our results are valid for a wide class of distributions including normal, beta, variance-gamma, generalized gamma and many more. Our operators are $k$th degree differential operators with polynomial coefficients; they are straightforward to obtain even when the target density bears no explicit handle. As an application, we derive a new formula for the density of the product of $k$ independent symmetric variance-gamma distributed random variables., Comment: 20 pages

Fix some $n \in \mathbb{N}$ and let $X_1, X_2,\dots, X_n$ be independent random variables drawn from the uniform distribution on $[0,1]$. A decision maker is shown the variables sequentially and, after each observation, must decide whether or not to keep the current one, with payoff the overall rank of the selected observation. Decisions are final: no recall is allowed, no regret is tolerated. The objective is to act in such a way as to minimise the expected payoff. In this note we give the explicit solution to this problem, known as Robbins' problem of optimal stopping, when $n=4$.

A consequence of de Finetti's representation theorem is that for every infinite sequence of exchangeable 0-1 random variables $(X_k)_{k\geq1}$, there exists a probability measure $\mu$ on the Borel sets of $[0,1]$ such that $\bar X_n = n^{-1} \sum_{i=1}^n X_i$ converges weakly to $\mu$. For a wide class of probability measures $\mu$ having smooth density on $(0,1)$, we give bounds of order $1/n$ with explicit constants for the Wasserstein distance between the law of $\bar X_n$ and $\mu$. This extends a recent result {by} Goldstein and Reinert \cite{goldstein2013stein} regarding the distance between the scaled number of white balls drawn in a P\'olya-Eggenberger urn and its limiting distribution. We prove also that, in the most general cases, the distance between the law of $\bar X_n$ and $\mu$ is bounded below by $1/n$ and above by $1/\sqrt{n}$ (up to some multiplicative constants). For every $\delta \in [1/2,1]$, we give an example of an exchangeable sequence such that this distance is of order $1/n^\delta$.

In the first part of the paper we use a new Fourier technique to obtain a Stein characterizations for random variables in the second Wiener chaos. We provide the connection between this result and similar conclusions that can be derived using Malliavin calculus. We also introduce a new form of discrepancy which we use, in the second part of the paper, to provide bounds on the 2-Wasserstein distance between linear combinations of independent centered random variables. Our method of proof is entirely original. In particular it does not rely on estimation of bounds on solutions of the so-called Stein equations at the heart of Stein's method. We provide several applications, and discuss comparison with recent similar results on the same topic.

We provide the first in-depth study of the "smart path" interpolation between an arbitrary probability measure and the gamma-$(\alpha, \lambda)$ distribution. We propose new explicit representation formulae for the ensuing process as well as a new notion of relative Fisher information with a gamma target distribution. We use these results to prove a differential and an integrated De Bruijn identity which hold under minimal conditions, hereby extending the classical formulae which follow from Bakry, Emery and Ledoux's $\Gamma$-calculus. Exploiting a specific representation of the "smart path", we obtain a new proof of the logarithmic Sobolev inequality for the gamma law with $\alpha\geq 1/2$ as well as a new type of HSI inequality linking relative entropy, Stein discrepancy and standardized Fisher information for the gamma law with $\alpha\geq 1/2$., Comment: Typos corrected

In this paper we propose tight upper and lower bounds for the Wasserstein distance between any two {{univariate continuous distributions}} with probability densities $p_1$ and $p_2$ having nested supports. These explicit bounds are expressed in terms of the derivative of the likelihood ratio $p_1/p_2$ as well as the Stein kernel $\tau_1$ of $p_1$. The method of proof relies on a new variant of Stein's method which manipulates Stein operators. We give several applications of these bounds. Our main application is in Bayesian statistics : we derive explicit data-driven bounds on the Wasserstein distance between the posterior distribution based on a given prior and the no-prior posterior based uniquely on the sampling distribution. This is the first finite sample result confirming the well-known fact that with well-identified parameters and large sample sizes, reasonable choices of prior distributions will have only minor effects on posterior inferences if the data are benign.

We propose a new general version of Stein's method for univariate distributions. In particular we propose a canonical definition of the Stein operator of a probability distribution {which is based on a linear difference or differential-type operator}. The resulting Stein identity highlights the unifying theme behind the literature on Stein's method (both for continuous and discrete distributions). Viewing the Stein operator as an operator acting on pairs of functions, we provide an extensive toolkit for distributional comparisons. Several abstract approximation theorems are provided. Our approach is illustrated for comparison of several pairs of distributions : normal vs normal, sums of independent Rademacher vs normal, normal vs Student, and maximum of random variables vs exponential, Frechet and Gumbel., Comment: 41 pages

We introduce a new formalism for computing expectations of functionals of arbitrary random vectors, by using generalised integration by parts formulae. In doing so we extend recent representation formulae for the score function introduced in Nourdin, Peccati and Swan (JFA, to appear) and also provide a new proof of a central identity first discovered in Guo, Shamai, and Verd{\'u} (IEEE Trans. Information Theory, 2005). We derive a representation for the standardized Fisher information of sums of i.i.d. random vectors which use our identities to provide rates of convergence in information theoretic central limit theorems (both in Fisher information distance and in relative entropy)., Comment: Accepted for publication in the Proceedings of the 2014 IEEE International Symposium on Information Theory

We develop Stein's method for the Fr\'echet distribution and apply it to compute rates of convergence in distribution of renormalized sample maxima to the Fr\'echet distribution.

Stein operators are differential operators which arise within the so-called Stein's method for stochastic approximation. We propose a new mechanism for constructing such operators for arbitrary (continuous or discrete) parametric distributions with continuous dependence on the parameter. We provide explicit general expressions for location, scale and skewness families. We also provide a general expression for discrete distributions. For specific choices of target distributions (including the Gaussian, Gamma and Poisson) we compare the operators hereby obtained with those provided by the classical approaches from the literature on Stein's method. We use properties of our operators to provide upper and lower variance bounds (only lower bounds in the discrete case) on functionals $h(X)$ of random variables $X$ following parametric distributions. These bounds are expressed in terms of the first two moments of the derivatives (or differences) of $h$. We provide general variance bounds for location, scale and skewness families and apply our bounds to specific examples (namely the Gaussian, exponential, Gamma and Poisson distributions). The results obtained via our techniques are systematically competitive with, and sometimes improve on, the best bounds available in the literature.

We develop a new method for bounding the relative entropy of a random vector in terms of its Stein factors. Our approach is based on a novel representation for the score function of smoothly perturbed random variables, as well as on the de Bruijn's identity of information theory. When applied to sequences of functionals of a general Gaussian field, our results can be combined with the Carbery-Wright inequality in order to yield multidimensional entropic rates of convergence that coincide, up to a logarithmic factor, with those achievable in smooth distances (such as the 1-Wasserstein distance). In particular, our findings settle the open problem of proving a quantitative version of the multidimensional fourth moment theorem for random vectors having chaotic components, with explicit rates of convergence in total variation that are independent of the order of the associated Wiener chaoses. The results proved in the present paper are outside the scope of other existing techniques, such as for instance the multidimensional Stein's method for normal approximations., Comment: 32 pages

While the asymptotic relative efficiency (ARE) of Wilcoxon rank-based tests for location and regression with respect to their parametric Student competitors can be arbitrarily large, Hodges and Lehmann (1961) have shown that the ARE of the same Wilcoxon tests with respect to their van der Waerden or normal-score counterparts is bounded from above by $6/\pi\approx 1.910$. In this paper, we revisit that result, and investigate similar bounds for statistics based on Student scores. We also consider the serial version of this ARE. More precisely, we study the ARE, under various densities, of the Spearman-Wald-Wolfowitz and Kendall rank-based autocorrelations with respect to the van der Waerden or normal-score ones used to test (ARMA) serial dependence alternatives.

Pinsker's inequality states that the relative entropy $d_{\mathrm{KL}}(X, Y)$ between two random variables $X$ and $Y$ dominates the square of the total variation distance $d_{\mathrm{TV}}(X,Y)$ between $X$ and $Y$. In this paper we introduce generalized Fisher information distances $\mathcal{J}(X, Y)$ between discrete distributions $X$ and $Y$ and prove that these also dominate the square of the total variation distance. To this end we introduce a general discrete Stein operator for which we prove a useful covariance identity. We illustrate our approach with several examples. Whenever competitor inequalities are available in the literature, the constants in ours are at least as good, and, in several cases, better., Comment: This is a revised version of our paper "Discrete Stein characterizations and discrete information distances" (arXiv reference : arXiv:1201.0143). Essential changes have been made. Certain elements of the previous version remain relevant to the literature and have not been included in the present version, therefore we upload this as a new arXiv submission

Classical estimation techniques for linear models either are inconsistent, or perform rather poorly, under $\alpha$-stable error densities; most of them are not even rate-optimal. In this paper, we propose an original one-step R-estimation method and investigate its asymptotic performances under stable densities. Contrary to traditional least squares, the proposed R-estimators remain root-$n$ consistent (the optimal rate) under the whole family of stable distributions, irrespective of their asymmetry and tail index. While parametric stable-likelihood estimation, due to the absence of a closed form for stable densities, is quite cumbersome, our method allows us to construct estimators reaching the parametric efficiency bounds associated with any prescribed values $(\alpha_0, \ b_0)$ of the tail index $\alpha$ and skewness parameter $b$, while preserving root-$n$ consistency under any $(\alpha, \ b)$ as well as under usual light-tailed densities. The method furthermore avoids all forms of multidimensional argmin computation. Simulations confirm its excellent finite-sample performances.

We provide a new perspective on Stein's so-called density approach by introducing a new operator and characterizing class which are valid for a much wider family of probability distributions on the real line. We prove an elementary factorization property of this operator and propose a new Stein identity which we use to derive information inequalities in terms of what we call the \emph{generalized Fisher information distance}. We provide explicit bounds on the constants appearing in these inequalities for several important cases. We conclude with a comparison between our results and known results in the Gaussian case, hereby improving on several known inequalities from the literature., Comment: This is a revised version of our paper "On a connection between Stein characterizations and Fisher information" (arXiv reference : arXiv:1111.2368). Essential changes have been made. Certain elements of the previous version remain relevant to the literature and have not been included in the present version, therefore we upload this as a new arXiv submission

A famous characterization theorem due to C.F. Gauss states that the maximum likelihood estimator (MLE) of the parameter in a location family is the sample mean for all samples of all sample sizes if and only if the family is Gaussian. There exist many extensions of this result in diverse directions, most of them focussing on location and scale families. In this paper, we propose a unified treatment of this literature by providing general MLE characterization theorems for one-parameter group families (with particular attention on location and scale parameters). In doing so, we provide tools for determining whether or not a given such family is MLE-characterizable, and, in case it is, we define the fundamental concept of minimal necessary sample size at which a given characterization holds. Many of the cornerstone references on this topic are retrieved and discussed in the light of our findings, and several new characterization theorems are provided. Of particular interest is that one part of our work, namely the introduction of so-called equivalence classes for MLE characterizations, is a modernized version of Daniel Bernoulli's viewpoint on maximum likelihood estimation., Comment: Published in at http://dx.doi.org/10.3150/13-BEJ506 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

In this paper we tackle the ANOVA problem for directional data (with particular emphasis on geological data) by having recourse to the Le Cam methodology usually reserved for linear multivariate analysis. We construct locally and asymptotically most stringent parametric tests for ANOVA for directional data within the class of rotationally symmetric distributions. We turn these parametric tests into semi-parametric ones by (i) using a studentization argument (which leads to what we call pseudo-FvML tests) and by (ii) resorting to the invariance principle (which leads to efficient rank-based tests). Within each construction the semi-parametric tests inherit optimality under a given distribution (the FvML distribution in the first case, any rotationally symmetric distribution in the second) from their parametric antecedents and also improve on the latter by being valid under the whole class of rotationally symmetric distributions. Asymptotic relative efficiencies are calculated and the finite-sample behavior of the proposed tests is investigated by means of a Monte Carlo simulation. We conclude by applying our findings on a real-data example involving geological data.

We construct two different Stein characterizations of discrete distributions and use these to provide a natural connection between Stein characterizations for discrete distributions and discrete information functionals.

This paper is concerned with a comparison of van der Waerden's and Wilcoxon's scores., Comment: This paper has been withdrawn due to an overlap with existing published material