Search

In the early 1980s Halbert White inaugurated a "model-robust'' form of statistical inference based on the "sandwich estimator'' of standard error. This estimator is known to be "heteroskedasticity-consistent", but it is less well-known to be "nonlinearity-consistent'' as well. Nonlinearity, however, raises fundamental issues because in its presence regressors are not ancillary, hence can't be treated as fixed. The consequences are deep: (1)~population slopes need to be re-interpreted as statistical functionals obtained from OLS fits to largely arbitrary joint $\xy$~distributions; (2)~the meaning of slope parameters needs to be rethought; (3)~the regressor distribution affects the slope parameters; (4)~randomness of the regressors becomes a source of sampling variability in slope estimates; (5)~inference needs to be based on model-robust standard errors, including sandwich estimators or the $\xy$~bootstrap. In theory, model-robust and model-trusting standard errors can deviate by arbitrary magnitudes either way. In practice, significant deviations between them can be detected with a diagnostic test., Comment: Submitted

We propose a general semi-supervised inference framework focused on the estimation of the population mean. As usual in semi-supervised settings, there exists an unlabeled sample of covariate vectors and a labeled sample consisting of covariate vectors along with real-valued responses (“labels”). Otherwise, the formulation is “assumption-lean” in that no major conditions are imposed on the statistical or functional form of the data. We consider both the ideal semi-supervised setting where infinitely many unlabeled samples are available, as well as the ordinary semi-supervised setting in which only a finite number of unlabeled samples is available. ¶ Estimators are proposed along with corresponding confidence intervals for the population mean. Theoretical analysis on both the asymptotic distribution and $\ell_{2}$-risk for the proposed procedures are given. Surprisingly, the proposed estimators, based on a simple form of the least squares method, outperform the ordinary sample mean. The simple, transparent form of the estimator lends confidence to the perception that its asymptotic improvement over the ordinary sample mean also nearly holds even for moderate size samples. The method is further extended to a nonparametric setting, in which the oracle rate can be achieved asymptotically. The proposed estimators are further illustrated by simulation studies and a real data example involving estimation of the homeless population.

We develop an empirical Bayes procedure for estimating the cell means in an unbalanced, two-way additive model with fixed effects. We employ a hierarchical model, which reflects exchangeability of the effects within treatment and within block but not necessarily between them, as suggested before by Lindley and Smith [J. R. Stat. Soc., B 34 (1972) 1–41]. The hyperparameters of this hierarchical model, instead of considered fixed, are to be substituted with data-dependent values in such a way that the point risk of the empirical Bayes estimator is small. Our method chooses the hyperparameters by minimizing an unbiased risk estimate and is shown to be asymptotically optimal for the estimation problem defined above, under suitable conditions. The usual empirical Best Linear Unbiased Predictor (BLUP) is shown to be substantially different from the proposed method in the unbalanced case and, therefore, performs suboptimally. Our estimator is implemented through a computationally tractable algorithm that is scalable to work under large designs. The case of missing cell observations is treated as well.

This paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function. We first introduce a class of semiparametric/parametric shrinkage estimators and establish their asymptotic optimality properties. Two specific cases, the location-scale family and the natural exponential family with quadratic variance function, are then studied in detail. We conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators. We also apply the method to real data and obtain encouraging results., Comment: Published at http://dx.doi.org/10.1214/15-AOS1377 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid ``post-selection inference'' by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing ``simultaneity insurance'' for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffe protection. Importantly it does not depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results., Published in at http://dx.doi.org/10.1214/12-AOS1077 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

A commonly used semiparametric partial linear model is considered. We propose analyzing this model using a difference based approach. The procedure estimates the linear component based on the differences of the observations and then estimates the nonparametric component by either a kernel or a wavelet thresholding method using the residuals of the linear fit. It is shown that both the estimator of the linear component and the estimator of the nonparametric component asymptotically perform as well as if the other component were known. The estimator of the linear component is asymptotically efficient and the estimator of the nonparametric component is asymptotically rate optimal. A test for linear combinations of the regression coefficients of the linear component is also developed. Both the estimation and the testing procedures are easily implementable. Numerical performance of the procedure is studied using both simulated and real data. In particular, we demonstrate our method in an analysis of an attitude data set., National Science Foundation (U.S.) (Grant DMS-1005539)

Most results in nonparametric regression theory are developed only for the case of additive noise. In such a setting many smoothing techniques including wavelet thresholding methods have been developed and shown to be highly adaptive. In this paper we consider nonparametric regression in exponential families with the main focus on the natural exponential families with a quadratic variance function, which include, for example, Poisson regression, binomial regression and gamma regression. We propose a unified approach of using a mean-matching variance stabilizing transformation to turn the relatively complicated problem of nonparametric regression in exponential families into a standard homoscedastic Gaussian regression problem. Then in principle any good nonparametric Gaussian regression procedure can be applied to the transformed data. To illustrate our general methodology, in this paper we use wavelet block thresholding to construct the final estimators of the regression function. The procedures are easily implementable. Both theoretical and numerical properties of the estimators are investigated. The estimators are shown to enjoy a high degree of adaptivity and spatial adaptivity with near-optimal asymptotic performance over a wide range of Besov spaces. The estimators also perform well numerically., Comment: Published in at http://dx.doi.org/10.1214/09-AOS762 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

In this paper we develop a nonparametric regression method that is simultaneously adaptive over a wide range of function classes for the regression function and robust over a large collection of error distributions, including those that are heavy-tailed, and may not even possess variances or means. Our approach is to first use local medians to turn the problem of nonparametric regression with unknown noise distribution into a standard Gaussian regression problem and then apply a wavelet block thresholding procedure to construct an estimator of the regression function. It is shown that the estimator simultaneously attains the optimal rate of convergence over a wide range of the Besov classes, without prior knowledge of the smoothness of the underlying functions or prior knowledge of the error distribution. The estimator also automatically adapts to the local smoothness of the underlying function, and attains the local adaptive minimax rate for estimating functions at a point. A key technical result in our development is a quantile coupling theorem which gives a tight bound for the quantile coupling between the sample medians and a normal variable. This median coupling inequality may be of independent interest., Published in at http://dx.doi.org/10.1214/07-AOS513 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Variance function estimation in nonparametric regression is considered and the minimax rate of convergence is derived. We are particularly interested in the effect of the unknown mean on the estimation of the variance function. Our results indicate that, contrary to the common practice, it is not desirable to base the estimator of the variance function on the residuals from an optimal estimator of the mean when the mean function is not smooth. Instead it is more desirable to use estimators of the mean with minimal bias. On the other hand, when the mean function is very smooth, our numerical results show that the residual-based method performs better, but not substantial better than the first-order-difference-based estimator. In addition our asymptotic results also correct the optimal rate claimed in Hall and Carroll [J. Roy. Statist. Soc. Ser. B 51 (1989) 3--14]., Comment: Published in at http://dx.doi.org/10.1214/009053607000000901 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Interval estimation of a binomial proportion is a very basic but very important problem with an extensive literature. Brown, Cai and DasGupta (2001) revisited this problem and showed that the performance of the standard Wald interval, which is used extensively in textbooks and in practice, is far more erratic and in adequate than is appreciated. Several natural alterna tive confidence intervals for p were recommended in Brown, Cai and DasGupta (2001). See also Agresti and Coull (1998). These intervals are all what the authors call crisp intervals. The coverage probability of these crisp confidence intervals contains significant oscillation, which is in trinsic in all crisp intervals due to the lattice struc ture of the binomial distributions. In the present paper, Geyer and Meeden introduce the notion of fuzzy con fidence intervals with the goal to eliminate oscillation and to have the exact coverage probability. The con fidence intervals are obtained by inverting families of randomized tests. In addition, the authors introduce the notion of fuzzy P-values. The introduction of the crit ical function 0 as a function of three variables x, a and 9 provides a unified description of fuzzy decision, fuzzy confidence interval and fuzzy P-values. Our discussion here will focus on four issues: (1) What is new in this paper?; (2) exact versus ap proximate coverage; (3) expected length; (4) general ization of abstract randomized confidence intervals to simultaneous inference. 1. WHAT IS NEW IN THIS PAPER?

This paper establishes the global asymptotic equivalence between a Poisson process with variable intensity and white noise with drift under sharp smoothness conditions on the unknown function. This equivalence is also extended to density estimation models by Poissonization. The asymptotic equivalences are established by constructing explicit equivalence mappings. The impact of such asymptotic equivalence results is that an investigation in one of these nonparametric models automatically yields asymptotically analogous results in the other models., Published at http://dx.doi.org/10.1214/009053604000000012 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We present a general proposal for testing for goodness of fit, based on resampling and subsampling methods, and illustrate it with graphical and analytical tests for the problems of testing for univariate or multivariate normality. The proposal shows promising, and in some cases dramatic, success in detecting nonnormality. Compared to common competitors, such as a Q-Q plot or a likelihood ratio test against a specified alternative, our proposal seems to be the most useful when the sample size is small, such as 10 or 12, or even very small, such as 6! We also show how our proposal provides tangible information about the nature of the true cdf from which one is sampling. Thus, our proposal also has data analytic value. Although only the normality problem is addressed here, the scope of application of the general proposal should be much broader.

Standard wavelet shrinkage procedures for nonparametric regression are restricted to equispaced samples. There, data are transformed into empirical wavelet coefficients and threshold rules are applied to the coefficients. The estimators are obtained via the inverse transform of the denoised wavelet coefficients. In many applications, however, the samples are nonequispaced. It can be shown that these procedures would produce suboptimal estimators if they were applied directly to nonequispaced samples. ¶ We propose a wavelet shrinkage procedure for nonequispaced samples. We show that the estimate is adaptive and near optimal. For global estimation, the estimate is within a logarithmic factor of the minimax risk over a wide range of piecewise Hölder classes, indeed with a number of discontinuities that grows polynomially fast with the sample size. For estimating a target function at a point, the estimate is optimally adaptive to unknown degree of smoothness within a constant. In addition, the estimate enjoys a smoothness property: if the target function is the zero function, then with probability tending to 1 the estimate is also the zero function.

A class of confidence sets with constant coverage probability for the mean of a p-variate normal distribution is proposed through a pseudo-empirical-Bayes construction. When the dimension is greater than 2, by combining analytical results with some exact numerical calculations the proposed sets are proved to have a uniformly smaller volume than the usual confidence region. Sufficient conditions for the connectedness of the proposed confidence sets are also derived. In addition, our confidence sets could be used to construct tests for point null hypotheses. The resultant tests have convex acceptance regions and hence are admissible by Birnbaum. Tabular results of the comparison between the proposed region and other confidence sets are also given.

The principal result is that, under conditions, to any nonparametric regression problem there corresponds an asymptotically equivalent sequence of white noise with drift problems, and conversely. This asymptotic equivalence is in a global and uniform sense. Any normalized risk function attainable in one problem is asymptotically attainable in the other, with the difference in normalized risks converging to zero uniformly over the entire parameter space. The results are constructive. A recipe is provided for producing these asymptotically equivalent procedures. Some implications and generalizations of the principal result are also discussed.

Preexperimental frequentist error probabilities are arguably inadequate, as summaries of evidence from data, in many hypothesis-testing settings. The conditional frequentist may respond to this by identifying certain subsets of the outcome space and reporting a conditional error probability, given the subset of the outcome space in which the observed data lie. Statistical methods consistent with the likelihood principle, including Bayesian methods, avoid the problem by a more extreme form of conditioning. In this paper we prove that the conditional frequentist's method can be made exactly equivalent to the Bayesian's in simple versus simple hypothesis testing: specifically, we find a conditioning strategy for which the conditional frequentist's reported conditional error probabilities are the same as the Bayesian's posterior probabilities of error. A conditional frequentist who uses such a strategy can exploit other features of the Bayesian approach--for example, the validity of sequential hypothesis tests (including versions of the sequential probability ratio test, or SPRT) even if the stopping rule is incompletely specified.

In many nonparametric problems, such as density estimation, nonparametric regression and so on, all the existing informative estimators are biased (asymptotic or finite sample). There has long been a suspicion that either informative unbiased estimators do not exist for such problems or they must be quite complicated. In this paper, we clarify the nonexistence of informative unbiased estimators in all singular problems both for fixed sample size and asymptotically (this includes most problems with optimal rate of convergence slower than $n^{-1/2}$). We also discuss situations in regular problems where such nonexistences can occur.

This paper presents lower bounds, derived from the information inequality, for the Bayes risk under scaled quadratic loss. Some numerical results are also presented which give some idea concerning the precision of these bounds. An appendix contains a proof of the information inequality without conditions on the estimator. This result is a direct extension of an earlier result of Fabian and Hannan.

This paper contains a complete class theorem (Theorem 3.2) which applies to most statistical estimation problems having a finite sample space. This theorem also applies to many other statistical problems with finite sample spaces. The description of this complete class involves a stepwise algorithm. At each step of the process it is necessary to construct the Bayes procedures in a suitably modified version of the original problem. The complete class is a minimal complete class if the loss function is strictly convex. Some examples are given to illustrate the application of this complete class theorem. Among these is a new result concerning the estimation of the parameters of a multinomial distribution under a normalized quadratic loss function. (See Example 4.5).

Assume the standard linear model $X_{n \times 1} = A_{n \times p} \theta_{p \times 1} + \varepsilon_{n \times 1},$ where $\varepsilon$ has an $n$-variate normal distribution with zero mean vector and identity covariance matrix. The least squares estimator for the coefficient $\theta$ is $\hat{\theta} \equiv (A'A)^{-1}A'X$. It is well known that $\hat{\theta}$ is dominated by James-Stein type estimators under the sum of squared error loss $|\theta - \hat{\theta}|^2$ when $p \geq 3$. In this article we discuss the possibility of improving upon $\hat{\theta}$, simultaneously under the "universal" class of losses: $\{L(|\theta - \hat{\theta}|): L(\cdot) \text{any nondecreasing function}\}.$ An estimator that can be so improved is called universally inadmissible ($U$-inadmissible). Otherwise it is called $U$-admissible. We prove that $\hat{\theta}$ is $U$-admissible for any $p$ when $A'A = I$. Furthermore, if $A'A \neq I$, then $\hat{\theta}$ is $U$-inadmissible if $p$ is "large enough." In a special case, $p \geq 4$ is large enough. The results are surprising. Implications are discussed.

Hypothesis testing problems in which the null hypothesis is simple, the parameter space is finite dimensional and the supports of the probability measures are independent of the parameter are considered. Essentially complete class results are obtained for characterizing the limits of Bayes tests. Conditions for tests to be admissible and the class to be complete are given. Results are then specialized to exponential families, along with some illustrative examples.

We prove in a discrete setting that if for all test functions, $t$, there is a $\mathbf{B}$ measurable test function, $s$, such that $E_p(t) = E_p(s)$ for all $p \in P$ then some subfield of $\mathbf{B}$ is sufficient for $P$.

Consider the problem of sequentially testing composite, contiguous hypotheses where the risk function is a linear combination of the probability of error in the terminal decision and the expected sample size. Assume that the common boundary of the closures of the null and the alternative hypothesis is compact. Observations are independent and identically distributed. We study properties of Bayes tests. One property is the exponential boundedness of the stopping time. Another property is continuity of the risk functions. The continuity property is used to establish complete class theorems as opposed to the essentially complete class theorems in Brown, Cohen and Strawderman.

We describe a general class of multivariate infinitely divisible distributions and their related stochastic processes. Then we prove inequalities which are the analogs of Slepian's inequality for these distributions. These inequalities are applied to the distributions of $M/G/\infty$ queues and of sample cumulative distribution functions for independent multivariate random variables.

Questions pertaining to the admissibility of fixed sample size tests of hypotheses, when sequential tests are available, are considered. For the normal case with unknown mean, suppose the risk function is a linear combination of probability of error and expected sample size. Then any fixed sample size test, with sample size $n \geqslant 2$, is inadmissible. On the other hand, suppose the risk function consists of the pair of components, probability of error and expected sample size. Then any optimal fixed sample size test for the one sided hypothesis is admissible. When the variance of the normal distribution is unknown, $t$-tests are studied. For one-sided hypotheses and componentwise risk functions the fixed sample size $t$-test is inadmissible if and only if the absolute value of the critical value of the test is greater than or equal to one. This implies that for the most commonly used sizes, the fixed sample size $t$-test is inadmissible. Other loss functions are discussed. Also an example for a normal mean problem is given where a nonmonotone test cannot be improved on by a monotone test when the risk is componentwise.

We consider problems of sequential testing when the loss function is the sum of a component due to an error in the terminal decision and a cost of observation component. In all cases we establish a characterization of a complete class or an essentially complete class. In order to obtain such results for testing a null hypothesis against an alternative hypothesis we establish complete class results for testing the closure of the null hypothesis against the closure of the alternative hypothesis. A complete class for testing closure of null against closure of alternative is an essentially complete class for testing null against alternative. Furthermore, a complete class for testing closure of null against closure of alternative is a complete class for testing null against alternative when the risks have certain continuity properties. Such continuity properties do hold in many cases. Three models are treated. The first is when the closure of the null space is compact and the cost of the first observation is positive. Under very unrestrictive conditions it is shown that the Bayes tests form a complete class. This result differs considerably from most fixed sample analogues that have been studied. The second model is when the closure of the null space is compact, the distributions are exponential family, and the cost of the first observation is zero. The third model is for the one dimensional exponential family case when the hypotheses are one sided.

Assume observations are from a subclass of a one parameter exponential family whose dominating measure is nonatomic. Consider a one-sided sequential testing problem where null and alternative parameter sets have one common boundary point. Let the risk function be a linear combination of probability of error and expected sample size. Our main result is that a sequential test is inadmissible if its continuation region has unbounded width in terms of the natural sufficient statistic. We apply this result to prove that weight function tests, with weight functions that contain the common boundary point in their support, are inadmissible. Furthermore any obstructive test is inadmissible, where obstructive means that the stopping time for the test does not have a finite moment generating function for some parameter point. Specific tests of the above type are cited.

We consider multivariate empirical processes $X_n(t) := \sqrt n (F_n(t) - F(t))$, where $F_n$ is an empirical distribution function based on i.i.d. variables with distribution function $F$ and $t \in \mathbb{R}^k$. For $X_F$ the weak limit of $X_n$, it is shown that $c(F, k)\lambda^{2(k-1)}e^{-2\lambda^2} \leq P\big\{\sup_t X_F(t) > \lambda\big\} \leq C(k)\lambda^{2(k-1)}e^{-2\lambda^2}$ for large $\lambda$ and appropriate constants $c, C$. When $k = 2$ these constants can be identified, thus permitting the development of Kolmogorov--Smirnov tests for bivariate problems. For general $k$ the bound can be used to obtain sharp upper-lower class results for the growth of $\sup_tX_n(t)$ with $n$.

The notion of Bahadur efficiency for test statistics is extended to the sequential case and illustrated in the specific context of testing one-sided hypotheses about a normal mean. An analog of Bahadur's theorem on the asymptotic optimality of the likelihood ratio statistic is seen to hold in the normal case. Some possible definitions of attained level for a sequential experiment are considered.

Consider the problem of sequentially testing the hypothesis that the mean of a normal distribution of known variance is less than or equal to a given value versus the alternative that it is greater than the given value. Impose the linear combination loss function under which the risk becomes a constant $c$, times the expected sample size, plus the probability of error. It is known that all admissible tests must be monotone--that is, they stop and accept if $S_n$, the sample sum at stage $n$, satisfies $S_n \leq a_n$; stop and reject if $S_n \geq b_n$. In this paper we show that any admissible test must in addition satisfy $b_n - a_n \leq 2\bar{b}(c)$. The bound $2\bar{b}(c)$ is sharp in the sense that the test with stopping bounds $a_n \equiv -\bar{b}(c), b_n \equiv \bar{b}(c)$ is admissible. As a consequence of the above necessary condition for admissibility of a sequential test, it is possible to characterize all sequential probability ratio tests (SPRT's) regarding admissibility. In other words necessary and sufficient conditions for the admissibility of an SPRT are given. Furthermore, an explicit numerical upper bound for $\bar{b}(c)$ is provided.

Suppose a random variable has a density belonging to a one parameter family which has strict monotone likelihood ratio. For inference regarding the parameter (or a monotone function of the parameter) consider the loss function to be bowl shaped for each fixed parameter and also to have each action be a "point of increase" or a "point of decrease" for some value of the parameter. Under these conditions, given any nonmonotone decision procedure, a unique monotone procedure is constructed which is strictly better than the given procedure for all the above loss functions. This result has application to the following areas: combining data problems, sufficiency, a multivariate one-sided testing problem.

Two examples are presented. In each, $p$ independent normal random variables having unit variance are observed. It is desired to estimate the unknown means, $\theta_i$, and the loss is of the form $L(\theta, a) = (\Sigma^p_{i=1} \nu(\theta_i))^{-1} \Sigma^p_{i=1}\nu(\theta_i)(\theta_i - a_i)^2$. The usual estimator, $\delta_0(x) = x$, is minimax with constant risk. In the first example $\nu(t) = e^{rt}$. It is shown that when $r \neq 0, \delta_0$ is inadmissible if and only if $p \geqslant 2$ whereas when $r = 0$ it is known to be inadmissible if and only if $p \geqslant 3$. In the second example $\nu(t) = (1 + t^2)^{r/2}$. It is shown that $\delta_0$ is inadmissible if $p > (2 - r)/(1 - r)$ and admissible if $p < (2 - r)/(1 - r)$. (In particular $\delta_0$ is admissible for all $p$ when $r \geqslant 1$ and only for $p = 1$ when $r < 0.$) In the first example the first order qualitative description of the better estimator when $\delta_0$ is inadmissible depends on $r$, while in the second example it does not. An example which is closely related to the first example, and which has more significance in applications, has been described by J. Berger.