Search

A class of e-variables is introduced and analyzed. Some examples are presented.

Missing data is a common issue in medical, psychiatry, and social studies. In literature, Multiple Imputation (MI) was proposed to multiply impute datasets and combine analysis results from imputed datasets for statistical inference using Rubin's rule. However, Rubin's rule only works for combined inference on statistical tests with point and variance estimates and is not applicable to combine general F-statistics or Chi-square statistics. In this manuscript, we provide a solution to combine F-test statistics from multiply imputed datasets, when the F-statistic has an explicit fractional form (that is, both the numerator and denominator of the F-statistic are reported). Then we extend the method to combine Chi-square statistics from multiply imputed datasets. Furthermore, we develop methods for two commonly applied F-tests, Welch's ANOVA and Type-III tests of fixed effects in mixed effects models, which do not have the explicit fractional form. SAS macros are also developed to facilitate applications., Comment: 21 pages

It has been observed that the sample mean of certain probability distributions in Billera-Holmes-Vogtmann (BHV) phylogenetic spaces is confined to a lower-dimensional subspace for large enough sample size. This non-standard behavior has been called stickiness and poses difficulties in statistical applications when comparing samples of sticky distributions. We extend previous results on stickiness to show the equivalence of this sampling behavior to topological conditions in the special case of BHV spaces. Furthermore, we propose to alleviate statistical comparision of sticky distributions by including the directional derivatives of the Fr\'echet function: the degree of stickiness., Comment: 8 Pages, 1 Figure, conference submission to GSI 2023

When analyzing data researchers make some decisions that are either arbitrary, based on subjective beliefs about the data generating process, or for which equally justifiable alternative choices could have been made. This wide range of data-analytic choices can be abused, and has been one of the underlying causes of the replication crisis in several fields. Recently, the introduction of multiverse analysis provides researchers with a method to evaluate the stability of the results across reasonable choices that could be made when analyzing data. Multiverse analysis is confined to a descriptive role, lacking a proper and comprehensive inferential procedure. Recently, specification curve analysis adds an inferential procedure to multiverse analysis, but this approach is limited to simple cases related to the linear model, and only allows researchers to infer whether at least one specification rejects the null hypothesis, but not which specifications should be selected. In this paper we present a Post-selection Inference approach to Multiverse Analysis (PIMA) which is a flexible and general inferential approach that accounts for all possible models, i.e., the multiverse of reasonable analyses. The approach allows for a wide range of data specifications (i.e. pre-processing) and any generalized linear model; it allows testing the null hypothesis of a given predictor not being associated with the outcome, by merging information from all reasonable models of multiverse analysis, and provides strong control of the family-wise error rate such that it allows researchers to claim that the null-hypothesis can be rejected for each specification that shows a significant effect. The inferential proposal is based on a conditional resampling procedure. To be continued..., Comment: 37 pages, 2 figures

We introduce a multiple testing procedure that controls the median of the proportion of false discoveries (FDP) in a flexible way. The procedure only requires a vector of p-values as input and is comparable to the Benjamini-Hochberg method, which controls the mean of the FDP. Our method allows freely choosing one or several values of alpha after seeing the data -- unlike Benjamini-Hochberg, which can be very liberal when alpha is chosen post hoc. We prove these claims and illustrate them with simulations. Our procedure is inspired by a popular estimator of the total number of true hypotheses. We adapt this estimator to provide simultaneously median unbiased estimators of the FDP, valid for finite samples. This simultaneity allows for the claimed flexibility. Our approach does not assume independence. The time complexity of our method is linear in the number of hypotheses, after sorting the p-values.

We consider the problem of goodness-of-fit testing for a model that has at least one unknown parameter that cannot be eliminated by transformation. Examples of such problems can be as simple as testing whether a sample consists of independent Gamma observations, or whether a sample consists of independent Generalised Pareto observations given a threshold. Over time the approach to determining the distribution of a test statistic for such a problem has moved towards on-the-fly calculation post observing a sample. Modern approaches include the parametric bootstrap and posterior predictive checks. We argue that these approaches are merely approximations to integrating over the posterior predictive distribution that flows naturally from a given model. Further, we attempt to demonstrate that shortcomings which may be present in the parametric bootstrap, especially in small samples, can be reduced through the use of objective Bayes techniques, in order to more reliably produce a test with the correct size., Comment: Bayes, Distribution, Gamma, GPD, Hypothesis Testing, Objective Bayes, p-value, Predictive Posterior, Simulation

This paper reexamines the seminal Lagrange multiplier test for cross-section independence in a large panel model where both the number of cross-sectional units n and the number of time series observations T can be large. The first contribution of the paper is an enlargement of the test with two extensions: firstly the new asymptotic normality is derived in a simultaneous limiting scheme where the two dimensions (n, T) tend to infinity with comparable magnitudes; second, the result is valid for general error distribution (not necessarily normal). The second contribution of the paper is a new test statistic based on the sum of the fourth powers of cross-section correlations from OLS residuals, instead of their squares used in the Lagrange multiplier statistic. This new test is generally more powerful, and the improvement is particularly visible against alternatives with weak or sparse cross-section dependence. Both simulation study and real data analysis are proposed to demonstrate the advantages of the enlarged Lagrange multiplier test and the power enhanced test in comparison with the existing procedures., Comment: 39 papers, 8 figures

In this paper we present the results from an empirical power comparison of 40 goodness-of-fit tests for the univariate Laplace distribution, carried out using Monte Carlo simulations with sample sizes $n = 20, 50, 100, 200$, significance levels $\alpha = 0.01, 0.05, 0.10$, and 400 alternatives consisting of asymmetric and symmetric light/heavy-tailed distributions taken as special cases from 11 models. In addition to the unmatched scope of our study, an interesting contribution is the proposal of an innovative design for the selection of alternatives. The 400 alternatives consist of 20 specific cases of 20 submodels drawn from the main 11 models. For each submodel, the 20 specific cases corresponded to parameter values chosen to cover the full power range. An analysis of the results leads to a recommendation of the best tests for five different groupings of the alternative distributions. A real-data example is also presented, where an appropriate test for the goodness-of-fit of the univariate Laplace distribution is applied to weekly log-returns of Amazon stock over a recent four-year period., Comment: 48 pages, 2 figures, 38 tables

Statistical methods are based on model assumptions, and it is statistical folklore that a method's model assumptions should be checked before applying it. This can be formally done by running one or more misspecification tests of model assumptions before running a method that requires these assumptions; here we focus on model-based tests. A combined test procedure can be defined by specifying a protocol in which first model assumptions are tested and then, conditionally on the outcome, a test is run that requires or does not require the tested assumptions. Although such an approach is often taken in practice, much of the literature that investigated this is surprisingly critical of it. Our aim is to explore conditions under which model checking is advisable or not advisable. For this, we review results regarding such "combined procedures" in the literature, we review and discuss controversial views on the role of model checking in statistics, and we present a general setup in which we can show that preliminary model checking is advantageous, which implies conditions for making model checking worthwhile., Comment: 35 pages, 1 figure

The Wald test statistic has been shown to diverge (Dufour et al, 2013, 2017) under some conditions. This note links the divergence to eigenvalues of a polynomial matrix and establishes the divergence rate.

Although hypothesis tests play a prominent role in Science, their interpretation can be challenging. Three issues are (i) the difficulty in making an assertive decision based on the output of an hypothesis test, (ii) the logical contradictions that occur in multiple hypothesis testing, and (iii) the possible lack of practical importance when rejecting a precise hypothesis. These issues can be addressed through the use of agnostic tests and pragmatic hypotheses., Comment: 19 pages, 3 figures

This paper develops a smooth test of goodness-of-fit for elliptical distributions. The test is adaptively omnibus, invariant to affine-linear transformations and has a convenient expression that can be broken into components. These components have diagnostic capabilities and can be used to identify specific departures. This helps in correcting the null model when the test rejects. As an example, the results are applied to the multivariate normal distribution for which the R package ECGofTestDx is available. It is shown that the proposed test strategy encompasses and generalizes a number of existing approaches. Some other cases are studied, such as the bivariate Laplace, logistic and Pearson type II distribution. A simulation experiment shows the usefulness of the diagnostic tools., Comment: 35 p. pre-print

We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class encompasses methods controlling familywise error rate, generalized familywise error rate, false discovery exceedance, joint error rate, simultaneous control of all false discovery proportions, and others, as well as seemingly unrelated methods such as gene set testing in genomics and cluster inference methods in neuroimaging. We show that all such methods are either equivalent to a closed testing method, or are uniformly improved by one. Moreover, we show that a closed testing method is admissible as a method controlling tail probabilities of false discovery proportions if and only if all its local tests are admissible. This implies that, when designing such methods, it is sufficient to restrict attention to closed testing methods only. We demonstrate the practical usefulness of this design principle by constructing a uniform improvement of a recently proposed method.

Despite its common practice, statistical hypothesis testing presents challenges in interpretation. For instance, in the standard frequentist framework there is no control of the type II error. As a result, the non-rejection of the null hypothesis cannot reasonably be interpreted as its acceptance. We propose that this dilemma can be overcome by using agnostic hypothesis tests, since they can control the type I and II errors simultaneously. In order to make this idea operational, we show how to obtain agnostic hypothesis in typical models. For instance, we show how to build (unbiased) uniformly most powerful agnostic tests and how to obtain agnostic tests from standard p-values. Also, we present conditions such that the above tests can be made logically coherent. Finally, we present examples of consistent agnostic hypothesis tests., Comment: 21 pages, 8 figures

In this paper, our interest is in the problem of simultaneous hypothesis testing when the test statistics corresponding to the individual hypotheses are possibly correlated. Specifically, we consider the case when the test statistics together have a multivariate normal distribution (with equal correlation between each pair) with an unknown mean vector and our goal is to decide which components of the mean vector are zero and which are non-zero. This problem was taken up earlier in Bogdan et al. (2011) for the case when the test statistics are independent normals. Asymptotic optimality in a Bayesian decision theoretic sense was studied in this context, the optimal precodures were characterized and optimality of some well-known procedures were thereby established. The case under dependence was left as a challenging open problem. We have studied the problem both theoretically and through extensive simulations and have given some permutation invariant rules. Though in Bogdan et al. (2011), the asymptotic derivations were done in the context of sparsity of the non-zero means, our result does not require the assumption of sparsity and holds under a more general setup., Comment: 16 pages, 0 figures, 2 tables

This paper describes a method for performing inference on models chosen by cross-validation. When the test error being minimized in cross-validation is a residual sum of squares it can be written as a quadratic form. This allows us to apply the inference framework in Loftus et al. (2015) for models determined by quadratic constraints to the model that minimizes CV test error. Our only requirement on the model training pro- cedure is that its selection events are regions satisfying linear or quadratic constraints. This includes both Lasso and forward stepwise, which serve as our main examples throughout. We do not require knowledge of the error variance $\sigma^2$. The procedures described here are computationally intensive methods of selecting models adaptively and performing inference for the selected model. Implementations are available in an R package., Comment: 9 pages, 2 figures

Likelihood-based methods of statistical inference provide a useful general methodology that is appealing, as a straightforward asymptotic theory can be applied for their implementation. It is important to assess the relationships between different likelihood-based inferential procedures in terms of accuracy and adherence to key principles of statistical inference, in particular those relating to conditioning on relevant ancillary statistics. An analysis is given of the stability properties of a general class of likelihood-based statistics, including those derived from forms of adjusted profile likelihood, and comparisons are made between inferences derived from different statistics. In particular, we derive a set of sufficient conditions for agreement to $O_{p}(n^{-1})$, in terms of the sample size $n$, of inferences, specifically $p$-values, derived from different asymptotically standard normal pivots. Our analysis includes inference problems concerning a scalar or vector interest parameter, in the presence of a nuisance parameter., Comment: Accepted for publication in Statistica Sinica

When conducting a meta-analysis of standardized mean differences (SMDs), it is common to assume equal variances in the two arms of each study. This leads to Cohen's $d$ estimates for which interpretation is simple. However, this simplicity should not be used as a justification for the assumption of equal variances in situations where evidence may suggest that it is incorrect. Until now, researchers have either used an $F$-test for each individual study as a justification for the equality of variances or perhaps even conveniently ignored such tools altogether. In this paper we propose using a meta-analysis of F-test statistics to estimate the ratio of variances prior to the combination of SMD's. This procedure allows some studies to be included that might otherwise be omitted by individual fixed level tests for unequal variances, sometimes occur even when the assumption of equal variances holds. The estimated ratio of variances, as well as associated confidence intervals, can be used as guidance as to whether the assumption of equal variances is violated. The estimators considered include variance stabilization transformations (VST) of the $F$-test statistics as well as MLE estimators. The VST approaches enable the use of QQ-plots to visually inspect for violations of equal variances while the MLE estimator easily allows for the introduction of a random effect. When there is evidence of unequal variances, this work provides a means to formally justify the use of less common methods such as log ratio of means when studies are measured on a different scale.

Efforts to develop more efficient multiple hypothesis testing procedures for false discovery rate (FDR) control have focused on incorporating an estimate of the proportion of true null hypotheses (such procedures are called adaptive) or exploiting heterogeneity across tests via some optimal weighting scheme. This paper combines these approaches using a weighted adaptive multiple decision function (WAMDF) framework. Optimal weights for a flexible random effects model are derived and a WAMDF that controls the FDR for arbitrary weighting schemes when test statistics are independent under the null hypotheses is given. Asymptotic and numerical assessment reveals that, under weak dependence, the proposed WAMDFs provide more efficient FDR control even if optimal weights are misspecified. The robustness and flexibility of the proposed methodology facilitates the development of more efficient, yet practical, FDR procedures for heterogeneous data. To illustrate, two different weighted adaptive FDR methods for heterogeneous sample sizes are developed and applied to data.

We consider the problem of detecting a rectangle of activation in a grid of sensors in d-dimensions with noisy measurements. This has applications to massive surveillance projects and anomaly detection in large datasets in which one detects anomalously high measurements over rectangular regions, or more generally, blobs. Recently, the asymptotic distribution of a multiscale scan statistic was established in (Kabluchko, 2011) under the null hypothesis, using non-constant boundary crossing probabilities for locally-stationary Gaussian random fields derived in (Chan and Lai, 2006). Using a similar approach, we derive the exact asymptotic level and power of four variants of the scan statistic: an oracle scan that knows the dimensions of the activation rectangle; the multiscale scan statistic just mentioned; an adaptive variant; and an epsilon-net approximation to the latter, in the spirit of (Arias-Castro, 2005). This approximate scan runs in time near-linear in the size of the grid and achieves the same asymptotic power as the adaptive scan. We complement our theory with some numerical experiments.

Uplift, essentially being the difference between two probabilities, is a central number in marketing performance measurement. A frequent question in applications is whether the uplifts of two campaigns are significantly different. In this article we present a new $\chi^2$-statistic which allows to answer this question by performing a statistical test. We show that this statistic is asymptotically $\chi^2$-distributed and demonstrate its application in a real life example. By running simulations with this new and alternative approaches, we find our suggested test to exhibit a better decisive power., Comment: 18 pages, 7 figures

Large-scale multiple testing with highly correlated test statistics arises frequently in many scientific research. Incorporating correlation information in estimating false discovery proportion has attracted increasing attention in recent years. When the covariance matrix of test statistics is known, Fan, Han & Gu (2012) provided a consistent estimate of False Discovery Proportion (FDP) under arbitrary dependence structure. However, the covariance matrix is often unknown in many applications and such dependence information has to be estimated before estimating FDP (Efron, 2010). The estimation accuracy can greatly affect the convergence result of FDP or even violate its consistency. In the current paper, we provide methodological modification and theoretical investigations for estimation of FDP with unknown covariance. First we develop requirements for estimates of eigenvalues and eigenvectors such that we can obtain a consistent estimate of FDP. Secondly we give conditions on the dependence structures such that the estimate of FDP is consistent. Such dependence structures include sparse covariance matrices, which have been popularly considered in the contemporary random matrix theory. When data are sampled from an approximate factor model, which encompasses most practical situations, we provide a consistent estimate of FDP via exploiting this specific dependence structure. The results are further demonstrated by simulation studies and some real data applications., Comment: 39 pages, 7 figures

Often it is not easy to choose between estimators, based on the estimated MSE and bias using simulation studies. Normality in small samples and a variance of the estimator, which is correct and easy to calculate using a single sample, give the added advantage that hypotheses concerning the parameter can be tested in new samples. A procedure to check normality is proposed where previously published MSE and bias are used to perform a test for normality. A confidence interval for the index of the S&P500 index is found by applying the results to estimators of the generalized Pareto distribution.

In contrast to its common definition and calculation, interpretation of p-values diverges among statisticians. Since p-value is the basis of various methodologies, this divergence has led to a variety of test methodologies and evaluations of test results. This chaotic situation has complicated the application of tests and decision processes. Here, the origin of the divergence is found in the prior probability of the test. Effects of difference in Pr(H0 = true) on the character of p-values are investigated by comparing real microarray data and its artificial imitations as subjects of Student's t-tests. Also, the importance of the prior probability is discussed in terms of the applicability of Bayesian approaches. Suitable methodology is found in accordance with the prior probability and purpose of the test., Comment: 16 pages, 3 figures, and 1 table

In this paper we have suggested a family of estimators for the population mean when study variable itself is qualitative in nature. Expressions for the bias and mean square error (MSE) of the suggested family have been obtained. An empirical study has been carried out to show the superiority of the constructed estimator over others., Comment: 9 pages

The limitation of permutation tests is that they assume exchangeability. It is shown that in generalized linear models one can construct permutation tests from score statistics in particular cases. When under the null hypothesis the observations are not exchangeable, a representation in terms of Cox-Snell residuals allows to develop an approach based on an expected permutation p-value (Eppv); this is applied to the logistic regression model. A small simulation study and an illustration with real data are given., Comment: 15 pages, 1 table

The class of dual $\phi$-divergence estimators (introduced in Broniatowski and Keziou (2009) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criterions are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both noncontaminated and contaminated data.

Some ratio estimators for estimating the population mean of the variable under study, which make use of information regarding the population proportion possessing certain attribute, are proposed. Under simple random sampling without replacement (SRSWOR) scheme, the expressions of bias and mean-squared error (MSE) up to the first order of approximation are derived. The results obtained have been illustrated numerically by taking some empirical population considered in the literature., Comment: 7 pages

In this paper we obtain a test which minimizes the sum of the two error probabilities irrespective of whether $\sigma^2$ is known or unknown., Comment: 16 pages

We introduce estimation and test procedures through divergence optimization for discrete or continuous parametric models. This approach is based on a new dual representation for divergences. We treat point estimation and tests for simple and composite hypotheses, extending maximum likelihood technique. An other view at the maximum likelihood approach, for estimation and test, is given. We prove existence and consistency of the proposed estimates. The limit laws of the estimates and test statistics (including the generalized likelihood ratio one) are given both under the null and the alternative hypotheses, and approximation of the power functions is deduced. A new procedure of construction of confidence regions, when the parameter may be a boundary value of the parameter space, is proposed. Also, a solution to the irregularity problem of the generalized likelihood ratio test pertaining to the number of components in a mixture is given, and a new test is proposed, based on $\chi ^{2}$-divergence on signed finite measures and duality technique.

In this paper, a problem of testing is discussed when the samples have been drawn from the normal distribution. The study of hypothesis testing is also extended to Baye's set up., Comment: 5 pages

We compute the loss of power in likelihood ratio tests when we test the original parameter of a probability density extended by the first Lehmann alternative., Comment: 6 pages, one figure; Corrected minor typographic errors (pay special attention to one in equation 4)

We look at the semigroup generated by a system of heat equations. Applications to testing normality and option pricing are addressed., Comment: 16 pages, 4 figures

Let $\BS_1,...,\BS_n$ be independent identically distributed random variables each having the standardized Bernoulli distribution with parameter $p\in(0,1)$. Let $m_*(p):=(1+p+2p^2)/(2\sqrt{p-p^2}+4p^2)$ if $0