Search

Hierarchical nonparametric processes are popular tools for defining priors on collections of probability distributions, which induce dependence across multiple samples. In survival analysis problems, one is typically interested in modeling the hazard rates, rather than the probability distributions themselves, and the currently available methodologies are not applicable. Here, we fill this gap by introducing a novel, and analytically tractable, class of multivariate mixtures whose distribution acts as a prior for the vector of sample-specific baseline hazard rates. The dependence is induced through a hierarchical specification of the mixing random measures that ultimately corresponds to a composition of random discrete combinatorial structures. Our theoretical results allow to develop a full Bayesian analysis for this class of models, which can also account for right-censored survival data and covariates, and we also show posterior consistency. In particular, we emphasize that the posterior characterization we achieve is the key for devising both marginal and conditional algorithms for evaluating Bayesian inferences of interest. The effectiveness of our proposal is illustrated through some synthetic and real data examples.

The use of large datasets for targeted therapeutic interventions requires new ways to characterize the heterogeneity observed across subgroups of a specific population. In particular, models for partially exchangeable data are needed for inference on nested datasets, where the observations are assumed to be organized in different units and some sharing of information is required to learn distinctive features of the units. In this manuscript, we propose a nested common atoms model (CAM) that is particularly suited for the analysis of nested datasets where the distributions of the units are expected to differ only over a small fraction of the observations sampled from each unit. The proposed CAM allows a two-layered clustering at the distributional and observational level and is amenable to scalable posterior inference through the use of a computationally efficient nested slice sampler algorithm. We further discuss how to extend the proposed modeling framework to handle discrete measurements, and we conduct posterior inference on a real microbiome dataset from a diet swap study to investigate how the alterations in intestinal microbiota composition are associated with different eating habits. We further investigate the performance of our model in capturing true distributional structures in the population by means of a simulation study.

Protection against disclosure is a legal and ethical obligation for agencies releasing microdata files for public use. Consider a microdata sample of size $n$ from a finite population of size $\bar{n}=n+\lambda n$, with $\lambda>0$, such that each record contains two disjoint types of information: identifying categorical information and sensitive information. Any decision about releasing data is supported by the estimation of measures of disclosure risk, which are functionals of the number of sample records with a unique combination of values of identifying variables. The most common measure is arguably the number $\tau_{1}$ of sample unique records that are population uniques. In this paper, we first study nonparametric estimation of $\tau_{1}$ under the Poisson abundance model for sample records. We introduce a class of linear estimators of $\tau_{1}$ that are simple, computationally efficient and scalable to massive datasets, and we give uniform theoretical guarantees for them. In particular, we show that they provably estimate $\tau_{1}$ all of the way up to the sampling fraction $(\lambda+1)^{-1}\propto (\log n)^{-1}$, with vanishing normalized mean-square error (NMSE) for large $n$. We then establish a lower bound for the minimax NMSE for the estimation of $\tau_{1}$, which allows us to show that: i) $(\lambda+1)^{-1}\propto (\log n)^{-1}$ is the smallest possible sampling fraction; ii) estimators' NMSE is near optimal, in the sense of matching the minimax lower bound, for large $n$. This is the main result of our paper, and it provides a precise answer to an open question about the feasibility of nonparametric estimation of $\tau_{1}$ under the Poisson abundance model and for a sampling fraction $(\lambda+1)^{-1}

In the 1920s, the English philosopher W.E. Johnson introduced a characterization of the symmetric Dirichlet prior distribution in terms of its predictive distribution. This is typically referred to as Johnson’s “sufficientness” postulate, and it has been the subject of many contributions in Bayesian statistics, leading to predictive characterization for infinite-dimensional generalizations of the Dirichlet distribution, i.e., species-sampling models. In this paper, we review “sufficientness” postulates for species-sampling models, and then investigate analogous predictive characterizations for the more general feature-sampling models. In particular, we present a “sufficientness” postulate for a class of feature-sampling models referred to as Scaled Processes (SPs), and then discuss analogous characterizations in the general setup of feature-sampling models.

The mean density estimation of a random closed set in $$\mathbb {R}^d$$ , based on a single observation, is a crucial problem in several application areas. In the case of stationary random sets, a common practice to estimate the mean density is to take the n-dimensional volume fraction with observation window as large as possible. In the present paper, we provide large and moderate deviation results for these estimators when the random closed set $$\Theta _n$$ belongs to the quite general class of stationary Boolean models with Hausdorff dimension $$n

Etant donne un echantillon de taille n dans une population d’individus appartenant a differents types dont les proportions sont inconnues, comment estimer la probabilite de decouvrir un nouveau type au (n+1)-ieme tirage ? C’est un probleme classique en statistique, souvent appele le probleme de l’estimation de la masse manquante. Des resultats recents on montre : (i) l’impossibilite d’estimer la masse manquante sans imposer des hypotheses sur les proportions des types ; (ii) la convergence de l’estimateur de la masse manquante de Good–Turing sous l’hypothese que la queue des proportions des types decroit vers 0 comme une fonction reguliere de parametre α∈(0,1) ; (iii) la vitesse de convergence n−α/2 pour l’estimateur de Good–Turing pour la classe de probabilites a variation reguliere α∈(0,1). Dans cet article, nous proposons une preuve alternative, et remarquablement plus courte, de l’impossibilite de l’estimation de la masse manquante sans hypothese sur la distribution. Au dela de son interet propre, cette preuve alternative suggere une approche naturelle pour ameliorer et etendre les resultats de vitesse de convergence de l’estimateur de Good–Turing sous l’hypothese de proportions a variation reguliere α∈(0,1). En particulier, nous montrons que la vitesse de convergence n−α/2 est la meilleure que peut atteindre un estimateur, a une fonction a variation bornee pres. De plus, nous montrons qu’une borne inferieure a l’estimation du risque minimax est au moins d’echelle n−α/2, ce qui amene a la conjecture que l’estimateur de Good–Turing est l’estimateur minimax de vitesse optimale sous une hypothese de proportions a variation reguliere.

Post randomization methods are among the most popular disclosure limitation techniques for both categorical and continuous data. In the categorical case, given a stochastic matrix M and a specified variable, an individual belonging to category i is changed to category j with probability $$M_{i,j}$$ M i , j . Every approach to choose the randomization matrix M has to balance between two desiderata: (1) preserving as much statistical information from the raw data as possible; (2) guaranteeing the privacy of individuals in the dataset. This trade-off has generally been shown to be very challenging to solve. In this work, we use recent tools from the computer science literature and propose to choose M as the solution of a constrained maximization problems. Specifically, M is chosen as the solution of a constrained maximization problem, where we maximize the mutual information between raw and transformed data, given the constraint that the transformation satisfies the notion of differential privacy. For the general categorical model, it is shown how this maximization problem reduces to a convex linear programming and can be therefore solved with known optimization algorithms.

Feature allocation models generalize species sampling models by allowing every observation to belong to more than one species, now called features. Under the popular Bernoulli product model for feature allocation, given $n$ samples, we study the problem of estimating the missing mass $M_{n}$, namely the expected number hitherto unseen features that would be observed if one additional individual was sampled. This is motivated by numerous applied problems where the sampling procedure is expensive, in terms of time and/or financial resources allocated, and further samples can be only motivated by the possibility of recording new unobserved features. We introduce a simple, robust and theoretically sound nonparametric estimator $\hat{M}_{n}$ of $M_{n}$. $\hat{M}_{n}$ turns out to have the same analytic form of the popular Good-Turing estimator of the missing mass in species sampling models, with the difference that the two estimators have different ranges. We show that $\hat{M}_{n}$ admits a natural interpretation both as a jackknife estimator and as a nonparametric empirical Bayes estimator, we give provable guarantees for the performance of $\hat{M}_{n}$ in terms of minimax rate optimality, and we provide with an interesting connection between $\hat{M}_{n}$ and the Good-Turing estimator for species sampling. Finally, we derive non-asymptotic confidence intervals for $\hat{M}_{n}$, which are easily computable and do not rely on any asymptotic approximation. Our approach is illustrated with synthetic data and SNP data from the ENCODE sequencing genome project.

The mean density of a random closed set with integer Hausdorff dimension is a crucial notion in stochastic geometry, in fact it is a fundamental tool in a large variety of applied problems, such as image analysis, medicine, computer vision, etc. Hence the estimation of the mean density is a problem of interest both from a theoretical and computational standpoint. Nowadays different kinds of estimators are available in the literature, in particular here we focus on a kernel–type estimator, which may be considered as a generalization of the traditional kernel density estimator of random variables to the case of random closed sets. The aim of the present paper is to provide asymptotic properties of such an estimator in the context of Boolean models, which are a broad class of random closed sets. More precisely we are able to prove large and moderate deviation principles, which allow us to derive the strong consistency of the estimator of the mean density as well as asymptotic confidence intervals. Finally we underline the connection of our theoretical findings with classical literature concerning density estimation of random variables.

The definition and investigation of general classes of nonparametric priors has recently been an active research line in Bayesian statistics. Among the various proposals, the Gibbs-type family, which includes the Dirichlet process as a special case, stands out as the most tractable class of nonparametric priors for exchangeable sequences of observations. This is the consequence of a key simplifying assumption on the learning mechanism, which, however, has justification except that of ensuring mathematical tractability. In this paper, we remove such an assumption and investigate a general class of random probability measures going beyond the Gibbs-type framework. More specifically, we present a nonparametric hierarchical structure based on transformations of completely random measures, which extends the popular hierarchical Dirichlet process. This class of priors preserves a good degree of tractability, given that we are able to determine the fundamental quantities for Bayesian inference. In particular, we derive the induced partition structure and the prediction rules and characterize the posterior distribution. These theoretical results are also crucial to devise both a marginal and a conditional algorithm for posterior inference. An illustration concerning prediction in genomic sequencing is also provided.

The prediction of future outcomes of a random phenomenon is typically based on a certain number of “analogous” observations from the past. When observations are generated by multiple samples, a natural notion of analogy is partial exchangeability and the problem of prediction can be effectively addressed in a Bayesian nonparametric setting. Instead of confining ourselves to the prediction of a single future experimental outcome, as in most treatments of the subject, we aim at predicting features of an unobserved additional sample of any size. We first provide a structural property of prediction rules induced by partially exchangeable arrays, without assuming any specific nonparametric prior. Then we focus on a general class of hierarchical random probability measures and devise a simulation algorithm to forecast the outcome of m future observations, for any m≥1. The theoretical result and the algorithm are illustrated by means of a real dataset, which also highlights the “borrowing strength” behavior across samples induced by the hierarchical specification.

The problem of the evaluation and estimation of the mean density of random closed sets in $\mathbb{R} ^{d}$ with integer Hausdorff dimension $0