Search

In the interest of business innovation, social network companies often carry out experiments to test product changes and new ideas. In such experiments, users are typically assigned to one of two experimental conditions with some outcome of interest observed and compared. In this setting, the outcome of one user may be influenced by not only the condition to which they are assigned but also the conditions of other users via their network connections. This challenges classical experimental design and analysis methodologies and requires specialized methods. We introduce the general additive network effect (GANE) model, which encompasses many existing outcome models in the literature under a unified model-based framework. The model is both interpretable and flexible in modeling the treatment effect as well as the network influence. We show that (quasi) maximum likelihood estimators are consistent and asymptotically normal for a family of model specifications. Quantities of interest such as the global treatment effect are defined and expressed as functions of the GANE model parameters, and hence inference can be carried out using likelihood theory. We further propose the “power-degree” (POW-DEG) specification of the GANE model. The performance of POW-DEG and other specifications of the GANE model are investigated via simulations. Under model misspecification, the POW-DEG specification appears to work well. Finally, we study the characteristics of good experimental designs for the POW-DEG specification. We find that graph-cluster randomization and balanced designs are not necessarily optimal for precise estimation of the global treatment effect, indicating the need for alternative design strategies.

After more than a decade since the introduction of Statistical Engineering by Roger Hoerl and Ronald Snee, a group of leading applied statisticians from academia, industry, and government were invited to discuss their perspectives on progress made, the current status of this important movement, and what future Statistical Engineering holds on the path forward in a series of two panel discussion papers. In this first article, the invited panelists focus their discussion on the past and present of Statistical Engineering. They discuss notable advances and current obstacles to progress. They also consider the unique value added by Statistical Engineering, and the possible addition of decision making to the body of knowledge. The format of the article consists of the posed questions from the moderators, a summary of key ideas from all the panelists, and then the individual detailed answers. The goal of this series of articles is to inspire statisticians to consider their possible role to advance the adoption of Statistical Engineering to solve important problems.

This study uses coronavirus disease 2019 (COVID-19) case counts and Google mobility data for 12 of Ontario's largest Public Health Units from Spring 2020 until the end of January 2021 to evaluate the effects of non-pharmaceutical interventions (NPIs; policy restrictions on business operations and social gatherings) and population mobility on daily cases. Instrumental variables (IV) estimation is used to account for potential simultaneity bias, because both daily COVID-19 cases and NPIs are dependent on lagged case numbers. IV estimates based on differences in lag lengths to infer causal estimates imply that the implementation of stricter NPIs and indoor mask mandates are associated with reductions in COVID-19 cases. Moreover, estimates based on Google mobility data suggest that increases in workplace attendance are correlated with higher case counts. Finally, from October 2020 to January 2021, daily Ontario forecasts from Box-Jenkins time-series models are more accurate than official forecasts and forecasts from a susceptible-infected-removed epidemiology model.Cette étude cherche à évaluer les effets des interventions non pharmaceutiques (INPs; restrictions sur les activités commerciales et rassemblements sociaux) et de la mobilité de la population sur le nombre de cas d’infection par jour, en utilisant les nombres de cas d’infection par la maladie à coronavirus 2019 (COVID-19) et les données de mobilité de Google pour 12 des plus grands Bureaux de Santé publique de l’Ontario entre le printemps 2020 et la fin janvier 2021. La méthode des variables instrumentales (VI) permet de rendre compte d’un biais potentiel de simultanéité puisque les taux quotidiens de COVID-19 et les INPs dépendent, tous les deux, du nombre de cas décalés. Les estimations par les VI basées sur les différences de durée des décalages d’ajustement pour inférer des estimations causales impliquent que de plus strictes INPs et le port obligatoire du masque dans les endroits fermés sont associés à une réduction de cas d’infection. Par ailleurs, Les estimations basées sur les données de mobilité de Google montrent que la présence accrue sur le lieu du travail est corrélée avec un plus grand nombre de cas d’infection. Finalement, d’octobre 2020 à Janvier 2021, les prévisions faites à partir de modèles de Box-Jenkins en série chronologique s’avèrent plus précises que les prévisions officielles et que celles utilisant le modèle épidémiologique susceptible – infecté – retiré.

In the second of two panel discussion articles focused on the evolution of statistical engineering (SE) as introduced by Roger Hoerl and Ronald Snee, a group of leading applied statisticians from academia, industry, and government present their perspectives on what the future might hold for this important movement. The invited panelists discuss the challenges and opportunities presented by the emergence of data science and the abundance of large amounts of data. They also consider the possible paths forward for SE, and the roles for statisticians in academia, industry, and government. The final question addresses what additional skills would be helpful to increase the effectiveness of the practice and advance SE. As with the first article, the format of the article follows the order of a posed question, a summary of key ideas, and then the detailed individual panelist answers. The article seeks to inspire statisticians to consider their possible role to leverage the potential of SE to solve important problems.

We propose a latent-lognormal variance components model for counts that is analogous to the standard normal model for continuous measurements. We analyze gauge repeatability and reproducibility stu...

Dorian Shainin’s group comparison procedure is a way to find the dominant cause(s) of variation in a process, based on an investigation plan that exploits the idea of leveraging. In this paper, we ...

A variety of random graph models have been proposed in the literature to model the associations within an interconnected system and to realistically account for various structures and attributes of...

We investigate efficient plans to assess the misclassification error rates of a binary measurement system used as an in-line inspection protocol. We assume that parts can be inspected repeatedly and that each part has its own (latent) misclassification rate. We propose a three-phase assessment plan. Phase I consists of data from recent inspection history. In Phase II, we select a sample of failed parts that we re-measure multiple times with the binary measurement system of interest. In Phase III, we verify a carefully selected subsample of the parts from Phase II with the aid of a binary gold standard measurement system. We show that the proposed plan is a substantial improvement over existing assessment plans in terms of cost and/or precision.

We consider a measurement system replacement case study. We show that with the collected data we can determine a decision rule for the corresponding 100 percent inspection scheme with the n...

There are various methods for measuring flow rates in rivers, but all of them have practical issues and challenges. A period of exceptionally high water levels revealed substantial discrepancies be...

Consider the assessment of a binary measurement system with multiple operators when a gold standard measurement system is also available (for the assessment study). Data are collected as in a gauge repeatability and reproducibility plan for a continuous measurement system and each operator in the study measures a number of parts multiple times. We characterize the performance of the measurement system by estimating the probabilities of accepting a non-conforming part and of rejecting a conforming part. To model the data, we assume that some parts are more difficult to correctly classify than others and so choose to use random part effects. We consider two cases, modeling the operator effects as fixed or random. For each, we study a conditional and marginal model and their corresponding estimates of the parameters of interest. We also provide guidance on the planning of the assessment study in terms of the number of parts, number of operators and number of repeated measurements.

To start I would like to thank Dr. Fogel for his interesting and informative paper as well as for the very well received talk he gave at the recent Stu Hunter conference in Roanoke Virginia. His wo...

We recommend an approach to estimate a process performance measure (or parameter) at the present time from a stream of data where the performance may drift slowly over time. It is common practice to estimate current process performance using either present-time data only or including all historical data. When sample sizes by time period are small, an estimate based only on present-time data is imprecise. When the performance changes over time, including historical data in estimation trades more bias for less variability. We propose to regulate the bias/variance trade-off using estimating equations that down-weight past data. We derive approximations for the variance of the estimator and the distribution of a test statistic involving the estimator. The work is motivated by estimation of a customer loyalty measure where realistic data demonstrates the proposed approach.

Monitoring several correlated quality characteristics of a process is common in modern manufacturing and service industries. Although a lot of attention has been paid to monitoring the multivariate process mean, not many control charts are available for monitoring the covariance matrix. This paper presents a comprehensive overview of the literature on control charts for monitoring the covariance matrix in a multivariate statistical process monitoring (MSPM) framework. It classifies the research that has previously appeared in the literature. We highlight the challenging areas for research and provide some directions for future research.

Usage-based insurance schemes provide new opportunities for insurers to accurately price and manage risk. These schemes have the potential to better identify risky drivers which not only allows insurance companies to better price their products but it allows drivers to modify their behaviour to make roads safer and driving more efficient. However, for Usage-based insurance products, we need to better understand how driver behaviours influence the risk of a crash or an insurance claim. In this article, we present our analysis of automotive telematics data from over 28 million trips. We use a case control methodology to study the relationship between crash drivers and crash-free drivers and introduce an innovative method for determining control (crash-free) drivers. We fit a logistic regression model to our data and found that speeding was the most important driver behaviour linking driver behaviour to crash risk.

To assess a binary measurement system the authors consider the estimation of the average error probabilities of such a system when a gold standard (error-free) system is available to verify the status of any part.

Industry uses inspection protocols to protect customers from receiving non-conforming product. The two error rates of these systems are the chance of shipping non-conforming product (customer's risk) and the chance of rejecting good product (producer's risk). We investigate the properties of two inspection protocols. In these protocols, the customer uses a gold standard measurement system that determines if received components are conforming. We show that with the first inspection protocol, we can estimate its error rates using only production data. With the second protocol, we propose adding a small measurement assessment study to allow estimation of the error rates.

The statistical evaluation of the reliability of binary tests and inspections is a challenging endeavor. In this paper, we propose an approach for the common situation where the true condition of the inspected items is unobservable (“gold-standard unavailable”), the probabilities of false acceptance and false rejection vary across items, and rejections are relatively rare. Our approach fits a latent-variable model, where the variability in misclassification probabilities is driven by a continuous property of a part. To deal with the low prevalence of rejections, we propose sampling items from multiple sources. The performance and properties of the estimators are assessed using simulation, asymptotic approximations, and a real-life case at a car-parts manufacturer.

In this expository paper, we review methods for monitoring medical outcomes with a focus on surgical quality. We discuss the importance and role of risk adjustment. We give the advantages and disad...

Summary We compare two binary diagnostic tests when each subject is measured more than once with each test and with a gold standard. We introduce a new model that allows the correlation between two measurements on a single subject by the same test to be different from the correlation between two measurements by different tests. We show that moment estimators of the population parameters for the mean sensitivities and specificities are virtually identical to the maximum likelihood estimates from our random-effects model. We apply the model to data comparing two rapid malaria tests and provide guidance for choosing the number of subjects and repeated measurements.

Risk-adjusted cumulative sum (RACUSUM) charts are popular for the surveillance of binary health care outcomes such as 30-day mortality rates following cardiac surgery. RACUSUM charts are built on the assumptions that the binary outcomes are independent and the baseline rates are known constants. However, these two assumptions are often violated, thus undermining the validity of the surveillance procedure. In this paper, the authors propose risk-adjusted surveillance procedures using a binary logistic regression model which allows -type autocorrelations among the binary outcomes. Two versions are presented: one with known, the other with estimated baseline parameters. The authors use Monte Carlo experiments to evaluate the power and the probability of false alarm (Type I error) of the surveillance procedures. Data on 30-day mortality rates following cardiac surgery are used for illustration. The Canadian Journal of Statistics xx: 1–17; 2015 © 2015 Statistical Society of Canada Resume Les cartes a somme cumulee ajustees au risque (RACUSUM) sont des outils populaires pour la surveillance d’evenements binaires tels que le deces dans les 30 jours suivant une chirurgie cardiaque. La construction des cartes RACUSUM est faite sous l'hypothese que les evenements binaires surveilles sont independants et que le risque de base est connu et constant. Ces deux hypotheses sont souvent violees, ce qui mine la validite de la procedure de surveillance. Les auteurs proposent une procedure de surveillance ajustee au risque en utilisant un modele de regression logistique qui permet une autocorrelation de type entre les evenements binaires modelises. Ils presentent deux versions de leur modele: dans la premiere, le risque de base est connu, mais dans la deuxieme, il est estime. A l'aide d'experiences de Monte Carlo, ils evaluent la puissance et la probabilite d'une fausse alarme (erreur de type I) de la procedure de surveillance. Ils illustrent leur methode a l'aide de donnees sur la mortalite dans les 30 jours suivant une chirurgie cardiaque. La revue canadienne de statistique xx: 1–17; 2015 © 2015 Societe statistique du Canada

Deciding whether two measurement systems agree well enough to be used interchangeably is important in medical and clinical contexts. Recently, the probability of agreement was proposed as an alternative to comparison techniques such as correlation, regression, and the limits of agreement approach, when the systems’ measurement errors are homoscedastic. However, in medical and clinical contexts, it is common for measurement variability to increase proportionally with the magnitude of measurement. In this article, we extend the probability of agreement analysis to accommodate heteroscedastic measurement errors, demonstrating the versatility of this simple metric. We illustrate its use with two examples: one involving the comparison of blood pressure measurement devices, and the other involving the comparison of serum cholesterol assays.

An overview and perspective is provided on the Phase I collection and analysis of data for use in process improvement and control charting.

Through simulations, the authors assess the performance of the proposed methods and show the upper EWMA chart exhibit good performance in most of the off-target scenarios considered.

We consider the problem of monitoring and comparing medical outcomes, such as surgical performance, over time. Performance is subject to change due to a variety of reasons including patient heterogeneity, learning, deteriorating skills due to aging, etc. For instance, we expect inexperienced surgeons to improve their skills with practice. We propose a graphical method to monitor surgical performance that incorporates risk adjustment to account for patient heterogeneity. The procedure gives more weight to recent outcomes and down-weights the influence of outcomes further in the past. The chart is clinically interpretable as it plots an estimate of the failure rate for a "standard" patient. The chart also includes a measure of uncertainty in this estimate. We can implement the method using historical data or start from scratch. As the monitoring proceeds, we can base the estimated failure rate on a known risk model or use the observed outcomes to update the risk model as time passes. We illustrate the proposed method with an example from cardiac surgery.

This discussion focuses on on the assessment of a single binary measurement system in a specific context that is commonly found in the manufacturing industry

Statistical engineering as proposed by Hoerl and Snee (2010a) aims to develop a discipline devoted to better understanding how to use statistical tools to support project goals. Existing examples abound but more work is needed. We discuss the use of statistical engineering to improve problem solving—that is, reducing variation in processes—and note that this requires a series of empirical investigations where we should use information gained to help plan subsequent investigations. The systematic use of prior/existing information, especially baseline information, in problem solving is illustrated using a crossbar dimension case study. The baseline results are used to help plan and analyze all subsequent investigations both when looking for a dominant cause of the variation and when assessing a possible solution. The effective use of prior statistical information and the consequences of its use in the variation reduction context are not commonly taught and thus opportunities for more efficient prob...

The standard plan for gauge reproducibility and repeatability studies is for each of r operators to measure k parts n times for a total of N = krn measurements. These studies are usually planned and conducted in isolation ignoring available baseline data generated by the measurement system used for inspection or process control. This article has two goals. First, it quantifies the substantial benefits of incorporating baseline data into the analysis of measurement study data. Second, it searches for good standard plans with a fixed total number of measurements that take into account available baseline data. With operator effects being considered to be fixed, situations where the part by operator interaction is excluded from or included in the model are investigated. The analysis of the combined data is based on maximum likelihood estimation and the ranking of plans on the approximate standard errors of the estimates using the Fisher information matrix. The benefit of incorporating baseline data into the a...

Occupation coding, an important task in official statistics, refers to coding a respondent’s text answer into one of many hundreds of occupation codes. To date, occupation coding is still at least partially conducted manually, at great expense. We propose three methods for automatic coding: combining separate models for the detailed occupation codes and for aggregate occupation codes, a hybrid method that combines a duplicate-based approach with a statistical learning algorithm, and a modified nearest neighbor approach. Using data from the German General Social Survey (ALLBUS), we show that the proposed methods improve on both the coding accuracy of the underlying statistical learning algorithm and the coding accuracy of duplicates where duplicates exist. Further, we find defining duplicates based on ngram variables (a concept from text mining) is preferable to one based on exact string matches.

Background There is considerable recent interest in the monitoring of individual surgeon or hospital surgical outcomes. If one aggregates data over time and assesses performance with a funnel plot, then the detection of any process deterioration or improvement could be delayed. The variable life adjusted display (VLAD) is widely used for monitoring on a case-by-case basis, but we show that use of the risk-adjusted Bernoulli cumulative sum (RA-CUSUM) chart leads to much better performance. Discussion We use simulation to illustrate that the RA-CUSUM chart has better performance than the VLAD in detecting changes in the rates of adverse events. Summary We recommend the RA-CUSUM approach over the VLAD approach for monitoring surgical performance. If the VLAD is used, we recommend running the RA-CUSUM chart in the background to generate signals that the process performance has changed. Published version