Search

Statistical engineering (SE) is a term that has been around in the statistical literature for more than 60 years. Over the years, it has been defined and used by a number of different groups and or...

Experts from diverse areas of industry, government, and academia are asked about the changing roles for statisticians in the SE workplace and discuss some of the opportunities and challenges for the future.

Background. Risk-adjusted control charts have become popular for monitoring processes that involve the management and treatment of patients in hospitals or other healthcare institutions. However, to date, the effect of estimation error on risk-adjusted control charts has not been studied. Methods. We studied the effect of estimation error on risk-adjusted binary cumulative sum (CUSUM) performance using actual and simulated data on patients undergoing coronary artery bypass surgery and assessed for mortality up to 30 days post-surgery. The effect of estimation error was indicated by the variability of the ‘true’ average run lengths (ARLs) obtained using repeated sampling of the observed data under various realistic scenarios. Results. Results showed that estimation error can have a substantial effect on risk-adjusted CUSUM chart performance in terms of variation of true ARLs. Moreover, the performance was highly dependent on the number of events used to derive the control chart parameters and the specified ARL for an in-control process (ARL0). However, the results suggest that it is the uncertainty in the overall adverse event rate that is the main component of estimation error. Conclusions. When designing a control chart, the effect of estimation error could be taken into account by generating a number of bootstrap samples of the available Phase I data and then determining the control limit needed to obtain an ARL0 of a pre-specified level 95% of the time. If limited Phase I data are available, it may be advisable to continue to update model parameters even after prospective patient monitoring is implemented.

Monitoring binary outcomes when evaluating health care performance has recently become common. Classical statistical methodologies such as cumulative sum (CUSUM) charts have been refined and used for this purpose. For instance, the risk-adjusted CUSUM chart (RA-CUSUM) for monitoring binary outcomes was proposed for monitoring 30-day mortality following cardiac surgery. The RA-CUSUM inherits optimality properties of the original CUSUM charts in the sense of signaling early when there is change. However, although the RA-CUSUM is a powerful monitoring tool, it will always eventually signal a change with probability 1 even when there is no real change. In other words, the probability of a type I error for the RA-CUSUM is 1. It also turns out that, because of the skewed distribution of the run lengths of the RA-CUSUM, the median is often well below the mean, and as a consequence more than half of all its false alarms occur before the designed average run length. In addition, when the change to be detected occurs at a later time in the series of observations being monitored, the rate of false alarms increases, and the RA-CUSUM may not be appropriate. Therefore, if the price of false alarms is high, it is preferable to use methods that control the rate of false alarms. In this paper, we propose alternative sequential curtailed and risk-adjusted charts that control the type I error rate in the context of monitoring 30-day mortality following cardiac surgery. We explore the merits of each of these methodologies in terms of average run lengths as well as in terms of type I error probabilities, and we compare them to the RA-CUSUM chart. We illustrate the methodologies by using data on monitoring performance of seven surgeons from a medical center. Copyright © 2011 John Wiley & Sons, Ltd.

In the traditional plan for assessing the reliability of a measurement system, a number of raters each measure the same group of subjects. If the system has a large number of raters, we recommend a new set of plans that has two advantages over the traditional plan. First, the proposed plans provide greater precision for estimating the intraclass correlation coefficient with the same total number of measurements. Second, the plans are flexible and can be adapted to constraints on the number of times any subject can be assessed or the number of times any rater can make an assessment. We provide a simple tool for planning a reliability study, access to the software for the planning in the case where there are constraints and an example to demonstrate the analysis of data from the proposed plans. The Canadian Journal of Statistics 39: 344–355; 2011 © 2011 Statistical Society of Canada Dans un plan traditionnel pour determiner la fiabilite d'un systeme de mesures, plusieurs evaluateurs mesurent tous les sujets d'un meme groupe. Lorsqu'il y a un grand nombre d'evaluateurs, nous recommandons un nouvel ensemble de plans qui possede deux avantages par rapport au plan traditionnel. Premierement, les plans proposes procurent une plus grande precision pour l'estimation du coefficient de correlation intraclasse avec un meme nombre de mesures. Deuxiemement, ces plans sont flexibles et ils peuvent etre modifies pour contraindre le nombre d'evaluations par sujet ou encore le nombre de mesures faites par un evaluateur. Nous suggerons un outil facile d'utilisation pour planifier une etude de fiabilite et pour utiliser le logiciel de planification lorsqu'il y a des contraintes. Nous presentons aussi un exemple pour illustrer l'analyse de donnees partir des plans proposes. La revue canadienne de statistique 39: 344–355; 2011 © 2011 Societe statistique du Canada

Purpose: Timely identification of systematic changes in radiation delivery of an imaging system can lead to a reduction in risk for the patients involved. However, existing quality assurance programs involving the routine testing of equipment performance using phantoms are limited in their ability to effectively carry out this task. To address this issue, the authors propose the implementation of an ongoing monitoring process that utilizes procedural data to identify unexpected large or small radiation exposures for individual patients, as well as to detect persistent changes in the radiation output of imaging platforms. Methods: Data used in this study were obtained from records routinely collected during procedures performed in the cardiac catheterization imaging facility at St. Andrew's War Memorial Hospital, Brisbane, Australia, over the period January 2008-March 2010. A two stage monitoring process employing individual and exponentially weighted moving average (EWMA) control charts was developed and used to identify unexpectedly high or low radiation exposure levels for individual patients, as well as detect persistent changes in the radiation output delivered by the imaging systems. To increase sensitivity of the charts, we account for variation in dose area product (DAP) values due to other measured factors (patient weight, fluoroscopy time, and digitalmore » acquisition frame count) using multiple linear regression. Control charts are then constructed using the residual values from this linear regression. The proposed monitoring process was evaluated using simulation to model the performance of the process under known conditions. Results: Retrospective application of this technique to actual clinical data identified a number of cases in which the DAP result could be considered unexpected. Most of these, upon review, were attributed to data entry errors. The charts monitoring the overall system radiation output trends demonstrated changes in equipment performance associated with relocation of the equipment to a new department. When tested under simulated conditions, the EWMA chart was capable of detecting a sustained 15% increase in average radiation output within 60 cases (

Analysis and reporting of among-institution aggregated hospital-acquired infection data are necessary for transparency and accountability. Different analytical methods are required for ensuring transparency and accountability for within-institution sequential analysis. In addition, unbiased summary information is needed for planning and informing the public. We believe that implementation of systems based on evidence is the key to improving institutional performance and safety. This must be accompanied by compliance, outcome audit and sequential analysis of outcome data, e.g. using statistical process control methods. Checklists can be a valuable aid for ensuring implementation of evidence-based systems. Aggregated outcome data analysis for transparency and accountability should concentrate primarily on accurately presenting the outcomes together with their precision. We describe tabulations, funnel plots and random-effects (shrinkage) analysis and avoid comparisons using league tables, star ratings and confidence intervals.

The standard plan for the assessment of the variation due to a measurement system involves a number of operators repeatedly measuring a number of parts in a balanced design. In this article, we consider the performance of two types of (unbalanced) assessment plans. In each type, we use a standard plan augmented with a second component. In type A augmentation, each operator measures a different set of parts once each. In type B augmentation, each operator measures the same set of parts once each. The goal of the paper is to identify good augmented plans for estimating the gauge repeatability and reproducibility (GR&R), a ratio that compares the contribution of the measurement system to the overall process variation. We show that, if there are three or more operators or if we include the possibility of part-by-operator interaction, then use of an appropriate augmented plan can produce substantial gains in efficiency for estimating GR&R compared with the best standard plan with the same total number of measurements.

Summary Sequential analysis of uncommon adverse outcomes (AEs) such as surgical site infections (SSIs) is desirable. Short postoperative lengths of stay (LOS) result in many SSIs occurring after discharge and they are often superficial. Deep and organ space (complex) SSIs occur less frequently but are detected more reliably and are suitable for monitoring wound care. Those occurring post-discharge usually require readmissison and can be counted accurately. Sequential analysis of meticillin-resistant Staphylococcus aureus bacteraemia is also needed. The key to prevention is to implement systems based on evidence, e.g. using ‘bundles' and checklists. Regular mortality and morbidity audit meetings are required and these may need to be followed by independent audits. Sequential statistical analysis is desirable for data presentation, to detect changes, and to discourage tampering with processes when occasional AEs occur in a reliable system. Tabulations and cumulative observed minus expected ( O − E ) charts and funnel plots are valuable, supplemented in the presence of apparent ‘runs' of AEs by cumulative sum analysis. Used prospectively, they may enable staff to visualise and detect patterns or shifts in rates and counts that might not otherwise be apparent.

To assess measurement system variation, we propose an alternative to the standard gauge reproducibility and repeatability (GR&R) study. The new plan, called a leveraged GR&R Study, is conducted in two stages. In the baseline stage, we select a sample of parts that are measured once only each using a fixed number of operators. Then we deliberately select extreme parts for the second stage where each operator measures each selected part a number of times. We demonstrate the advantages of the leveraged over the standard plan by comparing the standard deviations of the estimators of the parameters of interest. For a fixed number of operators and total number of measurements, we recommend leveraged plans with a baseline size that is roughly half the total number of measurements. We also recommend that the number of parts selected for the second stage be set to an integer multiple of the number of operators and that each of these parts be measured two or three times by each operator. This article has supplement...

Binary measurement systems that classify parts as pass or fail are widely used in industry, especially for systematic inspection in high-volume processes. In this context, we are likely to have available a large number of previously measured passed and failed parts. To support production and quality improvement, it is important to assess the misclassification rates, e.g., the probability of failing a conforming part or passing a nonconforming part. We may also want to estimate the unknown conforming rate. Here we focus on the assessment of a binary measurement system when no gold-standard measurement system is available. The standard assessment plan is to repeatedly measure a sample of parts and use a latent class model. We demonstrate the substantial benefit of supplementing the standard plan with the available data from the previously measured parts. We propose new sampling plans and compare them with the standard plan with respect to the precision of the estimators of the misclassification rates. We also give recommendations for planning an assessment study when we can sample from a population of previously measured parts.

Finding and controlling the causes of variation in product features is one way to control variability, but it may not be cost effective. Another method to control variation is to reduce the sensitivity of the process to the main sources of variation wit..

Monitoring medical outcomes is desirable to help quickly detect performance changes. Previous applications have focused mostly on binary outcomes, such as 30-day mortality after surgery. However, in many applications the survival time data are routinely collected. In this paper, we propose an updating exponentially weighted moving average (EWMA) control chart to monitor risk-adjusted survival times. The updating EWMA (uEWMA) operates in a continuous time; hence, the scores for each patient always reflect the most up-to-date information. The uEWMA can be implemented based on a variety of survival-time models and can be set up to provide an ongoing estimate of a clinically interpretable average patient score. The efficiency of the uEWMA is shown to compare favorably with the competing methods.

This article describes the virtual manufacturing environment Watfactory (freely available at http://services03.student.math.uwaterloo.ca:8080/~stat435/login.htm) and discusses its use in teaching process improvement. Watfactory provides a rich and realistic simulation of a manufacturing process and is accessed through a website requiring no software other than a Web browser. With Watfactory, students select, plan, and analyze the data from a sequence of empirical investigations of many different types, with the ultimate goal of reducing variation in the process output. We have found that using Watfactory addresses many shortcomings in traditional teaching methods for both undergraduate and industrial short courses.

We consider the assessment of an automated continuous measurement system used for 100% inspection in a high- volume manufacturing process. Because of the automation, we assume that there are no operator effects. If the system stores the measured values, we effectively know the current process mean and standard deviation. Because of the high volume, we have parts available with values spread across the whole distribution. The standard plan for measurement-system assessment is to select k parts at random from the process and measure each of the selected parts n times. We then estimate the repeatability of the system using ANOVA. We propose two improvements. First, we demonstrate the substantial value of using the known process characteristics in the analysis. Second, we describe an alternative sampling plan where we deliberately select parts with extreme values from the population of measured parts to remeasure. We call this selection leveraging. We discuss the analysis of the leveraged plan and show that it is more efficient than the standard plan. We also discuss the planning and implementation of a leveraged assessment study and some associated issues and extensions.

A case study presents some important aspects to consider when conducting experiments to reduce variation in an existing process. One thing to consider is that the search for a cause of the variation should be kept separate from the search for a solution..

A robust Hotelling’s T²-type control chart is proposed to monitor individual observations in a multivariate process. The chart is based on reweighted minimum covariance determinant estimators. It is shown how to set the control limits for the proposed c..

Despite the widespread use of binary measurement systems, little has been written about assessing them. Two plans are proposed here for assessing binary measurement systems that may be used under the assumption that the pass rate of the system is known..

The monitoring of multiple streams in parallel to detect changes in the overall process mean and in the individual stream of gauge means is discussed. Two new control chart statistics are proposed based on an F test and a likelihood ratio test. These s..

Occupation coding, an important task in official statistics, refers to coding a respondent's text answer into one of many hundreds of occupation codes. To date, occupation coding is still at least partially conducted manually at great expense. We propose two new methods for automatic coding: a hybrid method that combines a rule-based approach based on duplicates with a statistical learning algorithm, and a modified nearest neighbor approach. Using data from the German General Social Survey (ALLBUS), we show that both methods improve on both the coding accuracy of the underlying statistical learning algorithm and the coding accuracy of duplicates where duplicates exist. We also find that statistical learning is improved by combining separate models for the detailed occupation codes and for aggregate occupation codes. Further, we and defing duplicates based on n-gram variables (a concept from text mining) is preferable to one based on exact string matches.

The Shainin SystemTM (SS) is the name given to a problem solving system, with its associated strategies and tools, developed by Dorian Shainin, and widely used and promoted in the manufacturing sector.

The comparison of two measurement systems is important in medical and other contexts. A common goal is to decide if a new measurement system agrees suitably with an existing one, and hence whether the two can be used interchangeably. Various methods for assessing interchangeability are available, the most popular being the limits of agreement approach due to Bland and Altman. In this article, we review the challenges of this technique and propose a model-based framework for comparing measurement systems that overcomes those challenges. The proposal is based on a simple metric, the probability of agreement, and a corresponding plot which can be used to summarize the agreement between two measurement systems. We also make recommendations for a study design that facilitates accurate and precise estimation of the probability of agreement.

Until now, mixture errors that cause the actual mixture proportions to differ from those intended have not been considered in the analysis of mixture experiments. In this article, we show how a Bayesian approach can account for such errors. A simulation study shows the problems with ignoring these errors and also the benefits of accounting for them. The proposed approach is illustrated with a semirealistic glass mixture experiment.

This research was supported by grants from the Natural Sciences and Engineering Research Council of Canada.

SUMMARY In industry, process monitoring is widely employed to detect process changes rapidly. However, in some industrial applications observations are censored. For example, when testing breaking strengths and failure times often a limited stress test is performed. With censored observations, a direct application of traditional monitoring procedures is not appropriate. When the censoring occurs due to competing risks, we propose a control chart based on conditional expected values to detect changes in the mean strength. To protect against possible confounding caused by changes in the mean of the censoring mechanism we also suggest a similar chart to detect changes in the mean censoring level. We provide an example of monitoring bond strength to illustrate the application of this methodology.

The cumulative sum (CUSUM) procedure is a graphical method that is widely used for quality monitoring in industrial settings. More recently it has been used to monitor surgical outcomes whereby it 'signals' if sufficient evidence has accumulated that there has been a change in the surgical failure rate. A limitation of the standard CUSUM procedure in this context is that since it is simply based on the observed surgical outcomes, it may signal as a result of changes in the referral pattern, such as an increased proportion of high-risk patients, rather than due to a change in the actual surgical performance. We describe a new CUSUM procedure that adjusts for each patient's pre-operative risk of surgical failure through the use of a likelihood-based scoring method. The procedure is therefore ideally suited for settings where there is a variable mix of patients over time.

The need for process monitoring in industry is ubiquitous. By monitoring process output, process changes may be rapidly detected and problems corrected. However, in many industrial and medical appl...

Traditional X and R control charts are used widely in industry, but do not respond quickly to small or moderate changes in the process output. Confirmation sample control (CSC) charts to detect changes in the process mean and process variability are proposed. These new control charts have substantially better operating characteristics than X and R charts. CSC charts require that any unusual observed sample be confirmed through an independent confirmation sample taken from the process. This makes CSC charts appealing to production personnel since the charts require the verification of bad news. The implementation of CSC charts is illustrated, and figures are given that allow the determination of appropriate design parameters.

Correlated binary data are encountered in many areas of medical research, system reliability and quality control. For monitoring failures rates in such situations, simultaneous bivariate cumulative sum (CUSUM) charts with the addition of secondary control limits are proposed. Using an approach based on a Markov chain model, the run length properties of such a monitoring scheme can be determined for sudden, or gradual, changes in the failure rates. The proposed control charts are easy to implement, and are shown to be very effective at detecting small changes in the rate of undesirable outcomes, especially when the changes are gradual. This procedure is illustrated using bivariate outcome data arising from a series of paediatric surgeries. The methodology is sufficiently general that it may be adapted for multivariate normal, binomial or Poisson responses.

The control limits of an exponentially weighted moving average (EWMA) control chart should vary with time, approaching asymptotic limits as time increases. However, previous analyses of EWMA charts consider only asymptotic control limits. In this articl..

SUMMARY In the manufacture of metal fasteners in a progressive die operation, and other industrial situations, important quality dimensions cannot be measured on a continuous scale, and manufactured parts are classified into groups by using a step gauge. This paper proposes a version of exponentially weighted moving average (EWMA) control charts that are applicable to monitoring the grouped data for process shifts. The run length properties of this new grouped data EWMA chart are compared with similar results previously obtained for EWMA charts for variables data and with those for cumulative sum (CUSUM) schemes based on grouped data. Grouped data EWMA charts are shown to be nearly as efficient as variables-based EWMA charts and are thus an attractive alternative when the collection of variables data is not feasible. In addition, grouped data EWMA charts are less affected by the discreteness that is inherent in grouped data than are grouped data CUSUM charts. In the metal fasteners application, grouped data EWMA charts were simple to implement and allowed the rapid detection of undesirable process shifts.

Mixture experiments involve the mixing or blending of two or more ingredients to form an end product. Typically, the quality of the end product is a function of the relative proportions of the ingredients and other extraneous process factors such as hea..

An important goal of quality improvement in manufacturing is the reduction of variability in product characteristics. Producing more consistent output improves product performance and may reduce manufacturing costs. This article discusses and contrasts ..

Pre-control, also called stoplight control, is a quality monitoring scheme similar to a control chart. However, the goal of Pre-control is the prevention of nonconforming units rather than the detection of a lack of stability. The appeal of Pre-control ..

A Shewhart control chart is proposed based on gauging theoretically continuous observations into multiple groups. This chart is designed to monitor the process mean and standard deviation for deviations from stability. By assuming an underlying normal distribution, we derive the optimal grouping criterion that maximizes the expected statistical information available in a sample. Control charts based on grouped observations are superior to standard control charts based on variables, such as X and R charts, when the quality characteristic is difficult or expensive to measure precisely, but economical to gauge.

Methodology is presented for the design of single and double compressed-limit sequential probability ratio tests (SPRT) and cumulative sum (CUSUM) control charts to detect one-sided mean shifts in a symmetric probability distribution. We also show how to evaluate the average run length properties with the fast initial response (FIR) feature. The resulting CUSUM plans have a simple scoring procedure, and are extremely simple to derive and implement. The use of two compressed-limit gauges is more efficient than a single compressed-limit gauge. In the case of SPRTs, the use of two compressed limit gauges minimizes the average sampling number required for specified operating characteristics. In the case of CUSUM, the gain in efficiency reduces the out-of-control average run length for a given in-control average run length.

Two simple proof-load testing procedures are suggested to estimate the correlation between two variables that can individually only be determined destructively. The first procedure assumes that the means and variances of the variables are known. The second testing procedure is more flexible and requires no prior information. By assuming that the variables have a bivariate normal distribution and considering only the number of units that fail at each proof-load, we determine the maximum likelihood estimate for the correlation coefficient. Theoretical and simulation results compare favorably with previously suggested more complicated procedures and guide the practitioner in appropriate choices for the proof-load levels.

Acceptance control charts establish control limits based on specification limits for very capable processes where the mean is permitted to vary. This paper deals with designing such charts when a product has two or more correlated characteristics that must all simultaneously conform to specification limits for the product to be acceptable. We extend past work by solving the general correlated multivariate case, and by allowing a more general hypothesis test. We find the control limits and sample sizes for each characteristic that minimize sampling costs. The problem is a convex nonlinear optimization problem, and optimal solutions are found using multidimensional search.

It is often more economical to classify a continuous quality characteristic into several groups than it is to measure it exactly. We propose a control chart based on gauging theoretically continuous observations into multiple groups. This chart is designed to detect one-directional shifts in the mean of a normal distribution with specified operating characteristics. We show how to minimize the sample size required by optimizing the criteria used to group the quality characteristic. Control charts based on grouped observations may be superior to standard control charts based on variables when the quality characteristic is difficult or expensive to measure precisely but economical to gauge.

Timely identification of systematic changes in radiation delivery of an imaging system can lead to a reduction in risk for the patients involved. However, existing quality assurance programs involving the routine testing of equipment performance using phantoms are limited in their ability to effectively carry out this task. To address this issue, the authors propose the implementation of an ongoing monitoring process that utilizes procedural data to identify unexpected large or small radiation exposures for individual patients, as well as to detect persistent changes in the radiation output of imaging platforms.Data used in this study were obtained from records routinely collected during procedures performed in the cardiac catheterization imaging facility at St. Andrew's War Memorial Hospital, Brisbane, Australia, over the period January 2008-March 2010. A two stage monitoring process employing individual and exponentially weighted moving average (EWMA) control charts was developed and used to identify unexpectedly high or low radiation exposure levels for individual patients, as well as detect persistent changes in the radiation output delivered by the imaging systems. To increase sensitivity of the charts, we account for variation in dose area product (DAP) values due to other measured factors (patient weight, fluoroscopy time, and digital acquisition frame count) using multiple linear regression. Control charts are then constructed using the residual values from this linear regression. The proposed monitoring process was evaluated using simulation to model the performance of the process under known conditions.Retrospective application of this technique to actual clinical data identified a number of cases in which the DAP result could be considered unexpected. Most of these, upon review, were attributed to data entry errors. The charts monitoring the overall system radiation output trends demonstrated changes in equipment performance associated with relocation of the equipment to a new department. When tested under simulated conditions, the EWMA chart was capable of detecting a sustained 15% increase in average radiation output within 60 cases (1 month of operation), while a 33% increase would be signaled within 20 cases.This technique offers a valuable enhancement to existing quality assurance programs in radiology that rely upon the testing of equipment radiation output at discrete time frames to ensure performance security.

This article is a commentary on an article by Ron S. Kenett, "Statistics: A Life Cycle View," Quality Engineering, Volume 27, Number 1 (January 2015), pages 111-121.

The intraclass correlation is often used to assess the reliability of a measurement system. There is a considerable literature devoted to optimizing the standard assessment plan in which a number of subjects are measured repeatedly. We propose a two-stage investigation, here called a leveraged plan (LP), where in Stage I, we measure a number of subjects once. Then in Stage II, we select a subset of subjects with extreme initial measurements for repeated measurement. For a fixed total number of measurements, we show that the optimal LP provides a more precise estimate of the intraclass correlation coefficient than does the optimal standard plan (SP). We provide a table for finding the optimal LP given the true intraclass correlation and a specified precision for the estimate. For a fixed total number of measurements N, a nearly optimal LP makes roughly N/2 measurements in Stage I and then selects roughly N/6 extreme subjects to re-measure thrice each in Stage II. We also compare optimal leveraged with optimal SPs when there is a limit on the number of times each subject can be re-measured.

This article gives an example of how student-conducted experiments can enhance a course in the design of experiments. We focus on a project whose aim is to find a good mixture of water, soap and glycerin for making soap bubbles. This project is relatively straightforward to implement and understand. At its most basic level the project introduces students to mixture experiments and general issues in experimental design such as choosing and measuring an appropriate response, selecting a design, the effect of using repeats versus replicates, model building, making predictions, etc. To accommodate more advanced students, the project can be easily enhanced to draw on various areas of statistics, such as generalized linear models, robust design, and optimal design. Therefore it is ideal for a graduate level course as it encourages students to look beyond the basics presented in class.

Objective We previously applied non–risk-adjusted cumulative sum methods to analyze coronary bypass outcomes. The objective of this study was to assess the incremental advantage of risk-adjusted cumulative sum methods in this setting. Methods Prospective data were collected in 793 consecutive patients who underwent coronary bypass grafting performed by a single surgeon during a period of 5 years. The composite occurrence of an "adverse outcome" included mortality or any of 10 major complications. An institutional logistic regression model for adverse outcome was developed by using 2608 contemporaneous patients undergoing coronary bypass. The predicted risk of adverse outcome in each of the surgeon's 793 patients was then calculated. A risk-adjusted cumulative sum curve was then generated after specifying control limits and odds ratio. This risk-adjusted curve was compared with the non–risk-adjusted cumulative sum curve, and the clinical significance of this difference was assessed. Results The surgeon's adverse outcome rate was 96 of 793 (12.1%) versus 270 of 1815 (14.9%) for all the other institution's surgeons combined ( P = .06). The non–risk-adjusted curve reached below the lower control limit, signifying excellent outcomes between cases 164 and 313, 323 and 407, and 667 and 793, but transgressed the upper limit between cases 461 and 478. The risk-adjusted cumulative sum curve never transgressed the upper control limit, signifying that cases preceding and including 461 to 478 were at an increased predicted risk. Furthermore, if the risk-adjusted cumulative sum curve was reset to zero whenever a control limit was reached, it still signaled a decrease in adverse outcome at 166, 653, and 782 cases. Conclusions Risk-adjusted cumulative sum techniques provide incremental advantages over non–risk-adjusted methods by not signaling a decrement in performance when preoperative patient risk is high.

Purpose Ruptured abdominal aortic aneurysms (RAAAs) continue to result in early mortality in up to 50% of patients. Additionally, it remains difficult to compare outcomes given the variability in patient comorbidities and presentation. The purpose of this study was to describe an instrument that permits the prospective analysis of outcomes after RAAA repair while adjusting for the variability in preoperative risk. Methods Consecutive patients undergoing attempted open RAAA repair over a 5-year period (1999 to 2003) at our center were reviewed. Thirty-day or in-hospital mortality was the main outcome variable. Preoperative mortality risk was estimated for each patient by using a validated modification of the POSSUM scoring system (V-POSSUM). A risk-adjusted cumulative sum method (RA-CUSUM) was used to compare observed versus predicted outcomes by assigning a risk-adjusted score, based on log-likelihood ratios, to each patient. These scores were sequentially plotted with preset control limits to allow for "signaling" when results were substantially different from expected (doubling or halving of odds ratios). Results A total of 136 patients were reviewed, with an early mortality rate of 45.6%. V-POSSUM scores were accurate in predicting mortality for the entire cohort, with an observed-to-predicted mortality ratio of 0.92 ( P = .80). Each patient's risk-adjusted score was plotted sequentially. In one segment of the resulting plot, the graph adopted a negative slope and crossed the lower control limit, indicating improved results compared with predicted. Conclusions V-POSSUM scores in this series accurately predicted early mortality after RAAA surgery. The RA-CUSUM method allows for the prospective evaluation of outcomes, while taking into account patient variability. In the current study, this resulted in the identification of a series of patients who had improved outcomes compared with predicted.

Many processes have defective rates measured in parts per million (PPM). When the process yields such a high level of quality, traditional methods of process improvement, such as designed experiments, and control charts on the number of defectives or the time between defectives, are no longer effective. However, it may still be desirable to monitor and/or improve such processes. In this article, we take a critical look at attempts to apply control charts in this situation. As an alternative, we suggest that since defectives are so rare, we should carefully study any that are observed. By comparing the characteristics of the defectives to good units, both in terms of their physical dimensions and properties, and the process records from their production, we may be able to identify the key differences. Using this type of retrospective study, the goal is to identify an explanatory continuous variable or variables that can be monitored instead of the number of defectives or the time between defectives.