Search

Objective: Statistical process control charts have been advocated for use in monitoring of lung function in asthma. We aimed to evaluate their application in asthma using existing data from a randomized trial., Study Design and Setting: Patients on optimal inhaled corticosteroid/bronchodilator therapy (n=81) were randomized to continue the same or change to corticosteroid alone. Baseline statistical control was assessed from 20 days of electronically recorded lung function (peak expiratory flow [PEF], forced expiratory volume in one second [FEV(1)]). The ability to detect lung function changes was assessed during 10 days after randomization., Results: PEF measurements were in statistical control during baseline for only 59-79% of patients for different combinations of five control chart rules (e.g., Rule 1: >3 standard deviations outside mean and 95% expected to be in control), with similar proportions for FEV(1). After randomization, among those previously in statistical control, Rule 1 signaled lower FEV(1) for 35% of patients randomized to treatment change compared with 6% continuing baseline treatment (P=0.004). Control charts performed poorly for PEF, signaling a decrease for 8% of patients randomized to treatment change compared with 11% continuing treatment (P=0.7). A paradoxical increase was signaled for many patients continuing treatment., Conclusion: Control charts may not be suitable for use in many patients with asthma., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Background and Objective: In systematic reviews of the reliability of diagnostic tests, no quality assessment tool has been used consistently. The aim of this study was to develop a specific quality appraisal tool for studies of diagnostic reliability., Methods: Key principles for the quality of studies of diagnostic reliability were identified with reference to epidemiologic principles, existing quality appraisal checklists, and the Standards for Reporting of Diagnostic Accuracy (STARD) and Quality Assessment of Diagnostic Accuracy Studies (QUADAS) resources. Specific items that encompassed each of the principles were developed. Experts in diagnostic research provided feedback on the items that were to form the appraisal tool. This process was iterative and continued until consensus among experts was reached., Results: The Quality Appraisal of Reliability Studies (QAREL) checklist includes 11 items that explore seven principles. Items cover the spectrum of subjects, spectrum of examiners, examiner blinding, order effects of examination, suitability of the time interval among repeated measurements, appropriate test application and interpretation, and appropriate statistical analysis., Conclusions: QAREL has been developed as a specific quality appraisal tool for studies of diagnostic reliability. The reliability of this tool in different contexts needs to be evaluated., (Copyright (c) 2010 Elsevier Inc. All rights reserved.)

Objective: To explain which measures of accuracy and which statistical methods should be used in studies to assess the value of a new binary test as a replacement test, an add-on test, or a triage test., Study Design and Setting: Selection and explanation of statistical methods, illustrated with examples., Results: Statistical methods for comparative diagnostic accuracy studies are described that take into account the purpose of the new diagnostic test. Methods are described within a framework that defines the major purpose of test comparison: assessing the value of a new test as a replacement test, an add-on test, or a triage test. Methods appropriate for both unpaired and paired study designs for binary test data are given, including regression modeling of diagnostic test accuracy. Implications for efficient study designs are also discussed., Conclusions: Appropriate selection of existing statistical methods is necessary to address research questions about the comparative accuracy of new tests., (Copyright (c) 2010 Elsevier Inc. All rights reserved.)

Objective: To demonstrate how mixed models may be used to estimate treatment effects, and inform decisions on the need for monitoring initial response., Study Design and Setting: Mixed models were used to analyze data from the Perindopril Protection Against Recurrent Stroke Study (PROGRESS), which examined the effects of perindopril and indapamide in 6,105 patients at high risk of a cerebrovascular event., Results: The mean effect of perindopril was to lower blood pressure (BP) (systolic/diastolic) by 6/3 mmHg. The mean effects of perindopril/indapamide varied according to baseline BP, and lowering of BP ranged from 9/5 to 14/5 mmHg (for individuals with a baseline systolic BP <140 and >150 mmHg, respectively). We found no variation in the effects of treatment on BP for either perindopril alone or in combination with indapamide. The effects of treatment on the individual can be predicted from the mean effect of treatment for the group (perindopril) or baseline systolic BP subgroup (perindopril/indapamide)., Conclusion: Monitoring initial treatment response is unnecessary for antihypertensives similar to those examined in this study. To address this issue for other therapies, we suggest that trials should report estimates of treatment effects from mixed models, and the CONSORT statement should be expanded to include this item.

Objective: To compare three predictive models based on logistic regression to estimate adjusted likelihood ratios allowing for interdependency between diagnostic variables (tests)., Study Design and Setting: This study was a review of the theoretical basis, assumptions, and limitations of published models; and a statistical extension of methods and application to a case study of the diagnosis of obstructive airways disease based on history and clinical examination., Results: Albert's method includes an offset term to estimate an adjusted likelihood ratio for combinations of tests. Spiegelhalter and Knill-Jones method uses the unadjusted likelihood ratio for each test as a predictor and computes shrinkage factors to allow for interdependence. Knottnerus' method differs from the other methods because it requires sequencing of tests, which limits its application to situations where there are few tests and substantial data. Although parameter estimates differed between the models, predicted "posttest" probabilities were generally similar., Conclusion: Construction of predictive models using logistic regression is preferred to the independence Bayes' approach when it is important to adjust for dependency of tests errors. Methods to estimate adjusted likelihood ratios from predictive models should be considered in preference to a standard logistic regression model to facilitate ease of interpretation and application. Albert's method provides the most straightforward approach.

Objective: Whether tests such as imaging should be read with or without access to prior clinical information is controversial. Naïve comparisons may suggest that the provision of prior information improves test accuracy, whereas in fact the opposite may be true. This is because provision of clinical background may actually bias test readers to over- or underinterpret relevant test findings, and they may suboptimally integrate the previous and current evidence. We propose comparing the combined accuracy of prior information and a test read (i) with or (ii) without knowledge of prior information. Analysis methods include simple decision rules and logistic regression., Study Design and Setting: A study of cancer detection in women presenting with breast symptoms, in whom ultrasound could be read with or without reviewing prior mammography., Results: Naïve analysis gave an area under the receiver operating characteristics curve (AUC) for ultrasound read with mammography on view that was 4.6% higher (P < .01) than without mammography on view. Our approach, comparing the combined accuracy of mammography and ultrasound read i) with and ii) without knowledge of mammographic findings, showed much smaller differences., Conclusion: Our approach is more appropriate than naïve analyses. The particular choice of analytic method depends on the study size and the diagnostic accuracy of combinations of the prior information and the test reading.

Background and Objective: Publication bias and other sample size effects are issues for meta-analyses of test accuracy, as for randomized trials. We investigate limitations of standard funnel plots and tests when applied to meta-analyses of test accuracy and look for improved methods., Methods: Type I and type II error rates for existing and alternative tests of sample size effects were estimated and compared in simulated meta-analyses of test accuracy., Results: Type I error rates for the Begg, Egger, and Macaskill tests are inflated for typical diagnostic odds ratios (DOR), when disease prevalence differs from 50% and when thresholds favor sensitivity over specificity or vice versa. Regression and correlation tests based on functions of effective sample size are valid, if occasionally conservative, tests for sample size effects. Empirical evidence suggests that they have adequate power to be useful tests. When DORs are heterogeneous, however, all tests of funnel plot asymmetry have low power., Conclusion: Existing tests that use standard errors of odds ratios are likely to be seriously misleading if applied to meta-analyses of test accuracy. The effective sample size funnel plot and associated regression test of asymmetry should be used to detect publication bias and other sample size related effects.

Background and Objective: A range of fixed-effect and random-effects meta-analytic methods are available to obtain summary estimates of measures of diagnostic test accuracy. The hierarchical summary receiver operating characteristic (HSROC) model proposed by Rutter and Gatsonis in 2001 represents a general framework for the meta-analysis of diagnostic test studies that allows different parameters to be defined as a fixed effect or random effects within the same model. The Bayesian method used for fitting the model is complex, however, and the model is not widely used. The objective of this report is to show how the model may be fitted using the SAS procedure NLMIXED and to compare the results to the fully Bayesian analysis using an example., Methods: The HSROC model, its assumptions, and its interpretation are described. The advantages of this model over the usual summary ROC (SROC) regression model are outlined. A complex example is used to compare the estimated SROC curves, expected operating points, and confidence intervals using the alternative approaches to fitting the model., Results: The empirical Bayes estimates obtained using NLMIXED agree closely with those obtained using the fully Bayesian analysis., Conclusion: This alternative and more straightforward method for fitting the HSROC model makes the model more accessible to meta-analysts.

Objective: To develop and demonstrate efficient methods to estimate the relative true positive and false positive rates of two cervical screening tests (conventional cytology and PAPNET)., Methods: We designed the study to meet stringent methodologic criteria for comparison of two tests while simultaneously minimizing the numbers requiring reference standard verification. We used a cytology reference standard and also assessed histology when available. For the primary analysis, slides with discordant results around the test threshold (CIN 1) were reviewed by a panel of two cytopathologists, blind to previous results, to establish the reference standard result (reference standard threshold for abnormality CIN2). Where histology was available, a secondary analysis was conducted with the reference standard based on the highest grade lesion (either cytology or histology)., Results: Among 21,747 Pap smears, 372 were discordant around the test threshold, requiring verification. In the primary analysis PAPNET detected four more true positives than conventional reading; difference in sensitivity 1.29% (95%CI -5.79 to 8.36%, P=.40). There were two extra false positives using PAPNET; difference in the false positive rate 0.0097% (95%CI -0.122 to 0.142%, P=.47). The results of the combined cytology and histology analysis were similar; difference in true positive rate 0.29% (95%CI -6.76 to 7.34%, P=.50) and difference in false positive rate 0.024% (95%CI -0.098 to 0.15%, P=.39)., Conclusion: This is an efficient and valid study design where the objective is to examine the comparative accuracy of two tests. The design provides an efficient means of estimating the difference between true positive and false positive detection by the two tests, which often is sufficient information for policy decisions.