Search

Receiver operating characteristic (ROC) curve analysis is widely used in evaluating the effectiveness of a diagnostic test/biomarker or classifier score. A parametric approach for statistical inference on ROC curves based on a Box–Cox transformation to normality has frequently been discussed in the literature. Many investigators have highlighted the difficulty of taking into account the variability of the estimated transformation parameter when carrying out such an analysis. This variability is often ignored and inferences are made by considering the estimated transformation parameter as fixed and known. In this paper, we will review the literature discussing the use of the Box–Cox transformation for ROC curves and the methodology for accounting for the estimation of the Box–Cox transformation parameter in the context of ROC analysis, and detail its application to a number of problems. We present a general framework for inference on any functional of interest, including common measures such as the AUC, the Youden index, and the sensitivity at a given specificity (and vice versa). We further developed a new R package (named 'rocbc') that carries out all discussed approaches and is available in CRAN. [ABSTRACT FROM AUTHOR]

Many diseases are heterogeneous, comprised of multiple disease subgroups. It is of great interest but highly unlikely to find a single biomarker that can accurately detect such heterogeneous diseases across different subgroups. In this article, we propose to estimate a personalized diagnostic rule (PDR) to tailor more effective biomarkers to each individual according to a linear combination of his or her profiles. A standard grid search algorithm can be used to estimate the optimal linear PDR that maximizes the area under the receiver operating characteristics curve (AUC) among all the linear PDRs, but it is time‐consuming especially when the number of variables is large. Alternatively, we developed an efficient grid rotation algorithm that provides a nearly suboptimal solution and studied its variation to find the optimal solution. We implemented the cross‐validated forward variable selection method to find a subset of useful variables while avoid overfitting. Extensive simulations show that our proposed method reduces bias and variance. Analysis of a gastric cancer biomarker study and censored survival outcome data illustrates the practical utility of our proposed method. The proposed method is implemented in the open‐source R package persDx. [ABSTRACT FROM AUTHOR]

When evaluating a diagnostic test, it is common that a gold standard may not be available. One example is the diagnosis of SARS‐CoV‐2 infection using saliva sampling or nasopharyngeal swabs. Without a gold standard, a pragmatic approach is to postulate a "reference standard," defined as positive if either test is positive, or negative if both are negative. However, this pragmatic approach may overestimate sensitivities because subjects infected with SARS‐CoV‐2 may still have double‐negative test results even when both tests exhibit perfect specificity. To address this limitation, we propose a Bayesian hierarchical model for simultaneously estimating sensitivity, specificity, and disease prevalence in the absence of a gold standard. The proposed model allows adjusting for study‐level covariates. We evaluate the model performance using an example based on a recently published meta‐analysis on the diagnosis of SARS‐CoV‐2 infection and extensive simulations. Compared with the pragmatic reference standard approach, we demonstrate that the proposed Bayesian method provides a more accurate evaluation of prevalence, specificity, and sensitivity in a meta‐analytic framework. [ABSTRACT FROM AUTHOR]

Incorporating promising biomarkers into cancer screening practices for early‐detection is increasingly appealing because of the unsatisfactory performance of current cancer screening strategies. The matched case‐control design is commonly adopted in biomarker development studies to evaluate the discriminative power of biomarker candidates, with an intention to eliminate confounding effects. Data from matched case‐control studies have been routinely analyzed by the conditional logistic regression, although the assumed logit link between biomarker combinations and disease risk may not always hold. We propose a conditional concordance‐assisted learning method, which is distribution‐free, for identifying an optimal combination of biomarkers to discriminate cases and controls. We are particularly interested in combinations with a clinically and practically meaningful specificity to prevent disease‐free subjects from unnecessary and possibly intrusive diagnostic procedures, which is a top priority for cancer population screening. We establish asymptotic properties for the derived combination and confirm its favorable finite sample performance in simulations. We apply the proposed method to the prostate cancer data from the carotene and retinol efficacy trial (CARET). [ABSTRACT FROM AUTHOR]

Diagnostic tests usually need to operate at a high sensitivity or specificity level in practice. Accordingly, specificity at the controlled sensitivity, or vice versa, is a clinically sensible performance metric for evaluating continuous biomarkers. Meanwhile, the performance of a biomarker may vary across sub‐populations as defined by covariates, and covariate‐specific evaluation can be informative. In this article, we develop a novel modeling and estimation method for covariate‐specific specificity at a controlled sensitivity level. Unlike existing methods which typically adopt elaborate models of covariate effects over the entire biomarker distribution, our approach models covariate effects locally at a specific sensitivity level of interest. We also extend our proposed model to handle the whole continuum of sensitivities via dynamic regression and derive covariate‐specific ROC curves. We provide the variance estimation through bootstrapping. The asymptotic properties are established. We conduct extensive simulation studies to evaluate the performance of our proposed methods in comparison with existing methods, and further illustrate the applications in two clinical studies for aggressive prostate cancer. [ABSTRACT FROM AUTHOR]

Binary classification problems are ubiquitous in health and social sciences. In many cases, one wishes to balance two competing optimality considerations for a binary classifier. For instance, in resource-limited settings, an human immunodeficiency virus prevention program based on offering pre-exposure prophylaxis (PrEP) to select high-risk individuals must balance the sensitivity of the binary classifier in detecting future seroconverters (and hence offering them PrEP regimens) with the total number of PrEP regimens that is financially and logistically feasible for the program. In this article, we consider a general class of constrained binary classification problems wherein the objective function and the constraint are both monotonic with respect to a threshold. These include the minimization of the rate of positive predictions subject to a minimum sensitivity, the maximization of sensitivity subject to a maximum rate of positive predictions, and the Neyman-Pearson paradigm, which minimizes the type II error subject to an upper bound on the type I error. We propose an ensemble approach to these binary classification problems based on the Super Learner methodology. This approach linearly combines a user-supplied library of scoring algorithms, with combination weights and a discriminating threshold chosen to minimize the constrained optimality criterion. We then illustrate the application of the proposed classifier to develop an individualized PrEP targeting strategy in a resource-limited setting, with the goal of minimizing the number of PrEP offerings while achieving a minimum required sensitivity. This proof of concept data analysis uses baseline data from the ongoing Sustainable East Africa Research in Community Health study. Copyright © 2017 John Wiley & Sons, Ltd.

Statistical methods have been well‐developed for comparing two binary screening tests in the presence of verification bias. However, the complexity of existing methods and the computational difficulty in implementing them have restricted their use. A simple and easily implemented statistical method is therefore needed. In this paper, we propose a weighted McNemar's test statistic for comparing two sensitivities(specificities). The proposed test statistics are intuitive and simple to compute, only involving some minor modification of a McNemar's test statistic using the estimated verification probabilities for discordant pairs. Simulations demonstrate that the proposed weighted McNemar's test statistics preserve type I error as well as or better than the existing statistics. Furthermore, unlike the existing methods, the proposed weighted McNemar's test statistics can still be applied even when none of the accordant pairs are verified. [ABSTRACT FROM AUTHOR]

The development of a new diagnostic test ideally follows a sequence of stages which, among other aims, evaluate technical performance. This includes an analytical validity study, a diagnostic accuracy study, and an interventional clinical utility study. In this article, we propose a novel Bayesian approach to sample size determination for the diagnostic accuracy study, which takes advantage of information available from the analytical validity stage. We utilize assurance to calculate the required sample size based on the target width of a posterior probability interval and can choose to use or disregard the data from the analytical validity study when subsequently inferring measures of test accuracy. Sensitivity analyses are performed to assess the robustness of the proposed sample size to the choice of prior, and prior-data conflict is evaluated by comparing the data to the prior predictive distributions. We illustrate the proposed approach using a motivating real-life application involving a diagnostic test for ventilator associated pneumonia. Finally, we compare the properties of the approach against commonly used alternatives. The results show that, when suitable prior information is available, the assurance-based approach can reduce the required sample size when compared to alternative approaches. [ABSTRACT FROM AUTHOR]

False negative rates of severe acute respiratory coronavirus 2 diagnostic tests, together with selection bias due to prioritized testing can result in inaccurate modeling of COVID‐19 transmission dynamics based on reported "case" counts. We propose an extension of the widely used Susceptible‐Exposed‐Infected‐Removed (SEIR) model that accounts for misclassification error and selection bias, and derive an analytic expression for the basic reproduction number R0 as a function of false negative rates of the diagnostic tests and selection probabilities for getting tested. Analyzing data from the first two waves of the pandemic in India, we show that correcting for misclassification and selection leads to more accurate prediction in a test sample. We provide estimates of undetected infections and deaths between April 1, 2020 and August 31, 2021. At the end of the first wave in India, the estimated under‐reporting factor for cases was at 11.1 (95% CI: 10.7,11.5) and for deaths at 3.58 (95% CI: 3.5,3.66) as of February 1, 2021, while they change to 19.2 (95% CI: 17.9, 19.9) and 4.55 (95% CI: 4.32, 4.68) as of July 1, 2021. Equivalently, 9.0% (95% CI: 8.7%, 9.3%) and 5.2% (95% CI: 5.0%, 5.6%) of total estimated infections were reported on these two dates, while 27.9% (95% CI: 27.3%, 28.6%) and 22% (95% CI: 21.4%, 23.1%) of estimated total deaths were reported. Extensive simulation studies demonstrate the effect of misclassification and selection on estimation of R0 and prediction of future infections. A R‐package SEIRfansy is developed for broader dissemination. [ABSTRACT FROM AUTHOR]

Before a new screening test can be used in routine screening, its performance needs to be compared to the standard screening test. This comparison is generally done in population screening trials with a screen‐positive design where participants undergo one or both screening tests after which disease verification takes place for those positive on at least one screening test. We consider the randomized paired screen‐positive design of Alonzo and Kittelson where participants are randomized to receive one of the two screening tests and only participants with a positive screening test subsequently receive the other screening test followed by disease verification. The tests are usually offered in an unblinded fashion in which case the screening uptake may differ between arms, in particular when one test is more burdensome than the other. When uptake is associated with disease, the estimator for the relative sensitivity derived by Alonzo and Kittelson may be biased and the type I error of the associated statistical test is no longer guaranteed to be controlled. We present methods for comparing sensitivities of screening tests in randomized paired screen‐positive trials that are robust to differential screening uptake. In a simulation study, we show that our methods adequately control the type I error when screening uptake is associated with disease. We apply the developed methods to data from the IMPROVE trial, a nonblinded cervical cancer screening trial comparing the accuracy of HPV testing on self‐collected versus provider‐collected samples. In this trial, screening uptake was higher among participants randomized to self‐collection. [ABSTRACT FROM AUTHOR]

This article considers how to estimate the accuracy of a diagnostic test when there are repeated observations, but without the availability of a gold standard or reference test. We identify conditions under which the structure of the observed data is rich enough to provide sufficient degrees of freedom, such that a suitable latent class model can be fitted with identifiable accuracy parameters. We show that a Rule of Three applies, specifying that accuracy can be evaluated as long as there are at least three observations per individual with the given test. This rule also applies if the three observations arise from combinations of different test methods, or from a sequential design in which individuals are tested for a maximum number of times with the same test but stopping if a positive (or negative) result occurs. The rule pertains to tests having an arbitrary number of response categories. Accuracy is evaluated by parameters reflecting rates of misclassification among the response categories, and the model also provides estimates of the underlying distribution of the true disease state. These ideas are illustrated by data from two medical studies. Issues discussed include the advantages and disadvantages of analyzing the response variable as binary or multinomial, as well as the feasibility of testing goodness of fit when the model incorporates a large number of parameters. Comparisons are possible between models that do or do not assume equal accuracy rates for the observations, and between models where certain misclassification parameters are or are not assumed to be zero. [ABSTRACT FROM AUTHOR]

It is difficult to estimate sensitivity and specificity of diagnostic tests when there is no gold standard. Latent class models have been proposed as a potential solution as they provide estimates without the need for a gold standard. Using a motivating example of the evaluation of point of care tests for leptospirosis in Tanzania, we show how a realistic violation of assumptions underpinning the latent class model can lead directly to substantial bias in the estimates of the parameters of interest. In particular, we consider the robustness of estimates of sensitivity, specificity, and prevalence, to the presence of additional latent states when fitting a two-state latent class model. The violation is minor in the sense that it cannot be routinely detected with goodness-of-fit procedures, but is major with regard to the resulting bias. [ABSTRACT FROM AUTHOR]

This article proposes a powerful method to compare two samples. The proposed method handles comparison of data by drawing inference from ROC curve model parameters. The method estimates parameters from a linear model framework on the empirical sensitivities and specificities. The consistent ROC parameters are then used to give a more powerful test than existing methods in several situations. In addition, we present a comprehensive statistic based on the Cauchy combination, which works well in all scenarios considered in this article. We also offer an efficient one‐layer wild permutation procedure to calculate the P‐value of our statistic. The method is particularly useful when the underlying continuous biomarker results are non‐normal. We illustrate the proposed methods in a neonatal audiology diagnostic example. [ABSTRACT FROM AUTHOR]

High-volume testing of clinical specimens for sexually transmitted diseases is performed frequently by a process known as group testing. This algorithmic process involves testing portions of specimens from separate individuals together as one unit (or "group") to detect diseases. Retesting is performed on groups that test positively in order to differentiate between positive and negative individual specimens. The overall goal is to use the least number of tests possible across all individuals without sacrificing diagnostic accuracy. One of the most efficient group testing algorithms is array testing. In its simplest form, specimens are arranged into a grid-like structure so that row and column groups can be formed. Positive-testing rows/columns indicate which specimens to retest. With the growing use of multiplex assays, the increasing number of diseases tested by these assays, and the availability of subject-specific risk information, opportunities exist to make this testing process even more efficient. We propose specific specimen arrangements within an array that can reduce the number of retests needed when compared with other array testing algorithms. We examine how to calculate operating characteristics, including the expected number of tests and the SD for the number of tests, and then subsequently find a best arrangement. Our methods are illustrated for chlamydia and gonorrhea detection with the Aptima Combo 2 Assay. We also provide R functions to make our research accessible to laboratories. [ABSTRACT FROM AUTHOR]

During the early stage of biomarker discovery, high throughput technologies allow for simultaneous input of thousands of biomarkers that attempt to discriminate between healthy and diseased subjects. In such cases, proper ranking of biomarkers is highly important. Common measures, such as the area under the receiver operating characteristic (ROC) curve (AUC), as well as affordable sensitivity and specificity levels, are often taken into consideration. Strictly speaking, such measures are appropriate under a stochastic ordering assumption, which implies, without loss of generality, that higher measurements are more indicative for the disease. Such an assumption is not always plausible and may lead to rejection of extremely useful biomarkers at this early discovery stage. We explore the length of a smooth ROC curve as a measure for biomarker ranking, which is not subject to directionality. We show that the length corresponds to a ϕ divergence, is identical to the corresponding length of the optimal (likelihood ratio) ROC curve, and is an appropriate measure for ranking biomarkers. We explore the relationship between the length measure and the AUC of the optimal ROC curve. We then provide a complete framework for the evaluation of a biomarker in terms of sensitivity and specificity through a proposed ROC analogue for use in improper settings. In the absence of any clinical insight regarding the appropriate cutoffs, we estimate the sensitivity and specificity under a two‐cutoff extension of the Youden index and we further take into account the implied costs. We apply our approaches on two biomarker studies that relate to pancreatic and esophageal cancer. [ABSTRACT FROM AUTHOR]

In medical diagnostic studies, verification of the true disease status might be partially missing based on results of diagnostic tests and other characteristics of subjects. Because estimates of area under the ROC curve (AUC) based on partially validated subjects are usually biased, it is usually necessary to estimate AUC from a bias‐corrected ROC curve. In this article, various direct estimation methods of the AUC based on hybrid imputation [full imputations and mean score imputation (MSI)], inverse probability weighting, and the semiparametric efficient (SPE) approach are proposed and compared in the presence of verification bias when the test result is continuous under the assumption that the true disease status, if missing, is missing at random. Simulation results show that the proposed estimators are accurate for the biased sampling if the disease and verification models are correctly specified. The SPE and MSI based estimators perform well even under the misspecified disease/verification models. Numerical studies are performed to compare the finite sample performance of the proposed approaches with existing methods. A real dataset of neonatal hearing screening study is analyzed. [ABSTRACT FROM AUTHOR]

Group testing is an indispensable tool for laboratories when testing high volumes of clinical specimens for infectious diseases. An important decision that needs to be made prior to implementation is determining what group sizes to use. In best practice, an objective function is chosen and then minimized to determine an optimal set of these group sizes, known as the optimal testing configuration (OTC). There are a few options for objective functions, and they differ based on how the expected number of tests, assay characteristics, and testing constraints are taken into account. These varied options have led to a recent controversy in the literature regarding which of two different objective functions is better. In our paper, we examine these objective functions over a number of realistic situations for infectious disease testing. We show that this controversy may be much ado about nothing because the OTCs and corresponding results (eg, number of tests and accuracy) are largely the same for standard testing algorithms in a wide variety of situations. [ABSTRACT FROM AUTHOR]

Tests for disease often produce a continuous measure, such as the concentration of some biomarker in a blood sample. In clinical practice, a threshold C is selected such that results, say, greater than C are declared positive and those less than C negative. Measures of test accuracy such as sensitivity and specificity depend crucially on C, and the optimal value of this threshold is usually a key question for clinical practice. Standard methods for meta‐analysis of test accuracy (i) do not provide summary estimates of accuracy at each threshold, precluding selection of the optimal threshold, and furthermore, (ii) do not make use of all available data. We describe a multinomial meta‐analysis model that can take any number of pairs of sensitivity and specificity from each study and explicitly quantifies how accuracy depends on C. Our model assumes that some prespecified or Box‐Cox transformation of test results in the diseased and disease‐free populations has a logistic distribution. The Box‐Cox transformation parameter can be estimated from the data, allowing for a flexible range of underlying distributions. We parameterise in terms of the means and scale parameters of the two logistic distributions. In addition to credible intervals for the pooled sensitivity and specificity across all thresholds, we produce prediction intervals, allowing for between‐study heterogeneity in all parameters. We demonstrate the model using two case study meta‐analyses, examining the accuracy of tests for acute heart failure and preeclampsia. We show how the model can be extended to explore reasons for heterogeneity using study‐level covariates. [ABSTRACT FROM AUTHOR]

Group testing, as a cost-effective strategy, has been widely used to perform large-scale screening for rare infections. Recently, the use of multiplex assays has transformed the goal of group testing from detecting a single disease to diagnosing multiple infections simultaneously. Existing research on multiple-infection group testing data either exclude individual covariate information or ignore possible retests on suspicious individuals. To incorporate both, we propose a new regression model. This new model allows us to perform a regression analysis for each infection using multiple-infection group testing data. Furthermore, we introduce an efficient variable selection method to reveal truly relevant risk factors for each disease. Our methodology also allows for the estimation of the assay sensitivity and specificity when they are unknown. We examine the finite sample performance of our method through extensive simulation studies and apply it to a chlamydia and gonorrhea screening data set to illustrate its practical usefulness. [ABSTRACT FROM AUTHOR]

The receiver operating characteristic (ROC) curve is typically employed when one wants to evaluate the discriminatory capability of a continuous or ordinal biomarker in the case where two groups are to be distinguished, commonly the "healthy" and the "diseased." There are cases for which the disease status has three categories. Such cases employ the ROC surface, which is a natural generalization of the ROC curve for three classes. In this paper, we explore new methodologies for comparing two continuous biomarkers that refer to a trichotomous disease status, when both markers are applied to the same patients. Comparisons based on the volume under the surface have been proposed before, but that measure is often not clinically relevant. Here, we focus on comparing two correlated ROC surfaces at given pairs of true classification rates, which are more relevant to patients and physicians. We propose delta-based parametric techniques, power transformations to normality, and bootstrap-based smooth nonparametric techniques to investigate the performance of an appropriate test. We evaluate our approaches through an extensive simulation study and apply them to a real dataset from prostate cancer screening. [ABSTRACT FROM AUTHOR]

Evaluating the accuracy (ie, estimating the sensitivity and specificity) of new diagnostic tests without the presence of a gold standard is of practical meaning and has been the subject of intensive study for several decades. Existing methods use 2 or more diagnostic tests under several basic assumptions and then estimate the accuracy parameters via the maximum likelihood estimation. One of the basic assumptions is the conditional independence of the tests given the disease status. This assumption is impractical in many real applications in veterinary research. Several methods have been proposed with various dependence models to relax this assumption. However, these methods impose subjective dependence structures, which may not be practical and may introduce additional nuisance parameters. In this article, we propose a simple method for addressing this problem without the conditional independence assumption, using an empirical conditioning approach. The proposed method reduces to the popular Hui-Walter model in the case of conditional independence. Also, our likelihood function is of order-2 polynomial in parameters, while that of Hui-Walter is of order-3. The reduced model complexity increases the stability in estimation. Simulation studies are conducted to evaluate the performance of the proposed method, which shows overall smaller biases in estimation and is more stable than the existing method, especially when tests are conditionally dependent. Two real data examples are used to illustrate the proposed method. [ABSTRACT FROM AUTHOR]

Cancer patients, chronic kidney disease patients, and subjects infected with HIV are routinely monitored over time using biomarkers that represent key health status indicators. Furthermore, biomarkers are frequently used to guide initiation of new treatments or to inform changes in intervention strategies. Since key medical decisions can be made on the basis of a longitudinal biomarker, it is important to evaluate the potential accuracy associated with longitudinal monitoring. To characterize the overall accuracy of a time-dependent marker, we introduce a summary ROC curve that displays the overall sensitivity associated with a time-dependent threshold that controls time-varying specificity. The proposed statistical methods are similar to concepts considered in disease screening, yet our methods are novel in choosing a potentially time-dependent threshold to define a positive test, and our methods allow time-specific control of the false-positive rate. The proposed summary ROC curve is a natural averaging of time-dependent incident/dynamic ROC curves and therefore provides a single summary of net error rates that can be achieved in the longitudinal setting. [ABSTRACT FROM AUTHOR]

Group testing, where specimens are tested initially in pools, is widely used to screen individuals for sexually transmitted diseases. However, a common problem encountered in practice is that group testing can increase the number of false negative test results. This occurs primarily when positive individual specimens within a pool are diluted by negative ones, resulting in positive pools testing negatively. If the goal is to estimate a population-level regression model relating individual disease status to observed covariates, severe bias can result if an adjustment for dilution is not made. Recognizing this as a critical issue, recent binary regression approaches in group testing have utilized continuous biomarker information to acknowledge the effect of dilution. In this paper, we have the same overall goal but take a different approach. We augment existing group testing regression models (that assume no dilution) with a parametric dilution submodel for pool-level sensitivity and estimate all parameters using maximum likelihood. An advantage of our approach is that it does not rely on external biomarker test data, which may not be available in surveillance studies. Furthermore, unlike previous approaches, our framework allows one to formally test whether dilution is present based on the observed group testing data. We use simulation to illustrate the performance of our estimation and inference methods, and we apply these methods to 2 infectious disease data sets. [ABSTRACT FROM AUTHOR]

Two key aims of diagnostic research are to accurately and precisely estimate disease prevalence and test sensitivity and specificity. Latent class models have been proposed that consider the correlation between subject measures determined by different tests in order to diagnose diseases for which gold standard tests are not available. In some clinical studies, several measures of the same subject are made with the same test under the same conditions (replicated measurements), and thus, replicated measurements for each subject are not independent. In the present study, we propose an extension of the Bayesian latent class Gaussian random effects model to fit the data with binary outcomes for tests with replicated subject measures. We describe an application using data collected on hookworm infection carried out in the municipality of Presidente Figueiredo, Amazonas State, Brazil. In addition, the performance of the proposed model was compared with that of current models (the subject random effects model and the conditional (in)dependent model) through a simulation study. As expected, the proposed model presented better accuracy and precision in the estimations of prevalence, sensitivity and specificity. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

We introduce new binocular accuracy measures as alternatives to conventional marginal measures that can be used to evaluate screening tests in diagnostic studies involving paired organs (e.g. eyes and ears). Specifically, we consider screening studies based on a cross-sectional design, where both diagnosis and disease status are determined after study enrolment or sampling, yielding paired binocular binary data described via two models, namely, the extended common correlation model and the Gaussian copula probit model. The first relies on the assumption of exchangeability of fellow organs, while the second is more flexible. Binocular versions of sensitivity and specificity are defined, respectively, as the probability of at least one correct positive diagnosis in patients with one or both organs truly diseased and the probability of two correct negative diagnoses for patients with both organs truly un-diseased. Comparisons between the conventional marginal and binocular sensitivities and specificities are illustrated for both models using data from a diabetic retinopathy study. We show that our methodology provides a viable alternative to conventional ways of assessing diagnostic accuracy of screening tests for paired organs. The binocular versions of sensitivity and specificity reflect the way screening tests are conducted in practice, and they overcome the shortcomings of conventional measures. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

From the statistical learning perspective, this paper shows a new direction for the use of growth mixture modeling (GMM), a method of identifying latent subpopulations that manifest heterogeneous outcome trajectories. In the proposed approach, we utilize the benefits of the conventional use of GMM for the purpose of generating potential candidate models based on empirical model fitting, which can be viewed as unsupervised learning. We then evaluate candidate GMM models on the basis of a direct measure of success; how well the trajectory types are predicted by clinically and demographically relevant baseline features, which can be viewed as supervised learning. We examine the proposed approach focusing on a particular utility of latent trajectory classes, as outcomes that can be used as valid prediction targets in clinical prognostic models. Our approach is illustrated using data from the Longitudinal Assessment of Manic Symptoms study. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The receiver operating characteristic (ROC) curve is the most popular statistical tool for evaluating the discriminatory capability of a given continuous biomarker. The need to compare two correlated ROC curves arises when individuals are measured with two biomarkers, which induces paired and thus correlated measurements. Many researchers have focused on comparing two correlated ROC curves in terms of the area under the curve (AUC), which summarizes the overall performance of the marker. However, particular values of specificity may be of interest. We focus on comparing two correlated ROC curves at a given specificity level. We propose parametric approaches, transformations to normality, and nonparametric kernel-based approaches. Our methods can be straightforwardly extended for inference in terms of ROC-1 (t). This is of particular interest for comparing the accuracy of two correlated biomarkers at a given sensitivity level. Extensions also involve inference for the AUC and accommodating covariates. We evaluate the robustness of our techniques through simulations, compare them with other known approaches, and present a real-data application involving prostate cancer screening. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In this paper, we propose a Bayesian method to address misclassification errors in both independent and dependent variables. Our work is motivated by a study of women who have experienced new breast cancers on two separate occasions. We call both cancers primary, because the second is usually not considered as the result of a metastasis spreading from the first. Hormone receptors (HRs) are important in breast cancer biology, and it is well recognized that the measurement of HR status is subject to errors. This discordance in HR status for two primary breast cancers is of concern and might be an important reason for treatment failure. To sort out the information on true concordance rate from the observed concordance rate, we consider a logistic regression model for the association between the HR status of the two cancers and introduce the misclassification parameters (i.e., sensitivity and specificity) accounting for the misclassification in HR status. The prior distribution for sensitivity and specificity is based on how HR status is actually assessed in laboratory procedures. To account for the nonlinear effect of one error-free covariate, we introduce the B-spline terms in the logistic regression model. Our findings indicate that the true concordance rate of HR status between two primary cancers is greater than the observed value. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Testing protocols in large-scale sexually transmitted disease screening applications often involve pooling biospecimens (e.g., blood, urine, and swabs) to lower costs and to increase the number of individuals who can be tested. With the recent development of assays that detect multiple diseases, it is now common to test biospecimen pools for multiple infections simultaneously. Recent work has developed an expectation-maximization algorithm to estimate the prevalence of two infections using a two-stage, Dorfman-type testing algorithm motivated by current screening practices for chlamydia and gonorrhea in the USA. In this article, we have the same goal but instead take a more flexible Bayesian approach. Doing so allows us to incorporate information about assay uncertainty during the testing process, which involves testing both pools and individuals, and also to update information as individuals are tested. Overall, our approach provides reliable inference for disease probabilities and accurately estimates assay sensitivity and specificity even when little or no information is provided in the prior distributions. We illustrate the performance of our estimation methods using simulation and by applying them to chlamydia and gonorrhea data collected in Nebraska. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

A sequential design is proposed to test whether the accuracy of a binary diagnostic biomarker meets the minimal level of acceptance. The accuracy of a binary diagnostic biomarker is a linear combination of the marker's sensitivity and specificity. The objective of the sequential method is to minimize the maximum expected sample size under the null hypothesis that the marker's accuracy is below the minimal level of acceptance. The exact results of two-stage designs based on Youden's index and efficiency indicate that the maximum expected sample sizes are smaller than the sample sizes of the fixed designs. Exact methods are also developed for estimation, confidence interval and p-value concerning the proposed accuracy index upon termination of the sequential testing. Published 2016. This article is a U.S. Government work and is in the public domain in the USA. [ABSTRACT FROM AUTHOR]

Composite reference standards (CRSs) have been advocated in diagnostic accuracy studies in the absence of a perfect reference standard. The rationale is that combining results of multiple imperfect tests leads to a more accurate reference than any one test in isolation. Focusing on a CRS that classifies subjects as disease positive if at least one component test is positive, we derive algebraic expressions for sensitivity and specificity of this CRS, sensitivity and specificity of a new (index) test compared with this CRS, as well as the CRS-based prevalence. We use as a motivating example the problem of evaluating a new test for Chlamydia trachomatis, an asymptomatic disease for which no gold-standard test exists. As the number of component tests increases, sensitivity of this CRS increases at the expense specificity, unless all tests have perfect specificity. Therefore, such a CRS can lead to significantly biased accuracy estimates of the index test. The bias depends on disease prevalence and accuracy of the CRS. Further, conditional dependence between the CRS and index test can lead to over-estimation of index test accuracy estimates. This commonly-used CRS combines results from multiple imperfect tests in a way that ignores information and therefore is not guaranteed to improve over a single imperfect reference unless each component test has perfect specificity, and the CRS is conditionally independent of the index test. When these conditions are not met, as in the case of C. trachomatis testing, more realistic statistical models should be researched instead of relying on such CRSs. [ABSTRACT FROM AUTHOR]

Summarizing the information of many studies using a meta-analysis becomes more and more important, also in the field of diagnostic studies. The special challenge in meta-analysis of diagnostic accuracy studies is that in general sensitivity and specificity are co-primary endpoints. Across the studies both endpoints are correlated, and this correlation has to be considered in the analysis. The standard approach for such a meta-analysis is the bivariate logistic random effects model. An alternative approach is to use marginal beta-binomial distributions for the true positives and the true negatives, linked by copula distributions. In this article, we propose a new, nonparametric approach of analysis, which has greater flexibility with respect to the correlation structure, and always converges. In a simulation study, it becomes apparent that the empirical coverage of all three approaches is in general below the nominal level. Regarding bias, empirical coverage, and mean squared error the nonparametric model is often superior to the standard model, and comparable with the copula model. The three approaches are also applied to two example meta-analyses. [ABSTRACT FROM AUTHOR]

Group testing, through the use of pooling, has been widely implemented as a more efficient means to screen individuals for infectious diseases. Typically, in these settings, practitioners are tasked with the complimentary goals of both case identification and estimation. For these purposes, many group testing strategies have been proposed, which address issues such as preserving anonymity in estimation studies, quality control, and classification. In general, these strategies require that a significant number of the individuals be retested, either in pools or individually. In order to provide practitioners with a general methodology that can be used to accurately and precisely analyze data of this form, herein, we propose a binary regression framework that can incorporate data arising from any group testing strategy. Further, we relax previously made assumptions regarding testing error rates by relating the diagnostic testing results to the latent biological marker levels of the individuals being tested. We investigate the finite sample performance of our proposed methodology through simulation and by applying our techniques to hepatitis B data collected as part of a study involving Irish prisoners. [ABSTRACT FROM AUTHOR]

In this work, we describe a two-stage sampling design to estimate the infection prevalence in a population. In the first stage, an imperfect diagnostic test was performed on a random sample of the population. In the second stage, a different imperfect test was performed in a stratified random sample of the first sample. To estimate infection prevalence, we assumed conditional independence between the diagnostic tests and develop method of moments estimators based on expectations of the proportions of people with positive and negative results on both tests that are functions of the tests' sensitivity, specificity, and the infection prevalence. A closed-form solution of the estimating equations was obtained assuming a specificity of 100% for both tests. We applied our method to estimate the infection prevalence of visceral leishmaniasis according to two quantitative polymerase chain reaction tests performed on blood samples taken from 4756 patients in northern Ethiopia. The sensitivities of the tests were also estimated, as well as the standard errors of all estimates, using a parametric bootstrap. We also examined the impact of departures from our assumptions of 100% specificity and conditional independence on the estimated prevalence. [ABSTRACT FROM AUTHOR]

Although statistical methodology is well-developed for comparing diagnostic tests in terms of their sensitivity and specificity, comparative inference about predictive values is not. In this paper, we consider the analysis of studies comparing operating characteristics of two diagnostic tests that are measured on all subjects and have test outcomes from multiple sites with varying number of sites among subjects. We have developed a new approach for comparing sensitivity, specificity, positive predictive value, and negative predictive value with simple variance calculation and, in particular, focus on comparing tests using difference of positive and negative predictive values. Simulation studies are conducted to show the performance of our approach. We analyze real data on patients with lung cancer, based on their diagnostic tests, to illustrate the methodology. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In real life and somewhat contrary to biostatistical textbook knowledge, sensitivity and specificity (and not only predictive values) of diagnostic tests can vary with the underlying prevalence of disease. In meta-analysis of diagnostic studies, accounting for this fact naturally leads to a trivariate expansion of the traditional bivariate logistic regression model with random study effects. In this paper, a new model is proposed using trivariate copulas and beta-binomial marginal distributions for sensitivity, specificity, and prevalence as an expansion of the bivariate model. Two different copulas are used, the trivariate Gaussian copula and a trivariate vine copula based on the bivariate Plackett copula. This model has a closed-form likelihood, so standard software (e.g., sas proc nlmixed) can be used. The results of a simulation study have shown that the copulamodels perform at least as good but frequently better than the standard model. The methods are illustrated by two examples. [ABSTRACT FROM AUTHOR]

Youden index is widely utilized in studies evaluating accuracy of diagnostic tests and performance of predictive, prognostic, or risk models. However, both one and two independent sample tests on Youden index have been derived ignoring the dependence (association) between sensitivity and specificity, resulting in potentially misleading findings. Besides, paired sample test on Youden index is currently unavailable. This article develops efficient statistical inference procedures for one sample, independent, and paired sample tests on Youden index by accounting for contingency correlation, namely associations between sensitivity and specificity and paired samples typically represented in contingency tables. For one and two independent sample tests, the variances are estimated by Delta method, and the statistical inference is based on the central limit theory, which are then verified by bootstrap estimates. For paired samples test, we show that the estimated covariance of the two sensitivities and specificities can be represented as a function of kappa statistic so the test can be readily carried out. We then show the remarkable accuracy of the estimated variance using a constrained optimization approach. Simulation is performed to evaluate the statistical properties of the derived tests. The proposed approaches yield more stable type I errors at the nominal level and substantially higher power (efficiency) than does the original Youden's approach. Therefore, the simple explicit large sample solution performs very well. Because we can readily implement the asymptotic and exact bootstrap computation with common software like R, the method is broadly applicable to the evaluation of diagnostic tests and model performance. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Expert opinion plays an important role when choosing clusters of chemical compounds for further investigation. Often, the process by which the clusters are assigned to the experts for evaluation, the so-called selection process, and the qualitative ratings given by the experts to the clusters (chosen/not chosen) need to be jointly modeled to avoid bias. This approach is referred to as the joint modeling approach. However, misspecifying the selection model may impact the estimation and inferences on parameters in the rating model, which are of most scientific interest. We propose to incorporate the selection process into the analysis by adding a new set of random effects to the rating model and, in this way, avoid the need to model it parametrically. This approach is referred to as the combined model approach. Through simulations, the performance of the combined and joint models was compared in terms of bias and confidence interval coverage. The estimates from the combined model were nearly unbiased, and the derived confidence intervals had coverage probability around 95% in all scenarios considered. In contrast, the estimates from the joint model were severely biased under some form of misspecification of the selection model, and fitting the model was often numerically challenging. The results show that the combined model may offer a safer alternative on which to base inferences when there are doubts about the validity of the selection model. Importantly, thanks to its greater numerical stability, the combined model may outperform the joint model even when the latter is correctly specified. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In biomedical research and practice, continuous biomarkers are often used for diagnosis and prognosis, with a cut-point being established on the measurement to aid binary classification. When survival time is examined for the purposes of disease prognostication and is found to be related to the baseline measure of a biomarker, employing a single cut-point on the biomarker may not be very informative. Using survival time-dependent sensitivity and specificity, we extend a concordance probability-based objective function to select survival time-related cut-points. To estimate the objective function with censored survival data, we adopt a non-parametric procedure for time-dependent receiver operational characteristics curves, which uses nearest neighbor estimation techniques. In a simulation study, the proposed method, when used to select a cut-point to optimally predict survival at a given time within a specified range, yields satisfactory results. We apply the procedure to estimate survival time-dependent cut-point on the prognostic biomarker of serum bilirubin among patients with primary biliary cirrhosis. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

We consider the problem of estimating the prevalence of a disease under a group testing framework. Because assays are usually imperfect, misclassification of disease status is a major challenge in prevalence estimation. To account for possible misclassification, it is usually assumed that the sensitivity and specificity of the assay are known and independent of the group size. This assumption is often questionable, and substitution of incorrect values of an assay's sensitivity and specificity can result in a large bias in the prevalence estimate, which we refer to as the mis-substitution bias. In this article, we propose simple designs and methods for prevalence estimation that do not require known values of assay sensitivity and specificity. If a gold standard test is available, it can be applied to a validation subsample to yield information on the imperfect assay's sensitivity and specificity. When a gold standard is unavailable, it is possible to estimate assay sensitivity and specificity, either as unknown constants or as specified functions of the group size, from group testing data with varying group size. We develop methods for estimating parameters and for finding or approximating optimal designs, and perform extensive simulation experiments to evaluate and compare the different designs. An example concerning human immunodeficiency virus infection is used to illustrate the validation subsample design. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The performance of a diagnostic test is best evaluated against a reference test that is without error. For many diseases, this is not possible, and an imperfect reference test must be used. However, diagnostic accuracy estimates may be biased if inaccurately verified status is used as the truth. Statistical models have been developed to handle this situation by treating disease as a latent variable. In this paper, we conduct a systematized review of statistical methods using latent class models for estimating test accuracy and disease prevalence in the absence of complete verification. Published 2014. This article is a U.S. Government work and is in the public domain in the USA. [ABSTRACT FROM AUTHOR]

The evaluation, comparison, and public report of health care provider performance is essential to improving the quality of health care. Hospitals, as one type of provider, are often classified into quality tiers (e.g., top or suboptimal) based on their performance data for various purposes. However, potential misclassification might lead to detrimental effects for both consumers and payers. Although such risk has been highlighted by applied health services researchers, a systematic investigation of statistical approaches has been lacking. We assess and compare the expected accuracy of several commonly used classification methods: unadjusted hospital-level averages, shrinkage estimators under a random-effects model accommodating between-hospital variation, and two others based on posterior probabilities. Assuming that performance data follow a classic one-way random-effects model with unequal sample size per hospital, we derive accuracy formulae for these classification approaches and gain insight into how the misclassification might be affected by various factors such as reliability of the data, hospital-level sample size distribution, and cutoff values between quality tiers. The case of binary performance data is also explored using Monte Carlo simulation strategies. We apply the methods to real data and discuss the practical implications. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

An emerging concern with new therapeutic agents, especially treatments for type 2 diabetes, a prevalent condition that increases an individual's risk of heart attack or stroke, is the likelihood of adverse events, especially cardiovascular events, that the new agents may cause. These concerns have led to regulatory requirements for demonstrating that a new agent increases the risk of an adverse event relative to a control by no more than, say, 30% or 80% with high (e.g., 97.5%) confidence. We describe a Bayesian adaptive procedure for determining if the sample size for a development program needs to be increased and, if necessary, by how much, to provide the required assurance of limited risk. The decision is based on the predictive likelihood of a sufficiently high posterior probability that the relative risk is no more than a specified bound. Allowance can be made for between-center as well as within-center variability to accommodate large-scale developmental programs, and design alternatives (e.g., many small centers, few large centers) for obtaining additional data if needed can be explored. Binomial or Poisson likelihoods can be used, and center-level covariates can be accommodated. The predictive likelihoods are explored under various conditions to assess the statistical properties of the method. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In this paper, we consider the design for comparing the performance of two binary classification rules, for example, two record linkage algorithms or two screening tests. Statistical methods are well developed for comparing these accuracy measures when the gold standard is available for every unit in the sample, or in a two-phase study when the gold standard is ascertained only in the second phase in a subsample using a fixed sampling scheme. However, these methods do not attempt to optimize the sampling scheme to minimize the variance of the estimators of interest. In comparing the performance of two classification rules, the parameters of primary interest are the difference in sensitivities, specificities, and positive predictive values. We derived the analytic variance formulas for these parameter estimates and used them to obtain the optimal sampling design. The efficiency of the optimal sampling design is evaluated through an empirical investigation that compares the optimal sampling with simple random sampling and with proportional allocation. Results of the empirical study show that the optimal sampling design is similar for estimating the difference in sensitivities and in specificities, and both achieve a substantial amount of variance reduction with an over-sample of subjects with discordant results and under-sample of subjects with concordant results. A heuristic rule is recommended when there is no prior knowledge of individual sensitivities and specificities, or the prevalence of the true positive findings in the study population. The optimal sampling is applied to a real-world example in record linkage to evaluate the difference in classification accuracy of two matching algorithms. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

There are still challenges when meta-analyzing data from studies on diagnostic accuracy. This is mainly due to the bivariate nature of the response where information on sensitivity and specificity must be summarized while accounting for their correlation within a single trial. In this paper, we propose a new statistical model for the meta-analysis for diagnostic accuracy studies. This model uses beta-binomial distributions for the marginal numbers of true positives and true negatives and links these margins by a bivariate copula distribution. The new model comes with all the features of the current standard model, a bivariate logistic regression model with random effects, but has the additional advantages of a closed likelihood function and a larger flexibility for the correlation structure of sensitivity and specificity. In a simulation study, which compares three copula models and two implementations of the standard model, the Plackett and the Gauss copula do rarely perform worse but frequently better than the standard model. We use an example from a meta-analysis to judge the diagnostic accuracy of telomerase (a urinary tumor marker) for the diagnosis of primary bladder cancer for illustration. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Medical diagnostic tests must enjoy appropriate validity and high reliability in order to qualify as adequate assessment tools. Without a gold standard test, available medical diagnostic tests are not perfect; hence, the reliability of such tests must be evaluated precisely. Kappa coefficient statistics are often utilized to assess reliability of tests when there are two or more medical diagnostic tests. However, the statistics are imprecise for a typical case when the prevalence rate of a target disease is unknown. Although latent class models could be used to assess reliability, the models cannot estimate reliability in the case of two tests, due to unidentifiability or the lack of degrees of freedom. An alternative approach to assess reliability for the case of two tests is stratifying a two-by-two contingency table under the assumption that sensitivities and specificities between the two tests be equal over all strata and that prevalence rates in the strata be different from each other. Because stratification is basically a multi-sample analysis, it should not be applied to the situation where subsamples (i.e., centers) are randomly selected from a larger population. In this article, a type of mixed-effect model is proposed to evaluate the reliability of two tests for trials in randomly selected multiple centers. Several types of distributions for prevalence rates over subpopulations are considered. Simulation studies show that our proposed method performs nicely. Analysis of real data is also reported. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Intermediate test results often occur with diagnostic tests. When assessing diagnostic accuracy, it is important to properly report and account for these results. In the literature, these results are commonly discarded prior to analysis or treated as either a positive or a negative result. Although such adjustments allow sensitivity and specificity to be computed in the standard way, these forced decisions limit the interpretability and usefulness of the results. Estimation of diagnostic accuracy is further complicated when tests are evaluated without a gold standard. Although traditional latent class modeling can be readily applied to analyze these data and account for intermediate results, these models assume that tests are independent conditional on the true disease status, which is rarely valid in practice. We extend both the log-linear latent class model and the probit latent class model to accommodate the conditional dependence among tests while taking the intermediate results into consideration. We illustrate our methods using a simulation study and a published medical study on the detection of epileptiform activity in the brain. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

When an existing standard marker does not have sufficient classification accuracy on its own, new markers are sought with the goal of yielding a combination with better performance. The primary criterion for selecting new markers is that they have good performance on their own and preferably be uncorrelated with the standard. Most often linear combinations are considered. In this paper, we investigate the increment in performance that is possible by combining a novel continuous marker with a moderately performing standard continuous marker under a variety of biologically motivated models for their joint distribution. We find that an uncorrelated continuous marker with moderate performance on its own usually yields only minimally improved performance. We identify other settings that lead to large improvements, including a novel marker that has very poor performance on its own but is highly correlated with the standard and a novel marker with poor to moderate performance that is highly correlated with the standard but only in one class category. These results suggest changing current strategies for identifying markers to be included in panels for possible combination. Using simulated and real datasets, we examine the merits of a broadened strategy that selects panels of markers as candidates on the basis of their joint performance with existing markers, compared with the standard strategy that selects markers on the basis of their marginal performance. We find that a broadened strategy can be fruitful but necessitates using studies with large numbers of subjects. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

From the patients' management perspective, a good diagnostic test should contribute to both reflecting the true disease status and improving clinical outcomes. The diagnostic randomized clinical trial is designed to combine both diagnostic tests and therapeutic interventions. Evaluation of diagnostic tests is carried out with therapeutic outcomes as the primary endpoint rather than test accuracy. We lay out the probability framework for evaluating such trials. We compare two commonly referred designs-the two-arm design and the paired design-in a formal statistical hypothesis testing setup and identify the causal connection between the two tests. The paired design is shown to be more efficient than the two-arm design. The efficiency gains vary depending on the discordant rates of test results. We derive sample size formulas for both binary and continuous endpoints. We derive estimation of important quantities under the paired design and also conduct simulation studies to verify the theoretical results. We illustrate the method with an example of designing a randomized study on preoperative staging of bladder cancer. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This work focuses on the assessment of the discrimination ability of a binary marker to identify patients that will relapse in time. We consider the cumulative definition of sensitivity and dynamic definition of specificity at a time horizon, that is, the probability of a positive marker in the population that will relapse ( cases) and that will not relapse ( controls). In the presence of censoring, sensitivity and specificity cannot be estimated by proportions because it is not known whether censored subjects should be considered as cases or controls. The solutions proposed do not enable to obtain asymptotic confidence intervals. We explore the use of inverse probability of censoring weighting/imputation (borrowed from the methodology used to correct for verification bias) to adjust the classification matrix for the presence of censoring. The adjustment based on weights estimated conditional on the marker turned to be equivalent to the adjustment based on imputation. These approaches, which address for the presence of marker-dependent censoring, showed a better performance than the adjustment based on weights estimated on the entire sample, even in the case of marker-independent censoring. We derived single intervals and confidence region for sensitivity and 1-specificity using the delta method. The confidence region is particularly useful for a binary marker because the marker has some ability to discriminate among cases and controls only if the region does not intersect the first quadrant bisector. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]