Search

Multicancer detection (MCD) tests use blood specimens to detect preclinical cancers. A major concern is overdiagnosis, the detection of preclinical cancer on screening that would not have developed into symptomatic cancer in the absence of screening. Because overdiagnosis can lead to unnecessary and harmful treatments, its quantification is important. A key metric is the screen overdiagnosis fraction (SOF), the probability of overdiagnosis at screen detection. Estimating SOF is notoriously difficult because overdiagnosis is not observed. This estimation is more challenging with MCD tests because short‐term results are needed as the technology is rapidly changing. To estimate average SOF for a program of yearly MCD tests, I introduce a novel method that requires at least two yearly MCD tests given to persons having a wide range of ages and applies only to cancers for which there is no conventional screening. The method assumes an exponential distribution for the sojourn time in an operational screen‐detectable preclinical cancer (OPC) state, defined as once screen‐detectable (positive screen and work‐up), always screen‐detectable. Because this assumption appears in only one term in the SOF formula, the results are robust to violations of the assumption. An SOF plot graphs average SOF versus mean sojourn time. With lung cancer screening data and synthetic data, SOF plots distinguished small from moderate levels of SOF. With its unique set of assumptions, the SOF plot would complement other modeling approaches for estimating SOF once sufficient short‐term observational data on MCD tests become available. [ABSTRACT FROM AUTHOR]

This article explores the importance of calibration in prediction models when applied to new populations. It compares different calibration approaches and introduces Elkan's general updating approach (GUA), which accounts for differences in base rates between populations without re-estimation. The article explains how calibrated probability can be expressed as a function of base rates, which are estimated using proportions of subjects with a certain outcome in training and calibration data. The text discusses two approaches for estimating base probability in a training data set: Elkan's GUA calculates the average of the target variable, while Baker's GUA suggests using the average of the conditional probability. Both approaches assume that the only difference between populations is the change in base rate and require a calibration data set. A simulation study comparing different calibration methods for machine learning models found that beta calibration performed the best, while isotonic calibration and IVAP calibration were similar. Elkan calibration and Baker calibration also showed similar performance. The article provides detailed information on the simulation scenarios and the performance of each calibration method. [Extracted from the article]

There is growing interest in multicancer detection tests, which identify molecular signals in the blood that indicate a potential preclinical cancer. A key stage in evaluating these tests is a prediagnostic performance study, in which investigators store specimens from asymptomatic individuals and later test stored specimens from patients with cancer and a random sample of controls to determine predictive performance. Performance metrics include rates of cancer-specific true-positive and false-positive findings and a cancer-specific positive predictive value, with the latter compared with a decision-analytic threshold. The sample size trade-off method, which trades imprecise targeting of the true-positive rate for precise targeting of a zero-false-positive rate can substantially reduce sample size while increasing the lower bound of the positive predictive value. For a 1-year follow-up, with ovarian cancer as the rarest cancer considered, the sample size trade-off method yields a sample size of 163 000 compared with a sample size of 720 000, based on standard calculations. These design and analysis recommendations should be considered in planning a specimen repository and in the prediagnostic evaluation of multicancer detection tests. [ABSTRACT FROM AUTHOR]

Background: The test tradeoff curve helps investigators decide if collecting data for risk prediction is worthwhile when risk prediction is used for treatment decisions. At a given benefit-cost ratio (the number of false-positive predictions one would trade for a true positive prediction) or risk threshold (the probability of developing disease at indifference between treatment and no treatment), the test tradeoff is the minimum number of data collections per true positive to yield a positive maximum expected utility of risk prediction. For example, a test tradeoff of 3,000 invasive tests per true-positive prediction of cancer may suggest that risk prediction is not worthwhile. A test tradeoff curve plots test tradeoff versus benefit-cost ratio or risk threshold. The test tradeoff curve evaluates risk prediction at the optimal risk score cutpoint for treatment, which is the cutpoint of the risk score (the estimated risk of developing disease) that maximizes the expected utility of risk prediction when the receiver-operating characteristic (ROC) curve is concave. Methods: Previous methods for estimating the test tradeoff required grouping risk scores. Using individual risk scores, the new method estimates a concave ROC curve by constructing a concave envelope of ROC points, taking a slope-based moving average, minimizing a sum of squared errors, and connecting successive ROC points with line segments. Results: The estimated concave ROC curve yields an estimated test tradeoff curve. Analyses of 2 synthetic data sets illustrate the method. Conclusion: Estimating the test tradeoff curve based on individual risk scores is straightforward to implement and more appealing than previous estimation methods that required grouping risk scores. Highlights: The test tradeoff curve helps investigators decide if collecting data for risk prediction is worthwhile when risk prediction is used for treatment decisions. At a given benefit-cost ratio or risk threshold, the test tradeoff is the minimum number of data collections per true positive to yield a positive maximum expected utility of risk prediction. Unlike previous estimation methods that grouped risk scores, the method uses individual risk scores to estimate a concave ROC curve, which yields an estimated test tradeoff curve. [ABSTRACT FROM AUTHOR]

Chen and Heitjan (Sensitivity of estimands in clinical trials with imperfect compliance. Int J Biostat. 2023) used linear extrapolation to estimate the population average causal effect (PACE) from the complier average causal effect (CACE) in multiple randomized trials with all-or-none compliance. For extrapolating from CACE to PACE in this setting and in the paired availability design involving different availabilities of treatment among before-and-after studies, we recommend the sensitivity analysis in Baker and Lindeman (J Causal Inference, 2013) because it is not restricted to a linear model, as it involves various random effect and trend models. [ABSTRACT FROM AUTHOR]