Search

Objectives: An increase in analytical imprecision and/or the introduction of bias can affect the interpretation of quantitative laboratory results. In this study, we explore the impact of varying assay imprecision and bias introduction on the classification of patients based on fixed thresholds., Methods: Simple spreadsheets (Microsoft Excel) were constructed to simulate conditions of assay deterioration, expressed as coefficient of variation and bias (in percentages). The impact on patient classification was explored based on fixed interpretative limits. A combined matrix of imprecision and bias of 0%, 2%, 4%, 6%, 8%, and 10% (tool 1) as well as 0%, 2%, 5%, 10%, 15%, and 20% (tool 2) was simulated, respectively. The percentage of patients who were reclassified following the addition of simulated imprecision and bias was summarized and presented in tables and graphs., Results: The percentage of patients who were reclassified increased with increasing/decreasing magnitude of imprecision and bias. The impact of imprecision lessens with increasing bias such that at high biases, the bias becomes the dominant cause for reclassification., Conclusions: The spreadsheet tools, available as Supplemental Material, allow laboratories to visualize the impact of additional analytical imprecision and bias on the classification of their patients when applied to locally extracted historical results., (© The Author(s) 2023. Published by Oxford University Press on behalf of American Society for Clinical Pathology. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Between-reagent lot, bias, bile acid, drift, lot-to-lot variation, pregnancy, reagent, reagent lot, lot-to-lot verification, shift Keywords: between-reagent lot; bias; bile acid; drift; lot-to-lot variation; lot-to-lot verification; pregnancy; reagent; reagent lot; shift EN between-reagent lot bias bile acid drift lot-to-lot variation lot-to-lot verification pregnancy reagent reagent lot shift e108 e111 4 05/31/23 20230601 NES 230601 To the Editor, Bile acids (BA) are synthesised in the liver through a multi-step process, beginning with cholesterol. [Extracted from the article]

Lot-to-lot verification is an integral component for monitoring the long-term stability of a measurement procedure. The practice is challenged by the resource requirements as well as uncertainty surrounding experimental design and statistical analysis that is optimal for individual laboratories, although guidance is becoming increasingly available. Collaborative verification efforts as well as application of patient-based monitoring are likely to further improve identification of any differences in performance in a relatively timely manner. Appropriate follow up actions of failed lot-to-lot verification is required and must balance potential disruptions to clinical services provided by the laboratory. Manufacturers need to increase transparency surrounding release criteria and work closer with laboratory professionals to ensure acceptable reagent lots are released to end users. A tripartite collaboration between regulatory bodies, manufacturers, and laboratory medicine professional bodies is key to developing a balanced system where regulatory, manufacturing, and clinical requirements of laboratory testing are met, to minimize differences between reagent lots and ensure patient safety. Clinical Chemistry and Laboratory Medicine has served as a fertile platform for advancing the discussion and practice of lot-to-lot verification in the past 60 years and will continue to be an advocate of this important topic for many more years to come. [ABSTRACT FROM AUTHOR]

• Bias assessment can be based on regression slope, intercept and paired difference. • They have high false rejection rates and/ or low probability of bias detection. • Paired t -test performed best in low range ratio and low imprecision scenarios. • Mean difference performed better in all other range ratio and imprecision scenarios. • Mean difference and paired- t test combined increases power, false rejection rates. This simulation study was undertaken to assess the statistical performance of six commonly used rejection criteria for bias detection. The false rejection rate (i.e. rejection in the absence of simulated bias) and the probability of bias detection were assessed for the following: difference in measurements for individual sample pair, the mean of the paired differences, t -statistics (paired t -test), slope < 0.9 or > 1.1, intercept > 50% of the lower limit of measurement range, and coefficient of determination (R2) > 0.95. The linear regressions evaluated were ordinary least squares, weighted least squares and Passing-Bablok regressions. A bias detection rate of < 50% and false rejection rates of >10% are considered unacceptable for the purpose of this study. Rejection criteria based on regression slope, intercept and paired difference (10%) for individual samples have high false rejection rates and/ or low probability of bias detection. T -statistics (α = 0.05) performed best in low range ratio (lowest-to-highest concentration in measurement range) and low imprecision scenarios. Mean difference (10%) performed better in all other range ratio and imprecision scenarios. Combining mean difference and paired- t test improves the power of bias detection but carries higher false rejection rates. This study provided objective evidence on commonly used rejection criteria to guide laboratory on the experimental design and statistical assessment for bias detection during method evaluation or reagent lot verification. [ABSTRACT FROM AUTHOR]

Objectives: Automated qualitative serology assays often measure quantitative signals that are compared against a manufacturer-defined cutoff for qualitative (positive/negative) interpretation. The current general practice of assessing serology assay performance by overall concordance in a qualitative manner may not detect the presence of analytical shift/drift that could affect disease classifications.Methods: We describe an approach to defining bias specifications for qualitative serology assays that considers minimum positive predictive values (PPVs) and negative predictive values (NPVs). Desirable minimum PPVs and NPVs for a given disease prevalence are projected as equi-PPV and equi-NPV lines into the receiver operator characteristic curve space of coronavirus disease 2019 serology assays, and the boundaries define the allowable area of performance (AAP).Results: More stringent predictive values produce smaller AAPs. When higher NPVs are required, there is lower tolerance for negative biases. Conversely, when higher PPVs are required, there is less tolerance for positive biases. As prevalence increases, so too does the allowable positive bias, although the allowable negative bias decreases. The bias specification may be asymmetric for positive and negative direction and should be method specific.Conclusions: The described approach allows setting bias specifications in a way that considers clinical requirements for qualitative assays that measure signal intensity (eg, serology and polymerase chain reaction). [ABSTRACT FROM AUTHOR]

• Reagent verification may use ordinary least squares or Deming regressions. • Weighted forms of regression better detected lot-to-lot bias than unweighted forms. • Larger sample sizes and more replicates improve bias detection capability. • Effect of sample size and replicates depends on range ratios and assay imprecision. • Laboratories should tailor verification protocols balanced with assay performance. We investigate the simulated impact of varying sample size and replicate number using ordinary least squares (OLS) and Deming regression (DR) in both weighted and unweighted forms, when applied to paired measurements in lot-to-lot verification. Simulation parameter investigated in this study were: range ratio, analytical coefficient of variation, sample size, replicates, alpha (level of significance) and constant and proportional biases. For each simulation scenario, 10,000 iterations were performed, and the average probability of bias detection was determined. Generally, the weighted forms of regression significantly outperformed the unweighted forms for bias detection. At the low range ratio (1:10), for both weighted OLS and DR, improved bias detection was observed with greater number of replicates, than increasing the number of comparison samples. At the high range ratio (1:1000), for both weighted OLS and DR, increasing the number of replicates above two is only slightly more advantageous in the scenarios examined. Increasing the numbers of comparison samples resulted in better detection of smaller biases between reagent lots. The results of this study allow laboratories to determine a tailored approach to lot-to-lot verification studies, balancing the number of replicates and comparison samples with the analytical performance of measurement procedures involved. [ABSTRACT FROM AUTHOR]

Detection of between-lot reagent bias is clinically important and can be assessed by application of regression-based statistics on several paired measurements obtained from the existing and new candidate lot. Here, the bias detection capability of six regression-based lot-to-lot reagent verification assessments, including an extension of the Bland–Altman with regression approach are compared. Least squares and Deming regression (in both weighted and unweighted forms), confidence ellipses and Bland–Altman with regression (BA-R) approaches were investigated. The numerical simulation included permutations of the following parameters: differing result range ratios (upper:lower measurement limits), levels of significance (alpha), constant and proportional biases, analytical coefficients of variation (CV), and numbers of replicates and sample sizes. The sample concentrations simulated were drawn from a uniformly distributed concentration range. At a low range ratio (1:10, CV 3%), the BA-R performed the best, albeit with a higher false rejection rate and closely followed by weighted regression approaches. At larger range ratios (1:1,000, CV 3%), the BA-R performed poorly and weighted regression approaches performed the best. At higher assay imprecision (CV 10%), all six approaches performed poorly with bias detection rates <50%. A lower alpha reduced the false rejection rate, while greater sample numbers and replicates improved bias detection. When performing reagent lot verification, laboratories need to finely balance the false rejection rate (selecting an appropriate alpha) with the power of bias detection (appropriate statistical approach to match assay performance characteristics) and operational considerations (number of clinical samples and replicates, not having alternate reagent lot). [ABSTRACT FROM AUTHOR]