Search

Purpose: To develop a model that simulates radiologist assessments and use it to explore whether pairing readers based on their individual performance characteristics could optimize screening performance., Methods: Logistic regression models were designed and used to model individual radiologist assessments. For model evaluation, model-predicted individual performance metrics and paired disagreement rates were compared against the observed data using Pearson correlation coefficients. The logistic regression models were subsequently used to simulate different screening programs with reader pairing based on individual true-positive rates (TPR) and/or false-positive rates (FPR). For this, retrospective results from breast cancer screening programs employing double reading in Sweden, England, and Norway were used. Outcomes of random pairing were compared against those composed of readers with similar and opposite TPRs/FPRs, with positive assessments defined by either reader flagging an examination as abnormal., Results: The analysis data sets consisted of 936,621 (Sweden), 435,281 (England), and 1,820,053 (Norway) examinations. There was good agreement between the model-predicted and observed radiologists' TPR and FPR ( r  ≥ 0.969). Model-predicted negative-case disagreement rates showed high correlations ( r  ≥ 0.709), whereas positive-case disagreement rates had lower correlation levels due to sparse data ( r  ≥ 0.532). Pairing radiologists with similar FPR characteristics (Sweden: 4.50% [95% confidence interval: 4.46%-4.54%], England: 5.51% [5.47%-5.56%], Norway: 8.03% [7.99%-8.07%]) resulted in significantly lower FPR than with random pairing (Sweden: 4.74% [4.70%-4.78%], England: 5.76% [5.71%-5.80%], Norway: 8.30% [8.26%-8.34%]), reducing examinations sent to consensus/arbitration while the TPR did not change significantly. Other pairing strategies resulted in equal or worse performance than random pairing., Conclusions: Logistic regression models accurately predicted screening mammography assessments and helped explore different radiologist pairing strategies. Pairing readers with similar modeled FPR characteristics reduced the number of examinations unnecessarily sent to consensus/arbitration without significantly compromising the TPR., Highlights: A logistic-regression model can be derived that accurately predicts individual and paired reader performance during mammography screening reading.Pairing screening mammography radiologists with similar false-positive characteristics reduced false-positive rates with no significant loss in true positives and may reduce the number of examinations unnecessarily sent to consensus/arbitration., Competing Interests: The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J.J.J.G. no relevant relationships. C.K.A. activities related to the present article: no relevant relationships; activities not related to the present article: consultant at Canon Medical and in the advisory board of Izotropic Corp. F.S. activities related to the present article: no relevant relationships; activities not related to the present article: received personal speaker fees from Lunit Inc. S.T.P. activities related to the present article: no relevant relationships; activities not related to the present article: funded by an NIHR Postdoctoral Fellowship and an NIHR Career Development Fellowship (CDF-2016-09-018). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. D.J.J. no relevant relationships. M.L. no relevant relationships. S.H. activities related to the present article: no relevant relationships; activities not related to the present article: head of Breastcreen Norway. The interpretation and reporting of these data are the sole responsibility of the authors, and no endorsement by Cancer Registry of Norway is intended nor should be inferred. M.J.M.B. activities related to the present article: no relevant relationships; activities not related to the present article: has grants/grants pending with Screenpoint Medical, Sectra Benelux, Hologic, Volpara Solutions, Lunit Inc., and iCAD; received personal speaker fees from Hologic and Siemens Healthcare. I.S. activities related to the present article: no relevant relationships; activities not related to the present article: has grants/grants pending with Siemens Healthcare, Canon Medical, Screenpoint Medical, Sectra Benelux, Hologic, Volpara Solutions, Lunit Inc., and iCAD; received payment for lectures including service on speakers’ bureaus from Siemens Healthcare and is on the Scientific Advisory Board of Koning Corp. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided entirely by a grant from the Dutch Cancer Society and NWO Domain AES, as part of their joint strategic research program: Technology for Oncology ll. The collaboration project is co-funded by the PPP Allowance made available by Health-Holland, Top Sector Life Sciences & Health, to stimulate public-private partnerships. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Objectives: To analyze discordant and false-negatives of double reading digital breast tomosynthesis (DBT) versus digital mammography (DM) including reading times in the Oslo Tomosynthesis Screening Trial (OTST), and reclassify these in a retrospective reader study as missed, minimal sign, or true-negatives., Methods: The prospective OTST comparing double reading DBT vs. DM had paired design with four parallel arms: DM, DM + computer aided detection, DBT + DM, and DBT + synthetic mammography. Eight radiologists interpreted images in batches using a 5-point scale. Reading time was automatically recorded. A retrospective reader study including four radiologists classified screen-detected cancers with at least one false-negative score and screening examinations of interval cancers as negative, non-specific minimal sign, significant minimal sign, and missed; the two latter groups are defined "actionable." Statistics included chi-square, Fisher's exact, McNemar's, and Mann-Whitney U tests., Results: Discordant rate (cancer missed by one reader) for screen-detected cancers was overall comparable (DBT (31% [71/227]) and DM (30% [52/175]), p = .81), significantly lower at DBT for spiculated cancers (DBT, 19% [20/106] vs. DM, 36% [38/106], p = .003), but high (28/49 = 57%, p = 0.001) for DBT-only detected spiculated cancers. Reading time and sensitivity varied among readers. False-negative DBT-only detected spiculated cancers had shorter reading time than true-negatives in 46% (13/28). Retrospective evaluation classified the following DBT exams "actionable": three missed by both readers, 95% (39/41) of discordant cancers detected by both modes, all 30 discordant DBT-only cancers, 25% (13/51) of interval cancers., Conclusions: Discordant rate was overall comparable for DBT and DM, significantly lower at DBT for spiculated cancers, but high for DBT-only detected spiculated lesions. Most false-negative screen-detected DBT were classified as "actionable.", Clinical Relevance Statement: Retrospective evaluation of false-negative interpretations from the Oslo Tomosynthesis Screening Trial shows that most discordant and several interval cancers could have been detected at screening. This underlines the potential for modern AI-based reading aids and triage, as high-volume screening is a demanding task., Key Points: • Digital breast tomosynthesis (DBT) screening is more sensitive and has higher specificity compared to digital mammography screening, but high-volume DBT screening is a demanding task which can result in high discordance rate among readers. • Independent double reading DBT screening had overall comparable discordance rate as digital mammography, lower for spiculated masses seen on both modalities, and higher for small spiculated cancer seen only on DBT. • Almost all discordant digital breast tomosynthesis-detected cancers (72 of 74) and 25% (13 of 51) of the interval cancers in the Oslo Tomosynthesis Screening Trial were retrospectively classified as actionable and could have been detected by the readers., (© 2023. The Author(s).)