Comparison of visual criteria for amyloid-PET reading: could criteria merging reduce inter-rater variability?

Background: Three different amyloid tracers labeled with 18-flourine have been introduced into clinical use. The leaflets of tracers indicate different visual criteria for PET reporting. In clinical practice, it is not yet ascertained whether these criteria are equivalent in terms of diagnostic accuracy or if anyone is better than another. We aimed to evaluate the inter and intra-rater variability of visual assessment of ¹⁸ F-Florbetapir PET/CT images among six independent readers with different clinical experience.
Methods: We analyzed 252 PET/CT scans, visually assessed by each reader three times, applying independently the three different reading criteria proposed. Each reader evaluated the regional uptake specifying for each cortical region a numeric value of grading of positivity in order to assign a final score. At the end of each reading a level of confidence was determined by assigning a score from 0 (negative) to 4 (positive). After first reading, those cases in which the evaluations by two experienced readers did not match (discordant cases) were independently reevaluated merging all the three different visual interpretation criteria.
Results: Good agreement was observed for visual interpretation among the six readers' confidence-level using independently the three visual reading criteria: ICC=0.83 (0.80-0.86) for ¹⁸ F-florbetapir, ICC=0.84 (0.81-0.87) for ¹⁸ F-florbetaben, and ICC=0.86 (0.83-0.88) for ¹⁸ F-flutemetamol reading. A good inter-rater agreement was observed for final-score too: ICC=0.74 (0.70-0.78) for ¹⁸ F-florbetapir; ICC=0.82 (0.79-0.85) for ¹⁸ F-florbetaben; ICC=0.84 (0.81-0.87) for ¹⁸ F-flutemetamol. Intra-rater agreement was good for final-score (from 0.76 to 0.90; P<0.001) and confidence-level (Spearman's rho from 0.89 to 1.00; P<0.001). Disagreement between the two experienced readers was observed in 22 of 252 cases (9%). The agreement converged over a second round of independent reading in 12 of 22 cases (54%), by merging all the criteria.
Conclusions: All the criteria proposed are useful to determine the grading of positivity or negativity of amyloid deposition and their merging improves the diagnostic confidence and provides a better agreement.