Search

Background: This study aimed to assess the interobserver variability of semi-automatic diameter and volumetric measurements versus manual diameter measurements for small lung nodules identified on computed tomography scans., Methods: The radiological patient database was searched for CT thorax examinations with at least one noncalcified solid nodule (∼3-10 mm). Three radiologists with four to six years of experience evaluated each nodule in accordance with the Fleischner Society guidelines using standard diameter measurements, semi-automatic lesion diameter measurements, and volumetric assessments. Spearman's correlation coefficient measured intermeasurement agreement. We used descriptive Bland-Altman plots to visualize agreement in the measured data. Potential discrepancies were analyzed., Results: We studied a total of twenty-six nodules. Spearman's test showed that there was a much stronger relationship ( p < 0.05) between reviewers for the semi-automatic diameter and volume measurements (avg. r = 0.97 ± 0.017 and 0.99 ± 0.005, respectively) than for the manual method (avg. r = 0.91 ± 0.017). In the Bland-Altman test, the semi-automatic diameter measure outperformed the manual method for all comparisons, while the volumetric method had better results in two out of three comparisons. The incidence of reviewers modifying the software's automatic outline varied between 62% and 92%., Conclusions: Semi-automatic techniques significantly reduced interobserver variability for small solid nodules, which has important implications for diagnostic assessments and screening. Both the semi-automatic diameter and semi-automatic volume measurements showed improvements over the manual measurement approach. Training could further diminish observer variability, given the considerable diversity in the number of adjustments among reviewers., Competing Interests: The authors declare no conflicts of interest.

Background: Approximately, 55% of breast carcinomas are reported to be HER-2 low breast carcinomas. Trastuzumab-Deruxtecan is a new FDA-approved targeted therapy for HER-2 low metastatic breast carcinomas, making it essential that all efforts are made to identify these tumors in specimens submitted for pathologic examination. Cytology specimens are often the first and only modality of this assessment due to the ease of specimen procurement. This study aimed to determine the variability in HER-2 immunostaining interpretation among observers using cytologic specimens from metastatic sites., Design: A pathology database search was made to identify metastatic breast carcinoma reported in cytology specimens. A manual search was then done to identify cases of HER-2 low category, H&E cell block and HER-2 neu immunostain slides were retrieved for a total of 50 cases. Reviewer #1 and #2 independently interpreted HER-2 immunostain of all 50 cases. Only discordant cases were sent for reviewer-3 interpretation. All three were blinded by the metastatic site, and original HER-2 interpretation., Results: Of 50 cases, 11 cases (22%) were reported as concordant scores between reviewer #1 and reviewer #2 but had a discordant original IHC report. Additionally, 4 cases (8%) had discordant reporting of HER2 IHC stain between reviewer #1 and reviewer #2 making a total of 15 cases (30%) with overall discordant results., Conclusion: This study highlights the interobserver variability of HER-2 immunostain interpretation for HER-2 low category of breast carcinomas. We recommend the need for more robust laboratory techniques including molecular for uniform identification of these unique targetable metastatic breast carcinoma groups., (© 2024 Wiley Periodicals LLC.)

Background: Endoscopy scoring is a key component in the diagnosis of ulcerative colitis (UC) and Crohn's disease (CD). Variability in endoscopic scoring can impact patient trial eligibility and treatment effect measurement. In this study, we examine inter- and intraobserver variability of inflammatory bowel disease endoscopic scoring systems in a systematic review and meta-analysis., Methods: We included observational studies that evaluated the inter- and intraobserver variability using UC (endoscopic Mayo Score [eMS], Ulcerative Colitis Endoscopic Index of Severity [UCEIS]) or CD (Crohn's Disease Endoscopic Index of Severity [CDEIS], Simple Endoscopic Score for Crohn's Disease [SES-CD]) systems among adults (≥18 years of age) and were published in English. The strength of agreement was categorized as fair, moderate, good, and very good., Results: A total of 6003 records were identified. After screening, 13 studies were included in our analysis. The overall interobserver agreement rates were 0.58 for eMS, 0.66 for UCEIS, 0.80 for CDEIS, and 0.78 for SES-CD. The overall heterogeneity (I2) for these systems ranged from 93.2% to 99.2%. A few studies assessed the intraobserver agreement rate. The overall effect sizes were 0.75 for eMS, 0.87 for UCEIS, 0.89 for CDEIS, and 0.91 for SES-CD., Conclusions: The interobserver agreement rates for eMS, UCEIS, CDEIS, and SES-CD ranged from moderate to good. The intraobserver agreement rates for eMS, UCEIS, CDEIS, and SES-CD ranged from good to very good. Solutions to improve interobserver agreement could allow for more accurate patient assessment, leading to richer, more accurate clinical management and clinical trial data., (© The Author(s) 2024. Published by Oxford University Press on behalf of Crohn’s & Colitis Foundation. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.)

Background: In patients undergoing breast-conserving therapy without surgical clip implantation, the accuracy of tumor bed identification and the consistency of clinical target volume (CTV) delineation under computed tomography (CT) simulation remain suboptimal. This study aimed to investigate the feasibility of implementing preoperative magnetic resonance (MR) simulation on delineations by assessing interobserver variability (IOV)., Methods: Preoperative MR and postoperative CT simulations were performed in patients who underwent breast-conserving surgery with no surgical clips implanted. Custom immobilization pads were used to ensure the same supine position. Three radiation oncologists independently delineated the CTV of tumor bed on the images acquired from MR and CT simulation registration and CT simulation alone. Cavity visualization score (CVS) was assigned to each patient based on the clarity of the tumor bed on CT simulation images. IOV was indicated by generalized conformity index (CI gen ), denoted as CI gen-CT and CI gen-MR/CT , and the distance between the centroid of mass (dCOM), denoted as dCOM CT and dCOM MR/CT . The variation of IOV in different CVS subgroups was analyzed., Results: A total of 10 patients were enrolled in this study. The median and interquartile range (IQR) of maximum pathological diameter of the tumors in all patients were 1.55 (0.80-1.92) cm. No statistical significance was found between the volumes of CTVs on CT simulation and on MR/CT simulation registration images (p = 0.387). CI gen-MR/CT was significantly larger than CI gen-CT (p = 0.005). dCOM MR/CT was significantly smaller than dCOM CT (p = 0.037). The median and IQR of CVS in all patients were 2.34 (2.00-3.08). The difference of CI gen between CI gen-MR/CT and CI gen-CT was larger in the low CVS group (p = 0.016). The difference of dCOM showed a decreasing trend when CVS was lower, although it did not reach statistical significance (p = 0.095)., Conclusions: For patients who underwent breast-conserving surgery without surgical clip implantation, the use of preoperative MR simulation in delineating the CTV of tumor bed decreased the IOV among observers. The consistency of tumor bed identification was improved especially in cases where the margins of tumor bed were challenging to visualize on CT simulation images. The study findings offer potential benefits in reducing local recurrence and minimizing tissue irritation in the surrounding areas. Future investigation in a larger patient cohort to validate our results is warranted., (© 2024. The Author(s).)

Grading lung squamous cell carcinoma (LUSC) is controversial and not universally accepted. The histomorphologic feature of tumor budding (TB) is an established independent prognostic factor in colorectal cancer, and its importance is growing in other solid cancers, making it a candidate for inclusion in tumor grading schemes. We aimed to compare TB between preoperative biopsies and resection specimens in pulmonary squamous cell carcinoma and assess interobserver variability. A retrospective cohort of 249 consecutive patients primarily resected with LUSC in Bern (2000-2013, n = 136) and Lausanne (2005-2020, n = 113) with available preoperative biopsies was analyzed for TB and additional histomorphologic parameters, such as spread through airspaces and desmoplasia, by 2 expert pathologists (M.M., C.N.). Results were correlated with clinicopathologic parameters and survival. In resection specimens, peritumoral budding (PTB) score was low (0-4 buds/0.785 mm 2 ) in 47.6%, intermediate (5-9 buds/0.785 mm 2 ) in 27.4%, and high (≥10 buds/0.785 mm 2 ) in 25% of cases (median bud count, 5; IQR, 0-26). Both the absolute number of buds and TB score were similar when comparing tumor edge and intratumoral zone (P = .192) but significantly different from the score obtained in the biopsy (P < .001). Interobserver variability was moderate, regardless of score location (Cohen kappa, 0.59). The discrepant cases were reassessed, and consensus was reached in all cases with identification of causes of discordance. TB score was significantly associated with stage (P = .002), presence of lymph node (P = .033), and distant metastases (P = .020), without significant correlation with overall survival, tumor size, or pleural invasion. Desmoplasia was significantly associated with higher PTB (P < .001). Spread through airspaces was present in 34% and associated with lower PTB (P < .001). To conclude, despite confirming TB as a reproducible factor in LUSC, we disclose areas of scoring ambiguity. Preoperative biopsy evaluation was insufficient in establishing the final TB score of the resected tumor., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: To explore the topic of Prostate Imaging-Reporting and Data System (PI-RADS) interobserver variability, including a discussion of major sources, mitigation approaches, and future directions., Methods: A narrative review of PI-RADS interobserver variability., Results: PI-RADS was developed in 2012 to set technical standards for prostate magnetic resonance imaging (MRI), reduce interobserver variability at interpretation, and improve diagnostic accuracy in the MRI-directed diagnostic pathway for detection of clinically significant prostate cancer. While PI-RADS has been validated in selected research cohorts with prostate cancer imaging experts, subsequent prospective studies in routine clinical practice demonstrate wide variability in diagnostic performance. Radiologist and biopsy operator experience are the most important contributing drivers of high-quality care among multiple interrelated factors including variability in MRI hardware and technique, image quality, and population and patient-specific factors such as prostate cancer disease prevalence. Iterative improvements in PI-RADS have helped flatten the curve for novice readers and reduce variability. Innovations in image quality reporting, administrative and organisational workflows, and artificial intelligence hold promise in improving variability even further., Conclusion: Continued research into PI-RADS is needed to facilitate benchmark creation, reader certification, and independent accreditation, which are systems-level interventions needed to uphold and maintain high-quality prostate MRI across entire populations., (© 2024 The Author(s). BJU International published by John Wiley & Sons Ltd on behalf of BJU International.)

Introduction: The reliability of interpretation of coronary angiography as a diagnostic tool was investigated. Furthermore, the impact of interobserver variability of coronary lesions on clinical decision-making was assessed. One of our motivations to do this research was the research gaps and our aim to have up-to-date information regarding interobserver variability among different cardiologists., Methods: Our objective was to quantify interobserver variability among cardiologists who have seen angiograms independently. Disagreement among cardiologists in the visual assessment of invasive coronary angiography of coronary artery stenosis is not uncommon in previous studies. Three cardiologists with extensive experience in coronary angiography, including the primary cardiologist of each patient, read the angiograms of 200 patients from Toronto General Hospital independently., Results: Our research showed the mean agreement among all participating observers was 77.4%; therefore, the interobserver variability of coronary angiography interpretation was 22.6%., Discussion: Coronary angiography is still the gold-standard technique for guidance regarding coronary lesions. Sometimes, coronary angiography results in underestimation or overestimation of a lesion's functional severity. Interobserver variability should also be considered when interpreting the severity of coronary stenoses via invasive coronary angiography. This research shows that interobserver variability regarding coronary angiograms is still present (22.6%)., Competing Interests: The authors report no conflicts of interests for this work., (© 2024 Shivaie et al.)

Purpose: Traffic accidents persist as a leading cause of death. European law mandates the integration of automatic emergency call systems (eCall). Our project focuses on an automated injury prediction device for car accidents, correlating technical and epidemiological input data, such as age, gender, seating position, impact on the passenger compartment, seatbelt usage, impact direction, EES, vehicle class, and airbag deployment. This study aims to explore interobserver variability in data collection quality in real accident scenarios. The assessment will evaluate the impact of user training and measure the time needed for data collection to inform user recommendations for the prehospital assessment. Insights from this study can aid in evaluating the ability of different professional groups to identify potential accident-independent parameters at accident scenes. This includes, among other things, relaying information to dispatchers at rescue control centers, also within the context of telemedicine approaches., Methods: During group sessions, real accident scenarios were presented both before and after a training presentation. Participants, including laypersons, accident research staff, emergency services, hospital physicians, and emergency physicians, visually assessed injury prediction parameters within a time limit. Training involved defining and explaining parameters using accident images. The study analyzed participant demographics, prediction accuracy, and time required, comparing assessment quality between professional groups and before and after training., Results: In summary, the study demonstrates that training had a significantly positive impact on the quality of assessment for technical accident parameters. The processing time decreased significantly after training. A notable training effect was observed, particularly for the parameters of rigid collision object, affected passenger compartment, energy equivalent speed (EES), and front and side airbags. It was recommended that individuals without prior knowledge should receive training on assessing EES. Overall, it was evident that technical parameters following a traffic accident can be well assessed through training, irrespective of the professional group., Conclusion: Significant differences in the assessment quality of technical accident parameters were observed based on technical and medical expertise. After user training, interdisciplinary differences were reconciled, and all professional groups yielded comparable results, indicating that training can enhance the assessment abilities of all participants in the rescue chain, while the time required for assessing accident parameters was significantly reduced with training., (© 2024. The Author(s).)

Purpose: Ultrasound evaluation of thyroid nodules is the preferred technique, but it is dependent on operator interpretation, leading to inter-observer variability. The current study aimed to determine the inter-physician consensus on nodular characteristics, risk categorization in the classification systems, and the need for fine needle aspiration puncture., Methods: Four endocrinologists from the same center blindly evaluated 100 ultrasound images of thyroid nodules from 100 different patients. The following ultrasound features were evaluated: composition, echogenicity, margins, calcifications, and microcalcifications. Nodules were also classified according to ATA, EU-TIRADS, K-TIRADS, and ACR-TIRADS classifications. Krippendorff's alpha test was used to assess interobserver agreement., Results: The interobserver agreement for ultrasound features was: Krippendorff's coefficient 0.80 (0.71-0.89) for composition, 0.59 (0.47-0.72) for echogenicity, 0.73 (0.57-0.88) for margins, 0.55 (0.40-0.69) for calcifications, and 0.50 (0.34-0.67) for microcalcifications. The concordance for the classification systems was 0.7 (0.61-0.80) for ATA, 0.63 (0.54-0.73) for EU-TIRADS, 0.64 (0.55-0.73) for K-TIRADS, and 0.68 (0.60-0.77) for K-TIRADS. The concordance in the indication of fine needle aspiration puncture (FNA) was 0.86 (0.71-1), 0.80 (0.71-0.88), 0.77 0.67-0.87), and 0.73 (0.64-0.83) for systems previously described respectively., Conclusions: Interobserver agreement was acceptable for the identification of nodules requiring cytologic study using various classification systems. However, limited concordance was observed in risk stratification and many ultrasonographic characteristics of the nodules., (© 2024. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

Purpose: Trochlear dysplasia is one of the main risk factors for recurrent patellar dislocation. The Dejour classification identifies four categories that can be used to classify trochlear dysplasia. The purpose of this study is to evaluate the inter- and intraobserver reliability of the Dejour classification for trochlear dysplasia. The hypothesis was that both intra- and interobserver reliability would be at least moderate., Methods: This is a cross-sectional, reliability study. Twenty-eight examiners from the International Patellofemoral Study Group 2022 meeting evaluated lateral radiographs of the knee and axial magnetic resonance images from 15 cases of patellofemoral instability with trochlear dysplasia. They classified each case according to Dejour's classification for trochlear dysplasia (A-D). There were three rounds: one with only computed radiograph (CR), one with only magnetic resonance imaging (MRI) and one with both. Inter- and intraobserver reliability were calculated using κ coefficient (0-1)., Results: The mean age of patients was: 14.6 years; 60% were female and 53% had open physis. The interobserver reliability κ probabilities were 0.2 (CR), 0.13 (MRI) and 0.12 (CR and MRI). The intraobserver reliability κ probabilities were 0.45 (CR), 0.44 (MRI) and 0.65 (CR and MRI)., Conclusion: The Dejour classification for trochlear dysplasia has slight interobserver reliability and substantial intraobserver reliability., Level of Evidence: Level I., (© 2024 European Society of Sports Traumatology, Knee Surgery and Arthroscopy.)

Introduction: Semi-quantitative scoring of various parameters in renal biopsy is accepted as an important tool to assess disease activity and prognostication. There are concerns on the impact of interobserver variability in its prognostic utility, generating a need for computerized quantification., Methods: We studied 94 patients with renal biopsies, 45 with native diseases and 49 transplant patients with index biopsies for Polyomavirus nephropathy. Chronicity scores were evaluated using two methods. A standard definition diagram was agreed after international consultation and four renal pathologists scored each parameter in a double-blinded manner. Interstitial fibrosis (IF) score was assessed with five different computerized and AI-based algorithms on trichrome and PAS stains., Results: There was strong prognostic correlation with renal function and graft outcome at a median follow-up ranging from 24 to 42 months respectively, independent of moderate concordance for pathologists scores. IF scores with two of the computerized algorithms showed significant correlation with estimated glomerular filtration rate (eGFR) at biopsy but not at the end of follow-up. There was poor concordance for AI based platforms., Conclusion: Chronicity scores are robust prognostic tools despite interobserver reproducibility. AI-algorithms have absolute precision but are limited by significant variation when different hardware and software algorithms are used for quantification., Competing Interests: Declaration of competing interest All authors declare that there is no conflict of interest of any kind. No funding was obtained for the study., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Background: In radiotherapy inaccuracy in organ at risk (OAR) delineation can impact treatment plan optimisation and treatment plan evaluation. Brouwer et al. showed significant interobserver variability (IOV) in OAR delineation in head and neck cancer (HNC) and published international consensus guidelines (ICG) for OAR delineation in 2015. The aim of our study was to evaluate IOV in the presence of these guidelines., Methods: HNC radiation oncologists (RO) from each Belgian radiotherapy centre were invited to complete a survey and submit contours for 5 HNC cases. Reference contours (OARref) were obtained by a clinically validated artificial intelligence-tool trained using ICG. Dice similarity coefficients (DSC), mean surface distance (MSD) and 95% Hausdorff distances (HD95) were used for comparison., Results: Fourteen of twenty-two RO (64%) completed the survey and submitted delineations. Thirteen (93%) confirmed the use of delineation guidelines, of which six (43%) used the ICG. The OARs whose delineations agreed best with the OARref were mandible [median DSC 0.9, range (0.8-0.9); median MSD 1.1 mm, range (0.8-8.3), median HD95 3.4 mm, range (1.5-38.7)], brainstem [median DSC 0.9 (0.6-0.9); median MSD 1.5 mm (1.1-4.0), median HD95 4.0 mm (2.3-15.0)], submandibular glands [median DSC 0.8 (0.5-0.9); median MSD 1.2 mm (0.9-2.5), median HD95 3.1 mm (1.8-12.2)] and parotids [median DSC 0.9 (0.6-0.9); median MSD 1.9 mm (1.2-4.2), median HD95 5.1 mm (3.1-19.2)]. Oral cavity, cochleas, PCMs, supraglottic larynx and glottic area showed more variation. RO who used the consensus guidelines showed significantly less IOV (p = 0.008)., Conclusions: Although ICG for delineation of OARs in HNC exist, they are only implemented by about half of RO participating in this study, which partly explains the delineation variability. However, this study highlights that guidelines alone do not suffice to eliminate IOV and that more effort needs to be done to accomplish further treatment standardisation, for example with artificial intelligence.

Purpose: To investigate the interobserver variability (IOV) in target volume delineation of definitive radiotherapy for thoracic esophageal cancer (TEC) among cancer centers in China, and ultimately improve contouring consistency as much as possible to lay the foundation for multi-center prospective studies., Methods: Sixteen cancer centers throughout China participated in this study. In Phase 1, three suitable cases with upper, middle, and lower TEC were chosen, and participants were asked to contour a group of gross tumor volume (GTV-T), nodal gross tumor volume (GTV-N) and clinical target volume (CTV) for each case based on their routine experience. In Phase 2, the same clinicians were instructed to follow a contouring protocol to re-contour another group of target volume. The variation of the target volume was analyzed and quantified using dice similarity coefficient (DSC)., Results: Sixteen clinicians provided routine volumes, whereas ten provided both routine and protocol volumes for each case. The IOV of routine GTV-N was the most striking in all cases, with the smallest DSC of 0.37 (95% CI 0.32-0.42), followed by CTV, whereas GTV-T showed high consistency. After following the protocol, the smallest DSC of GTV-N was improved to 0.64 (95% CI 0.45-0.83, P = 0.005) but the DSC of GTV-T and CTV remained constant in most cases., Conclusion: Variability in target volume delineation was observed, but it could be significantly reduced and controlled using mandatory interventions.

The diagnosis of inflammatory bowel disease (IBD)-associated dysplasia is challenging, and past studies have demonstrated considerable interobserver variability in such diagnoses. This study aimed to assess interobserver agreement in IBD dysplasia diagnoses among subspecialty GI pathologists and to explore the impact of mentorship on diagnostic variability. Twelve GI pathologist mentees and 7 GI pathologist mentors reviewed 163 digitized slides. Participants rendered a diagnosis of negative for dysplasia, indefinite for dysplasia, low-grade dysplasia, or high-grade dysplasia and provided a confidence level for each case. Interobserver agreement and reliability were assessed using Cohen's and Fleiss' kappa (κ) statistics and intraclass correlation coefficient (ICC) analysis. The overall κ coefficient was 0.42 (95% CI: 0.38-0.46). The overall ICC was 0.67 (95% CI: 0.62-0.72). Κ coefficients ranged from 0.31 to 0.49 for mentor/mentee pairs and from 0.34 to 0.55 for pairs of mentees of the same mentor. The combined κ coefficient was 0.44 (95% CI: 0.39-0.48) for all mentees and 0.39 (95% CI: 0.34-0.43) for all mentors. Common features in low agreement cases included mucosal atrophy, areas of stark contrast, serrations, decreased goblet cells, absent surface epithelium, and poor orientation. Participants were confident in most diagnoses, and increased confidence levels generally correlated with higher interobserver agreement. Interobserver agreement among subspecialist GI pathologists in this curated cohort of IBD dysplasia cases was fair to moderate. Mentorship during GI pathology fellowship does not appear to be a significant factor contributing to interobserver variability, but increased experience also does not seem to improve interobserver agreement.

Background: The Abbreviated Injury Scale (AIS) and Injury Severity Score (ISS) are widely used to assess trauma patients. In this study, the interobserver variability of the injury severity assessment for severely injured patients was analyzed based on different injured anatomical regions, and the various demographic backgrounds of the observers., Methods: A standardized questionnaire was presented to surgical experts and participants of clinical polytrauma courses. It contained medical information and initial X-rays/CT-scans of 10 cases of severely injured patients. Participants estimated the severity of each injury based on the AIS. Interobserver variability for the AIS, ISS, and New Injury Severity Score (NISS) was calculated by employing the statistical method of Krippendorff's α coefficient., Results: Overall, 54 participants were included. The major contributing medical specialties were orthopedic trauma surgery (N = 36, 67%) and general surgery (N = 13, 24%). The measured interobserver variability in the assessment of the overall injury severity was high (α ISS : 0.33 / α NISS : 0.23). Moreover, there were differences in the interobserver variability of the maximum AIS (MAIS) depending on the anatomical region: α head and neck : 0.06, α thorax : 0.45, α abdomen : 0.27 and α extremities : 0.55., Conclusions: Interobserver agreement concerning injury severity assessment appears to be low among clinicians. We also noted marked differences in variability according to injury anatomy. The study shows that the assessment of injury severity is also highly variable between experts in the field. This implies the need for appropriate education to improve the accuracy of trauma evaluation in the respective trauma registries.

Background: The indeterminate categories in the Milan System for Reporting Salivary Gland Cytopathology (MSRSGC) are diagnostically challenging because of inherent heterogeneity and complexity, with wide interobserver variability (IOV). Herein, the authors explore the concordance rate (CR) between cytopathologists (CPs) and cytotechnologists (CTs) in interpreting indeterminate salivary gland lesions using the MSRSGC., Methods: Between 2011 and 2016, 86 indeterminate fine-needle aspirations had slides available for review, of which 48 had follow-up. Four CPs and 2 CTs performed an independent, blinded review of these slides and categorized them according to the MSRSGC. The CRs between CTs and CPs with the final sign-out cytopathologist (FCP) were assessed, and interobserver agreement was categorized into uniform, majority, divided, minimal, or no agreement., Results: The overall CR with the FCP ranged from 48.8% to 60.5% for CPs and from 22.1% to 36% for CTs. IOV κ scores for the entire group were 0.314 and, with the FCP as the reference, ranged from 0.403 to 0.539 for CPs and from 0.091 to 0.254 for CTs. Uniform, majority, divided, minimal, and no agreement was noted in 12.8%, 31.4%, 38.4%, 10.5%, and 6.9%, respectively, of all cases and in 16.7%, 35.4%, 31.3%, 8.3%, and 6.3%, respectively, of the cases with follow-up. Diagnostic challenges included distinguishing lymphoma from a reactive process and distinguishing mucin from mucin-like material., Conclusions: CPs had modestly higher CRs compared with CTs; and, although the variable CRs highlight indeterminate IOV, the MSRSGC enables reproducibility. Characterizing larger cohorts in the indeterminate categories will further improve MSRSGC criteria. Moreover, education on the MSRSGC should include CTs and CPs to improve overall diagnostic accuracy., (© 2020 American Cancer Society.)

Aims and Objectives: To compare and evaluate interobserver (nurses and physicians) agreement for dengue clinical signs and symptoms, including the World Health Organization diagnostic algorithm., Background: Agreement of clinical history defines the capacity of the examiner to measure a given clinical parameter in a reproducible and consistent manner, which is prerequisite for diagnosis validity. Nurses play a major role in the triage and care of dengue patients in many countries., Study Design: This is a sub-study on interobserver agreement performed as part of a cross-sectional diagnostic accuracy study for acute febrile illness (AFI) using the checklist STARD., Methods: A previously validated semi-structured sign and symptom standardised questionnaire for AFI was independently administered to 374 patients by physician and nurse pairs. The interobserver agreement was estimated using kappa statistics., Results: For a set of 27 signs and symptoms, we found six interobserver discrepancies (examiner detected red eyes, lethargy, exanthema, dyspnoea, bleeding and myalgia) as identified by regular and moderate kappa indexes. Four signs (patient observed red eyes, cough, diarrhoea and vomiting) and one symptom (earache) had near-perfect agreement. Most signs and symptoms showed substantial agreement. The WHO (Dengue guidelines for diagnosis, treatment, prevention and control: new edition, World Health Organization, 2009) clinical criteria for dengue comprise a group of symptoms known as "pains and aches." Interobserver agreement for abdominal pain, retro-orbital pain and arthralgia exceed that found for headache and myalgia., Conclusions: During a dengue outbreak, the interobserver agreement for most of the signs and symptoms used to assess AFI was substantial., Relevance to Clinical Practice: This result suggests good potential applicability of the tool by health professionals following training. A well-trained health professional is qualified to apply the standardised questionnaire to evaluate suspected dengue cases during outbreaks., (© 2020 John Wiley & Sons Ltd.)

Objective: Gleason scoring is the grading system which strongly predicts the prognosis of prostate cancer. However, even being one of the most commonly used systems, the presence of different interobserver agreement rates push the uropathologists update the definitons of the Gleason patterns. In this study, we aimed to determine the interobserver agreement variability among 7 general pathologists, and one expert uropathologist from 6 different centers., Methods: A set of 50 Hematoxylin & Eosin stained slides from 41 patients diagnosed as prostate cancer were revised by 8 different pathologists. The pathologists were also grouped according to having their residency at the same institute or working at the same center. All pathologists' and the subgroups' Gleason scores were then compared for interobserver variability by Fleiss' and Cohen's kappa tests using R v3.2.4., Results: There were about 8 pathologists from 6 different centers revised all the slides. One of them was an expert uropathologist with experience of 18 years. Among 7 general pathologists 4 had surgical pathology experience for over 5 years whilst 3 had under 5 years. The Fleiss' kappa was found as 0.54 for primary Gleason pattern, and 0.44 for total Gleason score (moderate agreement). The Fleiss' kappa was 0.45 for grade grouping system., Conclusion: Assigning a Gleason score for a patient can be problematic because of different interobserver agreement rates among pathologists even though the patterns were accepted as well-defined., Competing Interests: None

Aims: Tumour budding is considered to be a good marker for progression and prognosis in colorectal carcinomas. A uniform classification system has been established recently. The natural element of uncertainty in the practice of human medicine is also exhibited in the assessment of tumour budding. We tested the hypothesis that interobserver variability can be estimated during the assessment process and investigated its potential clinical implication., Methods and Results: Six investigators with different levels of experience could perceive different levels of difficulty (LOD) and estimated different levels of interobserver variability (LOIV) (Li1, lower than average; Li2, average; Li3, higher than average) during the assessment of tumour budding in 244 cases of colon cancer (pT3/4). In total, the LOIV showed following distribution: Li1: 36.1%, Li2: 43.9% and Li3: 20.0%. The LOIV was correlated significantly with the LOD given by the investigator. In total, the agreement rates with the final consensus classification were: Li1: 93.4%, Li2: 78.5% and Li3: 58.4%. The relative risk of disagreement with the final consensus classification was more than six times higher when a case was estimated to have a high rather than a low interobserver variability., Conclusion: Our data show that the investigator can estimate the interobserver variability during the ongoing rating process in pT3/4 colon cancer. The LOIV/LOD seems to be a valuable parameter of the assessment quality. For Li3 cases further measures seem mandatory., (© 2018 John Wiley & Sons Ltd.)

Purpose: To assess dimension measurement variability of liver metastases from neuroendocrine tumors (LMNET) on different magnetic resonance imaging (MRI) sequences., Material and Methods: In this institutional review board-approved retrospective study from January 2011 to December 2012, all liver MRI examinations performed at our department in patients with at least one measurable LMNET according to response evaluation criteria in solid tumors (RECIST1.1) were included. Up to two lesions were selected on T2-weighted MR images. Three reviewers independently measured long axes of 135 hepatic metastases in 30 patients (16 men, 14 women, mean age 61±11.4 (SD) years; range 28-78 years), during two separate reading sessions, on T2-weighted, diffusion-weighted MRI (DWI) (b; 50, 400, 800 s/mm 2 ) and arterial, portal and late phases after intravenous administration of a gadolinium chelate. Intraclass-correlation coefficients and Bland-Altman plots were used to assess intra-and interobserver variability., Results: Intra- and interobserver agreements ranged between 0.87-0.98, and 0.88-0.97, respectively. Intersequence agreements ranged between 0.92 [95%CI: 0.82-0.98] and 0.98 [95%CI: 0.93-0.99]. 95% limits of agreement for measurements were -10.2%,+8.9% for DWI (b=50s/mm 2 ) versus -21.9%,+24.2% and -15.8,+17.2% for arterial and portal phases, respectively., Conclusion: An increase<9% in measurement and a decrease of -10% on DWI should not be considered as true changes, with 95% confidence, versus 24% and -22% on arterial and 17%, -16% on portal phases, respectively. DWI might thus be the most reliable MR sequence for monitoring size variations of LMNETs., (Copyright © 2017 Editions françaises de radiologie. Published by Elsevier Masson SAS. All rights reserved.)

Objectives: The purpose of this study was to assess the intraobserver and interobserver variability in ultrasound (US) measurements of thyroid nodules., Methods: We performed a prospective study of the US examinations of 73 patients with 122 thyroid nodules greater than 5 mm in size. Ultrasound measurements in 4 dimensions (anteroposterior, transverse, longitudinal, and maximum diameters) and measurement of the estimated volume (using the ellipsoid formula) of each thyroid nodule were performed twice by 2 independent radiologists (A and B, with 10 years and 6 months of experience, respectively). The intraobserver and interobserver variability in measurements of thyroid nodules was assessed by a Bland-Altman analysis of agreement. The absolute values for intraobserver and interobserver variability were compared by a paired t test., Results: The 95% intraobserver and interobserver limits of agreement for the anteroposterior, transverse, longitudinal, and maximum diameters and estimated volume of thyroid nodules were ±18.2%, ± 14.3%, and ±21.0%; ± 17.2%%, ± 17.3%, and 18.2%; ± 14.6%, ± 15.5%, and ±22.3%; ± 13.8%, ± 15.5%, and ±19.6%; and ±30.2%, ± 27.7%, and ±44.1%, respectively. The absolute values for intraobserver variability were lower than those for interobserver variability for all measurements., Conclusions: There was considerable intraobserver and interobserver variability in US measurement of thyroid nodules, which must be taken into account during follow-up US examinations of patients with thyroid nodules., (© 2017 by the American Institute of Ultrasound in Medicine.)

Study Objectives: Inter-scorer variability in sleep staging of polysomnograms (PSGs) results primarily from difficulty in determining whether: (1) an electroencephalogram pattern of wakefulness spans > 15 sec in transitional epochs, (2) spindles or K complexes are present, and (3) duration of delta waves exceeds 6 sec in a 30-sec epoch. We hypothesized that providing digitally derived information about these variables to PSG scorers may reduce inter-scorer variability., Methods: Fifty-six PSGs were scored (five-stage) by two experienced technologists, (first manual, M1). Months later, the technologists edited their own scoring (second manual, M2). PSGs were then scored with an automatic system and the same two technologists and an additional experienced technologist edited them, epoch-by-epoch (Edited-Auto). This resulted in seven manual scores for each PSG. The two M2 scores were then independently modified using digitally obtained values for sleep depth and delta duration and digitally identified spindles and K complexes., Results: Percent agreement between scorers in M2 was 78.9 ± 9.0% before modification and 96.5 ± 2.6% after. Errors of this approach were defined as a change in a manual score to a stage that was not assigned by any scorer during the seven manual scoring sessions. Total errors averaged 7.1 ± 3.7% and 6.9 ± 3.8% of epochs for scorers 1 and 2, respectively, and there was excellent agreement between the modified score and the initial manual score of each technologist., Conclusions: Providing digitally obtained information about sleep depth, delta duration, spindles and K complexes during manual scoring can greatly reduce interrater variability in sleep staging by eliminating the guesswork in scoring epochs with equivocal features., (© 2016 American Academy of Sleep Medicine)

The diagnosis of microscopic colitis (MC) is based on histologic findings and includes collagenous colitis (CC) and lymphocytic colitis (LC). Incomplete MC (MCi) denotes patients with chronic diarrhea and a normal endoscopy and morphological changes that do not completely meet the histologic criteria of LC or CC. The aim of this study was to investigate the intraobserver and interobserver agreement on the MC subtypes of CC, LC, and MCi and the ability to discriminate MCi from normal and inflammatory bowel disease/nonspecific reactive changes. A single hematoxylin and eosin-stained specimen from biopsies of the following 5 groups were randomly selected and blinded: CC, LC, MCi, inflammatory bowel disease, and normal. Three pathologists independently reviewed the specimens. The specimens were relabeled and reinterpreted 4 months later. Intraobserver and interobserver agreement was evaluated by κ statistics. κ values for intraobserver agreement were good for 5 diagnostic groups varying from 0.70 to 0.83 and very good when simplifying to only 3 diagnostic groups varying from 0.88 to 0.96, separating MC/MCi from non-MC. κ values for interobserver agreement varied from 0.60 to 0.75 for 5 diagnostic groups and 0.81 to 0.89 for 3 diagnostic groups. The study shows that the intraobserver and interobserver agreement is high for discriminating between MC/MCi and non-MC, whereas the ability to discriminate MCi from CC and LC is lower. A revision and consensus on the histologic criteria of the MC subtypes seem warranted., (© 2013.)

BackgroundComputerized methodologies standardize the myocardial perfusion imaging (MPI) interpretation process.MethodsTo develop an automated relative perfusion quantitation approach for 18F-flurpiridaz, PET MPI studies from all phase III trial participants of 18F-flurpiridaz were divided into 3 groups. Count distributions were obtained in N = 40 normal patients undergoing pharmacological or exercise stress. Then, N = 90 additional studies were selected in a derivation group. Following receiver operating characteristic curve analysis, various standard deviations below the mean normal were used as cutoffs for significant CAD, and interobserver variability determined. Finally, diagnostic performance was compared between blinded visual readers and blinded derivations of automated relative quantitation in the remaining N = 548 validation patients.ResultsBoth approaches yielded comparable accuracies for the detection of global CAD, reaching 71% and 72% by visual reads, and 72% and 68% by automated relative quantitation, when using CAD ≥ 70% or ≥ 50% stenosis for significance, respectively. Similar results were observed when analyzing individual coronary territories. In both pharmacological and exercise stress, automated relative quantitation demonstrated significantly more interobserver agreement than visual reads.ConclusionsOur automated method of 18F-flurpiridaz relative perfusion analysis provides a quantitative, objective, and highly reproducible assessment of PET MPI in normal and CAD subjects undergoing either pharmacological or exercise stress.

BackgroundSynoptic reporting is recommended by many guideline committees to encourage the thorough histologic documentation necessary for optimal management of patients with melanoma.MethodsOne hundred fifty-one pathologists from 40 US states interpreted 41 invasive melanoma cases. For each synoptic reporting factor, the authors identified cases with "complete agreement" (all participants recorded the same value) versus any disagreement. Pairwise agreement was calculated for each case as the proportion of pairs of responses that agreed, where paired responses were generated by the comparison of each reviewer's response with all others.ResultsThere was complete agreement among all reviewers for 22 of the 41 cases (54%) on Breslow thickness dichotomized at 0.8 mm, with pairwise agreement ranging from 49% to 100% across the 41 cases. There was complete agreement for "no ulceration" in 24 of the 41 cases (59%), with pairwise agreement ranging from 42% to 100%. Tumor transected at base had complete agreement for 26 of the 41 cases (63%), with pairwise agreement ranging from 31% to 100%. Mitotic rate, categorized as 0/mm2 , 1/mm2 , or 2/mm2 , had complete agreement for 17 of the 41 cases (41%), with pairwise agreement ranging from 36% to 100%. Regression saw complete agreement for 14 of 41 cases (34%), with pairwise agreement ranging from 40% to 100%. Lymphovascular invasion, perineural invasion, and microscopic satellites were rarely reported as present. Respectively, these prognostic factors had complete agreement for 32 (78%), 37 (90%), and 18 (44%) of the 41 cases, and the ranges of pairwise agreement were 47% to 100%, 70% to 100%, and 53% to 100%, respectively.ConclusionsThese findings alert pathologists and clinicians to the problem of interobserver variability in recording critical prognostic factors.Lay summaryThis study addresses variability in the assessment and reporting of critical characteristics of invasive melanomas that are used by clinicians to guide patient care. The authors characterize the diagnostic variability among pathologists and their reporting methods in light of recently updated national guidelines. Results demonstrate considerable variability in the diagnostic reporting of melanoma with regard to the following: Breslow thickness, mitotic rate, ulceration, regression, and microscopic satellites. This work serves to alert pathologists and clinicians to the existence of variability in reporting these prognostic factors.

Background and purpose Volume delineation is a well-recognised potential source of error in radiotherapy. Whilst it is important to quantify the degree of interobserver variability (IOV) in volume delineation, the resulting impact on dosimetry and clinical outcomes is a more relevant endpoint. We performed a literature review of studies evaluating IOV in target volume and organ-at-risk (OAR) delineation in order to analyse these with respect to the metrics used, reporting of dosimetric consequences, and use of statistical tests. Methods and materials Medline and Pubmed databases were queried for relevant articles using keywords. We included studies published in English between 2000 and 2014 with more than two observers. Results 119 studies were identified covering all major tumour sites. CTV ( n = 47) and GTV ( n = 38) were most commonly contoured. Median number of participants and data sets were 7 (3–50) and 9 (1–132) respectively. There was considerable heterogeneity in the use of metrics and methods of analysis. Statistical analysis of results was reported in 68% ( n = 81) and dosimetric consequences in 21% ( n = 25) of studies. Conclusion There is a lack of consistency in conducting and reporting analyses from IOV studies. We suggest a framework to use for future studies evaluating IOV. [ABSTRACT FROM AUTHOR]