Search

Metrics derived from continuous glucose monitoring (CGM) systems are often discordant between systems. A major cause is that CGM systems are not standardized; they use various algorithms and calibration methods, leading to discordant CGM readings across systems. This discordance can be addressed by standardizing CGM performance assessments: If manufacturers aim their CGM systems at the same target, then CGM readings will align across systems. This standardization should include the comparator device, sample origin, and study procedures. With better aligned CGM readings, CGM-derived metrics will subsequently also align better between systems., Competing Interests: Declaration of Conflicting InterestsThe author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: SP and ME are employees of Institute for Diabetes Technology (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm; IfDT), Ulm, Germany.EEB has no disclosures.MF received lecture fees from Menarini.RH—Independent medical & scientific consultant, former employee of Roche Diabetes Care.JJ has been a lecturer/member of the scientific advisory boards at the following companies: Abbott, AstraZeneca, Boehringer Ingelheim, Eli Lilly, Medtronic, Nordic InfuCare, Novo Nordisk, and Sanofi.DCK is a consultant for Afon, Better Therapeutics, Integrity, Lifecare, Nevro, Novo, Samsung, and Thirdwayv. KM has nothing to disclose.JHN has received research support from Abbott.JP—Advisory panel for ROCHE Diabetes Care and Abbott; speaker fees from Insulet and Dexcom.ES is supported by grants from the US National Institutes of Health (NIH) and has received donated materials related to NIH-supported research from Abbott Diabetes Care, Roche Diagnostics, Siemens Diagnostics, Ortho Clinical Diagnostics, Abbott Diagnostics, Asahi Kasei Pharma Corp, GlycoMark Corp.RJS is chair of the Clinical Chemistry Department of Isala which carries out clinical studies, eg, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. RJS has received speaker’s honoraria or consulting fees in the last 3 years from Roche and Menarini.AT is a freelance consultant. He has received fees for lectures or consultancy fees from Abbott, Berlin Chemie, Dexcom, Evivamed, Menarini, Novo Nordisk, Roche and Sanofi in the last 3 years.NKT is a consultant for Roche Diagnostics and Roche Molecular Systems, received honoraria from Nova Biomedical and Thermo Fisher, and Chair (2024-2026), Critical and Point-of-Care Testing (CPOCT) Division, Association for Diagnostics and Laboratory Medicine (ADLM).LW has nothing to disclose.GF is general manager and medical director of the IfDT, which carries out clinical studies, eg, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IfDT have received speakers’ honoraria or consulting fees in the last 3 years Abbott, Berlin Chemie, Boydsense, Dexcom, Glucoset, i-SENS, Lilly, Menarini, Novo Nordisk, Perfood, Pharmasens, Roche, Sinocare, Terumo, Ypsomed.

Background: In analytical performance studies, the choice of comparator method plays an important role, as studies have shown that there exist relevant systematic differences (bias) between laboratory analyzers. The feasibility of retrospective recalibration of measurement results through comparison with methods or materials of higher metrological order to minimize bias was therefore assessed., Method: Existing data from performance studies of continuous and blood glucose monitoring systems were retrospectively analyzed. Comparison with a higher-order method was performed for two different data sets. In both cases, subject samples were measured, and a subset was also measured on a higher-order method. Recalibration based on higher-order materials (standard reference material [SRM]) was conducted for two different data sets containing results from SRM and subject samples. Linear regression analysis was performed for each device separately. Resulting equations were applied to the respective complete data set of subject samples. Bias between devices in a data set across all subject samples was assessed before and after recalibration., Results: Bias between devices was reduced from -3.6% to +0.6% in one data set and from +11.0% to +0.3% in the other by recalibration based on higher-order method. Using higher-order materials, bias was also reduced by recalibration, but mixed results were found: Bias was reduced from -3.1% to -0.1% in one data set and from -4.3% to -2.7% in the other., Conclusions: Recalibration did lead to a decrease in bias and thus can reduce the impact of the choice of comparator method. The procedure should be verified in a prospectively designed setting., Competing Interests: Declaration of Conflicting InterestsThe author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: G.F. is general manager and medical director of the IfDT (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies on the evaluation of BG meters, with CGM systems and medical devices for diabetes therapy on its own initiative and on behalf of various companies. G.F./IfDT have received speakers’ honoraria or consulting fees from Abbott, Ascensia, Berlin Chemie, Beurer, BOYDsense, CRF Health, Dexcom, i-SENS, Lilly, Metronom, MySugr, Novo Nordisk, Pharmasens, Roche, Sanofi, Sensile, Terumo, and Ypsomed. S.P., M.E., and C.H. are employees of the IfDT. T.G. is an employee of Diabetes Center Berne, Switzerland. E.E.B. is an employee of Equalis AB, Sweden, and also working in the Scandinavian Evaluation of Laboratory Equipment for Point-of-Care Testing (SKUP) group. K.M. is an employee of KAT General Hospital, Kifissia, Athens, Greece.

Comparing the performance of different continuous glucose monitoring (CGM) systems is challenging due to the lack of comprehensive guidelines for clinical study design. In particular, the absence of concise requirements for the distribution of comparator (reference) blood glucose (BG) concentrations and their rate of change (RoC) that are used to evaluate CGM performance, impairs comparability. For this article, several experts in the field of CGM performance testing have collaborated to propose characteristics of the distribution of comparator measurements that should be collected during CGM performance testing. Specifically, it is proposed that at least 7.5% of comparator BG concentrations are <70 mg/dL (3.9 mmol/L) and >300 mg/dL (16.7 mmol/L), respectively, and that at least 7.5% of BG-RoC combinations indicate fast BG changes with impending hypo- or hyperglycemia, respectively. These proposed characteristics of the comparator data can facilitate the harmonization of testing conditions across different studies and CGM systems and ensure that the most relevant scenarios representing real-life situations are established during performance testing. In addition, a study protocol and testing procedure for the manipulation of glucose levels are suggested that enable the collection of comparator data with these characteristics. This work is an important step toward establishing a future standard for the performance evaluation of CGM systems.

The use of different approaches for design and results presentation of studies for the clinical performance evaluation of continuous glucose monitoring (CGM) systems has long been recognized as a major challenge in comparing their results. However, a comprehensive characterization of the variability in study designs is currently unavailable. This article presents a scoping review of clinical CGM performance evaluations published between 2002 and 2022. Specifically, this review quantifies the prevalence of numerous options associated with various aspects of study design, including subject population, comparator (reference) method selection, testing procedures, and statistical accuracy evaluation. We found that there is a large variability in nearly all of those aspects and, in particular, in the characteristics of the comparator measurements. Furthermore, these characteristics as well as other crucial aspects of study design are often not reported in sufficient detail to allow an informed interpretation of study results. We therefore provide recommendations for reporting the general study design, CGM system use, comparator measurement approach, testing procedures, and data analysis/statistical performance evaluation. Additionally, this review aims to serve as a foundation for the development of a standardized CGM performance evaluation procedure, thereby supporting the goals and objectives of the Working Group on CGM established by the Scientific Division of the International Federation of Clinical Chemistry and Laboratory Medicine., Competing Interests: Declaration of Conflicting InterestsThe author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: G.F. is the general manager and medical director of the Institute for Diabetes Technology (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies, eg, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. G.F./IfDT have received research support, speakers’ honoraria, or consulting fees in the last three years from Abbott, Ascensia, Berlin Chemie, Boydsense, Dexcom, Lilly, Metronom, Medtronic, Menarini, MySugr, Novo Nordisk, PharmaSens, Roche, Sanofi, Terumo. M.E., D.W., S.P., S.W., and C.H. are employees of IfDT. R.S. is the chair of the Clinical Chemistry Department of Isala which carries out clinical studies, eg, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. R.S. has received speakers’ honoraria or consulting fees in the last three years from Roche and Menarini. J.J. has for the last three years been a lecturer/member of the scientific advisory board for Abbott, AstraZeneca, Boehringer Ingelheim, Eli Lilly, Medtronic, Nordic InfuCare, NovoNordisk A/S, and Sanofi. R.H. is an employee of Roche Diabetes Care GmbH. L.W. has received funding from the Diabetes Center Berne. E.E.B. and K.M. have no disclosures. D.C.K. is a consultant for Atropos Health, Better Therapeutics, Eoflow, Integrity, Lifecare, Nevro, Novo, Sanofi, and Thirdwayv. N.T. is a consultant for Roche Diagnostics and Radiometer. J.H.N. has received research support from Abbott. P.D. is a board member of PharmaSens. A.T. is a freelance consultant. He has received fees for lectures or consultancy fees from Abbott, Berlin Chemie, Dexcom, Evivamed, Menarini, Novo Nordisk, Roche, and Sanofi in the last three years.

Many decisions within healthcare are based on results from laboratory tests. These results all have an uncertainty, which can be either neglectable or large enough to affect the clinical assessment. In external quality assessment (EQA) the laboratories make independent observations on the same control sample, with values unknown for the participants. The results can be used to determine if quality improvement is needed. We present EQA data for laboratory tests with three different measurement scales; quantitative, ordinal and nominal. Results from EQA should be used in the discussions of quality in test results and necessary analytical performance specifications should be agreed upon.