Search

Objectives: To evaluate and improve "Making Alternative Treatment Choices Intuitive and Trustworthy" (MATCH-IT)-a digital, interactive decision support tool displaying structured evidence summaries for multiple comparisons-to help physicians interpret and apply evidence from network meta-analysis (NMA) for their clinical decision-making., Study Design and Setting: We conducted a qualitative user testing study, applying principles from user-centered design in an iterative development process. We recruited a convenience sample of practicing physicians in Norway, Belgium, and Canada, and asked them to interpret structured evidence summaries for multiple comparisons-linked to clinical guideline recommendations-displayed in MATCH-IT. User testing included (a) introduction of a clinical scenario, (b) a think-aloud session with participant-tool interaction, and (c) a semistructured interview. We video recorded, transcribed, and analyzed user tests using directed content analysis. The results informed new updates in MATCH-IT., Results: Distributed across 5 development cycles we tested MATCH-IT with 26 physicians. Of these, 24 (94%) reported either no or sparse prior experience with interpretation of NMA. Physicians perceived MATCH-IT as easy to interpret and navigate, and appreciated its ability to provide an overview of the evidence. Visualization of effects in pictograms and inclusion of information on burden of treatment ("practical issues") were highlighted as potentially useful features in interacting with patients. We also identified problems, including undiscovered functionalities (drag and drop), suboptimal tutorial, and cumbersome navigation of the tool. In addition, physicians wanted definition/explanation of key terms (eg, outcomes and "certainty"), and there were concerns that overwhelming evidence from a large NMA would complicate applicability to clinical practice. This led to several updates with development of a new start page, tutorial, updated user interface for more efficient maneuvering, solutions to display definition of key terms and a "frequently asked questions" section. To facilitate interpretation of large networks, we improved categorization of results using color coding and added filtering functionality. These modifications allowed physicians to focus on interventions of interest and reduce information overload., Conclusion: This study provides proof of concept that physicians can use MATCH-IT to understand NMA evidence. Key features of MATCH-IT in a clinical context include providing an overview of the evidence, visualization of effects, and the display of information on burden of treatments. However, unfamiliarity with the Grading of Recommendations Assessment, Development and Evaluation concepts, time constraints, and accessibility at the point of care may be challenges for use. To what extent our results are transferable to real-world clinical contexts remains to be explored., Competing Interests: Declaration of competing interest T.A. is the Chair of the board and Deputy CEO of the nonprofit organization MAGIC Evidence Ecosystem Foundation (https://www.magicevidence.org). F.A. is a paid consultant for the MAGIC Evidence Ecosystem Foundation and is a part of the research team in MAGIC conducting the research project “Making Alternative Treatment Choices Intuitive and Trustworthy” (MATCH-IT). P.O.V is the CEO of the nonprofit organization MAGIC Evidence Ecosystem Foundation (https://www.magicevidence.org). P.O.L, B.S.H, and A.F.H are all part of the research team in MAGIC conducting the research project Making Alternative Treatment Choices Intuitive and Trustworthy (MATCH-IT). J.W.B. is partly supported by the Canadian Institutes of Health Research Canada Research Chair in the prevention and management of chronic pain. There are no competing interests for any other author., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

In this study, we identify existing interactive knowledge translation tools that could help patients and health-care professionals to prevent diabetes complications in the Canadian context. We conducted an environmental scan in collaboration with researchers and 4 patient partners across Canada. We conducted searches among the research team members, their networks and Twitter, and through searches in databases and Google. To be included, interactive knowledge translation tools had to meet the following criteria: used to prevent diabetes complications; used in a real-life setting; used any instructional method or material; had relevance in the Canadian context, written in English or French; developed and/or published by experts in diabetes complications or by a recognized organization; created in 2013 or after; and accessibility online or on paper. Two reviewers independently screened each record for selection and extracted the following data: authorship, objective(s), patients' characteristics, type of diabetes complications targeted, type of knowledge users targeted and tool characteristics. We used simple descriptive statistics to summarize our results. Thirty-one of the 1,700 potentially eligible interactive knowledge translation tools were included in the scan. Tool formats included personal notebook, interactive case study, risk assessment tool, clinical pathway, decision support tool, knowledge quiz and checklist. Diabetes complications targeted by the tools included foot-related neuropathy, cardiovascular diseases, mental disorders and distress and any complications related to diabetes and kidney disease. Our results inform Canadian stakeholders interested in the prevention of diabetes complications to avoid unnecessary duplication, identify gaps in knowledge and support implementation of these tools in clinical and patients' decision-making., (Copyright © 2020 Canadian Diabetes Association. Published by Elsevier Inc. All rights reserved.)