Search

Background: Uncertainty about the applicability of controlled trial findings is an increasing concern for clinicians and policy decision makers. This study aimed to determine whether information reported in studies included in systematic reviews was adequate enough to assess their applicability. Methods: We used the databases of four recently conducted systematic reviews on the comparative efficacy and safety of second-generation antidepressants, inhaled corticosteroids, Alzheimer's drugs, and targeted immune modulators. We developed and pilot-tested a questionnaire to assess the adequacy of reporting with respect to seven previously validated criteria of study design that distinguish explanatory from pragmatic studies. For each of the 137 included studies, two reviewers independently assessed the adequacy of reporting. Results: Overall, only 12 percent of the included studies provided sufficient information to reliably distinguish explanatory from pragmatic studies. The areas with the greatest lack of reporting were the setting of the study, methods of adverse event assessment, and sample size considerations to determine a minimally important difference from a patient perspective. Conclusions: Substantial shortcomings in reporting exist in aspects of study design important to determine whether a study is applicable to specific populations of interest. [ABSTRACT FROM AUTHOR]

Abstract: Objective: To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials. Study Design: Currently no validated definition of effectiveness studies exists. We asked the directors of 12 Evidence-based Practice Centers to select six studies each: four that they considered to be examples of effectiveness trials and two considered efficacy studies. We then applied our proposed criteria to test the construct validity using the selected studies as if they had been identified by a gold standard. Results: Based on the rationale to identify effectiveness studies reliably with minimal false positives (i.e., a high specificity), a cutoff of six criteria produced the most desirable balance between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of 0.72. Conclusion: When applied in a standardized manner, our proposed criteria can provide a valid and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic reviews can improve when analysts place more emphasis on the generalizability of included studies. In addition, clinicians can also use our criteria to determine the external validity of individual studies, given an appropriate population of interest. [Copyright &y& Elsevier]

Objectives: The objective of this study was to clarify a difference between two approaches while evaluating the diagnostic accuracy of medical tests, labeled here as "pragmatic" vs. "explanatory" studies., Methods: Using the definitions and characteristics described by Schwartz and Lellouch for randomized trials of interventions, and Schwartz' more general distinction between a pragmatic and an explanatory approach in medical research, we define a similar continuum for diagnostic accuracy studies. Explanatory studies aim to better understand the behavior of a test; pragmatic ones are done to support recommendations or decisions about using the test in clinical practice., Results: Pragmatic test accuracy studies differ from explanatory test accuracy studies in several ways. The difference in aims has implications for key elements of study design, such as the study eligibility criteria, the recruitment of patients, the reference standard, and the choice of the statistical analysis. Explanatory accuracy studies are often designed to test a hypothesis. They are typically selective in recruitment, may include "healthy controls," with a small sample size, often recruited at a single center. They ignore testing failures in the analysis and more often present their results as ROC curves. By contrast, pragmatic studies are designed to guide decision making. They ideally will recruit a single, large, and representative group of patients at multiple sites and will more often present their results as estimates of sensitivity and specificity or predictive values at a prespecified threshold., Conclusion: Distinguishing between a pragmatic and an explanatory approach can help in the design, analysis, and interpretation of diagnostic accuracy studies. It can clarify debates about the appropriateness of design features to the study purpose and about the validity and applicability of study findings., (Copyright © 2019 Elsevier Inc. All rights reserved.)