Search

Background: The grades of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in systematic reviews, health technology assessment and guideline development organisations throughout the world. We have previously reported on the development of the Semi-Automated Quality Assessment Tool (SAQAT), which enables a semi-automated validity assessment based on GRADE criteria. The main advantage to our approach is the potential to improve inter-rater agreement of GRADE assessments particularly when used by less experienced researchers, because such judgements can be complex and challenging to apply without training. This is the first study examining the inter-rater agreement of the SAQAT. Methods: We conducted two studies to compare: a) the inter-rater agreement of two researchers using the SAQAT independently on 28 meta-analyses and b) the inter-rater agreement between a researcher using the SAQAT (who had no experience of using GRADE) and an experienced member of the GRADE working group conducting a standard GRADE assessment on 15 meta-analyses. Results: There was substantial agreement between independent researchers using the Quality Assessment Tool for all domains (for example, overall GRADE rating: weighted kappa 0.79; 95% CI 0.65 to 0.93). Comparison between the SAQAT and a standard GRADE assessment suggested that inconsistency was parameterised too conservatively by the SAQAT. Therefore the tool was amended. Following amendment we found fair-to-moderate agreement between the standard GRADE assessment and the SAQAT (for example, overall GRADE rating: weighted kappa 0.35; 95% CI 0.09 to 0.87). Conclusions: Despite a need for further research, the SAQAT may aid consistent application of GRADE, particularly by less experienced researchers. [ABSTRACT FROM AUTHOR]

Background: The grades of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in systematic reviews, health technology assessment and guideline development organisations throughout the world. A key advantage to this approach is that it aids transparency regarding judgments on the quality of evidence. However, the intricacies of making judgments about research methodology and evidence make the GRADE system complex and challenging to apply without training. Methods: We have developed a semi-automated quality assessment tool (SAQAT) l based on GRADE. This is informed by responses by reviewers to checklist questions regarding characteristics that may lead to unreliability. These responses are then entered into the Bayesian network to ascertain the probabilities of risk of bias, inconsistency, indirectness, imprecision and publication bias conditional on review characteristics. The model then combines these probabilities to provide a probability for each of the GRADE overall quality categories. We tested the model using a range of plausible scenarios that guideline developers or review authors could encounter. Results: Overall, the model reproduced GRADE judgements for a range of scenarios. Potential advantages over standard assessment are use of explicit and consistent weightings for different review characteristics, forcing consideration of important but sometimes neglected characteristics and principled downgrading where small but important probabilities of downgrading are accrued across domains. Conclusions: Bayesian networks have considerable potential for use as tools to assess the validity of research evidence. The key strength of such networks lies in the provision of a statistically coherent method for combining probabilities across a complex framework based on both belief and evidence. In addition to providing tools for less experienced users to implement reliability assessment, the potential for sensitivity analyses and automation may be beneficial for application and the methodological development of reliability tools. [ABSTRACT FROM AUTHOR]

Background: Modifiable lifestyle risk behaviours such as smoking, unhealthy diet, physical inactivity and alcohol misuse are the leading causes of major, non-communicable diseases worldwide. It is increasingly being recognised that interventions which target more than one risk behaviour may be an effective and efficient way of improving people’s lifestyles. To date, there has been no attempt to summarise the global evidence base for interventions targeting multiple risk behaviours. Objective: To identify and map the characteristics of studies evaluating multiple risk behaviour change interventions targeted at adult populations in any country. Methods: Seven bibliographic databases were searched between January, 1990, and January/ May, 2013. Authors of protocols, conference abstracts, and other relevant articles were contacted. Study characteristics were extracted and inputted into Eppi-Reviewer 4. Results: In total, 220 studies were included in the scoping review. Most were randomised controlled trials (62%) conducted in the United States (49%), and targeted diet and physical activity (56%) in people from general populations (14%) or subgroups of general populations (45%). Very few studies had been conducted in the Middle East (2%), Africa (0.5%), or South America (0.5%). There was also a scarcity of studies conducted among young adults (1%), or racial and minority ethnic populations (4%) worldwide. Conclusions: Research is required to investigate the interrelationships of lifestyle risk behaviours in varying cultural contexts around the world. Cross-cultural development and evaluation of multiple risk behaviour change interventions is also needed, particularly in populations of young adults and racial and minority ethnic populations. [ABSTRACT FROM AUTHOR]