Search

Introduction: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhancements made to the MMAT. Development: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. Tool Description: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. Conclusion: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.

Abstract Background Participatory Research (PR) entails the co-governance of research by academic researchers and end-users. End-users are those who are affected by issues under study (e.g., community groups or populations affected by illness), or those positioned to act on the knowledge generated by research (e.g., clinicians, community leaders, health managers, patients, and policy makers). Systematic reviews assessing the generalizable benefits of PR must address: the diversity of research topics, methods, and intervention designs that involve a PR approach; varying degrees of end-user involvement in research co-governance, both within and between projects; and the complexity of outcomes arising from long-term partnerships. Methods We addressed the above mentioned challenges by adapting realist review methodology to PR assessment, specifically by developing inductively-driven identification, selection, appraisal, and synthesis procedures. This approach allowed us to address the non-uniformity and complexity of the PR literature. Each stage of the review involved two independent reviewers and followed a reproducible, systematic coding and retention procedure. Retained studies were completed participatory health interventions, demonstrated high levels of participation by non-academic stakeholders (i.e., excluding studies in which end-users were not involved in co-governing throughout the stages of research) and contained detailed descriptions of the participatory process and context. Retained sets are being mapped and analyzed using realist review methods. Results The librarian-guided search string yielded 7,167 citations. A total of 594 citations were retained after the identification process. Eighty-three papers remained after selection. Principle Investigators (PIs) were contacted to solicit all companion papers. Twenty-three sets of papers (23 PR studies), comprising 276 publications, passed appraisal and are being synthesized using realist review methods. Discussion The systematic and stage-based procedure addressed challenges to PR assessment and generated our robust understanding of complex and heterogeneous PR practices. To date, realist reviews have focussed on evaluations of relatively uniform interventions. In contrast our PR search yielded a wide diversity of partnerships and research topics. We therefore developed tools to achieve conceptual clarity on the PR field, as a beneficial precursor to our theoretically-driven synthesis using realist methods. Findings from the ongoing review will be provided in forthcoming publications.

Little research has assessed the fidelity, adaptation or integrity of activities implemented within community-based obesity prevention initiatives. To address this gap, a mixed-method process evaluation was undertaken in the context of the South Australian Obesity Prevention and Lifestyle (OPAL) initiative. An ecological coding procedure assessed fidelity and adaptation of activity settings, targets and strategies implemented in the second year of four communities. Implementation integrity reflected fidelity and adaptation to local context, whereas efforts resulting in significant deviations from the original plan were deemed to lack fidelity and integrity. Staff implemented 284 strategies in 205 projects. Results show that 68.3 and 2.1% of strategies were implemented with fidelity or adapted, respectively. Overall, 70.4% of all strategies were implemented with integrity. Staff experienced barriers with 29.6% of strategies. Chi-square analyses show statistically significant associations between implementation integrity and strategy type, intervention and behavioural targets. These relationships are weak to modest. The strongest relationship was found between implementation integrity and proximal target. Staff experienced implementation barriers at the coalition, policy, organization, interpersonal and community levels. The greatest range of barriers was encountered working with organizations. To overcome these barriers, staff took greater ownership, invested more time, persisted and allocated more financial resources.

Aboriginal Australians, including Aboriginal Health Workers (AHWs), smoke at rates double the non-Aboriginal population. This study utilized concept mapping methodology to identify and prioritize culturally relevant strategies to promote smoking cessation in AHWs. Stakeholder participants included AHWs, other health service employees and tobacco control personnel. Smoking cessation strategies (n = 74) were brainstormed using 34 interviews, 3 focus groups and a stakeholder workshop. Stakeholders sorted strategies into meaningful groups and rated them on perceived importance and feasibility. A concept map was developed using multi-dimensional scaling and hierarchical cluster analyses. Ten unique clusters of smoking cessation strategies were depicted that targeted individuals, family and peers, community, workplace and public policy. Smoking cessation resources and services were represented in addition to broader strategies addressing social and environmental stressors that perpetuate smoking and make quitting difficult. The perceived importance and feasibility of clusters were rated differently by participants working in health services that were government-coordinated compared with community-controlled. For health service workers within vulnerable populations, these findings clearly implicate a need for contextualized strategies that mitigate social and environmental stressors in addition to conventional strategies for tobacco control. The concept map is being applied in knowledge translation to guide development of smoking cessation programs for AHWs.

This study reports on the development and evaluation of a rating tool to assess the scientific utility and cultural appropriateness of community-level indicators for application with Indigenous populations. Indicator criteria proposed by the U.S. Institute of Medicine were culturally adapted through reviewing the literature and consultations with academic and Indigenous stakeholders. Pre-testing and collaborator feedback drove the iterative development of the tool with stakeholder groups in Canada, Aotearoa/New Zealand, and Australia. Pilot testing with 17 raters across countries involved rating the same selection of six health and social indicators using a six-point ordinal scale. The final version of the rating tool includes 16 questions within three criterion domains: "importance," "soundness," and "viability." Academic and community stakeholder review established face and content validity. The indicator rating tool demonstrated good internal consistency and excellent inter-rater reliability for two of three pilot testing groups. Use of this instrument can strengthen collaborative research planning and evaluation with Indigenous communities through selection of relevant and culturally appropriate indicators for application to public health research, prevention programmes, and health and social policy.

Policy implementation reflects a complex change process where government decisions are transformed into programs, procedures, regulations, or practices aimed at social betterment. Three factors affecting contemporary implementation processes are explored: networked governance, sociopolitical context and the democratic turn, and new public management. This frame of reference invites evaluators to consider challenges present when evaluating macrolevel change processes, such as the inherent complexity of health and social problems, multiple actors with variable degrees of power and influence, and a political environment that emphasizes accountability. The evaluator requires a deep and cogent understanding of the health or social issues involved; strong analysis and facilitation skills to deal with a multiplicity of values, interests, and agendas; and a comprehensive toolbox of evaluation approaches and methods, including network analysis to assess and track the interconnectedness of key champions (and saboteurs) who might affect intervention effects and sustainability.

Democratic or equal participation in decision making is an ideal that community and academic stakeholders engaged in participatory research strive to achieve. This ideal, however, may compete with indigenous peoples' right to self-determination. Study objectives were to assess the perceived influence of multiple community (indigenous) and academic stakeholders engaged in the Kahnawake Schools Diabetes Prevention Project (KSDPP) across six domains of project decision making and to test the hypothesis that KSDPP would be directed by community stakeholders. Self-report surveys were completed by 51 stakeholders comprising the KSDPP Community Advisory Board (CAB), KSDPP staff, academic researchers and supervisory board members. KSDPP staff were perceived to share similar levels of influence with (i) CAB on maintaining partnership ethics and CAB activities and (ii) academic researchers on research and dissemination activities. KSDPP staff were perceived to carry significantly more influence than other stakeholders on decisions related to annual activities, program operations and intervention activities. CAB and staff were the perceived owners of KSDPP. The strong community leadership aligns KSDPP with a model of community-directed research and suggests that equitable participation--distinct from democratic or equal participation--is reflected by indigenous community partners exerting greater influence than academic partners in decision making. (Contains 4 tables and 1 figure.)

Health and social indicators that capture the distinct historical, social, and cultural contexts of Indigenous communities can play an important role in informing the planning and delivery of community interventions. There is currently considerable interest in cataloguing and vetting meaningful community-level health and social indicators that could be applied to research and health promotion activities in Indigenous communities in Australia, Canada, and New Zealand, inclusive of conventional indicators as well as measures developed specifically for use in or with Indigenous communities. To avoid haphazard selection of indicators, and to assure the comprehensiveness and relevance of any given set of indicators, a framework that can accommodate and conceptually classify indicators representing a full range of domains is required. We report here on the development of a conceptual framework, by which Indigenous community indicators, and more general community-level social indicators, can be sorted, catalogued, and systematically classified within four hierarchical levels. The indicator framework was developed across Canada, Australia and New Zealand in consultation with academic researchers and Indigenous community stakeholders, building from established health and social indicator systems. The Indigenous indicator framework permits Indigenous communities, public health researchers, and funding agencies to compare and select the most appropriate indicators for application in specific contexts from the multitude of existing indicators.

Purpose: Although implementation fidelity is an important component in the evaluation of school health promotion programs, it assumes that teaching is the most relevant teacher role. To understand the social context of program implementation, a qualitative study was undertaken with the aim of identifying the schoolteacher's role in implementing the objectives of the Kahnawake Schools Diabetes Prevention Project (KSDPP), a locally governed Kanienke:haka (Mohawk) community-based diabetes prevention program. Design/methodology/approach: Prospective semi-structured interviews were conducted cross-sectionally with 30 teachers, two administrators and one physical education teacher across four intervention years. Interviews were analysed retrospectively using qualitative thematic analysis. Findings: In implementing KSDPP objectives teachers adopted, to varying degrees, the roles of teaching the health education curriculum, enforcing the school nutrition policy, role modelling healthy lifestyles, and encouraging healthy lifestyles. Taken together, these roles point to a high-order role of teachers taking responsibility for enabling healthy lifestyles in their children, which is congruent with a wholistic approach to health. Study findings suggest that children in different classrooms were exposed to a different intervention dose based on the extent to which teachers applied each role. Research limitations/implications: The findings suggest that the current conceptualisation of implementation fidelity should be expanded to account for the influence of the social context (i.e. teachers' roles) on the implementation of health promotion program objectives. Originality/value: Consistent with an ecological approach to intervention, teachers would benefit from interventions that predispose, enable, and reinforce their capacity to adopt and apply health promotion roles. (Contains 1 figure and 1 table.)

Assessed how the youth empowerment process encouraged positive youth development. Observation, interview, and focus group data from an urban community health promotion project indicated that practitioners created environments conducive to adolescents taking responsibility for their quality of life by welcoming and enabling youth. Power was transferred to youth as responsibility for voicing, decision making, and action, resulting in positive changes in youth development and their social integration. (SM)