Search

Developing and evaluating the implementation of complex health interventions into routine health services in low and middle income countries is challenging. This presentation will explore how Theory of Change, a monitoring and evaluation approach, strengthened the design and evaluation of a complex health intervention (PRIME) which aimed to integrate mental health into primary health in districts or sub-districts in Ethiopia, India, Nepal, South Africa and Uganda. Erica will describe how PRIME used Theory of Change to co-develop district level mental healthcare plans with stakeholders to ensure local relevance within each implementation site while providing an overall framework for evaluation across sites. She will reflect on the strengths and weaknesses of this approach in the context of the evaluation of complex health interventions in global health.

Background: Many health interventions are complex and consist of several interacting components (Medical Research Council 2008). These components include multiple causal strands, outcomes and levels of governance and may result in unexpected outcomes and non-linear change (Glouberman and Zimmerman 2002). As such they present challenges to the design and evaluation of complex health interventions. Although broad theoretical guidance has been developed by the UK Medical Research Council (MRC) (Craig, Dieppe et al. 2008), it contains little practical guidance and has been criticised for not including theory driven approaches to evaluation such as Theory of Change (ToC) (Anderson 2008). De Silva, Breuer et al. (2014) have proposed that ToC may complement the MRC guidance on complex health interventions. Methods: This study explores how ToC can strengthen the design and evaluation of complex health interventions using the example of The Programme for Improving Mental Health Care (PRIME). PRIME is a research programme which aimed to develop, implement and evaluate the integration of mental health into primary health in districts or sub-districts in Ethiopia, India, Nepal, South Africa and Uganda. A ToC approach was used in addition to other approaches to design and evaluate these complex mental health interventions. Firstly, I conducted a systematic review to determine the extent to which ToC has been used to design and evaluate public health interventions. Secondly, I compared the process of developing the ToC between all five PRIME countries and reflected on the value of ToC workshops using a framework analysis of workshop documentation and interviews with facilitators. Thirdly, I explored the development of the ToCs within the programme as a whole and the implications for the development of the intervention and the choice of evaluation methods. Fourthly, I presented a ToC for the integration of mental health care in low and middle income countries. Fifthly, I demonstrated how ToC can be used as a framework for a qualitative comparative analysis of process and outcome data using longitudinal data from 10 PRIME implementation facilities in Nepal. Lastly, I provide a set of 10 key lessons learned from PRIME in the application of ToC to complex mental health interventions. Results I found that the ToC approach has been used for the design and evaluation of public health interventions since the 1990s. However, there is a lack of clear description of the use of ToC in the literature on public health interventions and inconsistency in how it is used. In applying the ToC approach to PRIME, I found that facilitators reported that ToC workshops were a valuable way to develop ToCs and that different stakeholders at the workshop contribute different types of information to the ToC process. Hierarchies within the health system are an important consideration for ToC workshops as power dynamics are likely to influence the functioning of the group. In addition, I found that the development of a cross country ToC can result in a programme theory which is relevant for complex multilevel intervention in different contexts. This ToC can provide a framework to map contextually relevant interventions and can be used to complement other intervention development approaches. The ToC can also be used to ensure indicators for all the short-, medium- and long-term outcomes are identified. However, combining process and outcome data analysis using the ToC is not straightforward. Qualitative Comparative Analysis (QCA) can be used to analyse process and outcome data in a single analysis in health services research. Conclusion: ToC can be used to strengthen the design and evaluation of complex health interventions and can be used to complement the MRC guidance in the design and evaluation of complex health interventions.

The use of Theory of Change to design, implement and evaluate Public Health Interventions: a systematic review protocol. (DOCX 202 kb)

Background The Medical Research Councils’ framework for complex interventions has been criticized for not including theory-driven approaches to evaluation. Although the framework does include broad guidance on the use of theory, it contains little practical guidance for implementers and there have been calls to develop a more comprehensive approach. A prospective, theory-driven process of intervention design and evaluation is required to develop complex healthcare interventions which are more likely to be effective, sustainable and scalable. Methods We propose a theory-driven approach to the design and evaluation of complex interventions by adapting and integrating a programmatic design and evaluation tool, Theory of Change (ToC), into the MRC framework for complex interventions. We provide a guide to what ToC is, how to construct one, and how to integrate its use into research projects seeking to design, implement and evaluate complex interventions using the MRC framework. We test this approach by using ToC within two randomized controlled trials and one non-randomized evaluation of complex interventions. Results Our application of ToC in three research projects has shown that ToC can strengthen key stages of the MRC framework. It can aid the development of interventions by providing a framework for enhanced stakeholder engagement and by explicitly designing an intervention that is embedded in the local context. For the feasibility and piloting stage, ToC enables the systematic identification of knowledge gaps to generate research questions that strengthen intervention design. ToC may improve the evaluation of interventions by providing a comprehensive set of indicators to evaluate all stages of the causal pathway through which an intervention achieves impact, combining evaluations of intervention effectiveness with detailed process evaluations into one theoretical framework. Conclusions Incorporating a ToC approach into the MRC framework holds promise for improving the design and evaluation of complex interventions, thereby increasing the likelihood that the intervention will be ultimately effective, sustainable and scalable. We urge researchers developing and evaluating complex interventions to consider using this approach, to evaluate its usefulness and to build an evidence base to further refine the methodology. Trial registration Clinical trials.gov: NCT02160249

Introduction: Depression in mothers can have debilitating consequences on the women themselves, their infants and their family. Thus, it is imperative to detect and treat perinatal depression early. Due to lack of awareness and stigma, help seeking, detection and treatment for perinatal depression in Nepal remains low. To counter barriers on lack of awareness, stigma and non-detection of mental health problems including depression, alcohol use disorder, psychosis and epilepsy, the PRogramme for Improving Mental Health carE (PRIME) developed and implemented a community sensitization programme and a Community Informant Detection Tool (CIDT). Neither of these programmes has focused on perinatal depression. This study aims to adapt the depression CIDT and the community sensitization programme to include perinatal depression. Methods: The CIDT and community sensitization programme were adapted using the following four steps. Firstly, a qualitative study was conducted with perinatal women with depressive symptoms visiting Meghauli and Dibyanagar health facilities or "gau-ghar clinic" (n=26) and service providers (n=34) to develop a culturally relevant content. Secondly, a draft CIDT and community sensitization manual were prepared based on the qualitative findings. Thirdly, a one-day workshop and several consultation meetings were held with mental health professionals (n=16) to ensure that the content was understandable and applicable to the context. Lastly, based on the workshop findings and consultation meetings, the manual and tools were modified and adapted for perinatal depression. Results: Due to poor awareness and stigma, none of our respondents had ever sought help for depression from the antenatal or postnatal service providers. Using local expressions for common depressive symptoms such as loss of interest, rumination, pessimistic views, extreme worries, restlessness, two separate CIDTs were developed each for antenatal and postnatal depression. Lack of support from the husband and family followed by poverty were the major contributing factors for depression. In addition, cultural factors such as the low position of women in patriarchal society and preference for son exacerbated problems in some women. The community sensitization manual was adapted to include local myths and facts about perinatal depression; causes with examples related to local beliefs; symptoms explained in local idioms; and role of the family. The heads of the families and key community members were recommended as key targets for the community sensitization programmes. Conclusion: It is important for any intervention to be responsive to local understanding and needs. The adapted CIDT and community sensitization manual has integrated the local issues and expressions of symptoms of perinatal depression for women in the Chitwan district.