Search

IntroductionThe first three years of a child’s life are the most critical to child development and have an impact on the future achievement of the child. Young children’s healthy development depends on nurturing care that ensures health, nutrition, responsive caregiving, safety, and security. Parents & other adult caregivers play a critical role in moderating children’s early experiences, which has a lasting impact be it positive or negative on the children’s future. Parenting education programs are proven to improve parental skills, capacity, and efficacy in a way that supports improved child development outcomes. Yet, most parents in low-middle-income countries such as Rwanda lack access to information and skills on how to support their children’s holistic development. In response, Save The Children implemented the First Steps “Intera za Mbere” holistic parenting education project in Rwanda from 2014 to 2021. This paper reflects on how monitoring, evaluation, accountability, and learning (MEAL) approaches were applied throughout the project cycle and their impact on program improvement and national policy and advocacy. This paper explores how the aspirations for measurement for change, considerations for innovation uptake and frameworks for learning about improvement are reflected in this project.MethodsThe project utilized qualitative and quantitative MEAL across the program cycle. Action research at the start of the project identified promoters and inhibitors of high-quality nurturing care and program delivery modalities. The project utilized a randomized control trial to provide insight into components that work better for parenting education. Evidence from surveys done remotely via phones was used to inform COVID-19 adaptations of the program.ResultsThe application of MEAL evidence led to the successful development and improvement of the program. At the policy level, evidence from the project influenced the review of the 2016 National Integrated ECD policy and the development of the national parenting education framework.ConclusionThe regular use of evidence from MEAL is critical for program improvement, scale-up, and policy influence.

In this review paper, we explore how on-the-ground Early Childhood Development (ECD) innovators are using monitoring, evaluation, and learning (MEL) systems to guide the design and implementation of ECD programs, as well as how MEL systems can influence policy and support the achievement of impact at scale. We reflect on articles in the Frontiers series “Effective delivery of integrated interventions in early childhood: innovations in evidence use, monitoring, evaluation, and learning.” The 31 contributions to the series reflect the breadth and depth of complexity that characterizes ECD, including global geographic spread, with studies from Asia, Europe, Africa, and Latin America and the Caribbean. Our synthesis finds that integrating MEL processes and systems into the fabric of a program or policy initiative can broaden the underlying value proposition. Specifically, ECD organizations sought to design their MEL systems to ensure programs fit the values, goals, experiences and conceptual frameworks of diverse stakeholders, so that participating makes sense to all. For example, formative, exploratory research identified the priorities and needs of the target population and frontline service providers, and informed the content and delivery of an intervention. ECD organizations also designed their MEL systems to support a shift of accountability toward broader ownership: They included delivery agents and program participants alike as subjects rather than objects, through active participation in data collection, and by providing opportunities for equitable discussion of results and decision-making. Programs collected data to respond to specialized characteristics, priorities and needs, embedding program activities into existing day-to-day routines. Further, papers pointed to the importance of intentionally involving a variety of stakeholders in national and international dialogues to ensure that diverse ECD data collection efforts are aligned and multiple perspectives are considered in the development of national ECD policies. And, several papers illustrate the value of creative methods and measurement tools to integrate MEL into a program or policy initiative. Finally, our synthesis concludes that these findings align with the five aspirations that were formulated as part of the Measurement for Change dialogue, which motivated the launch of the series.

Syphilis is one of the most common sexually transmitted infections (STIs) worldwide and has shown a rising trend in recent years, according to a report published by the World Health Organization (WHO) in 2021. Given this problem, the present study aims to develop a scoping review of what has been done in the world after the publication of the global strategy for the elimination of STIs, with a specific focus on syphilis. Thus, we searched for papers on health policies in response to syphilis in Pubmed, Scopus, ScienceDirect, and EBSCO by CINAHL, as well as in official documents from international health organizations. The period from January 1, 2016, to August 14, 2022 was considered. Our search returned 880 papers addressing “Syphilis,” “Health Policy,” and “Health Policies” combined. Twenty-three papers fulfilled the inclusion and exclusion criteria according to two research questions set out for this scoping review. Our findings suggest that Brazil and Peru presented the greatest adequacy of the strategies provided by WHO in 2016 and the Pan American Health Organization (PAHO) in 2017, aiming tothe goals set out in the UN's 2030 Agenda for sustainable development. Among the studies found, six countries (Cuba, Thailand, Belarus, Armenia, Moldova, and Puerto Rico) reported the elimination of mother-to-child transmission (MTCT) of syphilis, but the most recent data are from 2016. Furthermore, it is essential to mention that no country has been found that has presented a comprehensive response to syphilis, noting the control or elimination of the disease in all key populations. Thus, it is necessary to constantly monitor national policies based on in-depth studies on the quality of the response, the challenges, and the national, regional, and global perspectives for the control of the disease until 2030, the year in which the SDGs will be reviewed.Systematic review registrationhttps://osf.io/x9er5/?view_only=0cc0062222ec45dcb2f4d41484d285b6, identifier: 10.17605/OSF.IO/X9ER5.

BackgroundThere is wide acknowledgement in the literature that social connection is protective against loneliness and depression. More robust research, however, is needed to evaluate interventions that promote social connection. This protocol paper outlines the evaluation of a community-wide social connection program, Connect Local, in metropolitan Melbourne, Australia to support people 65 years and older to increase access to local community services/activities; and to ascertain impact on social connection, loneliness, depressive symptoms, physical and mental wellbeing, and use of health services.MethodsA Type 1 Hybrid design, including program effectiveness, cost-effectiveness, and implementation evaluation of the Connect Local program, will be undertaken. Eighty-eight participants aged ≥65 years with one or more chronic health condition, who are also either experiencing or at risk of loneliness, social isolation and depressive symptoms will be invited to participate in the evaluation. Outcomes, measured at baseline, 3, 6 and 12 months, include loneliness, social isolation, depressive symptoms, social anxiety, goal attainment, wellbeing, quality of life and health care utilisation. A gender and age matched comparator group of 88 individuals will be recruited from outside the intervention local government area. Impact of the intervention on community service providers in the target region will be evaluated using mixed methods, where triangulation will be used to combine the qualitative and quantitative data using a deductive-simultaneous design. Changes in wellbeing and quality of life of community volunteers will also be measured. All groups will be interviewed to ascertain their experience and perceptions of the program. The economic evaluation will use a Social Return on Investment (SROI) approach, to include outcomes at the individual, community, and system levels. Implementation outcomes will consider Reach, Adoption, Feasibility, Acceptability, Appropriateness, Fidelity, and Sustainability of the intervention.DiscussionThis study will provide a better understanding of the impacts of a community-wide social connection approach in older adults, the community and broader system.Clinical trial registrationhttps://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=385192; Identifier ACTRN12623000968673.

A lesson identified from the COVID-19 pandemic is that we need to extend existing best practice for intervention development. In particular, we need to integrate (a) state-of-the-art methods of rapidly coproducing public health interventions and messaging to support all population groups to protect themselves and their communities with (b) methods of rapidly evaluating co-produced interventions to determine which are acceptable and effective. This paper describes the Agile Co-production and Evaluation (ACE) framework, which is intended to provide a focus for investigating new ways of rapidly developing effective interventions and messaging by combining co-production methods with large-scale testing and/or real-world evaluation. We briefly review some of the participatory, qualitative and quantitative methods that could potentially be combined and propose a research agenda to further develop, refine and validate packages of methods in a variety of public health contexts to determine which combinations are feasible, cost-effective and achieve the goal of improving health and reducing health inequalities.

BackgroundThe US Department of Veterans Affairs (VA) has over 15 years of experience in delivery of evidence-based psychotherapies (EBPs). This paper describes strategies for using clinical documentation and administrative data to understand adherence and modifications to EBPs for Posttraumatic Stress Disorder (PTSD).MethodsThis study focused on two EBPs for PTSD, Cognitive Processing Therapy (CPT) and Prolonged Exposure (PE). The sample included VA therapists from across the US who provided CPT and PE and the patients they treated over a 1-year period. The data sources for this study were templated EBP chart notes and VA administrative data. We used a manual review of note content and administrative data rules to code therapy adherence and modifications in 7,297 EBP sessions for 1,257 patients seen by 182 therapists. Two trained coders rated each therapy note and resolved discrepancies through consensus. To contextualize and explain variation in adherence and modifications, we conducted brief 30–45-min semi-structured interviews with a purposive subsample of these therapists (n = 32).FindingsCombining manual chart review and administrative data allowed for identification of 11 types of modifications. Raters disagreed on adherence for 30% of notes. The disagreement stemmed from the presence of therapy modifications that were not clearly documented, necessitating the development of decision rules and strategies for modification coding. Both therapists and patients contributed to the variance in the extent to which different modifications occurred. Therapist interviews demonstrated therapist awareness of modifying the protocols in the ways identified through chart review.ConclusionHealthcare systems can use data collected as part of routine care to understand how and when EBPs are modified but need to develop scalable strategies to document adaptations and modifications to EBPs in routine care.

The document, National Medium and Long Term Plan for Actively Responding to the Aging of Population points out that in order to actively respond to the aging population in China, it is necessary to steadily increase the endowment wealth reserves. To achieve this goal, it is urgent to stochastically assess the future financial situation of the basic pension insurance in China, grasp its various possible conditions, trends, and corresponding confidence intervals, so that the government can take targeted measures to gradually consolidate the wealth reserves for this basic insurance first, and then steadily increase the social wealth reserves for the elderly. Thus, this paper first analyzes the characteristics of the aging population of insureds participating in basic pension insurance, and then randomly simulates the long-term financial situation of the basic pension insurance. The study found that the aging of the insureds has the characteristics of “fierce coming and slow decline” and “long-term seriousness.” Among the six indicators of the financial situation of basic pension insurance, Indicator 1 (the current year's expenditures as a proportion of current year's contributions), Indicator 2 (current year's balance of contributions and expenditures), Indicator 3 (current year's payment gap as a proportion of current year's contributions), Indicator 4 (accumulated balance), Indicator 5 (fund ratio), and Indicator 6 (accumulated payment gap as a proportion of current year's contributions) are respectively in the range of [0.73%, 1.80%], [−12.05, −0.12] trillion yuan, [0.29%, 3.89%], [−133.39, −5.62] trillion yuan, [2032, 2043] years and [6.72%, 215.63%] with a probability of 95%. We analyzed the influence direction and degree of main parameters on the financial situation of the fund and analyzed the impact of parameter value paths on the final financial status of the fund to improve the ability to strengthen fund reserve. The backtracking found that if the value path of the average salary growth rate shows a trend of rising first and then falling, then the final financial situation at the end of the period will be “worse.” If it shows a trend of falling first and then rising, the final financial situation will be “better.”

Evaluating the success of a public health campaign is critical. It helps policy makers to improve prevention strategies and close existing gaps. For instance, Brazil's “Syphilis No!” campaign reached many people, but how do we analyze its real impact on population awareness? Are epidemiologic variables sufficient? This study examined literature on using of information technology approaches to analyze the impact of public health campaigns. We began the systematic review with 276 papers and narrowed it down to 17, which analyzed campaigns. In addition to epidemiological variables, other types of variables of interest included: level of (i) access to the campaign website, (ii) subject knowledge and awareness, based on questionnaires, (iii) target population's interest, measured from both online search engine and engagement with Social Network Service, and (iv) campaign exposure through advertising, using data from television commercials. Furthermore, we evaluated the impact by considering several dimensions such as: communication, epidemiology, and policy enforcement. Our findings provide researchers with an overview of various dimensions, and variables-of-interest, for measuring public campaign impact, and examples of how and which campaigns have used them.

Tackling complex public health challenges requires integrated approaches to health, such as One Health (OH). A key element of these approaches is the integration of knowledge across sectors, disciplines and stakeholders. It is not yet clear which elements of knowledge integration need endorsement to achieve best outcomes. This paper assesses 15 OH initiatives in 16 African, Asian and European countries to identify opportunities to improve knowledge integration and to investigate geographic influences on knowledge integration capacities. Two related evaluation tools, both relying on semi-quantitative questionnaires, were applied to two sets of case studies. In one tool, the questions relate to operations and infrastructure, while the other assigns questions to the three phases of “design,” “implementation,” and “evaluation” of the project life cycle. In both, the question scores are aggregated using medians. For analysis, extreme values were identified to highlight strengths and weaknesses. Seven initiatives were assessed by a single evaluator external to the initiative, and the other eight initiatives were jointly assessed by several internal and external evaluators. The knowledge integration capacity was greatest during the project implementation stage, and lowest during the evaluation stage. The main weaknesses pointing towards concrete potential for improvement were identified to be a lack of consideration of systemic characteristics, missing engagement of external stakeholders and poor bridging of knowledge, amplified by the absence of opportunities to learn and evolve in a collective process. Most users were unfamiliar with the systems approach to evaluation and found the use of the tools challenging, but they appreciated the new perspective and saw benefits in the ensuing reflections. We conclude that systems thinking and associated practises for OH require not only specific education in OH core competencies, but also methodological and institutional measures to endorse broad participation. To facilitate meta-analyses and generic improvement of integrated approaches to health we suggest including knowledge integration processes as elements to report according to the COHERE guidelines.

Background: Vaccination coverages need to be constantly maintained and improved with the implementation of vaccination strategies. This paper describes the development of an evidence-based tool to guide their planning and evaluation.Methods: A scoping review was performed in MEDLINE and institutional websites to search for similar available tools. A first version of the tool was developed considering review results and a four-step method used for the control and continuous improvement of processes and products, namely the Deming cycle. A panel of eight experts was then involved in a Delphi study for the finalization of the tool that was eventually discussed in a face-to-face meeting.Results: The scoping review found only one document and the first version of the tool was composed of 30 items. After the Delphi first round, 11 additional items were suggested and 5 original items amended. After the Delphi second round 41 items were eventually included. During the face-to-face meeting, 7 items were recognized as requisites for setting vaccination strategies, whereas 17 as relevant ones.Conclusions: Current public health challenges impose the need for evidence-based tools to organize effective vaccination strategies. Our tool is a first proposal which aims to reflect this focus.

Introduction: Rabies is endemic in Vietnam and has been a statutory notifiable infectious disease since 1998. We, herein, assessed the performance of rabies surveillance in Southern Vietnam and identified areas for improvement.Materials and Methods: We analyzed data on human rabies cases reported during 1991–2018. We adapted guidelines from the U.S. Centers for Disease Control and Prevention to evaluate attributes of surveillance. Between June and November 2018, we interviewed a total of 145 staff from hospitals, preventive medicine centers, and animal health offices at provincial and district levels in five southern provinces.Results: Between 2009 and 2018, an average of nine cases of human rabies (range: 4–20 cases) was reported annually in Southern Vietnam, representing an incidence of 2.7 cases per 10 million population. The highest incidence was observed in 2018 (5.5 cases per 10 million population). Survey data suggested that only 24% (13/53) of participants agreed that the monthly report template was easy to complete and that 42% (23/55) indicated that the change from the paper-based to the electronic case notification systems was easy. Only 7% (2/29) of human rabies cases were reported timely, and 65% (13/20) successfully collected specimens. Approximately 39% (56/144) of staff were aware of turning surveillance data into prevention activities, and 21% (31/145) witnessed data used for strategic program decision making.Conclusions: Although rabies surveillance was quite simple, flexible, and accepted in southern Vietnam, simplifying the report forms, training staff, and improving the timeliness of reporting and data usage are highly recommended for a better implementation of rabies surveillance.

The “wecoach” is a web-application that builds the capacities of team leaders to improve working conditions that are positively related to the psychological health and well-being of their team members. The web-application works through an automated, rule-based chat enhanced by machine learning. This so-called conversational agent guides the team leader through a systematic project cycle, providing a mind map of work and health, training materials, self-assessments, and online tools to conduct team surveys and workshops, as well as self-evaluation of progress and effectiveness. In this paper, we present the development process of this web-application, which resulted in (1) a comprehensive intervention approach, (2) the prototype, and (3) the implementation of an evaluation design for a multi-level, randomized controlled trial.

This Perspective presents the five key aspirations of an approach to data use, decision making and monitoring, evaluation, and learning (MEL) in Early Childhood Development (ECD) referred to as Measurement for Change. The core ideas of Measurement for Change gave rise to this series, and many of the papers submitted in this series speak to this approach, whether directly or indirectly. The five aspirations describe interconnected concepts that advocate for practitioners and researchers within ECD to build the capacity to use data in their decision making, by establishing a monitoring, evaluation, and learning system that strives to be: Dynamic; Inclusive; Informative; Interactive; and People-centered.

Introduction: Health promotion delivery systems are increasingly being asked to implement policy, systems, and environmental interventions (PSEs). However, evaluating PSEs is challenging, especially in low-resource community settings. This paper describes the use of RE-AIM to evaluate a physical activity PSE delivered through University of Wyoming Extension and highlights challenges and opportunities in pragmatic, real-world program evaluation.Methods: Extension health educators adapted a point-of-decision prompt intervention encouraging stairway use through posters, called Take the Stairs, Wyoming! Reach was assessed through estimates of daily traffic, effectiveness was assessed through opportunistic interviews, adoption was calculated as the number and proportion of sites that agreed to hang posters, implementation was calculated as the proportion of sites with a poster in place at a 2-weeks follow-up visit, and maintenance was assessed through 6-months opportunistic interviews (individual level) and proportion of sites with a poster in place (organizational level).Results: Overall, the posters were widely adopted and most posters were implemented as intended. However, capturing reach, effectiveness, and maintenance was challenging, as health educators found the evaluation burdensome. Therefore, it was difficult to determine if the posters were effective at increasing physical activity levels.Discussion: Suggestions are provided for capturing reach, effectiveness, and maintenance data in community settings. Future efforts are needed to create evaluation tools to pragmatically measure effectiveness of PSEs on changing behaviors, as well as to prioritize program evaluation in Extension.

The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand “how” and “why” results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.

BackgroundCommunity-based processes addressing environment, housing, and health issues may decrease health inequities by addressing social, economic, and political health determinants more effectively. Yet little analysis of their effectiveness has been undertaken and their potential transfer to marginalized groups in rich country settings. In Europe, stark health inequalities are seen among the Roma, Europe’s most impoverished community who often reside in informal settlements suffering from illiteracy, inadequate housing, and lack of water and sanitation. This paper assesses a dry sanitation project in a Roma informal settlement in the Paris region to improve their living conditions.MethodsBetween 2014 and 2017, multiple stakeholders were involved in a participatory process of design, construction, and maintenance of toilets. Interviews, mapping, model construction, and facilitated discussion were used to identify design features and follow-up indicators. Field notes, videos, questionnaires, and observation provided data for monitoring and evaluation. For questionnaires delivered to women in the community, a cross section time series was conducted to due to migration.ResultsDespite issues related to maintenance, the overall quality of life of women improved after toilet construction. This included indicators for comfort, cleanliness, practicality, privacy, security, and menstrual hygiene management. Furthermore, fewer women restrained themselves from relieving themselves or from drinking less water to avoid urinating. Odors continued to be an issue. Self-reporting of illnesses, such as diarrhea and urinary tract infections, were not reliable due to the vague description of these illnesses and the potential recall bias. Appropriate sanitation in informal settlements is a necessity as shown by feedback from Roma women and the literature. However, a more sustainable toilet project would have required an adequate budget, good quality materials, accountability, and buy in from all inhabitants as well as good relationships between the chief and all stakeholders in the settlement. Furthermore, provision should have been made for vulnerable and less autonomous inhabitants to ensure proper follow-up and maintenance.Conclusion/recommendationsSafe water and sanitation is essential to decrease health inequities. Formative research, identifying natural leaders, sanitation committee formation, outcome indicator and baseline questionnaire development, appropriate messages and toilet design and placement, long-term follow-up, and political support are elements for effective interventions with the Roma. Potential attrition and reporting bias should be considered in monitoring and evaluation.

This paper developed an effective model for improving global health nursing competence among undergraduate students. A descriptive case study was conducted by implementing four programs. All programs were conducted with students majoring nursing and healthcare, where the researcher was a program director, professor, or facilitator. These programs were analyzed in terms of students’ needs assessment, program design, and implementation and evaluation factors. The concept and composition of global nursing competence, identified within previous studies, were deemed appropriate in all of our programs. Program composition varied from curricular to extracurricular domains. During the implementation phase, most of the programs included non-Korean students to improve cultural diversity and overcome language barriers. Qualitative and quantitative surveys were conducted to assess program efficacy. Data triangulation from students’ reflective journals was examined. Additionally, students’ awareness regarding changes within global health nursing, improved critical thinking, cultural understanding, and global leadership skills were investigated pre and post-program implementation. We discuss how identifying students’ needs regarding global nursing competence when developing appropriate curricula.