Search

"The racial landscape in the United States is intricate and deeply entwined with historical legacies of systemic inequalities, particularly impacting African American communities. This study researches the evaluation practices within programs specifically designed for African American communities, employing Critical Race Theory (CRT) and Culturally Responsive Evaluation (CRE) frameworks. The study focuses on the backgrounds, experiences, and identities of program evaluators engaged in these communities, seeking to understand how race influences their practices. Using a generic qualitative research design, 15 semi-structured interviews were conducted to elucidate evaluators' perspectives on issues such as race, culture, equity, and inclusion throughout the evaluation process. The findings highlighted the crucial role of identity in evaluation practice, discussing challenges and effective strategies for managing identity dynamics. The analysis also provided insight into various aspects of evaluation practices within Black and/or African American communities, including cultural context, diverse evaluation methodologies, integration of social justice principles, and strategies for enhancing evaluation. This research paves the way for future studies exploring evaluators' operations within Black ecosystems, suggesting potential areas for inquiry like purposive sampling and focus groups to capture diverse perspectives and roles. Future studies could explore evaluator competencies and roles, refining evaluation practices within Black communities."--Abstract from author supplied metadata.

"As the evaluation discipline moves away from the "black-box" evaluations, theory-based evaluation approaches such as Contribution Analysis (CA) have gained popularity. This study responded to explicit requests to probe deeper into CA (e.g., Budhwani & McDavid, 2017; Delahais & Toulemonde, 2012; Dybdal et al., 2012; Lemire, 2010) as well as a more general call for a systematic examination of evaluation cases in order to generate "practical knowledge (Schwandt, 2008) about the Contribution Analysis (CA) evaluation approach (Mayne, 2012). As such, this dissertation employed a multiple-method design of two segments, Phase I consisted a systematic review of the conceptual literature, and the ensuing Phase II called upon a multiple case study of seven empirical CA cases. Taken together, the study design allowed for a systematic inquiry into the theoretical translation and practice of contribution analysis (CA). Specifically, the study investigated how Contribution Analysis is conceptualized by theorists, and how this is understood and translated into practice by evaluation practitioners. In this endeavor, the dissertation identified elements of effective practice by characterizing adaptations, adjustments, and innovations, and identified conditions under which practices may be different (e.g., evaluand, contexts, or evaluators) (Smith, 1993). The study pursued a richer understanding of the contexts of practice and to probe deeper into the translation of theory to practice using empirical evidence to further develop and improve on the current understandings of CA theory and practice. The findings from this study contribute to the empirical body on evaluation theory and practice, as well as methodological contribution on conducting research on evaluation. Moreover, findings from this study seek to inform the development of CA contingency theories, which identify conditions under which practices are effective, which are considered to be the strongest types of evaluation approaches as they are buttressed by empirical knowledge of practice (Shadish et al., 1991; Smith, 1993). The dissertation findings illustrate the complexity of the contexts in which CA evaluations take place, and identified contextual factors related to the program theory, sector of practice, geographic scope, temporal interval, and the effect-object. It seems that for certain contexts and purposes, CA by itself may not be enough as it not amenable for the comparison of causal packages across contexts nor cases. Therefore, approaches like Process Tracing (PT) or Bayesian modeling, or Qualitative Comparative Analysis (QCA) are used to facilitate direct comparisons of effects/influences of impact pathways from case to case. I argue however, that the understated value of Contribution Analysis is in its function as a validation approach, which creates a framework to build robust and sound arguments in support of contribution claims."--Abstract from author supplied metadata

Describes the process of creating evaluation tools for Oddfellows Playhouse Youth Theater, Inc., a non-profit theater arts serving young people between the ages of 6 and 20. The process involved using the tools of community psychology and applying them to a non-profit organization so that it could more successfully represent itself to funders. A large part of the evaluation process involved focusing the project to meet the needs of the organization within the scope of the evaluator's time and expertise. The Oddfellows requested a literature review of the term resiliency, which helped to focus the evaluation. Using the term resiliency as a focus, the evaluation team was able to create questions that Oddfellows would want answered in an interview process. The Oddfellows group ended the relationship with the ability to use the evaluation tools that were created. A discussion of what the evaluator and the organization learned from this process is presented.