Search

This article is the fifth in a series on the relationship of evaluation theory and practice. It was commissioned by the Eastern Evaluation Research Society and the American Evaluation Association as part of The Chelimsky Forum, launched in 2013 to honor Eleanor Chelimsky, one of the most influential and respected program evaluators of our era, and the former director of the Program Evaluation and Methodology Division of the U.S. Government Accountability Office. Supported by the Robert Wood Johnson Foundation, the forum is aimed at discussing challenges in the fit between theory and practice and strategies for addressing them. This article focuses on the relationships between evaluators and their users, especially policy makers. It tries to see things through the eyes of these users, recognizing their values and concepts related to evidence, proof, and "the truth" as they see it. It therefore tries to come to grips with the common core of evaluation thinking among a host of such clients, aiming to speak to them in their own language, but without sacrificing evaluator independence. It also recognizes the modern world of electronic reports and the possibility of both widespread and targeted dissemination of evaluation results. Finally, it offers some hints about evaluator development in this modern world, especially in the volatile world of policy and politics.

The primary focus of this article is ''advocating for the results of a specific evaluation.'' To some extent, all evaluators are advocates, however the author notes, "there is no great mystery in how to use evaluations for advocacy, especially with respect to influencing policy making within federal and state governments." Evaluators who do this for a living use the same techniques as lobbyists, marketing professionals, and journalists--areas where advocacy and persuasion are simply part of their work. In fact, their evaluations are read by these private sector practitioners who write about them in their own newsletters and journals. The article provides an illustrative overview of the role of evaluators, as well as the mechanics and nitty-gritty of procedures that are designed to be responsive to policy makers needs, but to also maintain both the reality and appearance of professionalism and independence. In the remainder of this article, the author focuses on broader issues: How can evaluators command the attention of policy makers? What kind of evaluations will policy makers use when considering policy questions? How close should evaluators get with policy makers?

In 1997, American Evaluation Association member George Grob, now retired from the U.S. Department of Health and Human Services (HHS) and currently President of the Center for Public Program Evaluation, made a testimony on Medicare home health care fraud and abuse before the U.S. Senate Special Committee on Aging. The occasion was to announce the HHS Office of Inspector General's (OIG) release of two reports. The two reports that OIG profiled in the congressional testimony were part of a "large body of work including audits, investigations, inspections, and congressional testimony." In this article, the authors profile the HHS home health care evaluations, the political and organizational contexts in which they occurred, their short- and long-term effects, and their implications for evaluators' understanding of evaluation practice and the use of evaluation findings. The authors also present a narrative description of the evaluations and their background, follow it with an amalgamation of extensive evaluator-editor dialogue in the format of a single interview, and conclude with reflections on the case.

The author enjoyed reading Katharine Hay's ambitious and humbling visions for evaluation field building in South Asia. She has successfully positioned herself on a high mountain with a wonderful set of binoculars that enable her to see the entire evaluation landscape of South Asia. She magically sees and describes significant historical forces and professional movements of the past, helping one to appreciate how the current landscape was fashioned. Her insights about where South Asia is now evaluation wise are very helpful, because the influences of centralized planning in what she calls the postcolonial period and the evaluation research of liberalization period are still at play. But it is her wishes and suggestions for the future that the author finds most helpful--emphasis on government use of evaluation, accountability and learning focused on citizens and the poor, and emphasis on multiple users and citizen engagement. In this article, the author focuses on these three themes and reflects on what insights he could add to an evaluation future in South Asia. He argues that experience is a better teacher, and he is totally lacking it with respect to evaluation in South Asia. What has worked in the United States is something that South Asia evaluators are welcome to read about. However, their own experience will be their true teacher.

This article is the fifth in a series on the relationship of evaluation theory and practice. It was commissioned by the Eastern Evaluation Research Society and the American Evaluation Association as part of The Chelimsky Forum, launched in 2013 to honor Eleanor Chelimsky, one of the most influential and respected program evaluators of our era, and the former director of the Program Evaluation and Methodology Division of the U.S. Government Accountability Office. Supported by the Robert Wood Johnson Foundation, the forum is aimed at discussing challenges in the fit between theory and practice and strategies for addressing them. This article focuses on the relationships between evaluators and their users, especially policy makers. It tries to see things through the eyes of these users, recognizing their values and concepts related to evidence, proof, and “the truth” as they see it. It therefore tries to come to grips with the common core of evaluation thinking among a host of such clients, aiming to speak to them in their own language, but without sacrificing evaluator independence. It also recognizes the modern world of electronic reports and the possibility of both widespread and targeted dissemination of evaluation results. Finally, it offers some hints about evaluator development in this modern world, especially in the volatile world of policy and politics. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editor discusses various reports within the issue on topics including ways of evaluation planning, the life cycle of an academic program and a framework for evaluation planning.