New Directions in Evaluation Research: Implications for Vocational Education. Occasional Paper No. 55.

In the early 1970s educational evaluation's fusion of respect for rationality, power to implement rational procedure, and an altruistic objective (improved instruction) appealed to educational researchers. Dealing essentially with closed systems, it measured program success by student performance on measurement instruments. Critics exclaimed that program developers and evaluators took away personal decision rights from students and that outcome measures were incomplete and inaccurate. Changes occured due to reanalysis showing the futility of earlier research, operating focus on procedures (not outcomes), open student participation, and loss of stability required for longitudinal study. In present-day open system evaluation settings, emphasis is on multiple objectives, and selection of what is to be evaluated is left open. No longer committed to methodology and provision of clear information, evaluators use case studies and looser, more interactive designs. Their reactions to political demands are to use needs assessments, develop systematic procedures, and write more reports. Problems between politics and evaluation include these: (1) the technical disagreement that any evaluation is subject to erodes its credibility with its contracting agency, (2) evaluation results can be used to discredit politicians whose claims outstrip their programs, and (3) success of political efforts gives politicians a sense of personal power. (YLB)