Search

The Standards for Educational and Psychological Testing recommend an argument-based approach to validation that involves a clear statement of the intended interpretation and use of test scores, the identification of the underlying assumptions and inferences in that statement—termed the interpretation/use argument, and gathering of evidence to support or refute the assumptions and inferences. We present two approaches to articulating the interpretation/use argument. One approach uses the five sources of validity evidence in the Standards for Educational and Psychological Testing as a framework and the other approach uses Kane’s chain of assumptions/inferences as a framework. Through this process we identified aspects of these approaches that need to be further clarified for instrument developers to consistently implement either approach, identified important differences in the perspective each approach takes on validation, and highlight important questions for the measurement and mathematics educat...

The Standards for Educational and Psychological Testing indicate that multiple sources of validity evidence should be used to support the interpretation of test scores. In the past decade, examinee response processes, as a source of validity evidence, have received increased attention. However, there have been relatively few methodological studies of the accuracy and consistency of examinee response processes as measured by verbal reports in the context of educational measurement. The objective of the current study was to investigate the accuracy and consistency of examinee response processes—as measured by verbal reports—as a function of varying interviewer and item variables in a think aloud interview within an educational measurement context. Results indicate that the accuracy of responses may be undermined when students perceive the interviewer to be an expert in the domain. Further, the consistency of response processes may be undermined when items that are too easy or difficult are used to elicit re...

The Standards for Educational and Psychological Testing indicate that test instructions, and by extension item objectives, presented to examinees should be sufficiently clear and detailed to help ensure that they respond as developers intend them to respond (Standard 3.20; AERA, APA, & NCME, 1999). The present study investigates the use of verbal reports, one of many sources of evidence for validity arguments, as a way to evaluate the content clarity of 30 items from a large-scale science assessment. Student reports were used to edit items and create a student-modified test form. Evaluations from expert preservice teachers were used to edit the items and create an expert-modified test form. Both experimental forms, along with the original set of 30 items, were then randomly assigned to a sample of 264 examinees. Hierarchical regression analyses indicated that examinee performance on the student-modified and expert-modified forms was similar relative to performance on the original test items. Item statisti...

The Texas Education Agency (TEA) relied heavily on the test development process used for the Texas Assessment of Academic Skills (TAAS) in its defense presented during the GI Forum v. Texas Education Agency (1999) litigation. In this article, we present the Texas test development process. The process spans 3 years of assessment development that includes the 2 general areas of planning and maintenance activities. Each step of both the planning and maintenance stages is described in detail. In addition, each step of the process is either explicitly or implicitly linked to the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1985). The routine use of this process by TEA and its contractors (as well as the process itself) provided strong evidence of both content and curricular validity of the TAAS.

Environmental regulation, like educational policy, crucially depends on the establishment and enforcement of standards. One can observe numerous similarities in the interplay of social, political, and technical issues in educational and environmental standard setting. In this article, I review several major types of environmental standards (design, performance, exposure, safety, and behavioral) and discuss their points of contact with educational standards. In this article, I highlight areas of judgment common to both standard-setting processes and describe the principal mechanisms that are used to improve the credibility of environmental standards. In conclusion, I suggest ways in which experiences gained in the environmental arena could usefully be extended to the educational arena.

A great deal of measurement research is currently being utilized to improve instruction at the elementary and secondary levels, but many opportunities for improving utilization are still untapped. The author proposes specific approaches to increased collaboration between researchers and practitioners to meet the burgeoning research needs of the nations' schools and to assist in data-based decision-making efforts. Society's increasing demands for accurate, usable data have contributed to the urgent current need for viable, appropriate research. This need can be met only by reseachers and practitioners working together to provide answers to critical educational questions.