Search

Assessment and evaluation at all levels of educational systems have become policy priorities for many countries. Two common reasons for this are student learning expectations and accountability. Although much effort has been put into the creation and refinement of content standards, standardized tests, and methods for using testing results, there has been less attention paid to the development of performance standards (proficiency levels) that greatly affect policy decision making. The present study investigates the Objective Standard Setting Model (OSS) as an improved criterion-referenced method for setting multilevel performance standards. To demonstrate how OSS can be employed for performance standard setting, our study used data from preservice teachers completing an assessment literacy test in a pre- and posttest environment. Using OSS, performance standard levels of proficient and excellent were established with clear content-related descriptions of growth in student content mastery. [ABSTRACT FROM AUTHOR]

Objectives: The authors outline the difference between content and performance standards and the rationale for standard setting at a medical college. The principles of the college's standard setting processes for the written and objective structured clinical examination summative assessments are discussed in greater detail.Conclusion: There is no evidence of any single standard setting method to be the best. Multiple methods exist and will have varied results when applied. The judgement of a panel of subject experts remains an important component of the standard setting process. [ABSTRACT FROM AUTHOR]

A one-year follow-up study examined the role of developmental challenges, personality (dependency, self-criticism, and personal efficacy), and support systems in adaptation among Israeli emerging adults (N = 236) participating in a preparatory academic program. Participants were assessed during their enrollment in the preparatory academic program and one year later, after graduation, when their academic success or failure could be determined. Personal efficacy predicted higher levels of goal investment and goal progress, as well as more positive life events. In contrast, self-criticism predicted lower goal investment, higher goal stress, and a reduced number of positive events. Self-criticism also predicted elevated levels of a motivation and an increase in depressive and anxiety symptoms. Paternal social support predicted young people's more favorable adaptation one year later. Among young adult males, academic failure predicted later level of symptoms and more negative events. These findings encourage further examination of the role of personality and family relationships in the transition to adulthood. [ABSTRACT FROM AUTHOR]

Interest in end-of-year accountability exams has increased dramatically since the passing of the No Child Left Behind Act in 2001. With this increased interest comes a desire to use student data collected throughout the year to estimate student proficiency and predict how well they will perform on end-of-year exams. This article uses student performance on the Assistment System, an online mathematics tutor, to show that replacing percentage correct with an Item Response Theory estimate of student proficiency leads to better fitting prediction models. In addition, it uses other tutor performance metrics to further increase prediction accuracy. Prediction error bounds are also calculated to attain an absolute measure to which the models can be compared. [ABSTRACT FROM AUTHOR]

Standard-setting methods are widely used to determine cut scores on a test that examinees must meet for a certain performance standard. Because standard setting is a measurement procedure, it is important to evaluate variability of cut scores resulting from the standard-setting process. Generalizability theory is used in this study to estimate standard errors of cut scores resulting from two standard-setting methods: item rating (Angoff-based) and mapmark (bookmark-based) methods. In this study, two different generalizability (G) study designs and four different decision (D) study designs were examined, and the impact of varying different aspects of the study design and universes of generalization was examined. Results suggest that cut scores were generally consistent for both methods. The first round standard setting contributed the most to the overall variability for the mapmark method. Also, it is clear that there is no one standard error associated with a certain cut score. [ABSTRACT FROM AUTHOR]

Goal orientation research has developed two branches. One branch uses a two-factor dispositional model. The other branch uses a three-factor domain-specific model. The present multiwave study examines dispositional goal orientation constructs through both two- and three-factor models using repeated-measure confirmatory factor analysis. Results indicate dispositional performance goal orientation of the two-factor model can be better partitioned into the Prove and Avoid factors of the three-factor model. Further evidence is presented indicating the potential for more than three factors by further partitioning the Avoid factor into two separate constructs. Consistent with both branches of goal orientation research, learning goal orientation is largely independent of Performance, Prove, and Avoid factors in all tested models. [ABSTRACT FROM AUTHOR]

Special education teachers are routinely taught to collect and analyze student performance data. Data collection and analysis are assumed to be necessary for accurate analysis of three possible trends: (a) an upward trend (when performance trend (when performance worsens). Few studies have examined the accuracy of trend analysis when teachers simply observe consecutive training sessions, nor has there been sufficient comparison of the accuracy of trend analysis based on various forms of data (e.g., raw data vs. data in graphic form). The present study examined the effects of observation and various forms of data on teachers' ability to accurately analyze trends. A 2-factor experimental design was used with 4 groups of randomly assigned subjects. Subjects analyzed 15 samples of videotaped student performance. All subjects viewed the tapes with or without additional forms of data and made judgment regarding the trend of student performance following exposure to each sample. The data were analyzed using a two-way ANOVA. Both the main effect for form of data and the interaction between form of data and type of trends were statistically significant beyond the .01 level. All subjects were able to accurately analyze upward trends in performance, but data were required to accurately analyze level and downward trends. These results are interpreted as support for the practice of training teachers to collect and analyze student performance data. [ABSTRACT FROM AUTHOR]