Search

This paper describes how a state education system in Australia introduced standards-referenced assessments into its large-scale, high-stakes, curriculum-based examinations in a way that enables comparison of performance across time even though the examinations are different each year. It describes the multi-stage modified Angoff standard-setting procedure used to establish cut-off scores on subject examinations, and how the results from this exercise were then used to develop standards packages. These packages illustrate the performances of students at the borders between the various bands. The paper also shows how originally it was intended to use a Rasch measurement model to create the statistical feedback used in the standard-setting procedure. It also describes the modifications to the feedback that were necessary to meet the real-time constraints of this large-scale examination programme. It argues that consideration should now be given to using the Rasch model to provide this feedback instead of the current approach. (Contains 6 figures, 1 table and 1 note.)

Describes a two-part study involving 14- to 15-year-old high school students in New South Wales (Australia) that compared two modes of responding to free response examination questions: by computer and by pen and paper. Discusses results of analysis of variance. (Author/LRW)

This paper describes how a state education system in Australia introduced standards-referenced assessments into its large-scale, high-stakes, curriculum-based examinations in a way that enables comparison of performance across time even though the examinations are different each year. It describes the multi-stage modified Angoff standard-setting procedure used to establish cut-off scores on subject examinations, and how the results from this exercise were then used to develop standards packages. These packages illustrate the performances of students at the borders between the various bands. The paper also shows how originally it was intended to use a Rasch measurement model to create the statistical feedback used in the standard-setting procedure. It also describes the modifications to the feedback that were necessary to meet the real-time constraints of this large-scale examination programme. It argues that consideration should now be given to using the Rasch model to provide this feedback instead of the current approach. [ABSTRACT FROM PUBLISHER]

A two-part study, involving 14- to 15-year-old high school students, compared two modes of responding to free response examination questions: by computer and, the traditional way, by pen and paper. In Part 1, both the computer group ( n = 57) and the pen and paper group ( n = 52) were formed by random assignment. They answered three essay questions from a 1997 external English test which were holistically marked. In Part 2, the computer group ( n = 88) and the pen and paper group ( n = 53) chose their preferred method of response. They answered two essay questions from a 1999 external English test, which were analytically marked using standards-referenced criteria. For four of the five questions, there were no significant differences between the pen and paper response marks and the computer response marks when presented in their original formats. When the pen and paper responses were word-processed, markers tended to award higher marks to the handwritten scripts. [ABSTRACT FROM AUTHOR]