Search

This article outlines some key issues for writing effective computer-aided assessment (CAA) questions in subjects with substantial mathematical or statistical content, especially the importance of control of random parameters and the encoding of wrong methods of solution (mal-rules) commonly used by students. The pros and cons of using CAA and different question types are discussed. Issues surrounding the selection and encoding of mal-rules are highlighted, especially for multi-choice and responsive numerical input questions. These generate mal-rule-specific feedback, the mal-rule used being deduced from the student's selection or input. Student answer file data from the use of over 800 questions and their embedding within an overall assessment regime is analysed and presented to show that this has had a very beneficial effect on the examination performance of a large cohort of first-year economics students in their mathematics module over the last 6 years. Question analysis of over 270,000 question attempts, identifying the most difficult/discriminating questions, shows that the questions are robust, valid and span an appropriate range of difficulties. The idea of underlying mal-rules is examined to see how far this explains this range.

Computer-Aided Assessments (CAAs) have been used increasingly at Brunel University for over 10 years to test students' mathematical abilities. Recently, we have focussed on providing very rich feedback to the students; given the work involved in designing and coding such feedback, it is important to study the impact of the interaction between students and feedback. To make feedback more focussed, examination scripts have been analysed to identify common student errors and misconceptions. These have then been used to code distracters in multiple-choice and responsive numerical input-type questions. Since random parameters are used in all questions developed, distracters have to be coded as algebraic or algorithmic "mal-rules". This paper reports on the methodology used to identify students' errors and misconceptions and how the evidence collected was used to code the distracters. The paper also provides hard evidence that real learning has taken place while students have interacted with the CAAs. Statistical analyses of exam performance over eight years indicate that students are able to improve performance in subsequent formative and summative assessments provided that they have truly engaged with the CAA, especially by spending time studying the feedback provided. (Contains 3 tables and 3 figures.)

This opinion piece presents my experiences of using e-assessment for foundation year mathematics, analysing data from a 230-strong cohort. Strengths and weaknesses of using e-assessment are discussed, and I present two experiments that have been embedded this year to attempt to counteract poor mathematical skills and study attitudes amongst some students. Data from the last four years show that learning (as measured by end-of-year traditional exams) has been sustained. It seems probable that this is partly due to the efficacy of these measures, but results from the experiments in using staged tests and in appraising foundationers' mathematical confidence are mixed. [ABSTRACT FROM AUTHOR]