Search

Purpose This study aims to report the design, development and evaluation of a digital quality assurance application aimed at improving and ensuring the quality of assessment programmes in higher education. Design/methodology/approach The application was developed using a design-based research (DBR) methodology. The application’s design was informed by a literature search and needs assessment of quality assurance stakeholders to ensure compliance with daily practices and accreditation requirements. Stakeholders from three study programmes evaluated the application. Findings As part of the development of the application, module- and programme-level dashboards were created to provide an overview of the programme’s outcomes, assessment methods, assessment metrics, self-evaluated quality indicators and assessment documents. The application was evaluated by stakeholders at the module and programme levels. Overall, the results indicated that the dashboards aided them in gaining insight into the assessment programme and its alignment with underlying assessments. Practical implications Visualisation of the assessment programme’s structure and content identifies gaps and opportunities for improvement, which can be used to initiate a dialogue and further actions to improve assessment quality. Originality/value The application developed facilitates a cyclical and transparent assessment quality assurance procedure that is continuously available to various stakeholders in quality assurance.

How can feedback dialogues stimulate students' reflective thinking? This study aims to investigate: (1) the effects of feedback dialogues between teachers and students on students' perceptions of teacher feedback and (2) the relation between features of feedback dialogues and students' thinking activities as part of reflective thinking. A quasi-experiment was carried out in which 72 secondary education students received written teacher feedback. Half of them had an additional feedback dialogue with their teacher. The latter group perceived teachers' feedback as more useful than the group without a dialogue. The feedback dialogues could be clustered in 2 groups that varied in the number of segments containing teacher and student interaction. The number of segments containing interaction was positively correlated with students' use of thinking activities.

This study investigated the effects of feedback providing improvement strategies and a reflection assignment on students’ writing motivation, process, and performance. Students in the experimental feedback condition (n = 41) received feedback including improvement strategies, whereas students in the control feedback condition (n = 41) received feedback without improvement strategies. Within each feedback condition, half of the students received a reflection assignment on feedback use and the revision (experimental reflection condition), while the other half received a reflection assignment on feedback perception (control reflection condition). Results indicated that in the experimental feedback condition writing performance gained from the control reflection assignment, while in the control feedback condition it gained from the experimental reflection assignment. Improvement strategies negatively predicted self-efficacy beliefs, especially when initial self-efficacy beliefs were low, and positively predicted planning/revising. Reflections on feedback use and the revision positively predicted mastery goal when mastery goal initially was low or moderate.

Item does not contain fulltext The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and assessment, and an innovative school with extensive experience. The self-evaluation was based on 12 quality criteria for CAPs, including both validity and reliability, and criteria stressing the importance of the formative function of assessment, such as meaningfulness and educational consequences. In each school, teachers, management and examination board participated. Results show that the two schools use different approaches to assure assessment quality. The innovative school seems to be more aware of its own strengths and weaknesses, to have a more positive attitude towards teachers, students, and educational innovations, and to explicitly involve stakeholders (i.e., teachers, students, and the work field) in their assessments. This school also had a more explicit vision of the goal of competence-based education and could design its assessments in accordance with these goals. 01 augustus 2011

The effects of progress feedback on university students' writing mastery goal, self-efficacy beliefs, and writing performance were examined in this experiment. Students in the experimental condition (n = 42) received progress feedback on their writing assignment, whereas students in the control condition (n = 44) received feedback without progress information. The Motivated Strategies for Learning Questionnaire (MSLQ) was used to measure students' writing mastery goal and self-efficacy beliefs. Results showed that progress feedback did not affect students' writing mastery goal and performance but did affect students' self-efficacy beliefs. The effect depended on the number of progress comments.

Nievelstein, F., Van Gog, T., Boshuizen, H. P. A., & Prins, F. J. (2008). Expertise-related differences in conceptual and ontological knowledge in the legal domain. European Journal of Cognitive Psychology, 20, 1043–1064.

As assessment methods are changing, the way to determine their quality needs to be changed accordingly. This article argues for the use Competences Assessment Programs (CAPs), combinations of traditional tests and new assessment methods which involve both formative and summative assessments. To assist schools in evaluating their CAPs, a self-evaluation procedure was developed, based on 12 quality criteria for CAPs developed in earlier studies. A self-evaluation was chosen as it is increasingly used as an alternative to external evaluation. The CAP self-evaluation is carried out by a group of functionaries from the same school and comprises individual self-evaluations and a group interview. The CAP is rated on the 12 quality criteria and a piece of evidence is asked for to support these ratings. In this study, three functionaries from eight schools (N = 24) evaluated their CAP using the self-evaluation procedure. Results show that the group interview was very important as different perspectives on the CAP are assembled here into an overall picture of the CAP’s quality. Schools seem to use mainly personal experiences to support their ratings and need to be supported in the process of carrying out a self-evaluation.

A General Practitioner (GP) is no longer a loner, but a team player in either a group practice or a care centre. This change has led to a concomitant growth in curricular interest in skills essential for successful collaboration and for enhancing critical reflection towards colleagues' performance. Giving and receiving constructive feedback are examples of these skills. The aim of this study was to gain insight in the style and quality of feedback reports on consultation skills written by GPs-in-Training (GPiTs) and by their GP-trainers. Furthermore, the preferences of the GPiTs concerning feedback style were examined. Results show significant differences between GP-trainers and GPiTs in feedback style and quality. A ranking task indicated that GPiTs have a preference for reports characterised by a large number of reflective remarks. Questionnaire results indicate the added value of the use of peer feedback. Implications for the integration of peer feedback activities in the curriculum of GPiTs are discussed.

Sluijsmans, D. M. A., & Prins, F. (2006). A conceptual framework for integrating peer assessment in teacher education. Studies in Educational Evaluation, 32, 6-22.

Quantitative content analysis is increasingly used to surpass surface level analyses in Computer-Supported Collaborative Learning (e.g., counting messages), but critical reflection on accepted practice has generally not been reported. A review of CSCL conference proceedings revealed a general vagueness in definitions of units of analysis. In general, arguments for choosing a unit were lacking and decisions made while developing the content analysis procedures were not made explicit. In this article, it will be illustrated that the currently accepted practices concerning the ‘unit of meaning’ are not generally applicable to quantitative content analysis of electronic communication. Such analysis is affected by ‘unit boundary overlap’ and contextual constraints having to do with the technology used. The analysis of e-mail communication required a different unit of analysis and segmentation procedure. This procedure proved to be reliable, and the subsequent coding of these units for quantitative analysis yielded satisfactory reliabilities. These findings have implications and recommendations for current content analysis practice in CSCL research. (http://www.sciencedirect.com/science/article/pii/S0360131505000540)

In this case study our aim was to gain more insight in the possibilities of qualitative formative peer assessment in a computer supported collaborative learning (CSCL) environment. An approach was chosen in which peer assessment was operationalized in assessment assignments and assessment tools that were embedded in the course material. The course concerned a higher education case-based virtual seminar, in which students were asked to conduct research and write a report in small multidisciplinary teams. The assessment assignments contained the discussion of assessment criteria, the assessment of a group report of a fellow group and writing an assessment report. A list of feedback rules was one of the assessment tools. A qualitative oriented study was conducted, focusing on the attitude of students towards peer assessment and practical use of peer assessment assignments and tools. Results showed that students' attitude towards peer assessment was positive and that assessment assignments had added value. However, not all students fulfilled all assessment assignments. Recommendations for implementation of peer assessment in CSCL environments as well as suggestions for future research are discussed.

Background. In his dissertation, Jan Vermunt (1992) proposed four different learning styles: a meaning directed, a reproduction directed, an application directed and an undirected style. Aims. In a cross-sectional and longitudinal design the development of these learning styles during students' stay at the university is studied, as well as the relation of these styles with academic success. A systematic relation between year of study and learning style is expected, namely that the application and meaning directed learning style scores would be higher in the later years and the undirected and reproduction learning style scores would be the highest in the early years. Results. In the cross-sectional study it was not found that the application and meaning directed learning style scores were higher in the later years, nor that the undirected and reproduction learning style scores were the highest in the early years. In the longitudinal study the means of the meaning directed and application directed learning style scores indeed increased over the years, while the means of the reproduction and undirected learning style scores decreased. For the score on the meaning directed learning style, this change was significant, though a marginal effect size was found. Conclusion. It might not be concluded there is a systematic relation between year of study and learning style. No evidence is found for the implicit hierarchy that the meaning directed and application directed learning are better than the reproduction learning style. The undirected learning style, however, correlates negatively with academic success.