Search

Seventy students participated in an experiment to measure the effects of either providing explanations or listening during small group discussions on recall of related subject-matter studied after the discussion. They watched a video of a small group discussing a problem. In the first experimental condition, the video was stopped at various points in time, enabling the participants to verbally respond to the discussion. In the second condition, they listened to the same discussion, without contributing. In the control condition, they listened to a discussion that was not related to the subject-matter subsequently studied. After the discussion, all participants studied a text and answered questions that tested their recall of information from this text. No immediate differences in recall were found. One month later, participants who had actively engaged in explaining remembered more from the text. The conclusion appears justified that actively providing explanations during a discussion positively affects long-term memory. [ABSTRACT FROM AUTHOR]

Objective To determine the effectiveness of single-point benchmarking and longitudinal benchmarking for inter-school educational evaluation. Methods We carried out a mixed, longitudinal, cross-sectional study using data from 24 annual measurement moments (4 tests × 6 year groups) over 4 years for 4 annual progress tests assessing the graduation-level knowledge of all students from 3 co-operating medical schools. Participants included undergraduate medical students (about 5000) from 3 medical schools. The main outcome measures involved between-school comparisons of progress test results based on different benchmarking methods. Results Variations in relative school performance across different tests and year groups indicate instability and low reliability of single-point benchmarking, which is subject to distortions as a result of school-test and year group-test interaction effects. Deviations of school means from the overall mean follow an irregular, noisy pattern obscuring systematic between-school differences. The longitudinal benchmarking method results in suppression of noise and revelation of systematic differences. The pattern of a school’s cumulative deviations per year group gives a credible reflection of the relative performance of year groups. Conclusions Even with highly comparable curricula, single-point benchmarking can result in distortion of the results of comparisons. If longitudinal data are available, the information contained in a school’s cumulative deviations from the overall mean can be used. In such a case, the mean test score across schools is a useful benchmark for cross-institutional comparison. [ABSTRACT FROM AUTHOR]

In-training assessment (ITA), defined as multiple assessments of performance in the setting of day-to-day practice, is an invaluable tool in assessment programmes which aim to assess professional competence in a comprehensive and valid way. Research on clinical performance ratings, however, consistently shows weaknesses concerning accuracy, reliability and validity. Attempts to improve the psychometric characteristics of ITA focusing on standardisation and objectivity of measurement thus far result in limited improvement of ITA-practices. The aim of the paper is to demonstrate that the psychometric framework may limit more meaningful educational approaches to performance assessment, because it does not take into account key issues in the mechanics of the assessment process. Based on insights from other disciplines, we propose an approach to ITA that takes a constructivist, social-psychological perspective and integrates elements of theories of cognition, motivation and decision making. A central assumption in the proposed framework is that performance assessment is a judgment and decision making process, in which rating outcomes are influenced by interactions between individuals and the social context in which assessment occurs. The issues raised in the article and the proposed assessment framework bring forward a number of implications for current performance assessment practice. It is argued that focusing on the context of performance assessment may be more effective in improving ITA practices than focusing strictly on raters and rating instruments. Furthermore, the constructivist approach towards assessment has important implications for assessment procedures as well as the evaluation of assessment quality. Finally, it is argued that further research into performance assessment should contribute towards a better understanding of the factors that influence rating outcomes, such as rater motivation, assessment procedures and other contextual variables. [ABSTRACT FROM AUTHOR]

Background : Many studies have been conducted on tutor performance in problem-based curricula. In the past, the implicit assumption behind these studies was that tutor performance is a relatively stable characteristic. More recent studies demonstrate that a tutor's performance may be dependent on other circumstances, such as the level of structure in the curricular materials. The aim of this study was to investigate whether a tutor's performance is also dependent on the tutorial group's productivity. Purpose: The idea is that low-productive tutorial groups require much more input from a tutor than high-productive groups. In the problem-based curriculum under investigation, most tutors guide 2 tutorial groups within the same unit. A salient finding in this problem-based curriculum was that some tutors who guide 2 tutorial groups within the same unit have discrepancies in their tutor performance across the 2 groups. This finding might be explained by differences in both tutorial groups. In this study, first the scope of the discrepancy phenomena was studied. Second, the relation between the tutor's performance and the tutorial group's productivity was studied. Methods: The data set for this study included 136 tutors who, in total, ran 272 tutorial groups (each tutor ran 2 groups per unit). The analyses were conducted at the tutorial group level. Students were asked to judge the performance of their tutor. Low, medium, and high levels of tutor performance were distinguished. Tutors who were qualified as ''low level of performance in one tutorial group and ''medium level of performance in the other tutorial group were considered to have a discrepancy in their tutor performance: ''discrepancy tutors. The same holds for tutors with medium level of performance in one group and high level of performance in the other group or low level of performance in one group and high level of performance in the other group. All other tutors were considered ''nondiscrepancy tutors. The nondiscrepancy tutors had equal levels of performance in both groups: a low, medium, or high level. For each type of tutor (discrepancy tutors and nondiscrepancy tutors) the average tutorial group's productivity score was computed. Results: The results show that 39% of the tutors were classified as discrepancy tutors. In addition, it was found that a discrepancy tutor with a low level of tutor performance in one group also had a low productivity score in this group, whereas a high level of tutor performance corresponds with a high level of the tutorial group's productivity. Furthermore, the results show that nondiscrepancy tutors with a high level of tutor performance receive high tutor performance scores, irrespective of the tutorial group's level of productivity. Conclusions: These findings demonstrate that the tutorial group's productivity is another influencing factor in determining tutor performance. Low-productive groups require much more input from a tutor than high-productive groups. Nondiscrepancy tutors with consistent low levels of tutor performance and discrepancy tutors lack certain competencies that are needed when being confronted with a low-productive tutorial group. Nondiscrepancy tutors with a high level of tutor performance, on the contrary, know how to deal with low-productive tutorial groups, due to which their tutor performance is high irrespective of the tutorial group's productivity. Thus, a tutor's performance seems to be part tutor specific and part situation specific (i.e., dependent on the group's productivity). [ABSTRACT FROM AUTHOR]

Background: This research focuses on the relation between individual study and group discussion. In a problem-based curriculum, it is expected that the way students prepare themselves during individual study (i.e., search and prepare the literature) will influence the quality of the reporting phase. Purpose: To investigate whether searching for different literature resources and preparing the literature (by making summaries to explain the literature) affects the quality of the reporting phase. Method : A 23-item questionnaire was developed, reflecting 2 factors of the search phase and 1 factor of the preparing phase of the individual study. Two factors (breadth and depth of the discussion) reflected the reporting phase. Participants were 1st-year students (N = 195, 90%) at the Medical School of the Maastricht University in the Netherlands during the academic year 1997-1998. Results: Regression analyses show that the search phase and the preparing phase explained 29% of the variance of the breadth and 38% of the depth of the reporting phase. Conclusions: Searching different literature resources has a small and negative impact on the quality of the reporting phase. However, preparing extensively for the next tutorial meeting is very important, especially for the depth of the reporting phase. Therefore, when new knowledge must be integrated in the tutorial group, students must be taught to prepare themselves by making summaries that are useful and effective during the discussion. [ABSTRACT FROM AUTHOR]

Background: To bridge the gap between preclinical and clinical education that still exists in the problem-based medical school at Maastricht, tutorial groups in which clinicians guided and discussed students' interactions with real patients were introduced in the preclinical program. Purpose: To find out what students find instructive in these patient tutorial groups and how this should be interpreted in relation to the problem-based character of the curriculum. Methods: Two focus groups were composed of participants of the patient tutorial groups in 1996-1997. They discussed relevant issues in two sessions and approved reports of the outcomes. Results and Conclusions: Students found the real-patient contact in the tutorials very valuable. They had not practiced taking a history to formulate and test a differential diagnosis with patients before. Problem-based tutorials with written cases were said to frequently lack depth of discussion. Individual contacts with simulated patients were found to be important in testing one's own knowledge and skills. [ABSTRACT FROM AUTHOR]

Background: Self-directed learning requires accurate self-assessment, but research evidence shows poor validity of self-assessment. Training in self-assessment may improve validity. Purpose: To investigate if repeated personal feedbackbased on objective knowledge and skill scores enhances self-assessment skills of practicing general practitioners. Method: Participants were general practitioners (n = 60),who received skills training covering 4 clinicalskills at3 months (Group A)or 6 months (Group B)after enrollment in the study. Participants were tested at 3-month intervals with a knowledge test (60 items), a performance-based test(4 stations), and a self-assessmentquestionnaire (22 items), covering the four different clinical skills. They received personal feedback on the results. Results: At 3 months, mean scores on the self-assessment questionnaire and knowledge test had increased significantly more in Group Acompared to Group B, but at 6 months no differences in mean scores remained.Correlations between self-assessment rating and objective scores were low to moderate,with little overall improvementover time. Conclusions: Although self-assessment scores can to some extent be useful in measuring perceived changes in competence in groups, individual self-assessment scores on their own are an invalid source ofinformation concerning competence ofpracticing physicians, and this does not improve significantly with regular feedback. [ABSTRACT FROM AUTHOR]