Search

Context Small-group learning is advocated for enhancing higher-order thinking and the development of skills and attitudes. Teacher performance, group interaction and the quality of assignments have been shown to affect small-group learning in hybrid and problem-based curricula. Objectives This study aimed to examine the perceptions of student groups as to how teacher performance, group interaction and the quality of assignments are related to one another and to learning effects in seminars of 15–30 students in a hybrid curriculum. Methods We constructed a 28-item questionnaire and administered it to 639 students attending 32 seminars in Years 1–4 of an undergraduate veterinary curriculum. We performed factor analysis and reliability analysis of the questionnaire. We used correlation and regression analyses to explore the interactions of the four-factor model, with teacher performance, group interaction and quality of assignments as independent variables and the perceived learning effect of the seminars as the dependent variable. Results The response rate was 99%. Teacher performance (β = 0.78) and group interaction (− 0.28) significantly influenced the perceived learning effect. The total effect of the quality of assignments (through effects on teacher performance and group interaction) was 0.47. Discussion The strong relationship between teacher performance and learning effect suggests that students rely strongly on their teachers. The negative effect of group interaction may reflect poor alignment of teaching and assessment and poor organisation of group processes. This should be further examined. Comparative studies on seminars are also recommended. [ABSTRACT FROM AUTHOR]

Objective To test hypotheses regarding the longitudinal effects of problem-based learning (PBL) and conventional learning relating to students’ appreciation of the curriculum, self-assessment of general competencies, summative assessment of clinical competence and indicators of career development. Methods The study group included 2 complete cohorts of graduates who were admitted to the medical curriculum in 1992 (conventional curriculum, n = 175) and 1993 (PBL curriculum, n = 169) at the Faculty of Medicine, University of Groningen, the Netherlands. Data were obtained from student records, graduates’ self-ratings and a literature search. Gender and secondary school grade point average (GPA) scores were included as moderator variables. Data were analysed by a stepwise multiple and logistic regression analysis. Results Graduates of the PBL curriculum scored higher on self-rated competencies. Contrary to expectations, graduates of the PBL curriculum did not show more appreciation of their curriculum than graduates of the conventional curriculum and no differences were found on clinical competence. Graduates of the conventional curriculum needed less time to find a postgraduate training place. No differences were found for scientific activities such as reading scientific articles and publishing in peer- reviewed journals. Women performed better on clinical competence than did men. Grade point average did not affect any of the variables. Conclusions The results suggest that PBL affects self-rated competencies. These outcomes confirm earlier findings. However, clinical competence measures did not support this finding. [ABSTRACT FROM AUTHOR]

Objective To determine the effectiveness of single-point benchmarking and longitudinal benchmarking for inter-school educational evaluation. Methods We carried out a mixed, longitudinal, cross-sectional study using data from 24 annual measurement moments (4 tests × 6 year groups) over 4 years for 4 annual progress tests assessing the graduation-level knowledge of all students from 3 co-operating medical schools. Participants included undergraduate medical students (about 5000) from 3 medical schools. The main outcome measures involved between-school comparisons of progress test results based on different benchmarking methods. Results Variations in relative school performance across different tests and year groups indicate instability and low reliability of single-point benchmarking, which is subject to distortions as a result of school-test and year group-test interaction effects. Deviations of school means from the overall mean follow an irregular, noisy pattern obscuring systematic between-school differences. The longitudinal benchmarking method results in suppression of noise and revelation of systematic differences. The pattern of a school’s cumulative deviations per year group gives a credible reflection of the relative performance of year groups. Conclusions Even with highly comparable curricula, single-point benchmarking can result in distortion of the results of comparisons. If longitudinal data are available, the information contained in a school’s cumulative deviations from the overall mean can be used. In such a case, the mean test score across schools is a useful benchmark for cross-institutional comparison. [ABSTRACT FROM AUTHOR]

Objective To determine the differential effects of a paper-based versus a web-based portfolio in terms of portfolio quality, user-friendliness and student motivation. Methods An experimental design was used to compare Year 1 medical students' reflective portfolios. The portfolios differed in presentation medium only (i.e. web-based versus paper-based). Content analysis, a student questionnaire and mentor interviews were used to evaluate portfolio quality, user-friendliness and student motivation. A total of 92 portfolios were scored independently by 2 raters using a portfolio quality-rating instrument. Results Portfolio structure, quality of reflection and quality of evidence showed no significant effects of presentation medium. Multi-level analysis showed a significant effect for student motivation: web-based portfolios scored 0.39 more than paper-based portfolios ( P < 0.05; effect size 0.76). The mentors reported no differences in portfolio quality, except that there were more visuals in web-based portfolios. Students spent significantly more time preparing the web-based than the paper-based portfolios (15.4 hours versus 12.2 hours; t = 2.1, P < 0.05; effect size 0.46). The 2 student groups did not differ significantly in terms of their satisfaction with the portfolio. The mentors perceived the web-based portfolios as more user-friendly. Conclusions The web-based portfolios were found to enhance students' motivation, were more user-friendly for mentors, and delivered the same content quality compared with paper-based portfolios. This suggests that web-based presentation may promote acceptance of portfolios by students and teachers alike. [ABSTRACT FROM AUTHOR]

Context We addressed the assessment of professional behaviour in tutorial groups by investigating students' perceptions of the frequency and impact of critical incidents that impede this assessment and 5 factors underlying these critical incidents. Methods A questionnaire asking students to rate the frequency and impact of 40 critical incidents relating to effective assessment of professional behaviour on a 5-point Likert scale was developed and sent to all undergraduate medical students in Years 2–4 of a 6-year undergraduate curriculum. Results The response rate was 70% ( n = 393). Important factors underlying critical incidents are: lack of effective interaction; lack of thoroughness; tutors' failure to confront students with unprofessional behaviour; lack of effort to find solutions, and lack of student motivation. Confirmatory factor analysis showed a good model fit. Because the relationship between frequency of occurrence and degree of impediment varies, the best information about the true impact of critical incidents and the underlying factors is provided by the product of frequency and degree of impediment. Frequency of occurrence remains stable and degree of impediment increases in Years 2–4. Conclusions The results of this study can be used to design and improve faculty development programmes aimed at improving assessment of professional behaviour. Training programmes should motivate tutors by providing background information as to why and how sound assessment of professional behaviour is to be performed and encourage tutors to confront students with and discuss all aspects of professional behaviour, as well as provide appropriate feedback. [ABSTRACT FROM AUTHOR]

Purpose To ascertain whether the grade point average (GPA) of school-leaving examinations is related to study success, career development and scientific performance. The problem of restriction of range was expected to be partially reduced due to the use of a national lottery system weighted in favour of students with higher GPAs. Method We studied the students ( n = 398) admitted to the Faculty of Medicine, University of Groningen, the Netherlands in 1982 and 1983. Data concerning drop-out and study progress were derived from the student administration. Data about career development were obtained from annual interviews with graduates ( n = 318) between 1993 and 2000. Literature searches yielded data concerning scientific performance. Multiple linear regression and logistic regression were used to analyse the data. The variables ‘gender’ and ‘cohort’ and their interaction were included in the analyses to account for variation in the general level of the dependent variable and the effect of GPA on the dependent variable. Results GPA scores had no effect on drop-out rate. High GPA scores were associated with significantly less time to graduation, more chance of a preferred placement for specialist training and higher scientific output. GPA was not related to income. Gender differences were found for study duration and scientific output. Women graduated earlier and published less. Conclusion The GPA of school-leaving examinations was found to be related to study success, career development and scientific performance. In this study the usual problem of restriction of range was shown to be absent. The weighted lottery procedure even resulted in an over-dispersion of candidates relative to the applicants. The resulting effect sizes were in agreement with those reported in other studies. [ABSTRACT FROM AUTHOR]

Context The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence. Methods We developed an instrument, the Portfolio Analysis Scoring Inventory, to examine the impact of form and content aspects on portfolio assessment. The Inventory consists of 15 items derived from interviews with experienced mentors, the literature, and the criteria for reflective competence used in the regular portfolio assessment procedure. Forty portfolios, selected from 231 portfolios for which ratings from the regular assessment procedure were available, were rated by 2 researchers, independently, using the Inventory. Regression analysis was used to estimate the correlation between the ratings from the regular assessment and those resulting from the Inventory items. Results Inter-rater agreement ranged from 0.46 to 0.87. The strongest predictor of the variance in the regular ratings was ‘quality of reflection’ (R 0.80; R2 66%). No further items accounted for a significant proportion of variance. Irrelevant items, such as writing style and lay-out, had negligible effects. Conclusions The absence of an impact of irrelevant criteria appears to support the validity of the portfolio assessment procedure. Further studies should examine the portfolio's validity for the assessment of clinical competence. [ABSTRACT FROM AUTHOR]

Objective To investigate the effects of a virtual learning environment (VLE) on group interaction and consultation of information resources during the preliminary phase, self-study phase and reporting phase of the problem-based learning process in an undergraduate medical curriculum. Methods A questionnaire was administered to 355 medical students in Years 1 and 2 to ask them about the perceived usefulness of a virtual learning environment that was created with Blackboard for group interaction and the use of learning resources. Results The students indicated that the VLE supported face-to-face interaction in the preliminary discussion and in the reporting phase but did not stimulate computer-mediated distance interaction during the self-study phase. They perceived that the use of multimedia in case presentations led to a better quality of group discussion than if case presentations were exclusively text-based. They also indicated that the information resources that were hyperlinked in the VLE stimulated the consultation of these resources during self-study, but not during the reporting phase. Conclusions Students indicated that the use of a VLE in the tutorial room and the inclusion of multimedia in case presentations supported processes of active learning in the tutorial groups. However, if we want to exploit the full potential of asynchronous computer-mediated communication to initiate in-depth discussion during the self-study phase, its application will have to be selective and deliberate. Students indicated that the links in the VLE to selected information in library repositories supported their learning. [ABSTRACT FROM AUTHOR]

Objective To investigate whether adding a brief video case instead of an equivalent written text improves the cognitive and metacognitive processes (data exploration, theory building, theory evaluation and metareasoning) of residents in problem-based learning. Subjects Paediatric residents. Setting Department of Paediatrics, Århus University Hospital, Skejby, Denmark. Methods Eleven residents were each assigned to 1 of 2 groups. Both groups analysed an identical vignette of a patient case. Immediately after this, 1 group watched a 2.5-minute video recording and the other group read a description of the same video recording. The groups then reanalysed the case. Thinking processes were tapped by recording and analysing the verbal group interaction. The frequencies of clause categories were calculated and compared using chi-square tests. Results The verbal interaction showed statistically significant improvements in data exploration, theory building and theory evaluation after the video case. Conclusions The greater improvement in clause frequency after the video case compared with the equivalent text suggests an improvement in data exploration, theory building and theory evaluation. The use of patient video recordings may be a valuable supplement to postgraduate paediatric training, particularly pertaining to movement disorders. [ABSTRACT FROM AUTHOR]

CONTEXT Continuing professional development (CPD) of general practitioners. OBJECTIVE Criterion-referenced standards for assessing performance in the real practice of general practitioners (GPs) should be available to identify learning needs or poor performers for CPD. The applicability of common standard setting procedures in authentic assessment has not been investigated. METHODS To set a standard for assessment of GP-patient communication with video observation of daily practice, we investigated 2 well known examples of 2 different standard setting approaches. An Angoff procedure was applied to 8 written cases. A border-line regression method was applied to videotaped consultations of 88 GPs. The procedures and outcomes were evaluated by the applicability of the procedure, the reliability of the standards and the credibility as perceived by the stakeholders, namely, the GPs. RESULTS Both methods are applicable and reliable; the obtained standards are credible according to the GPs. CONCLUSIONS Both modified methods can be used to set a standard for assessment in daily practice. The context in which the standard will be used — i.e. the specific purpose of the standard, the moment the standard must be available or if specific feedback must be given — is important because methods differ in practical aspects. [ABSTRACT FROM AUTHOR]

For more than two decades the Medical School in Maastricht, the Netherlands, has used simulated patients (SPs) to provide students with opportunities to practise their skills in communication and physical examination. In this educational setting a student meets a SP in a videotaped session. Feedback by the SP to the student at the end of the session is considered an important educational feature. We found no instruments to assess individual SP performance during those sessions. To develop a valid, reliable and feasible instrument to evaluate the performance of SPs. The content of the instrument was validated through interviews with students, teachers and experts who are involved with SPs. They were asked to indicate key features of good SP performance. Based on the interviews, a written checklist was developed to measure individual SP performance. The instrument was evaluated in a regular SP session at the medical school, involving 152 students and their teachers. All interviewees considered the scale to be satisfactory and the instrument to be valid. The feasibility and reliability of the checklists were investigated using the data of 398 returned checklists. Cronbach's alpha was found to be 0.73. Generalizability analysis showed that 12 completed checklists were required to obtain a reliable assessment of one SP. The Maastricht Assessment of Simulated Patients (MaSP) appears to be a valid, reliable and feasible tool to assess the performance of SPs in an educational setting. [ABSTRACT FROM AUTHOR]