Search

Context: Resources for medical education are becoming more constrained, whereas accountability in medical education is increasing. In this constrictive environment, medical schools need to consider and justify their selection procedures in terms of costs and benefits. To date, there have been no studies focusing on this aspect of selection. Objectives: We aimed to examine and compare the costs and benefits of two different approaches to admission into medical school: a tailored, multimethod selection process versus a lottery procedure. Our goal was to assess the relative effectiveness of each approach and to compare these in terms of benefits and costs from the perspective of the medical school. Methods: The study was conducted at Maastricht University Medical School, at which the selection process and a weighted lottery procedure ran in parallel for 3 years (2011–2013). The costs and benefits of the selection process were compared with those of the lottery procedure over three student cohorts throughout the Bachelor's programme. The extra costs of selection represented the monetary investment of the medical school in conducting the selection procedure; the benefits were derived from the increase in income generated by the prevention of dropout and the reductions in extra costs facilitated by decreases in the repetition of blocks and objective structured clinical examinations. Results: The tailor‐made selection procedure cost about €139 000 when extrapolated to a full cohort of students (n = 286). The lottery procedure came with negligible costs for the medical school. However, the average benefits of selection compared with the lottery system added up to almost €207 000. Conclusions: This study not only shows that conducting a cost–benefit comparison is feasible in the context of selection for medical school, but also that an 'expensive' selection process can be cost‐beneficial in comparison with an 'inexpensive' lottery system. We encourage other medical schools to examine the cost‐effectiveness of their own selection processes in relation to student outcomes in order to extend knowledge on this important topic. It's too expensive! Although feasibility is a common concern about selection practice, this study shows through cost‐benefit comparison that 'expensive' admissions processes can be cheaper than 'inexpensive' lotteries [ABSTRACT FROM AUTHOR]

Objectives This study set out to examine how much time students and teachers devote to different learning-oriented interactions during seminar sessions and students’ and teachers’ perceptions about the occurrence and desirability of these interactions. Methods Students and teachers participating in eight seminar group sessions in Year 4 of an undergraduate veterinary curriculum completed an 11-item questionnaire which asked them to rate, on a 5-point Likert scale, the frequency of occurrence and level of desirability of three learning-oriented types of interaction: exploratory questioning; cumulative reasoning, and handling of conflict about knowledge. The questionnaire also invited positive and negative responses to aspects of group interactions and an overall mark (1–10) for the seminars and group interactions. Four group sessions were video-recorded and analysed using a coding scheme. The amount of time devoted to the different interactions was calculated. Results Both students and teachers gave scores of 3.0–3.5 for frequency of occurrence of exploratory questioning and cumulative reasoning and < 3.0 for occurrence of handling of conflict about knowledge. The desired occurrences of all interaction types were significantly higher than the actual occurrences according to students and teachers. Teachers were responsible for the majority of the interactions (93%). The percentages of session time devoted to teacher-centred cumulative reasoning, exploratory questioning and handling of conflict about knowledge were 65.8%, 15.6% and 3.1%, respectively. Conclusions Group interactions in seminar groups are dominated by the posing of questions by teachers to students. The moderate occurrence of group interactions as perceived by students and teachers may be explained by the inadequate preparation of teachers and students to stimulate group interactions. [ABSTRACT FROM AUTHOR]

Context Small-group learning is advocated for enhancing higher-order thinking and the development of skills and attitudes. Teacher performance, group interaction and the quality of assignments have been shown to affect small-group learning in hybrid and problem-based curricula. Objectives This study aimed to examine the perceptions of student groups as to how teacher performance, group interaction and the quality of assignments are related to one another and to learning effects in seminars of 15–30 students in a hybrid curriculum. Methods We constructed a 28-item questionnaire and administered it to 639 students attending 32 seminars in Years 1–4 of an undergraduate veterinary curriculum. We performed factor analysis and reliability analysis of the questionnaire. We used correlation and regression analyses to explore the interactions of the four-factor model, with teacher performance, group interaction and quality of assignments as independent variables and the perceived learning effect of the seminars as the dependent variable. Results The response rate was 99%. Teacher performance (β = 0.78) and group interaction (− 0.28) significantly influenced the perceived learning effect. The total effect of the quality of assignments (through effects on teacher performance and group interaction) was 0.47. Discussion The strong relationship between teacher performance and learning effect suggests that students rely strongly on their teachers. The negative effect of group interaction may reflect poor alignment of teaching and assessment and poor organisation of group processes. This should be further examined. Comparative studies on seminars are also recommended. [ABSTRACT FROM AUTHOR]

Context One goal of undergraduate assessment is to test students’ (future) performance. In the area of skills testing, the objective structured clinical examination (OSCE) has been of great value as a tool with which to test a number of skills in a limited time, with bias reduction and improved reliability. But can OSCEs measure undergraduate internship expertise in basic clinical skills? Methods Undergraduate students ( n = 32) were given a questionnaire listing 182 basic clinical skills. We asked them to score the number of times they had performed each skill during their internships (a 12-month period in Year 6). We assessed the students at the end of Year 5 (before the start of their internships) and again at the start of Year 7 (undergraduate training takes 7 years in Belgium, with internships during Year 6), using a 14-station OSCE assessing basic clinical skills. Global ratings were used to score performance. The relationship between internship experience and the OSCE Year 7 score was analysed using a linear regression model, controlling for variation in OSCE scores from Year 5. A multi-level analysis was performed considering students as level-1 units and stations as level-2 units. Results Year 7 OSCE scores (post-internships) were not affected by the number of times that students practised basic medical skills during their internships. Discussion Scores on OSCEs do not seem to reflect clinical expertise acquired during internships. Other more integrated assessment methods may prove to be more valid for testing final undergraduate skills levels. [ABSTRACT FROM AUTHOR]

Objective To test hypotheses regarding the longitudinal effects of problem-based learning (PBL) and conventional learning relating to students’ appreciation of the curriculum, self-assessment of general competencies, summative assessment of clinical competence and indicators of career development. Methods The study group included 2 complete cohorts of graduates who were admitted to the medical curriculum in 1992 (conventional curriculum, n = 175) and 1993 (PBL curriculum, n = 169) at the Faculty of Medicine, University of Groningen, the Netherlands. Data were obtained from student records, graduates’ self-ratings and a literature search. Gender and secondary school grade point average (GPA) scores were included as moderator variables. Data were analysed by a stepwise multiple and logistic regression analysis. Results Graduates of the PBL curriculum scored higher on self-rated competencies. Contrary to expectations, graduates of the PBL curriculum did not show more appreciation of their curriculum than graduates of the conventional curriculum and no differences were found on clinical competence. Graduates of the conventional curriculum needed less time to find a postgraduate training place. No differences were found for scientific activities such as reading scientific articles and publishing in peer- reviewed journals. Women performed better on clinical competence than did men. Grade point average did not affect any of the variables. Conclusions The results suggest that PBL affects self-rated competencies. These outcomes confirm earlier findings. However, clinical competence measures did not support this finding. [ABSTRACT FROM AUTHOR]

Objective To determine the effectiveness of single-point benchmarking and longitudinal benchmarking for inter-school educational evaluation. Methods We carried out a mixed, longitudinal, cross-sectional study using data from 24 annual measurement moments (4 tests × 6 year groups) over 4 years for 4 annual progress tests assessing the graduation-level knowledge of all students from 3 co-operating medical schools. Participants included undergraduate medical students (about 5000) from 3 medical schools. The main outcome measures involved between-school comparisons of progress test results based on different benchmarking methods. Results Variations in relative school performance across different tests and year groups indicate instability and low reliability of single-point benchmarking, which is subject to distortions as a result of school-test and year group-test interaction effects. Deviations of school means from the overall mean follow an irregular, noisy pattern obscuring systematic between-school differences. The longitudinal benchmarking method results in suppression of noise and revelation of systematic differences. The pattern of a school’s cumulative deviations per year group gives a credible reflection of the relative performance of year groups. Conclusions Even with highly comparable curricula, single-point benchmarking can result in distortion of the results of comparisons. If longitudinal data are available, the information contained in a school’s cumulative deviations from the overall mean can be used. In such a case, the mean test score across schools is a useful benchmark for cross-institutional comparison. [ABSTRACT FROM AUTHOR]

Objective To determine whether items of progress tests used for inter-curriculum comparison favour students from the medical school where the items were produced (i.e. whether the origin bias of test items is a potential confounder in comparisons between curricula). Methods We investigated scores of students from different schools on subtests consisting of progress test items constructed by authors from the different schools. In a cross-institutional collaboration between 3 medical schools, progress tests are jointly constructed and simultaneously administered to all students at the 3 schools. Test score data for 6 consecutive progress tests were investigated. Participants consisted of approximately 5000 undergraduate medical students from 3 medical schools. The main outcome measure was the difference between the scores on subtests of items constructed by authors from 2 of the collaborating schools (subtest difference score). Results The subtest difference scores showed that students obtained better results on items produced at their own schools. This effect was more pronounced in Years 2–5 of the curriculum than in Year 1, and diminished in Year 6. Conclusions Progress test items were subject to origin bias. As a consequence, all participating schools should contribute equal numbers of test items if tests are to be used for valid and fair inter-curriculum comparisons. [ABSTRACT FROM AUTHOR]

Objective To determine the differential effects of a paper-based versus a web-based portfolio in terms of portfolio quality, user-friendliness and student motivation. Methods An experimental design was used to compare Year 1 medical students' reflective portfolios. The portfolios differed in presentation medium only (i.e. web-based versus paper-based). Content analysis, a student questionnaire and mentor interviews were used to evaluate portfolio quality, user-friendliness and student motivation. A total of 92 portfolios were scored independently by 2 raters using a portfolio quality-rating instrument. Results Portfolio structure, quality of reflection and quality of evidence showed no significant effects of presentation medium. Multi-level analysis showed a significant effect for student motivation: web-based portfolios scored 0.39 more than paper-based portfolios ( P < 0.05; effect size 0.76). The mentors reported no differences in portfolio quality, except that there were more visuals in web-based portfolios. Students spent significantly more time preparing the web-based than the paper-based portfolios (15.4 hours versus 12.2 hours; t = 2.1, P < 0.05; effect size 0.46). The 2 student groups did not differ significantly in terms of their satisfaction with the portfolio. The mentors perceived the web-based portfolios as more user-friendly. Conclusions The web-based portfolios were found to enhance students' motivation, were more user-friendly for mentors, and delivered the same content quality compared with paper-based portfolios. This suggests that web-based presentation may promote acceptance of portfolios by students and teachers alike. [ABSTRACT FROM AUTHOR]

Context We addressed the assessment of professional behaviour in tutorial groups by investigating students' perceptions of the frequency and impact of critical incidents that impede this assessment and 5 factors underlying these critical incidents. Methods A questionnaire asking students to rate the frequency and impact of 40 critical incidents relating to effective assessment of professional behaviour on a 5-point Likert scale was developed and sent to all undergraduate medical students in Years 2–4 of a 6-year undergraduate curriculum. Results The response rate was 70% ( n = 393). Important factors underlying critical incidents are: lack of effective interaction; lack of thoroughness; tutors' failure to confront students with unprofessional behaviour; lack of effort to find solutions, and lack of student motivation. Confirmatory factor analysis showed a good model fit. Because the relationship between frequency of occurrence and degree of impediment varies, the best information about the true impact of critical incidents and the underlying factors is provided by the product of frequency and degree of impediment. Frequency of occurrence remains stable and degree of impediment increases in Years 2–4. Conclusions The results of this study can be used to design and improve faculty development programmes aimed at improving assessment of professional behaviour. Training programmes should motivate tutors by providing background information as to why and how sound assessment of professional behaviour is to be performed and encourage tutors to confront students with and discuss all aspects of professional behaviour, as well as provide appropriate feedback. [ABSTRACT FROM AUTHOR]

Purpose To ascertain whether the grade point average (GPA) of school-leaving examinations is related to study success, career development and scientific performance. The problem of restriction of range was expected to be partially reduced due to the use of a national lottery system weighted in favour of students with higher GPAs. Method We studied the students ( n = 398) admitted to the Faculty of Medicine, University of Groningen, the Netherlands in 1982 and 1983. Data concerning drop-out and study progress were derived from the student administration. Data about career development were obtained from annual interviews with graduates ( n = 318) between 1993 and 2000. Literature searches yielded data concerning scientific performance. Multiple linear regression and logistic regression were used to analyse the data. The variables ‘gender’ and ‘cohort’ and their interaction were included in the analyses to account for variation in the general level of the dependent variable and the effect of GPA on the dependent variable. Results GPA scores had no effect on drop-out rate. High GPA scores were associated with significantly less time to graduation, more chance of a preferred placement for specialist training and higher scientific output. GPA was not related to income. Gender differences were found for study duration and scientific output. Women graduated earlier and published less. Conclusion The GPA of school-leaving examinations was found to be related to study success, career development and scientific performance. In this study the usual problem of restriction of range was shown to be absent. The weighted lottery procedure even resulted in an over-dispersion of candidates relative to the applicants. The resulting effect sizes were in agreement with those reported in other studies. [ABSTRACT FROM AUTHOR]

Context The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence. Methods We developed an instrument, the Portfolio Analysis Scoring Inventory, to examine the impact of form and content aspects on portfolio assessment. The Inventory consists of 15 items derived from interviews with experienced mentors, the literature, and the criteria for reflective competence used in the regular portfolio assessment procedure. Forty portfolios, selected from 231 portfolios for which ratings from the regular assessment procedure were available, were rated by 2 researchers, independently, using the Inventory. Regression analysis was used to estimate the correlation between the ratings from the regular assessment and those resulting from the Inventory items. Results Inter-rater agreement ranged from 0.46 to 0.87. The strongest predictor of the variance in the regular ratings was ‘quality of reflection’ (R 0.80; R2 66%). No further items accounted for a significant proportion of variance. Irrelevant items, such as writing style and lay-out, had negligible effects. Conclusions The absence of an impact of irrelevant criteria appears to support the validity of the portfolio assessment procedure. Further studies should examine the portfolio's validity for the assessment of clinical competence. [ABSTRACT FROM AUTHOR]

Objective To investigate the effects of a virtual learning environment (VLE) on group interaction and consultation of information resources during the preliminary phase, self-study phase and reporting phase of the problem-based learning process in an undergraduate medical curriculum. Methods A questionnaire was administered to 355 medical students in Years 1 and 2 to ask them about the perceived usefulness of a virtual learning environment that was created with Blackboard for group interaction and the use of learning resources. Results The students indicated that the VLE supported face-to-face interaction in the preliminary discussion and in the reporting phase but did not stimulate computer-mediated distance interaction during the self-study phase. They perceived that the use of multimedia in case presentations led to a better quality of group discussion than if case presentations were exclusively text-based. They also indicated that the information resources that were hyperlinked in the VLE stimulated the consultation of these resources during self-study, but not during the reporting phase. Conclusions Students indicated that the use of a VLE in the tutorial room and the inclusion of multimedia in case presentations supported processes of active learning in the tutorial groups. However, if we want to exploit the full potential of asynchronous computer-mediated communication to initiate in-depth discussion during the self-study phase, its application will have to be selective and deliberate. Students indicated that the links in the VLE to selected information in library repositories supported their learning. [ABSTRACT FROM AUTHOR]

For more than two decades the Medical School in Maastricht, the Netherlands, has used simulated patients (SPs) to provide students with opportunities to practise their skills in communication and physical examination. In this educational setting a student meets a SP in a videotaped session. Feedback by the SP to the student at the end of the session is considered an important educational feature. We found no instruments to assess individual SP performance during those sessions. To develop a valid, reliable and feasible instrument to evaluate the performance of SPs. The content of the instrument was validated through interviews with students, teachers and experts who are involved with SPs. They were asked to indicate key features of good SP performance. Based on the interviews, a written checklist was developed to measure individual SP performance. The instrument was evaluated in a regular SP session at the medical school, involving 152 students and their teachers. All interviewees considered the scale to be satisfactory and the instrument to be valid. The feasibility and reliability of the checklists were investigated using the data of 398 returned checklists. Cronbach's alpha was found to be 0.73. Generalizability analysis showed that 12 completed checklists were required to obtain a reliable assessment of one SP. The Maastricht Assessment of Simulated Patients (MaSP) appears to be a valid, reliable and feasible tool to assess the performance of SPs in an educational setting. [ABSTRACT FROM AUTHOR]

In anesthetized dogs with chronic, complete AV block we studied the characteristics of ventricular repolarization and refractoriness. Therefore, we determined: (1) steady-state values of ventricular effective refractory period (VERP), action potential duration (APD), and stimulus T interval (STI) before and after d-sotalol treatment at various pacing cycle lengths (PCLs); and (2) the dynamics of VERP, APD, and STI before and after d-sotalol treatment after the abrupt PCL decreases. VEHP, APD, and STI showed a normal frequency dependency. All three parameters increased significantly after d-sotalol administration. During steady-state and dynamic measurements, STI was always longer than APD and APD was always longer than VERP in an individual animal, irrespective of PCL and conditions. Standard deviations of steady-state and dynamic values indicated a considerable interindividual variation. However, the dynamics of VERP, APD, and STI after an abrupt decrease in PCL were highly correlated (linear regression analysis: r² ≥ 0.93). The best mathematical model to describe these dynamics was a bi-exponential model (r² ≥ 0.98) with a very short first and a much longer second time constant. We found that there was a very consistent relation between VERP, APD, and STI, not only during steady-state but also in the dynamic situation after various abrupt PCL decreases. This relation does not change after the administration of d-sotalol. Therefore, STI could be used to predict steady-state and dynamic values of VERP and APD. Since STI can be made available online in implantable pacing systems this could lead to the development of new features in these devices. [ABSTRACT FROM AUTHOR]

Context: For more than two decades the Medical School in Maastricht, the Netherlands, has used simulated patients (SPs) to provide students with opportunities to practise their skills in communication and physical examination. In this educational setting a student meets a SP in a videotaped session. Feedback by the SP to the student at the end of the session is considered an important educational feature. We found no instruments to assess individual SP performance during those sessions., Objective: To develop a valid, reliable and feasible instrument to evaluate the performance of SPs., Methods: The content of the instrument was validated through interviews with students, teachers and experts who are involved with SPs. They were asked to indicate key features of good SP performance. Based on the interviews, a written checklist was developed to measure individual SP performance. The instrument was evaluated in a regular SP session at the medical school, involving 152 students and their teachers., Main Outcomes: All interviewees considered the scale to be satisfactory and the instrument to be valid. The feasibility and reliability of the checklists were investigated using the data of 398 returned checklists. Cronbach's alpha was found to be 0.73. Generalizability analysis showed that 12 completed checklists were required to obtain a reliable assessment of one SP., Conclusions: The Maastricht Assessment of Simulated Patients (MaSP) appears to be a valid, reliable and feasible tool to assess the performance of SPs in an educational setting.