Search

Background Cultural differences might challenge the acceptance of the implementation of assessment formats that are developed in other countries. Acceptance of assessment formats is essential for its effectiveness; therefore, we explored the views of students and specialists on the practicality and impact on learning of these formats. This study was conducted to explore Indonesian students’ and specialists’ appreciation of the implementation of the Mini-Clinical Evaluation Exercise (Mini-CEX) in Indonesian clerkships. Methods This study was conducted at the Universitas Gadjah Mada, Indonesia. Participants were 52 students and 21 specialists in neurology and 78 students and 50 specialists in internal medicine. They were asked to complete a 19-item questionnaire that covered the characteristics of the mini-CEX such as its practicality, and the impact on learning and professional development. We used a Mann-Whitney U test to analyse the data. Results In total, 124 students (46 from neurology and 78 from internal medicine) and 38 specialists (13 from neurology and 25 from internal medicine) participated in this study. Students and specialists were positive about the practicality of the mini-CEX and the impact of this assessment format on learning and on professional development. The Mann-Whitney U test showed that there were no significant differences between students’ and specialists’ opinions on the mini-CEX, except for 2 items: specialists’ appreciation of direct observation (mean rank = 93.16) was statistically significantly higher than students’ appreciation of it (mean rank = 77.93; z = 2.065; p p Conclusion Students and specialists were positive about the mini-CEX in Indonesian clerkships, although it was developed and validated in another culture. We found only small differences between their appreciations, which could be explained by the patterns of specialist-student interaction in Indonesian culture as large power distance and low individualism country.

Background: Assessment is an essential part of the educational system. Usually, assessment is used to measure students’ knowledge, but it can also be used to drive students’ learning and behaviors....

Objectives: The aim was to examine main reasons for students' medical school choice and their relationship with students' characteristics and motivation during the students' medical study.Methods: In this multisite cross-sectional study, all Year-1 and Year-4 students who had participated in a selection procedure in one of the three Dutch medical schools included in the study were invited to complete an online survey comprising personal data, their main reason for medical school choice and standard, validated questionnaires to measure their strength of motivation (Strength of Motivation for Medical School-Revised) and autonomous and controlled type of motivation (Academic Self-regulation Questionnaire). Four hundred seventy-eight students participated. We performed frequency analyses on the reasons for medical school choice and regression analyses and ANCOVAs to study their associations with students' characteristics and motivation during their medical study.Results: Students indicated 'city' (Year-1: 24.7%, n=75 and Year-4: 36.0%, n=52) and 'selection procedure' (Year-1: 56.9%, n=173 and Year-4: 46.9%, n=68) as the main reasons for their medical school choice. The main reasons were associated with gender, age, being a first-generation university student, ethnic background and medical school, and no significant associations were found between the main reasons and the strength and type of motivation during the students' medical study.Conclusions: Most students had based their medical school choice on the selection procedure. If medical schools desire to achieve a good student-curriculum fit and attract a diverse student population aligning the selection procedure with the curriculum and taking into account various students' different approaches is important.

In medical school selection, non-cognitive performance in particular correlates with performance in clinical practice. It is arguable, therefore, that selection should focus on non-cognitive aspects despite the predictive value of prior cognitive performance for early medical school performance. The aim of this study at Radboud University Medical Center, the Netherlands, is to determine the effects of admitting students through an autonomous non-cognitive procedure on early medical school performance. We compared their performance to the performance of students selected through an autonomous cognitive selection procedure, enrolling in the Bachelor's curriculum simultaneously. 574 students (2013 and 2014 cohorts), admitted through non-cognitive selection (based on portfolio, CASPer and MMI, n = 135) or cognitive selection (curriculum sample selection, n = 439) were included in the study. We compared dropout rates, course credits and grades, using logistic and linear regression. The dropout rate was the highest in the non-cognitive selection group (p

Situational Judgement Tests (SJTs) are increasingly implemented in medical school admissions. In this paper, we investigate the effects of vocational interests, previous academic experience, gender and age on SJT performance. The SJT was part of the selection process for the Bachelor's degree programme in Medicine at University of Groningen, the Netherlands. All applicants for the academic year 2015-2016 were included and had to choose between learning communities Global Health (n = 126), Sustainable Care (n = 149), Intramural Care (n = 225), or Molecular Medicine (n = 116). This choice was used as a proxy for vocational interest. In addition, all graduate-entry applicants for academic year 2015-2016 (n = 213) were included to examine the effect of previous academic experience on performance. We used MANCOVA analyses with Bonferroni post hoc multiple comparisons tests for applicant performance on a six-scenario SJT. The MANCOVA analyses showed that for all scenarios, the independent variables were significantly related to performance (Pillai's Trace: 0.02-0.47, p

Not all students cope successfully with the demands of medical school, and students' struggles may result in study delay or dropout. To prevent these outcomes, medical schools need to identify students who are experiencing academic difficulties and provide them with timely interventions through access to support programs. Although the importance of early identification and intervention is well recognized, less is known about successful strategies for identifying and supporting struggling students.Building on the literature and their own empirical findings, the authors propose an integrated, school-wide model for medical student success comprising a continuum of academic and behavioral support. This Four-Tier Continuum of Academic and Behavioral Support (4T-CABS) model focuses on improving both academic and behavioral outcomes by offering support for students at four levels, which range from adequate instruction for all, to targeted small-group interventions, to individualized support, and also include exit support for students who might be better off in another degree program.Additionally, medical schools should provide both academic and behavioral support; set high, yet realistic expectations and clearly communicate these to students; and intervene early, which requires timely identification of at-risk students who would benefit from the different types and tiers of support. Finally, interventions should be evidence based and fit the needs of the identified groups of students. The authors argue that adopting the core principles of the 4T-CABS model will enable medical schools to maximize academic engagement and performance for all students.

In the Netherlands, students are admitted to medical school through (1) selection, (2) direct access by high pre-university Grade Point Average (pu-GPA), (3) lottery after being rejected in the selection procedure, or (4) lottery. At Radboud University Medical Center, 2010 was the first year we selected applicants. We designed a procedure based on tasks mimicking the reality of early medical school. Applicants took an online course followed by an on-site exam, resembling courses and exams in early medical school. Based on the exam scores, applicants were selected or rejected. The aim of our study is to determine whether curriculum sample selection explains performance in medical school and is preferable compared to selection based on performance in secondary school. We gathered data on the performance of students of three consecutive cohorts (2010-2012, N = 954). We compared medical school performance (course credits and grade points) of selected students to the three groups admitted in other ways, especially lottery admissions. In regression analyses, we controlled for out of context cognitive performance by adjusting for pu-GPA. Selection-admitted students outperformed lottery-admitted students on most outcome measures, unadjusted as well as adjusted for pu-GPA (p aecurrency 0.05). They had higher grade points than non-selected lottery students, both unadjusted and adjusted for pu-GPA (p aecurrency 0.025). Adjusted for pu-GPA, selection-admitted students and high-pu-GPA students performed equally. We recommend this selection procedure as it adds to secondary school cognitive performance for the general population of students, is efficient for large numbers of applicants and not labour-intensive.

ContextStudents tend to postpone preparation for a test until the test is imminent, which raises various risks associated with cramming' behaviours, including that for suboptimal learning. Cumulative assessment utilises spaced testing to stimulate students to study more frequently and to prevent procrastination. This randomised controlled study investigated how cumulative assessment affects time spent on self-study and test performance compared with end-of-course assessment.MethodsA total of 78 undergraduate medical students in a Year2 pre-clinical course were randomly assigned to either of two conditions. Students in the cumulative assessment condition were assessed in weeks 4, 8 and 10. Students in the end-of-course assessment condition were assessed in week10 only. Each week, students reported the number of hours they spent on self-study.ResultsStudents in the cumulative assessment condition (n=25) spent significantly more time on self-study than students in the end-of-course assessment condition (n=37) in all weeks of the course except weeks 5, 9 and 10. Overall, the cumulative assessment group spent 69hours more on self-study during the course than did the control group, although the control group spent 7hours more in studying during the final week of the course than did the cumulative assessment group. Students in the cumulative assessment condition scored slightly higher on questions concerning the content of the last part of the course.ConclusionsCumulative assessment encourages students to distribute their learning activities over a course, which leaves them more opportunity to study the content of the last part of the course prior to the final examination. There was no evidence for a short-term effect of cumulative assessment on overall knowledge gain. We hypothesise that larger positive effects might be found if retention were to be measured in the long term.Discuss ideas arising from the article at discuss.

Marieke de Visser,1 Roland F Laan,1 Rik Engbers,1 Janke Cohen-Schotanus,2 Cornelia Fluit1 1Department for Research in Learning and Education, Radboudumc Health Academy, Radboud University Medical Center, Nijmegen, The Netherlands; 2Center for Research and Innovation in Medical Education, University of Groningen and University Medical Center Groningen, Groningen, The Netherlands Introduction: Research on selection for medical school does not explore selection as a learning experience, despite growing attention for the learning effects of assessment in general. Insight in the learning effects allows us to take advantage of selection as an inclusive part of medical students’ learning process to become competent professionals. The aims of this study at Radboud University Medical Center, the Netherlands, were 1) to determine whether students have learning experiences in the selection process, and, if so, what experiences; and 2) to understand what students need in order to utilize the learning effects of the selection process at the start of the formal curriculum. Materials and methods: We used focus groups to interview 30 students admitted in 2016 about their learning experiences in the selection process. Thematic analysis was used to explore the outcomes of the interviews and to define relevant themes. Results: In the selection process, students learned about the curriculum, themselves, their relation to others, and the profession they had been selected to enter, although this was not explicitly perceived as learning. Students needed a connection between selection and the curriculum as well as feedback to be able to really use their learning experiences for their further development. Discussion: Medical school selection qualifies as a learning experience, and students as well as medical schools can take advantage of this. We recommend a careful design of the selection procedure, integrating relevant selection learning experiences into the formal curriculum, providing feedback and explicitly approaching the selection and the formal curriculum as interconnected contributors to students’ development. Keywords: admission, assessment, medical school qualitative research, students, curriculum

In medical education, student distress is known to hamper learning and professional development. To address this problem, recent studies aimed at helping students cope with stressful situations. Undergraduate students in clinical practice frequently use experiences of surrounding peers to estimate their abilities to master such challenging situations. This use of the experiences of others, known as social comparison, may affect student distress both positively and negatively. To find characteristics of a beneficial use of social comparison, we examined differences in comparison behaviours between students expressing low and high levels of distress. The participants in our study, response rate 93 % (N = 301/321), were all medical students in their first year in clinical practice. They completed the General Health Questionnaire (GHQ-12) to measure distress, and three separate questionnaires to measure: (1) orientation to comparison, (2) motive for comparison, and (3) interpretation of comparison. Differences were analysed using multivariate analysis of variance. Although all students were oriented towards social comparison, the analyses showed that this orientation was less apparent among low-distress students. Besides, the low-distress students were less inclined to use motives indicative for comparisons with peers perceived as performing worse and were less negative in the interpretations of their comparisons. As social comparison is frequently used among all students, we recommend to make them aware of their comparison behaviours and inform them about the pros and cons of the distinguished aspects of the comparison process.

Introduction Many training programmes in postgraduate medical education (PGME) have introduced competency frameworks, but the effects of this change on preparedness for practice are unknown. Therefore, we explored how elements of competency-based programmes in PGME (educational innovations, attention to competencies and learning environment) were related to perceived preparedness for practice among new consultants. Methods A questionnaire was distributed among 330 new consultants. Respondents rated how well their PGME training programme prepared them for practice, the extent to which educational innovations (portfolio, Mini-CEX) were implemented, and how much attention was paid to CanMEDS competencies during feedback and coaching, and they answered questions on the learning environment and general self-efficacy. Multiple regression and mediation analyses were used to analyze data. Results The response rate was 43 % (143/330). Controlling for self-efficacy and gender, the learning environment was the strongest predictor of preparedness for practice (B = 0.42, p

Background: Progress testing is an assessment tool used to periodically assess all students at the end-of-curriculum level. Because students cannot know everything, it is important that they recognize their lack of knowledge. For that reason, the formula-scoring method has usually been used. However, where partial knowledge needs to be taken into account, the number-right scoring method is used. Research comparing both methods has yielded conflicting results. As far as we know, in all these studies, Classical Test Theory or Generalizability Theory was used to analyze the data. In contrast to these studies, we will explore the use of the Rasch model to compare both methods.Methods: A 2 x 2 crossover design was used in a study where 298 students from four medical schools participated. A sample of 200 previously used questions from the progress tests was selected. The data were analyzed using the Rasch model, which provides fit parameters, reliability coefficients, and response option analysis.Results: The fit parameters were in the optimal interval ranging from 0.50 to 1.50, and the means were around 1.00. The person and item reliability coefficients were higher in the number-right condition than in the formula-scoring condition. The response option analysis showed that the majority of dysfunctional items emerged in the formula-scoring condition.Conclusions: The findings of this study support the use of number-right scoring over formula scoring. Rasch model analyses showed that tests with number-right scoring have better psychometric properties than formula scoring. However, choosing the appropriate scoring method should depend not only on psychometric properties but also on self-directed test-taking strategies and metacognitive skills.

Many different medical school selection processes are used worldwide. In this paper, we examine the effect of (1) participation, and (2) selection in a voluntary selection process on study performance. We included data from two cohorts of medical students admitted to Erasmus MC, Rotterdam and VUmc, Amsterdam, The Netherlands and compared them to previously published data from Groningen medical school, The Netherlands. All included students were admitted based on either (1) a top pre-university grade point average, or (2) a voluntary selection process, or (3) weighted lottery. We distinguished between lottery-admitted students who had participated in the voluntary selection process and had been rejected, and lottery-admitted students who had not participated. Knowledge test scores, study progress, and professionalism scores were examined using ANCOVA modelling, logistic regression, and Bonferroni post hoc multiple-comparison tests, controlling for gender and cohort. For written test grades, results showed a participation effect at Groningen medical school and Erasmus MC (p

Medical schools seek ways to improve their admissions strategies, since the available methods prove to be suboptimal for selecting the best and most motivated students. In this multi-site cross-sectional questionnaire study, we examined the value of (different) selection procedures compared to a weighted lottery procedure, which includes direct admission based on top pre-university grade point averages (ae8 out of 10; top-pu-GPA). We also considered whether students had participated in selection, prior to being admitted through weighted lottery. Year-1 (pre-clinical) and Year-4 (clinical) students completed standard validated questionnaires measuring quality of motivation (Academic Self-regulation Questionnaire), strength of motivation (Strength of Motivation for Medical School-Revised) and engagement (Utrecht Work Engagement Scale-Student). Performance data comprised GPA and course credits in Year-1 and clerkship performance in Year-4. Regression analyses were performed. The response rate was 35% (387 Year-1 and 273 Year-4 students). Top-pu-GPA students outperformed selected students. Selected Year-1 students reported higher strength of motivation than top-pu-GPA students. Selected students did not outperform or show better quality of motivation and engagement than lottery-admitted students. Participation in selection was associated with higher engagement and better clerkship performance in Year-4. GPA, course credits and strength of motivation in Year-1 differed between students admitted through different selection procedures. Top-pu-GPA students perform best in the medical study. The few and small differences found raise questions about the added value of an extensive selection procedure compared to a weighted lottery procedure. Findings have to be interpreted with caution because of a low response rate and small group sizes.

Background: Job satisfaction is essential for physicians' well-being and patient care. The work ethic of long days and hard work that has been advocated for decades is acknowledged as a threat for physicians' job satisfaction, well-being, and patient safety. Our aim was to determine the actual and preferred job size of physicians and to investigate how these and the differences between them influence physicians' job satisfaction.Method: Data were retrieved from a larger, longitudinal study among physicians starting medical training at Groningen University in 1982/83/92/93 (N = 597). Data from 506 participants (85%) were available for this study. We used regression analysis to investigate the influence of job size on physicians' job satisfaction (13 aspects) and ANOVA to examine differences in job satisfaction between physicians wishing to retain, reduce or increase job size.Results: The majority of the respondents (57%) had an actual job size less than 1.0 FTE. More than 80% of all respondents preferred not to work full-time in the future. Respondents' average actual and preferred job sizes were.85 FTE and .81 FTE, respectively. On average, respondents who wished to work less (35% of respondents) preferred a job size reduction of 0.18 FTE and those who wished to work more (12%) preferred an increase in job size of 0.16 FTE. Job size influenced satisfaction with balance work-private hours most (beta = -.351). Physicians who preferred larger job sizes were - compared to the other groups of physicians - least satisfied with professional accomplishments.Conclusions: A considerable group of physicians reported a gap between actual and preferred job size. Realizing physicians' preferences as to job size will hardly affect total workforce, but may greatly benefit individual physicians as well as their patients and society. Therefore, it seems time for a shift in work ethic.

Background: Various feedback characteristics have been suggested to positively influence student learning. It is not clear how these feedback characteristics contribute to students' perceived learning value of feedback in cultures classified low on the cultural dimension of individualism and high on power distance. This study was conducted to validate the influence of five feedback characteristics on students' perceived learning value of feedback in an Indonesian clerkship context.Methods: We asked clerks in Neurology (n = 169) and Internal Medicine (n = 132) to assess on a 5-point Likert scale the learning value of the feedback they received. We asked them to record whether the feedback provider (1) informed the student what went well, (2) mentioned which aspects of performance needed improvement, (3) compared the student's performance to a standard, (4) further explained or demonstrated the correct performance, and (5) prepared an action plan with the student to improve performance. Data were analyzed using multilevel regression.Results: A total of 250 students participated in this study, 131 from Internal Medicine (response rate 99%) and 119 from Neurology (response rate 70%). Of these participants, 225 respondents (44% males, 56% females) completed the form and reported 889 feedback moments. Students perceived feedback as more valuable when the feedback provider mentioned their weaknesses (beta = 0.153, p Conclusions: In Indonesia, we could validate four out of the five characteristics for effective feedback. We argue that our findings relate to culture, in particular to the levels of individualism and power distance. The recognized characteristics of what constitutes effective feedback should be validated across cultures.

ObjectivesThis study was conducted to: (i) analyse whether students admitted to one medical school based on top pre-university grades, a voluntary multifaceted selection process, or lottery, respectively, differed in study performance; (ii) examine whether students who were accepted in the multifaceted selection process outperformed their rejected peers, and (iii) analyse whether participation in the multifaceted selection procedure was related to performance.MethodsWe examined knowledge test and professionalism scores, study progress and dropout in three cohorts of medical students admitted to the University of Groningen, the Netherlands in 2009, 2010 and 2011 (n=1055). We divided the lottery-admitted group into, respectively, students who had not participated and students who had been rejected in the multifaceted selection process. We used ancova modelling, logistic regression and Bonferroni post hoc multiple-comparison tests and controlled for gender and cohort.ResultsThe top pre-university grade group achieved higher knowledge test scores and more Year1 course credits than all other groups (pConclusionsA top pre-university grade point average was the best predictor of performance. For so-called non-academic performance, the multifaceted selection process was efficient in identifying applicants with suitable skills. Participation in the multifaceted selection procedure seems to be predictive of higher performance. Further research is needed to assess whether our results are generalisable to other medical schools.

In this longitudinal study, we investigated the relationship between physicians' prior achievements (before, during and after medical school) and job satisfaction, and tested the two lines of reasoning that prior achievements influence job satisfaction positively or negatively, respectively. The participants were graduates who started their medical training in 1982 (n = 147), 1983 (n = 154), 1992 (n = 143) and 1993 (n = 153). We operationalised job satisfaction as satisfaction (on a 10-point scale) with 13 cognitive, affective and instrumental aspects of the participants' jobs. The measures of achievement before, during and after medical school included pre-university grade point average, study progress and a residency position in the specialty of first choice, respectively. We included the effect of curriculum type (problem-based learning versus traditional), gender and years of experience as moderator variables. Higher achievers before and during medical school were more satisfied about their income (beta = .152, p

The influence of problem-based learning (PBL) and open-book tests on long-term knowledge retention is unclear and subject of discussion. Hypotheses were that PBL as well as open-book tests positively affect long-term knowledge retention. Four progress test results of fifth and sixth-year medical students (n = 1,648) of three medical schools were analyzed. Two schools had PBL driven curricula, and the third one had a traditional curriculum (TC). One of the PBL schools (PBLob) used a combination of open-book (assessing backup knowledge) and closed-book tests (assessing core knowledge); the other two schools (TC and PBLcb) only used closed-book tests. The items of the progress tests were divided into core and backup knowledge. T tests (with Bonferroni correction) were used to analyze differences between curricula. PBL students performed significantly better than TC students on core knowledge (average effect size (av ES) = 0.37-0.74) and PBL students tested with open-book tests scored somewhat higher than PBL students tested without such tests (av ES = 0.23-0.30). Concerning backup knowledge, no differences were found between the scores of the three curricula. Students of the two PBL curricula showed a substantially better long-term knowledge retention than TC students. PBLob students performed somewhat better on core knowledge than PBLcb students. These outcomes suggest that a problem-based instructional approach in particular can stimulate long-term knowledge retention. Distinguishing knowledge into core and backup knowledge and using open-book tests alongside closed-book tests could enhance long-term core knowledge retention.

OBJECTIVES: Traditional student feedback questionnaires are imperfect course evaluation tools, largely because they generate low response rates and are susceptible to response bias. Preliminary research suggests that prediction-based methods of course evaluation - in which students estimate their peers' opinions rather than provide their own personal opinions - require significantly fewer respondents to achieve comparable results and are less subject to biasing influences. This international study seeks further support for the validity of these findings by investigating: (i) the performance of the prediction-based method, and (ii) its potential for bias. METHODS: Participants (210 Year 1 undergraduate medical students at McGill University, Montreal, Quebec, Canada, and 371 Year 1 and 385 Year 3 undergraduate medical students at the University Medical Center Groningen [UMCG], University of Groningen, Groningen, the Netherlands) were randomly assigned to complete course evaluations using either the prediction-based or the traditional opinion-based method. The numbers of respondents required to achieve stable outcomes were determined using an iterative process. Differences between the methods regarding the number of respondents required were analysed using t-tests. Differences in evaluation outcomes between the methods and between groups of students stratified by four potentially biasing variables (gender, estimated general level of achievement, expected test result, satisfaction after examination completion) were analysed using multivariate analysis of variance (manova). RESULTS: Overall response rates in the three student cohorts ranged from 70% to 94%. The prediction-based method required significantly fewer respondents than the opinion-based method (averages of 26-28 and 67-79 respondents, respectively) across all samples (p < 0.001), whereas the outcomes achieved were fairly similar. Bias was found in four of 12 opinion-based condition comparisons (three sites, four variables), and in only one comparison in the prediction-based condition. CONCLUSIONS: Our study supports previous findings that prediction-based methods require significantly fewer respondents to achieve results comparable with those obtained through traditional course evaluation methods. Moreover, our findings support the hypothesis that prediction-based responses are less subject to bias than traditional opinion-based responses. These findings lend credence to prediction-based as an accurate and efficient method of course evaluation.

Context During clinical rotations, students move from one clinical situation to another. Questions exist about students’ strategies for coping with these transitions. These strategies may include a process of social comparison because in this context it offers the student an opportunity to estimate his or her abilities to master a novel rotation. These estimates are relevant for learning and performance because they are related to self-efficacy. We investigated whether student estimates of their own future performance are influenced by the performance level and gender of the peer with whom the student compares him- or herself. Methods We designed an experimental study in which participating students (n = 321) were divided into groups assigned to 12 different conditions. Each condition entailed a written comparison situation in which a peer student had completed the rotation the participant was required to undertake next. Differences between conditions were determined by the performance level (worse, similar or better) and gender of the comparison peer. The overall grade achieved by the comparison peer remained the same in all conditions. We asked participants to estimate their own future performance in that novel rotation. Differences between their estimates were analysed using analysis of variance (anova). Results Students’ estimates of their future performance were highest when the comparison peer was presented as performing less well and lowest when the comparison peer was presented as performing better (p

Background: Research from outside the medical field shows that leadership behaviours influence job satisfaction. Whether the same is true for the medical training setting needs to be explored. The aim of this study was to investigate the influence of residents' overall appreciation of their supervisor's leadership and observation of specific supervisor leadership behaviours on job satisfaction.Methods: We invited residents (N = 117) to rate how often they observed certain task and relation-oriented leadership behaviours in their supervisor and overall appreciation of their supervisor's leadership. Furthermore, they rated their satisfaction with 13 different aspects of their jobs on a 10-point scale. Using exploratory factor analysis we identified four factors covering different types of job satisfaction aspects: personal growth, autonomy, affective, and instrumental job satisfaction aspects. Influence of overall appreciation for supervisor leadership and observation of certain leadership behaviours on these job satisfaction factors were analysed using multiple regression analyses.Results: The affective aspects of job satisfaction were positively influenced by overall appreciation of leadership (B = 0.792, p = 0.017), observation of specific instructions (B = 0.972, p = 0.008) and two-way communication (B = 1.376, p = 0.008) and negatively by mutual decision-making (B = -1.285, p = 0.007). No effects were found for the other three factors of job satisfaction.Conclusions: We recommend that supervisors become more aware of whether and how their behaviours influence residents' job satisfaction. Especially providing specific instructions and using two-way communication seem important to help residents deal with their insecurities and to offer them support.

Introduction: Residents benefit from regular, high quality feedback on all CanMEDS roles during their training. However, feedback mostly concerns Medical Expert, leaving the other roles behind. A feedback system was developed to guide supervisors in providing feedback on CanMEDS roles. We analyzed whether feedback was provided on the intended roles and explored differences in quality of written feedback.Methods: In the feedback system, CanMEDS roles were assigned to five authentic situations: Patient Encounter, Morning Report, On-call, CAT, and Oral Presentation. Quality of feedback was operationalized as specificity and inclusion of strengths and improvement points. Differences in specificity between roles were tested with Mann-Whitney U tests with a Bonferroni correction (=0.003).Results: Supervisors (n=126) provided residents (n=120) with feedback (591 times). Feedback was provided on the intended roles, most frequently on Scholar (78%) and Communicator (71%); least on Manager (47%), and Collaborator (56%). Strengths (78%) were mentioned more frequently than improvement points (52%), which were lacking in 40% of the feedback on Manager, Professional, and Collaborator. Feedback on Scholar was more frequently (p=0.000) and on Reflective Professional was less frequently (p=0.003) specific.Discussion and conclusion: Assigning roles to authentic situations guides supervisors in providing feedback on different CanMEDS roles. We recommend additional supervisor training on how to observe and evaluate the roles.

Medical Education 2012: 46:575585 Context Research from numerous medical schools has shown that students from ethnic minorities underperform compared with those from the ethnic majority. However, little is known about why this underperformance occurs and whether there are performance differences among ethnic minority groups. Objectives This study aimed to investigate underperformance across ethnic minority groups in undergraduate pre-clinical and clinical training. Methods A longitudinal prospective cohort study of progress on a 6-year undergraduate medical course was conducted in a Dutch medical school. Participants included 1661 Dutch and 696 non-Dutch students who entered the course over a consecutive 6-year period (20022007). Main outcome measures were performance in Year 1 and in the pre-clinical and clinical courses. Odds ratios (ORs) with 95% confidence intervals (CIs) were estimated by logistic regression analysis for ethnic subgroups (Surinamese/Antillean, Turkish/Moroccan/African, Asian, Western) compared with Dutch students, adjusted for age, gender, pre-university grade point average (pu-GPA), additional socio-demographic variables (first-generation immigrant, urban background, first-generation university student, first language, medical doctor as parent) and previous performance at medical school. Results Compared with Dutch students, Surinamese and Antillean students specifically underperformed in the Year 1 course (pass rate: 37% versus 64%; adjusted OR 0.40, 95% CI 0.270.60) and the pre-clinical course (pass rate: 19% versus 41%; adjusted OR 0.57, 95% CI 0.350.93). On the clinical course all non-Dutch subgroups were less likely than Dutch students to receive a grade of = 8.0 (at least three of five grades: 5477% versus 88%; adjusted ORs: 0.170.45). Conclusions Strong ethnic disparities exist in medical school performance even after adjusting for age, gender, pu-GPA and socio-demographic variables. More subjective grading cannot be ruled out as a cause of lower grades in clinical training, but other possible explanations should be studied further to mitigate the disparities.

The extent to which students feel involved in their education positively influences academic achievement. Individual student–faculty meetings can foster student involvement. To be effective, faculty acknowledgement of the benefit of these meetings is a prerequisite. The aim of this study was to explore faculty perceptions of individual student–faculty meetings. In addition we investigated students’ perceptions. As part of the undergraduate programme, mandatory individual intake and follow-up meetings between first-year medical students (n = 425) and senior faculty members (n = 34) have been implemented from 2009 onwards. We administered a questionnaire on faculty perceptions of the benefit and impact of intake meetings. Subsequently, after both meetings had been held, strong and weak points of the mandatory programme were explored using open-ended questions. Students’ perceptions were investigated by open-ended questions as a part of the curriculum evaluation process. Faculty enjoyed the meetings (90 %), perceived the meetings to be beneficial (74 %) and expected a positive effect on student involvement (74 %). Faculty appreciated the opportunity to give advice tailored to students’ personal needs and levels of performance. The students appreciated the meetings and the attention given to their personal situation and study progress. Faculty and student appreciation of the meetings seems to support the assumption that the individual meetings increase students’ social and academic involvement. Further research should focus on the impact of individual student–faculty meetings on students’ learning behaviours.

CONTEXT Medical students often fail to finish medical school within the designated time. An academic dismissal (AD) policy aims to enforce satisfactory progress and to enable early identification and timely support or referral of struggling students. In this study, we assessed whether the implementation of an AD policy improved study progress in the first 2 years of medical school. Additionally, we analysed its effect on the help-seeking behaviour of struggling students. METHODS We compared two AD cohorts (entering in 2005 and 2006, respectively) and two non-AD cohorts (entering in 2003 and 2004, respectively) on dropout rates, Year 1 curriculum completion rates and the percentage of students with an optimal study rate (i.e. all modules completed) at 1 and 2 years after enrolment. We also measured the effect on study progress of attending the support meetings offered. RESULTS The AD (n = 809) and non-AD cohorts (n = 809) did not differ significantly in dropout rate at 5 months, in Year 1 completion rate at 2 years and in the percentage of optimally performing students at 1 year after enrolment. At 2 years after enrolment, more students from the AD cohorts had left and more non-AD students demonstrated optimal performance, but effect sizes (ESs) for these differences were small. Voluntary support at 4 months was attended by AD students more often than by non-AD students (68.9% versus 39.8%; chi(2)((1)) = 43.95, p

Context Teaching students in reflection calls for specific teacher competencies. We developed and validated a rating scale focusing on Student perceptions of their Teachers' competencies to Encourage Reflective Learning in small Groups (STERLinG). Methods We applied an iterative procedure to reduce an initial list of 241 items pertaining to teacher competencies to 47 items. Subsequently, we validated the instrument in two successive studies. In the first study, we invited 679 medical and speech and language therapy students to assess the teachers of their professional development groups with the STERLinG. Principal components analysis (PCA) with varimax rotation was used to investigate the internal structure of the instrument. In the second study, which involved 791 medical, dental, and speech and language therapy students, we performed a confirmatory factor analysis using the oblique multiple group (OMG) method to verify the original structure. Results In Study 1, 463 students (68%) completed the STERLinG. The PCA yielded three components: Supporting self-insight; Creating a safe environment, and Encouraging self-regulation. The final 36-item instrument explained 44.3% of the variance and displayed high reliability with alpha-values of 0.95 for the scale, and 0.91, 0.86 and 0.86 for the respective subscales. In Study 2, 501 students (63%) completed the STERLinG. The OMG confirmed the original structure of the STERLinG and explained 53.0% of the total variance with high alpha-values of 0.96 for the scale, and 0.94, 0.90 and 0.90 for the respective subscales. Conclusions The STERLinG is a practical and valid tool for gathering student perceptions of their teachers' competencies to facilitate reflective learning in small groups considering its stable structure, the correspondence of the STERLinG structure with educational theories and the coverage of important domains of reflection. In addition, our study may provide a theoretical framework for the practice of and research into reflective learning.

Background: The transition from preclinical to clinical training is perceived as stressful with a high workload being the main difficulty. To ease this transition, we implemented a dual learning year, where just-in-time skills training and clerkships alternated.Aims: To examine the effect of the dual learning year on students' stress and perceptions of workload and skills level, and to compare these data with a baseline measurement from a curriculum in which skills training was provided in advance of clerkships.Method: During the first Master year, students completed four rotations in which 5 weeks of skills training and 5-week clerkships alternated. In the second clerkship week of each rotation, students (n = 476) completed questionnaires measuring their experienced workload, perceived skills level and stress. Analysis of variance was used for trend analysis and to determine differences with the baseline measurement (n 83).Results: During the dual learning year, 'experienced workload' decreased (F(1,471) = 9.408, p Conclusions: Compared to the baseline measurement and the results of earlier studies on transition difficulties, alternating just-in-time skills training and clerkships seem to ease the transition from preclinical to clinical training.

Medical Education 2010: 44: 884–891 Objectives The ability to master discipline-specific knowledge is one of the competencies medical students must acquire. In this context, ‘mastering’ means being able to recall and apply knowledge. A way to assess this competency is to use both open- and closed-book tests. Student performance on both tests can be influenced by the way the student processes information. Deep information processing is expected to influence performance positively. The personal preferences of students in relation to how they process information in general (i.e. their level of need for cognition) may also be of importance. In this study, we examined the inter-relatedness of deep learning, need for cognition and preparation time, and scores on open- and closed-book tests. Methods This study was conducted at the University Medical Centre Groningen. Participants were Year 2 students (n = 423). They were asked to complete a questionnaire on deep information processing, a scale for need for cognition on a questionnaire on intellectualism and, additionally, to write down the time they spent on test preparation. We related these measures to the students’ scores on two tests, both consisting of open- and closed-book components and used structural equation modelling to analyse the data. Results Both questionnaires were completed by 239 students (57%). The results showed that need for cognition positively influenced both open- and closed-book test scores (β-coefficients 0.05 and 0.11, respectively). Furthermore, study outcomes measured by open-book tests predicted closed-book test results better than the other way around (β-coefficients 0.72 and 0.11, respectively). Conclusions Students with a high need for cognition performed better on open- as well as closed-book tests. Deep learning did not influence their performance. Adding open-book tests to the regularly used closed-book tests seems to improve the recall of knowledge that has to be known by heart. Need for cognition may provide a valuable addition to existing theories on learning.

The potential richness of the feedback for learners and teachers is one of the educational advantages of progress tests (PTs). Every test administration yields information on a student's knowledge level in each sub-domain of the test (cross-sectional information), and it adds a next point to the corresponding knowledge growth curve (longitudinal information). Traditional paper-based feedback has severe limitations and requires considerable effort from the learners to give meaning to the data. We reasoned that the PT data should be flexibly accessible in all pathways and with any available comparison data, according to the personal interest of the learner. For that purpose, a web-based tool (Progress test Feedback, the ProF system) was developed. This article presents the principles and features of the generated feedback and shows how it can be used. In addition to enhancement of the feedback, the ProF database of longitudinal PT-data also provides new opportunities for research on knowledge growth, and these are currently being explored.

Teachers involved in test development usually prefer criterion-referenced standard setting methods using panels. Since expert panels are costly, standards are often set by a pre-fixed percentage of questions answered correctly or norm-referenced methods aimed at ranking examinees.To discuss the (dis)advantages of commonly used criterion and norm-referenced methods and present a new compromise method: standards based on a fixed cut-off score using the best scoring students as reference point.Historical data from 54 Maastricht (norm-referenced) and 52 Groningen (criterion-referenced) tests were used to demonstrate huge discrepancies and variability in cut-off scores and failure rates. Subsequently, the compromise model - known as Cohen's method - was applied to the Groningen tests.The Maastricht norm-referenced method led to a large variation in required cut-off scores (15-46%), but a stable failure rate (about 17%). The Groningen method with a conventional, pre-fixed standard of 60% led to a large variation in failure rates (17-97%). The compromise method reduced variation in required cut-off scores as well as failure rates.Both the criterion and norm-referenced standards, used in practice, have disadvantages. The proposed compromise model reduces the disadvantages of both methods and is considered more acceptable. Last but not least, compared to standard setting methods using panels, this method is affordable.

Introduction The development of professional behaviour is an important objective for students in Health Sciences, with reflective skills being a basic condition for this development. Literature describes a variety of methods giving students opportunities and encouragement for reflection. Although the literature states that learning and working together in peer meetings fosters reflection, these findings are based on experienced professionals. We do not know whether participation in peer meetings also makes a positive contribution to the learning experiences of undergraduate students in terms of reflection. Aim The aim of this study is to gain an understanding of the role of peer meetings in students' learning experiences regarding reflection. Method A phenomenographic qualitative study was undertaken. Students' learning experiences in peer meetings were analyzed by investigating the learning reports in students' portfolios. Data were coded using open coding. Results The results indicate that peer meetings created an interactive learning environment in which students learned about themselves, their skills and their abilities as novice professionals. Students also mentioned conditions for a well-functioning group. Conclusion The findings indicate that peer meetings foster the development of reflection skills as part of professional behaviour.

ContextStudents' perceptions of their learning environment are of great importance to their learning process. In this study we assessed the time allocated by students to clerkship activities and the relationship between students' allocations of time and their perceptions of the quality of their clinical learning environment.MethodsParticipants were 133 undergraduate students from eight hospitals taking part in four clerkship rotations. All students recorded the time they spent on eight clerkship activities over 2 weeks and completed the Postgraduate Hospital Educational Environment Measure (PHEEM). Partial correlation analyses were undertaken to examine the relationship between the amount of time students spent on each activity and their PHEEM scores.ResultsStudents spent nearly 8 hours a day on clerkship activities. Most time was spent on observing doctors (40%), followed by participating in consultations without direct supervision (12%). The time students spent on observing doctors (r = 0.206, P ConclusionsThe results suggest that the time spent on activities involving direct patient contact is positively related to students' perceptions of the quality of their learning environment. None of the activities were significantly negatively related to the students' perceptions of their clinical learning environment. Future research should examine the optimal time allocations required to enhance the perceived quality of the clinical learning environment.

Background: Previous research revealed relationships between learning strategies and knowledge acquisition. During clerkships, however, students' focus widens beyond mere knowledge acquisition as they further develop overall competence. This shift in focus can influence learning strategy use.Aim: We explored which learning strategies were used during clerkships and their relationship to clinical performance.Methods: Participants were 113 (78%) clerks at the university hospital or one of six affiliated hospitals. Learning strategies were assessed using the 'Approaches to Learning at Work Questionnaire' (deep, surface-rational and surface-disorganised learning). Clinical performance was calculated by taking the mean of clinical assessment marks. The relationship between learning strategies and clinical performance was explored using regression analysis.Results: Most students (89%) did not clearly prefer a single learning strategy. No relationship was found between learning strategies and clinical performance.Discussion: Since overall competence comprises integration of knowledge, skills and professional behaviour, we assume that students without a clear preference use more than one learning strategy. Finding no relationship between learning strategies and clinical performance reflects the complexity of clinical learning. Depending on circumstances it may be important to obtain relevant information quickly (surface-rational) or understand material thoroughly (deep). In future research we will examine when and why students use different learning strategies.

OBJECTIVES We aimed to study the effects of active-learning curricula on graduation rates of students and on the length of time needed to graduate. METHODS Graduation rates for 10 generations of students enrolling in the eight Dutch medical schools between 1989 and 1998 were analysed. In addition, time needed to graduate was recorded. Three of the eight schools had curricula emphasising active learning, small-group instruction and limited numbers of lectures; the other five had conventional curricula to varying degrees. RESULTS Overall, the active-learning curricula graduated on average 8% more students per year, and these students graduated on average 5 months earlier than their colleagues from conventional curricula. CONCLUSIONS Four hypotheses potentially explaining the effect of active learning on graduation rate and study duration were considered: (i) active-learning curricula promote the social and academic integration of students; (ii) active-learning curricula attract brighter students; (iii) active-learning curricula retain more poor students, and (iv) the active engagement of students with their study required by active-learning curricula induces better academic performance and, hence, lower dropout rates. The first three hypotheses had to be rejected. It was concluded that the better-learning hypothesis provides the most parsimonious account for the data.

Background: Several authors assume that the supervisor’s role, observation of behaviour and students’ active participation are important factors in the instructiveness of feedback. Aim: This study aims to provide empirical evidence for these expectations. Methods: For two weeks, 142 clerks from eight hospitals recorded for each individual feedback event: who provided the feedback, whether the feedback was based on observation of behaviour, who initiated the feedback moment and the perceived instructiveness of the feedback. Data were analysed with multilevel techniques. Results: The perceived instructiveness of feedback provided by specialists and residents did not differ significantly. However, both were perceived to be more instructive than feedback from nursing and paramedical staff (specialists ¼ 0.862, p < 0.01; residents ¼ 0.853, p < 0.01). Feedback on behaviour that had been directly observed was reported to be more instructive than feedback on behaviour that had not been observed (observed ¼ 0.314, p < 0.001). Feedback which stemmed from student initiative or a joint initiative was experienced to be more instructive than feedback which ensued from the supervisor’s initiative (student ¼ 0.441, p < 0.01; joint ¼ 0.392, p < 0.01). Conclusions: The expectations concerning the influence of observation and student initiative on the instructiveness of feedback were confirmed in this empirical study. Expected differences in instructiveness between feedback from specialists and residents were not confirmed.

Background: The validation of educational instruments, in particular the employment of factor analysis, can be improved in many instances.Aims: To demonstrate the superiority of a sophisticated method of factor analysis, implying an integration of recommendations described in the factor analysis literature, over often employed limited applications of factor analysis. We demonstrate the essential steps, focusing on the Postgraduate Hospital Educational Environment Measure (PHEEM).Method: The PHEEM was completed by 279 clerks. We performed Principal Component Analysis (PCA) with varimax rotation. A combination of three psychometric criteria was applied: scree plot, eigenvalues > 1.5 and a minimum percentage of additionally explained variance of approximately 5%. Furthermore, four interpretability criteria were used. Confirmatory factor analysis was performed to verify the original scale structure.Results: Our method yielded three interpretable and practically useful dimensions: learning content and coaching, beneficial affective climate and external regulation. Additionally, combining several criteria reduced the risk of overfactoring and underfactoring. Furthermore, the resulting dimensions corresponded with three learning functions essential to high-quality learning, thus strengthening our findings. Confirmatory factor analysis disproved the original scale structure.Conclusions: Our sophisticated approach yielded several advantages over methods applied in previous validation studies. Therefore, we recommend this method in validation studies to achieve best practice.

CONTEXT Two learning approaches are consistently distinguished in the literature: deep and surface learning. The deep learning approach is considered preferable. Open-book tests are expected to stimulate deep learning and to offer a possible way of handling the substantial growth in medical knowledge. In this study we test the hypothesis that open-book tests stimulate deep learning more than closed-book tests.METHODS Medical students in Years 2 (n = 423) and 3 (n = 306) participated in this study. They evaluated their preparation for open- and closed-book tests using the test for Deep Information Processing (DIP). This questionnaire consists of 24 items divided into three subscales: Critical Reading; Broaden One's Context, and Structuring. A paired t-test was used to analyse the data.RESULTS Both cohorts scored significantly higher when preparing for closed-book tests for the overall DIP score and on the Broaden One's Context and Structuring scales. Year 3 students also scored significantly higher on the Critical Reading scale when preparing for closed-book tests. Gender differences were found: women used deeper learning approaches than men.CONCLUSIONS Our hypothesis was not supported. In fact, the opposite was found: closed-book tests stimulated a deep learning approach more than open-book tests. Three possible explanations are: deep learning is particularly necessary for remembering and recalling knowledge; students feel more confident when preparing for closed-book tests, and students are more motivated to study for closed-book tests. The debate on the concept of deep learning in higher education should probably be renewed.

Today's health sciences educational programmes have to deal with a growing and changing amount of knowledge. It is becoming increasingly important for students to be able to use and manage knowledge. We suggest incorporating open-book tests in assessment programmes to meet these changes. This view on the use of open-book tests is discussed and the influence on test quality is examined. To cope with the growing amount of medical knowledge, we have divided the body of knowledge into core knowledge, which students must know without need for references, and backup knowledge, which students need to understand and use properly with the help of references if so desired. As a result, all tests consist of a subtest for reproduction and understanding of core knowledge (a closed-book test) and a subtest for the ability to understand and manage backup knowledge (an open-book test). Statistical data from 14 such double-subtest exams for first and second-year students were analyzed for two cohorts (N = 435 and N = 449) with multilevel analysis, in accordance with generalizability theory. The reliability of the open and closed-book sections of the separate tests varied between 0.712 and 0.850. The open-book items reduce reliability somewhat. The estimated disattenuated correlation was 0.960 and 0.937 for cohorts 1 and 2 respectively. It is concluded that the use of open-book items with closed-book items slightly decreases test reliability but the overall index is acceptable. In addition, open and closed-book sections are strongly positively related. Therefore, open-book tests could be helpful in complementing today's assessment programmes.

OBJECTIVE: To test hypotheses regarding the longitudinal effects of problem-based learning (PBL) and conventional learning relating to students' appreciation of the curriculum, self-assessment of general competencies, summative assessment of clinical competence and indicators of career development.METHODS: The study group included 2 complete cohorts of graduates who were admitted to the medical curriculum in 1992 (conventional curriculum, n = 175) and 1993 (PBL curriculum, n = 169) at the Faculty of Medicine, University of Groningen, the Netherlands. Data were obtained from student records, graduates' self-ratings and a literature search. Gender and secondary school grade point average (GPA) scores were included as moderator variables. Data were analysed by a stepwise multiple and logistic regression analysis.RESULTS: Graduates of the PBL curriculum scored higher on self-rated competencies. Contrary to expectations, graduates of the PBL curriculum did not show more appreciation of their curriculum than graduates of the conventional curriculum and no differences were found on clinical competence. Graduates of the conventional curriculum needed less time to find a postgraduate training place. No differences were found for scientific activities such as reading scientific articles and publishing in peer- reviewed journals. Women performed better on clinical competence than did men. Grade point average did not affect any of the variables.CONCLUSIONS: The results suggest that PBL affects self-rated competencies. These outcomes confirm earlier findings. However, clinical competence measures did not support this finding.

OBJECTIVE To determine the effectiveness of single-point benchmarking and longitudinal benchmarking for inter-school educational evaluation. METHODS We carried out a mixed, longitudinal, cross-sectional study using data from 24 annual measurement moments (4 tests x 6 year groups) over 4 years for 4 annual progress tests assessing the graduation-level knowledge of all students from 3 co-operating medical schools. Participants included undergraduate medical students (about 5000) from 3 medical schools. The main outcome measures involved between-school comparisons of progress test results based on different benchmarking methods. RESULTS Variations in relative school performance across different tests and year groups indicate instability and low reliability of single-point benchmarking, which is subject to distortions as a result of school-test and year group-test interaction effects. Deviations of school means from the overall mean follow an irregular, noisy pattern obscuring systematic between-school differences. The longitudinal benchmarking method results in suppression of noise and revelation of systematic differences. The pattern of a school's cumulative deviations per year group gives a credible reflection of the relative performance of year groups. CONCLUSIONS Even with highly comparable curricula, single-point benchmarking can result in distortion of the results of comparisons. If longitudinal data are available, the information contained in a school's cumulative deviations from the overall mean can be used. In such a case, the mean test score across schools is a useful benchmark for cross-institutional comparison.

OBJECTIVE To determine whether items of progress tests used for inter-curriculum comparison favour students from the medical school where the items were produced (i.e. whether the origin bias of test items is a potential confounder in comparisons between curricula).METHODS We investigated scores of students from different schools on subtests consisting of progress test items constructed by authors from the different schools. In a cross-institutional collaboration between 3 medical schools, progress tests are jointly constructed and simultaneously administered to all students at the 3 schools. Test score data for 6 consecutive progress tests were investigated. Participants consisted of approximately 5000 undergraduate medical students from 3 medical schools. The main outcome measure was the difference between the scores on subtests of items constructed by authors from 2 of the collaborating schools (subtest difference score).RESULTS The subtest difference scores showed that students obtained better results on items produced at their own schools. This effect was more pronounced in Years 2-5 of the curriculum than in Year 1, and diminished in Year 6.CONCLUSIONS Progress test items were subject to origin bias. As a consequence, all participating schools should contribute equal numbers of test items if tests are to be used for valid and fair inter-curriculum comparisons.