Search

Introduction: Health professions education (HPE) has adopted the conceptualization of validity as an argument. However, the theoretical and practical aspects of how validity arguments should be developed, used and evaluated in HPE have not been deeply explored. Articulating the argumentation theory undergirding validity and validation can help HPE better operationalise validity as an argument. To better understand this, the authors explored how HPE validity scholars conceptualise assessment validity arguments and argumentation, seeking to understand potential consequences of these views on validation practices. Methods: The authors used critical case sampling to identify HPE assessment validity experts in three ways: (1) participation in a prominent validity research group, (2) appearing in a bibliometric study of HPE validity publications and (3) authorship of recent HPE validity literature. Qualitative semi‐structured interviews were conducted with 16 experts in HPE assessment validity from four different countries. The authors used reflexive thematic analysis to develop themes relevant to their research question. Results: The authors developed three themes grounded in participants' responses: (1) In theory, HPE validity is a social and situated argument. (2) In practice, the absence of audience and evaluation stymies the social nature of HPE validity. (3) Lack of validity argumentation creates and maintains power differentials within HPE. Participants articulated that current HPE validation practices are rooted in post‐positivist epistemology when they should be situated (i.e. context‐dependent), audience‐centric and inclusive. Discussion: When discussing validity argumentation in theory, participants' descriptions reflect an interpretivist lens for evaluation that is misaligned with real‐world validity practices. This misalignment likely arises from HPE's adoption of "validity as an argument" as a slogan, without integrating theoretical and practical principles of argumentation theory. How do HPE experts conceptualize the argumentation aspect of validity? @Midwest_MedPeds @ab_martini @LaraVarpio @erikdriessen @DrDanSchumacher explore the gap between theory and practice. [ABSTRACT FROM AUTHOR]

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen assessors appraised one of three competency-based mock portfolios while thinking aloud, before taking part in semi-structured interviews. A thematic analysis of the think-aloud protocols and interviews revealed that assessors reached judgments through a 3-phase cyclical cognitive process of acquiring, organizing, and integrating evidence. Upon conclusion of the first cycle, assessors reviewed the remaining portfolio evidence to look for confirming or disconfirming evidence. Assessors were inclined to stick to their initial judgments even when confronted with seemingly disconfirming evidence. Although assessors reached similar final (pass-fail) judgments of students' professional competence, they differed in their information-processing approaches and the reasoning behind their judgments. Differences sprung from assessors' divergent assessment beliefs, performance theories, and inferences about the student. Assessment beliefs refer to assessors' opinions about what kind of evidence gives the most valuable and trustworthy information about the student's competence, whereas assessors' performance theories concern their conceptualizations of what constitutes professional competence and competent performance. Even when using the same pieces of information, assessors furthermore differed with respect to inferences about the student as a person as well as a (future) professional. Our findings support the notion that assessors' reasoning in judgment and decision-making varies and is guided by their mental models of performance assessment, potentially impacting feedback and the credibility of decisions. Our findings also lend further credence to the assertion that portfolios should be judged by multiple assessors who should, moreover, thoroughly substantiate their judgments. Finally, it is suggested that portfolios be designed in such a way that they facilitate the selection of and navigation through the portfolio evidence.

During placements abroad, healthcare students are confronted with different personal and professional challenges, related to participation in practice. This study investigates when and how students respond to such challenges, and which coping and support mechanisms students use to overcome these. Twenty-five international students shared their experiences about physiotherapy placement in The Netherlands. Using a critical incident technique, we asked participants to recall events where participation was affected by an unforeseen situation, in or outside the clinic. Further, we explored students' strategies of seeking support within their social network to overcome individual challenges. Two researchers applied thematic analysis to the interview data, following an iterative approach. Team discussions supported focused direction of data collection and analysis, before conceptualizing results. Participants described a wide range of challenges. The scope and impact level of challenges varied widely, including intercultural differences, language barriers and inappropriate behaviour in the workplace, students' personal context and wellbeing. Mechanisms employed by students to overcome these challenges depended on the type of event (personal or professional), making purposeful use of their available network. Students involve clinical staff, peers, family and friends during placement abroad, to make deliberate use of their support network to overcome challenges in participation, whereas the academic network remains distant. Findings may help reflect on the roles and responsibilities of academic staff and other professionals involved with placements abroad. Healthcare programmes should ensure support before, during and after placement is within students' reach. [ABSTRACT FROM AUTHOR]

A tool used in student-centered education is discussion among students in small learning groups. The Western origin of student-centered education, coupled with cross-cultural differences in communication styles, may detract from its cross-cultural applicability. This study investigates how in student-centered education, students' cultural backgrounds are expressed in discussions and shape students' discussion behaviors and skills. A comparative case study using problem-based learning as a student-centered model was conducted in three medical schools located in East Asia, Western Europe and the Middle East. Four cultural factors were found to potentially cause students, especially those in the non-Western schools, to refrain from speaking up, asking questions, and challenging others in discussions. Six contextual factors mediated the influence of the cultural factors. The findings were incorporated in a conceptual model. The conclusion seems justified that student-centered education is feasible in different cultural contexts, but across these contexts, processes and outcomes are likely to differ.

Purpose: To conduct a bibliometric case study of the journal Perspectives on Medical Education (PME) to provide insights into the journal's inner workings and to "take stock" of where PME is today, where it has been, and where it might go. Methods: Data, including bibliographic metadata, reviewer and author details, and downloads, were collected for manuscripts submitted to and published in PME from the journal's Editorial Manager and Web of Science. Gender of authors and reviewers was predicted using Genderize.io. To visualize and analyze collaboration patterns, citation relationships and term co-occurrence social network analyses (SNA) were conducted. VOSviewer was used to visualize the social network maps. Results: Between 2012–2019 PME received, on average, 260 manuscripts annually (range = 73–402). Submissions were received from authors in 81 countries with the majority in the United States (US), United Kingdom, and the Netherlands. PME published 518 manuscripts with authors based in 31 countries, the majority being in the Netherlands, US, and Canada. PME articles were downloaded 717,613 times (mean per document: 1388). In total 1201 (55% women) unique peer reviewers were invited and 649 (57% women) completed reviews; 1227 (49% women) unique authors published in PME. SNA revealed that PME authors were quite collaborative, with most authoring articles with others and only a minority (n = 57) acting as single authors. Discussion: This case study provides a glimpse into PME and offers evidence for PME's next steps. In the future, PME is committed to growing the journal thoughtfully; diversifying and educating editorial teams, authors, and reviewers, and liberating and sharing journal data. [ABSTRACT FROM AUTHOR]

Students perceive reflective writing as difficult. Concept mapping may be an alternative format for reflection, which provides support while allowing students to freely shape their thoughts. We examined (1) the quality of reflection in reflective concept maps created by first-year medical students and (2) students' perceptions about concept mapping as a tool for reflection. Mixed-method study conducted within the medical curriculum of Maastricht University, The Netherlands, consisting of: (1) Analysis of the quality of reflection in 245 reflective concept maps created by 40 first-year students. Reflection quality was analysed by assessing focus of reflection (technical/practical/sensitising) and depth of reflection (description/justification/critique/discussion). (2) Semi-structured interviews with 22 students to explore perceived effectiveness of reflective concept mapping. Depth of reflection reached at least the level of critique in 82% of maps. Three factors appeared to affect the perceived effectiveness of concept mapping for reflection: (1) reflective concept map structure; (2) alertness to meaningful experiences in practice and (3) learning by doing. These results yielded supportive evidence for concept mapping as a useful technique to teach novice learners the basics of effective reflection. Meaningful implementation requires a delicate balance between providing a supportive structure and allowing flexibility for the student. [ABSTRACT FROM AUTHOR]

Objectives: We expect physicians to be lifelong learners. Participation in clinical practice is an important potential source of that learning. To support physicians in this process, a better understanding of how they learn in clinical practice is necessary. This study investigates how physicians recognise and use informal feedback from interactions with patients in outpatient settings as learning cues to adjust their communication behaviours in daily practice. Methods: To understand physicians' use of informal feedback, we combined non‐participant observations with semi‐structured interviews. We enrolled 10 respiratory physicians and observed 100 physician‐patient interactions at two teaching hospitals in the Netherlands. Data collection and analysis were performed iteratively according to the principles of constructivist grounded theory. Results: Following stages of open, axial and selective coding, we were able to conceptualise how physicians use cues to reflect on and adjust their communication. In addition to vast variations within and across patient encounters, we observed recurring adjustments in physicians' communication behaviours in response to specific informal feedback cues. Physicians recognised and used these cues to self‐monitor communication performance. They had established 'communication repertoires' based on multiple patient interactions, which many saw as learning opportunities contributing to the development of expertise. Our findings, however, show differences in physicians' individual levels of sensitivity in recognising and using learning opportunities in daily practice, which were further influenced by contextual, personal and interpersonal factors. Whereas some described themselves as having little inclination to change, others used critical incidents to fine‐tune their communication repertoires, and yet others constantly reshaped them, seeking learning opportunities in their daily work. Conclusions: There is large variation in how physicians use learning cues from daily practice. To enhance learning in and from daily practice, we propose turning workplace learning into a collaborative effort with the aim of increasing awareness and the use of informal performance‐relevant feedback. Sehlbach et al. use observations and interviews to examine how physicians use informal feedback for professional development, highlighting how workplace learning could be improved through collaboration. [ABSTRACT FROM AUTHOR]

Background: Despite increasing calls for patient and public involvement in health‐care quality improvement, the question of how patient evaluations can contribute to physician learning and performance assessment has received scant attention. Objective: The objective of this study was to explore, amid calls for patient involvement in quality assurance, patients' perspectives on their role in the evaluation of physician performance and to support physicians' learning and decision making on professional competence. Design: A qualitative study based on semi‐structured interviews. Setting and Participants: The study took place in a secondary care setting in the Netherlands. The authors selected 25 patients from two Dutch hospitals and through the Dutch Lung Foundation, using purposive sampling. Methods: Data were analysed according to the principles of template analysis, based on an a priori coding framework developed from the literature about patient empowerment, feedback and performance assessment. Results: The analysis unearthed three predominant patient perspectives: the proactive perspective, the restrained perspective and the outsider perspective. These perspectives differed in terms of perceived power dynamics within the doctor‐patient relationship, patients' perceived ability, and willingness to provide feedback and evaluate their physician's performance. Patients' perspectives thus affected the role patients envisaged for themselves in evaluating physician performance. Discussion and conclusion: Although not all patients are equally suitable or willing to be involved, patients can play a role in evaluating physician performance and continuing training through formative approaches. To involve patients successfully, it is imperative to distinguish between different patient perspectives and empower patients by ensuring a safe environment for feedback. [ABSTRACT FROM AUTHOR]

Background:Portfolios are used as tools to coach and assess students in the workplace. This study sought to evaluate the content validity of portfolios as reflected in their capacity to adequately assess achieved competences of medical students during clerkships. Methods: We reviewed 120 workplace portfolios at three medical universities (Belgium and the Netherlands). To validate their content, we developed a Validity Inventory for Portfolio Assessment (VIPA) based on the CanMEDS roles. Two raters evaluated each portfolio and indicated for each VIPA item whether the portfolio provided sufficient information to enable satisfactory assessment of the item. We ran a descriptive analysis on the validation data and computed Cohen’s Kappa to investigate interrater agreement. Results:The portfolios adequately covered the items pertaining to the communicator (90%) and professional (87%) roles. Coverage of the medical expert, collaborator, scholar and manager roles ranged between 75% and 85%. The health advocate role, covering 59%, was clearly less well represented. This role also exhibited little interrater agreement (Kappa < 0.4). Conclusions:This study lends further credence to the evidence that portfolios can indeed adequately assess the different CanMEDS roles during clerkships, the health advocate role, which was less well represented in the portfolio content, excepted. [ABSTRACT FROM PUBLISHER]

Collaborations between groups of professionals often have a long history, which can still influence contemporary practice. If problems in the collaboration occur, the search for effective interventions for these problems may be informed by analysing current practice as well as the historical development of the collaboration. The study focused on the collaboration between obstetricians and midwives in the Netherlands. We performed a secondary analysis of questionnaire data focusing on midwives evaluating the collaborative performance of obstetricians in the Netherlands. Template analysis was used to analyse the questionnaires. The initial template was based on a model for interprofessional collaboration. As a final step, we reflected on the results in light of the historical development of the collaboration. The midwives experienced a power imbalance and a lack of trust and mutual acquaintanceship in their collaboration with obstetricians. They also reported a need for interprofessional governance and formalization. Most of these reported problems in the collaboration have their origin in the historical development of both professions and in the development of the collaboration between both professional groups. Combining an exploration of contemporary interprofessional practice with a historical perspective on interprofessional collaboration is fruitful for understanding problems in collaboration between professional groups, and provides guidance for improving collaboration. [ABSTRACT FROM PUBLISHER]

Context It is widely acknowledged that assessment can affect student learning. In recent years, attention has been called to 'programmatic assessment', which is intended to optimise both learning functions and decision functions at the programme level of assessment, rather than according to individual methods of assessment. Although the concept is attractive, little research into its intended effects on students and their learning has been conducted. Objectives This study investigated the elements of programmatic assessment that students perceived as supporting or inhibiting learning, and the factors that influenced the active construction of their learning. Methods The study was conducted in a graduate-entry medical school that implemented programmatic assessment. Thus, all assessment information, feedback and reflective activities were combined into a comprehensive, holistic programme of assessment. We used a qualitative approach and interviewed students ( n = 17) in the pre-clinical phase of the programme about their perceptions of programmatic assessment and learning approaches. Data were scrutinised using theory-based thematic analysis. Results Elements from the comprehensive programme of assessment, such as feedback, portfolios, assessments and assignments, were found to have both supporting and inhibiting effects on learning. These supporting and inhibiting elements influenced students' construction of learning. Findings showed that: (i) students perceived formative assessment as summative; (ii) programmatic assessment was an important trigger for learning, and (iii) the portfolio's reflective activities were appreciated for their generation of knowledge, the lessons drawn from feedback, and the opportunities for follow-up. Some students, however, were less appreciative of reflective activities. For these students, the elements perceived as inhibiting seemed to dominate the learning response. Conclusions The active participation of learners in their own learning is possible when learning is supported by programmatic assessment. Certain features of the comprehensive programme of assessment were found to influence student learning, and this influence can either support or inhibit students' learning responses. [ABSTRACT FROM AUTHOR]

Collaboration between different groups of health care professionals is often rooted in a long and often difficult history. This history can exert a strong influence on how professionals collaborate and historical tensions can contribute to problems in contemporary practice. However, literature about interprofessional collaboration often ignores the historical underpinnings of collaboration. In this paper, the historical development of interprofessional collaboration between obstetricians and midwives within the setting of Dutch obstetrical care is explored using a review of Dutch and English literature for documents explicitly or implicitly describing the historical development of this collaboration. This literature delineates the establishment of professional boundaries and the formalization of the collaboration between the two professions. It also details the history of physician domination over the midwives both in midwifery practice and education and the relatively recent reversal of this situation. Moreover, the shift in collaborative partner from general practitioner to obstetrician and its effect on collaboration is examined. Insight into the historical foundations of Dutch maternity care collaboration may allow us to understand the origins, and thus formulate possible solutions, for contemporary problems within this collaboration. [ABSTRACT FROM AUTHOR]

Background: Despite acknowledgement that the Canadian Medical Educational Directives for Specialists (CanMEDS) framework covers the relevant competencies of physicians, many educators and medical professionals struggle to translate the CanMEDS roles into comprehensive training programmes for specific specialties. Aim: To gain insight into the applicability of the CanMEDS framework to guide the design of educational programmes for specific specialties by exploring stakeholders' perceptions of specialty specific competencies and examining differences between those competencies and the CanMEDS framework. Methods: This case study is a sequel to a study among ObsGyn specialists. It explores the perspectives of patients, midwives, nurses, general practitioners, and hospital boards on gynaecological competencies and compares these with the CanMEDS framework. Results: Clinical expertise, reflective practice, collaboration, a holistic view, and involvement in practice management were perceived to be important competencies for gynaecological practice. Although all the competencies were covered by the CanMEDS framework, there were some mismatches between stakeholders' perceptions of the importance of some competencies and their position in the framework. Conclusion: The CanMEDS framework appears to offer relevant building blocks for specialty specific postgraduate training, which should be combined with the results of an exploration of specialty specific competencies to arrive at a postgraduate curriculum that is in alignment with professional practice. [ABSTRACT FROM AUTHOR]

Background: Earlier studies suggested national culture to be a potential barrier to curriculum reform in medical schools. In particular, Hofstede's cultural dimension 'uncertainty avoidance' had a significant negative relationship with the implementation rate of integrated curricula. Aims: However, some schools succeeded to adopt curriculum changes despite their country's strong uncertainty avoidance. This raised the question: 'How did those schools overcome the barrier of uncertainty avoidance?' Method: Austria offered the combination of a high uncertainty avoidance score and integrated curricula in all its medical schools. Twenty-seven key change agents in four medical universities were interviewed and transcripts analysed using thematic cross-case analysis. Results: Initially, strict national laws and limited autonomy of schools inhibited innovation and fostered an 'excuse culture': ' It's not our fault. It is the ministry's'. A new law increasing university autonomy stimulated reforms. However, just this law would have been insufficient as many faculty still sought to avoid change. A strong need for change, supportive and continuous leadership, and visionary change agents were also deemed essential. Conclusions: In societies with strong uncertainty avoidance strict legislation may enforce resistance to curriculum change. In those countries opposition by faculty can be overcome if national legislation encourages change, provided additional internal factors support the change process. [ABSTRACT FROM AUTHOR]

Medical Education 2012: 46: 738-747 Context Medical schools worldwide are increasingly switching to student-centred methods such as problem-based learning (PBL) to foster lifelong self-directed learning (SDL). The cross-cultural applicability of these methods has been questioned because of their Western origins and because education contexts and learning approaches differ across cultures. Objectives This study evaluated PBL's cross-cultural applicability by investigating how it is applied in three medical schools in regions with different cultures in, respectively, East Asia, the Middle East and Western Europe. Specifically, it investigated how students' cultural backgrounds impact on SDL in PBL and how this impact affects students. Methods A qualitative, cross-cultural, comparative case study was conducted in three medical schools. Data were collected through 88 semi-structured, in-depth interviews with Year 1 and 3 students, tutors and key persons involved in PBL, 32 observations of Year 1 and 3 PBL tutorials, document analysis, and contextual information. The data were thematically analysed using the template analysis method. Comparisons were made among the three medical schools and between Year 1 and 3 students across and within the schools. Results The cultural factors of uncertainty and tradition posed a challenge to Middle Eastern students' SDL. Hierarchy posed a challenge to Asian students and achievement impacted on both sets of non-Western students. These factors were less applicable to European students, although the latter did experience some challenges. Several contextual factors inhibited or enhanced SDL across the cases. As students grew used to PBL, SDL skills increased across the cases, albeit to different degrees. Conclusions Although cultural factors can pose a challenge to the application of PBL in non-Western settings, it appears that PBL can be applied in different cultural contexts. However, its globalisation does not postulate uniform processes and outcomes, and culturally sensitive alternatives might be developed. [ABSTRACT FROM AUTHOR]

The differences of learning experiences in the workplace put challenges on the assessment: the assessment programme should be aligned with the general competency framework of the curriculum and also fit to the differences in learning contexts of the workplace. We used van der Vleuten's programmatic assessment model to develop a workplace-based assessment programme for final year clerkships. We aimed to design a programme that stimulates learning, supports assessment decision, is feasible and non-bureaucratic. The first experiences with the programme show that students think that the programme has high learning value and the assessment is sufficiently robust. Many of the commonly reported weaknesses of work-based assessment (not a good fit with the educational context, too complex, too bureaucratic and too much work) were not mentioned by the students. [ABSTRACT FROM AUTHOR]

Context The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence. Methods We developed an instrument, the Portfolio Analysis Scoring Inventory, to examine the impact of form and content aspects on portfolio assessment. The Inventory consists of 15 items derived from interviews with experienced mentors, the literature, and the criteria for reflective competence used in the regular portfolio assessment procedure. Forty portfolios, selected from 231 portfolios for which ratings from the regular assessment procedure were available, were rated by 2 researchers, independently, using the Inventory. Regression analysis was used to estimate the correlation between the ratings from the regular assessment and those resulting from the Inventory items. Results Inter-rater agreement ranged from 0.46 to 0.87. The strongest predictor of the variance in the regular ratings was ‘quality of reflection’ (R 0.80; R2 66%). No further items accounted for a significant proportion of variance. Irrelevant items, such as writing style and lay-out, had negligible effects. Conclusions The absence of an impact of irrelevant criteria appears to support the validity of the portfolio assessment procedure. Further studies should examine the portfolio's validity for the assessment of clinical competence. [ABSTRACT FROM AUTHOR]

Purpose: How mentors shape their mentoring is strongly influenced by their personal beliefs about the goals and purpose of mentoring, the possible activities associated with it, who decides on the focus of the mentoring relationship, and the strategies mentors choose to enact these beliefs in practice. In accordance with the personal interpretative framework, the authors operationalized mentors' beliefs as professional self-understanding (the what) and subjective educational theory (the how) of teaching and sought to identify different mentoring positions., Method: Using a qualitative approach, the authors conducted semistructured interviews between December 2017 and January 2018 with 18 undergraduate mentors from Maastricht University in Maastricht, the Netherlands. The aim of the interviews was to reconstruct their personal interpretative framework. Before building a general pattern of explanation in a cross-case analysis, the authors performed a within-case analysis of the data, analyzing individual mentors., Results: This approach resulted in the identification and description of 4 mentoring positions: the (1) facilitator (service providing and responsive), (2) coach (development supporting and responsive), (3) monitor (signaling and collaborative), and (4) exemplar (service providing or development supporting and directive). Each position represents a coherent pattern of normative beliefs about oneself as a mentor (professional self-understanding) and how to enact these beliefs in practice (subjective educational theory)., Conclusions: Awareness of their mentoring position can help mentors understand why they act the way they do in certain situations and how this behavior affects their mentees' learning and development. It can also help mentors identify personal learning needs and, consequently, provide opportunities for faculty development.

Introduction: Portfolio-based assessments require that learners' competence development is adequately reflected in portfolio documentation. This study explored how students select and document performance data in their portfolios and how they perceive these data to be representative for their competence development., Methods: Students uploaded performance data in a competency-based portfolio. During one clerkship period, twelve students also recorded an audio diary in which they reflected on experiences and feedback that they perceived to be indicants of their competence development. Afterwards, these students were interviewed to explore the extent to which the performance documentation in the portfolio corresponded with what they considered illustrative evidence of their development. The interviews were analyzed using thematic analysis., Results: Portfolios provide an accurate but fragmented picture of student development. Portfolio documentation was influenced by tensions between learning and assessment, student beliefs about the goal of portfolios, student performance evaluation strategies, the learning environment and portfolio structure., Discussion: This study confirms the importance of taking student perceptions into account when implementing a competency-based portfolio. Students would benefit from coaching on how to select meaningful experiences and performance data for documentation in their portfolios. Flexibility in portfolio structure and requirements is essential to ensure optimal fit between students' experienced competence development and portfolio content.

Purpose: An estimated 20% of health care expenditures are wasteful. Educational interventions aimed at reducing waste by delivering high-value, cost-conscious care (HV3C) often focus on the role of the physician. This study sought to understand how attending physicians, who have a central role in the workplace, prepare residents to provide HV3C., Method: Researchers from Maastricht University in Maastricht, the Netherlands, conducted semistructured interviews between September 2016 and August 2017 with 12 attending physicians who supervise residents in the workplace. Participants were purposefully sampled from 5 institutions throughout the Netherlands to include surgical and nonsurgical attending physicians and hospital- and nonhospital-based physicians. Data collection and analysis were iterative, using principles of grounded theory., Results: The attending physician's approach to providing HV3C was an important factor in preparing residents in the workplace. Three differences became apparent: priority of HV3C training, feedback on HV3C, and obstacles to HV3C delivery. Results indicate that attending physicians use 3 teaching methods to teach HV3C delivery: Socratic questioning, role modeling, and setting limits. Training was often implicit and ad hoc., Conclusions: How attending physicians deal with HV3C themselves influences how they prepare residents in the workplace. To optimize resident training, it may be important to create a supportive environment for HV3C delivery and training. Delivery could be supported by making HV3C a shared goal for attending physicians and residents, thereby providing insight into clinical practice behavior and minimizing the influence of obstacles. Training could be optimized by supporting a variety of teaching methods suitable for daily teaching to stimulate continuous learning in residents.

Introduction: International placements challenge students to find the right level of participation, as local practices, language and time pressure may affect their engagement in patient-related tasks or team activities. This study sought to unpack the initiation process during international clinical placements with the ultimate aim to achieve active student participation., Methods: Following a constructivist grounded theory approach, we conducted two individual interviews with 15 undergraduate healthcare students (before departure and whilst on placement). To identify emerging themes, we applied an iterative process of data collection and constant comparative analysis. Several team discussions informed further analysis, allowing us to reach a more conceptual level of theory., Results: From our findings we constructed a four-phase model of healthcare students' initiation of international clinical placements, which brings into focus how the phases of 'orientation', 'adjustment' and 'contribution to patient care' build up towards a 'sense of belonging'. We identified several factors that induced active student participation in practice, such as a favourable workplace setting, opportunities for learning and a local support network., Discussion: Active student participation is aimed at different goals, depending on the four phases of initiation that eventually lead to a sense of belonging and support workplace learning.

Background: Entrustment of residents has been formalized in many competency-based graduate medical education programs, but its relationship with informal decisions to entrust residents with clinical tasks is unclear. In addition, the effects of formal entrustment on training practice are still unknown., Objective: Our objective was to learn from faculty members in training programs with extensive experience in formal entrustment how formal entrustment relates to informal entrustment decisions., Methods: A questionnaire was e-mailed to all Dutch obstetrics and gynecology program directors to gather information on how faculty entrusts residents with clinical independence. We also interviewed faculty members to explore the relationship between formal entrustment and informal entrustment. Interviews were analyzed with conventional content analysis., Results: Of 92 programs, 54 program directors completed the questionnaire (59% response rate). Results showed that formal entrustment was seen as valuable for generating formative feedback and giving insight into residents' progress in technical competencies. Interviewed faculty members (n = 12) used both formal and informal entrustment to determine the level of resident independence. Faculty reported they tended to favor informal entrustment because it can be reconsidered. In contrast, formal entrustment was reported to feel like a fixed state., Conclusions: In a graduate medical education program where formal entrustment has been used for more than a decade, faculty used a combination of formal and informal entrustment. Informal entrustment is key in deciding if a resident can work independently. Faculty members reported being unsure how to optimally use formal entrustment in practice next to their informal decisions., Competing Interests: Conflict of interest: The authors declare they have no competing interests.

Background: Current postgraduate medical training programmes fall short regarding residents' development of generic competencies (communication, collaboration, leadership, professionalism) and reflective and deliberate practice. Paying attention to these non-technical skills in a structural manner during postgraduate training could result in a workforce better prepared for practice. A development-oriented performance assessment (PA), which assists residents with assessment of performance and deliberately planned learning activities, could potentially contribute to filling this gap. This study aims to explore residents experiences with the PA., Methods: We conducted a qualitative interview study with 16 residents from four different medical specialties who participated in the PA, scheduled halfway postgraduate training. The PA was conducted by an external facilitator, a psychologist, and focused specifically on professional development and career planning. Residents were interviewed 6 months after the PA. Data were analysed using the framework method for qualitative analysis., Results: Residents found the PA to be of additional value for their training. The overarching merit was the opportunity to evaluate competencies not usually addressed in workplace-based assessments and progress conversations. In addition, the PA proved a valuable tool for assisting residents with reflecting upon their work and formulating their learning objectives and activities. Residents reported increased awareness of capacity, self-confidence and enhanced feelings of career-ownership. An important factor contributing to these outcomes was the relationship of trust with the facilitator and programme director., Conclusion: The PA is a promising tool in fostering the development of generic competencies and reflective and deliberate practice. The participating residents, facilitator and programme directors were able to contribute to a safe learning environment away from the busy workplace. The facilitator plays an important role by providing credible and informative feedback. Commitment of the programme director is important for the implementation of developmental plans and learning activities.

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen assessors appraised one of three competency-based mock portfolios while thinking aloud, before taking part in semi-structured interviews. A thematic analysis of the think-aloud protocols and interviews revealed that assessors reached judgments through a 3-phase cyclical cognitive process of acquiring, organizing, and integrating evidence. Upon conclusion of the first cycle, assessors reviewed the remaining portfolio evidence to look for confirming or disconfirming evidence. Assessors were inclined to stick to their initial judgments even when confronted with seemingly disconfirming evidence. Although assessors reached similar final (pass-fail) judgments of students' professional competence, they differed in their information-processing approaches and the reasoning behind their judgments. Differences sprung from assessors' divergent assessment beliefs, performance theories, and inferences about the student. Assessment beliefs refer to assessors' opinions about what kind of evidence gives the most valuable and trustworthy information about the student's competence, whereas assessors' performance theories concern their conceptualizations of what constitutes professional competence and competent performance. Even when using the same pieces of information, assessors furthermore differed with respect to inferences about the student as a person as well as a (future) professional. Our findings support the notion that assessors' reasoning in judgment and decision-making varies and is guided by their mental models of performance assessment, potentially impacting feedback and the credibility of decisions. Our findings also lend further credence to the assertion that portfolios should be judged by multiple assessors who should, moreover, thoroughly substantiate their judgments. Finally, it is suggested that portfolios be designed in such a way that they facilitate the selection of and navigation through the portfolio evidence.

Purpose: Residency programs around the world use multisource feedback (MSF) to evaluate learners' performance. Studies of the reliability of MSF show mixed results. This study aimed to identify the reliability of MSF as practiced across occasions with varying numbers of assessors from different professional groups (physicians and nonphysicians) and the effect on the reliability of the assessment for different competencies when completed by both groups., Method: The authors collected data from 2008 to 2012 from electronically completed MSF questionnaires. In total, 428 residents completed 586 MSF occasions, and 5,020 assessors provided feedback. The authors used generalizability theory to analyze the reliability of MSF for multiple occasions, different competencies, and varying numbers of assessors and assessor groups across multiple occasions., Results: A reliability coefficient of 0.800 can be achieved with two MSF occasions completed by at least 10 assessors per group or with three MSF occasions completed by 5 assessors per group. Nonphysicians' scores for the "Scholar" and "Health advocate" competencies and physicians' scores for the "Health advocate" competency had a negative effect on the composite reliability., Conclusions: A feasible number of assessors per MSF occasion can reliably assess residents' performance. Scores from a single occasion should be interpreted cautiously. However, every occasion can provide valuable feedback for learning. This research confirms that the (unique) characteristics of different assessor groups should be considered when interpreting MSF results. Reliability seems to be influenced by the included assessor groups and competencies. These findings will enhance the utility of MSF during residency training.

Reforms in postgraduate medical education (PGME) exposed a gap between educational theory and clinical practice. Entrustable Professional Activities (EPAs) were introduced to assist clinicians in bridging this gap and to create better consonance between the intended and the enacted curriculum. In this viewpoint paper, we discuss the potential and the pitfalls of using EPAs in PGME. EPAs promise an effective way of teaching abstract competencies in a curriculum based on real-life professional activities that are suitable for clinical assessment. Summative judgement is used to entrust a resident step by step in a certain EPA, resulting in an increase of independent practice. However, we argue that the success of EPAs depends on (1) a balance: brief focussed descriptions against the requirements for detail and (2) a precondition: a mature and flexible workplace for learning.

Context: Mentors are increasingly involved in doctor performance assessments. Mentoring seems to be a key determinant in achieving the ultimate goal of those assessments, namely, improving doctor performance. Little is known, however, about how mentors perceive and fulfil this role., Objective: The aim of this paper is to expand understanding of the role of mentors in performance assessment., Methods: Thirty-eight mentors undertook formative performance assessments of their peers in a pilot study. A mixed-methods design was used, consisting of a postal survey (n = 28) and qualitative interviews with a subset of mentors (n = 11). Individual semi-structured interviews were completed and transcripts were analysed by two researchers using a grounded theory approach., Results: The results of the survey showed that 89% of mentors intended to continue in their mentorship role. Interviews revealed that mentors used several strategies in the assessments, including: contrasting and collating information; posing reflective questions, and goal setting. Mentors experienced difficulty in disregarding their views of the doctors evaluated. Some mentors noticed obstacles with specific interview skills such as 'paying attention to their colleagues' strengths' and 'enabling doctors to find their own solutions'. Mentors reported that they and their organisations benefited from the assessments. The perceived benefits included: improved interview skills; increased solidarity, and increased mutual respect., Conclusions: The study provides insights into what mentors can do to increase the chance that externally derived information is integrated into doctors' self-assessments. Mainly, mentors used strategies aimed at effectively delivering feedback and encouraging reflection. However, we found that mentors who took part in our study appeared to struggle with a number of obstacles related to: time investment; familiarity with the doctor assessed, and the acquiring of specific interview skills.

Aim: Portfolios are often used as an instrument with which to stimulate students to reflect on their experiences. Research has shown that working with portfolios does not automatically stimulate reflection. In this study we addressed the question: What are the conditions for successful reflective use of portfolios in undergraduate medical education? METHODOLOGY/RESEARCH DESIGN: We designed a portfolio that was aimed at stimulating reflection in early undergraduate medical education, using experiences described in the medical education literature and elsewhere. Conditions for reflective portfolio use were identified through interviews with 13 teachers (mentors), who were experienced in mentoring students in the process of developing their portfolios. The interviews were analysed according to the principles of grounded theory., Results: The conditions for successful reflective use of portfolios that emerged from the interviews fell into 4 categories: coaching; portfolio structure and guidelines; relevant experiences and materials, and summative assessment. According to the mentors, working with a portfolio designed to meet these conditions will stimulate students' reflective abilities., Conclusion: This study shows that portfolios are a potentially valuable method of assessing and developing students' reflective skills in undergraduate medical training, provided certain conditions for effective portfolios are recognised and met. Portfolios have a strong potential for enhancing learning and assessment but they are very vulnerable and may easily lead to disappointment. Before implementing portfolios in education, one should first consider whether the necessary conditions can be fulfilled, including an appropriate portfolio structure, an appropriate assessment procedure, the provision of enough new experiences and materials, and sufficient teacher capacity for adequate coaching and assessment.