Search

Objective: To compare postpartum hemorrhage (PPH) patient outcomes before and after an in situ instructional design based PPH simulation attended by obstetrics and gynecology (OBGYN) residents.Methods: This uncontrolled before-and-after study was conducted in Recife, Brazil including all 1388 women delivering from June to August 2012 and all 1357 delivering from June to August 2013. The 36 OBGYN residents were divided into13 teams of two or three participants and were trained through ID based PPH simulation training with the following eight steps: (1) prior knowledge activation, (2) video demonstration, (3) dual-coding PPH protocol discussion-an image association during the training, (4) training scenario # 1, (5) debriefing, (6) training scenario # 2 with immediate feedback, (7) training scenario # 3, and (8) debriefing with self-assessment. The training scenarios had an increasing level of complexity. The main goal of the training was the adequate management of PPH and situational awareness improvement-the ability to anticipate, recognize, and intercept unfolding error chains. The primary patient outcomes rates used for the before and after comparison were therapeutic uterotonics use within 24 h of birth and blood transfusion. Secondary outcomes were therapeutic oxytocin mean dosage IU within 24 h of birth, postpartum Hb < 6 g/dL, among others. Chi-square test was used for categorical variables comparison and independent t-test for continuous variables.Results: PPH rates were 100 (7.2% of 2012 deliveries) and 80 cases (5.9% of 2013 deliveries), respectively. Comparison of primary post- and pre-simulation outcomes revealed no significant differences. However, in the comparison for therapeutic oxytocin mean dosage IU within 24 h of birth, there was an increase found after the simulation (15.98 ± 7.4 versus 25.1 ± 12.3; p < .001). For all other outcome measures, there were no statistical differences.Conclusions:In situ ID based PPH simulation leads to an increase in the mean dosage of oxytocin after training, in selected cases. This may indicate better situational awareness when managing women with PPH. [ABSTRACT FROM AUTHOR]

Background: Consistent identification and prevention of failure for at-risk medical students is challenging, failing courses is costly to all stakeholders, and there is need for further research into duration, timing and structure of interventions to help students in difficulty. Aims: To verify the value of a new exam two weeks into medical school as a predictor of failure, and explore the requirements for a preventative intervention. Methods: Students who failed the two-week exam were invited to a series of large-group workshops and small-group follow-up meetings. Participants' subsequent exam performance was compared with non-participants. Results: About 71% of students who performed poorly in the new exam subsequently failed a course. Attendance at the workshops made no difference to short- or long-term pass rates. Attendance at more than three follow-up small group sessions significantly improved pass rates two semesters later, and was influenced by teacher experience. Conclusions: Close similarity between predictor task and target task is important for accurate prediction of failure. Consideration should be given to dose effect and class size in the prevention of failure of at-risk students, and we recommend a systemic approach to intervention/remediation programmes, involving a whole semester of mandatory, weekly small group meetings with experienced teachers. [ABSTRACT FROM AUTHOR]

Medical Education 2012: 46: 38-48 Context Programmatic assessment is a notion that implies that the strength of the assessment process results from a careful combination of various assessment instruments. Accordingly, no single instrument is superior to another, but each has its own strengths, weaknesses and purpose in a programme. Yet, in terms of psychometric methods, a one-size-fits-all approach is often used. Kane's views on validity as represented by a series of arguments provide a useful framework from which to highlight the value of different widely used approaches to improve the quality and validity of assessment procedures. Methods In this paper we discuss four inferences which form part of Kane's validity theory: from observations to scores; from scores to universe scores; from universe scores to target domain, and from target domain to construct. For each of these inferences, we provide examples and descriptions of approaches and arguments that may help to support the validity inference. Conclusions As well as standard psychometric methods, a programme of assessment makes use of various other arguments, such as: item review and quality control, structuring and examiner training; probabilistic methods, saturation approaches and judgement processes, and epidemiological methods, collation, triangulation and member-checking procedures. In an assessment programme each of these can be used. [ABSTRACT FROM AUTHOR]

There are no scientific theories that are uniquely related to assessment in medical education. There are many theories in adjacent fields, however, that can be informative for assessment in medical education, and in the recent decades they have proven their value. In this AMEE Guide we discuss theories on expertise development and psychometric theories, and the relatively young and emerging framework of assessment for learning. Expertise theories highlight the multistage processes involved. The transition from novice to expert is characterised by an increase in the aggregation of concepts from isolated facts, through semantic networks to illness scripts and instance scripts. The latter two stages enable the expert to recognise the problem quickly and form a quick and accurate representation of the problem in his/her working memory. Striking differences between experts and novices is not per se the possession of more explicit knowledge but the superior organisation of knowledge in his/her brain and pairing it with multiple real experiences, enabling not only better problem solving but also more efficient problem solving. Psychometric theories focus on the validity of the assessment - does it measure what it purports to measure and reliability - are the outcomes of the assessment reproducible. Validity is currently seen as building a train of arguments of how best observations of behaviour (answering a multiple-choice question is also a behaviour) can be translated into scores and how these can be used at the end to make inferences about the construct of interest. Reliability theories can be categorised into classical test theory, generalisability theory and item response theory. All three approaches have specific advantages and disadvantages and different areas of application. Finally in the Guide, we discuss the phenomenon of assessment for learning as opposed to assessment of learning and its implications for current and future development and research. [ABSTRACT FROM AUTHOR]

In assessment a considerable shift in thinking has occurred from assessment of learning to assessment for learning. This has important implications for the conceptual framework from which to approach the issue of assessment, but also with respect to the research agenda. The main conceptual changes pertain to programmes of assessment. This has led to a broadened perspective on the types of construct assessment tries to capture, the way information from various sources is collected and collated, the role of human judgement and the variety of psychometric methods to determine the quality of the assessment. Research into the quality of assessment programmes, how assessment influences learning and teaching, new psychometric models and the role of human judgement is much needed. [ABSTRACT FROM AUTHOR]

Background: Publications in a variety of journals have described the problem of medical students'' decreasing anatomical knowledge. Interestingly, the number of people making this assertion is growing, despite a lack of empirical evidence that today's medical graduates actually know less about anatomy than medical students in the past. Nevertheless, many people are claiming that students'' anatomical knowledge is impaired due to negative effects from several factors, including teaching by non-medically qualified teachers, diminished use of cadaver dissection as a teaching tool and neglect of vertical integration of anatomy teaching. Aim: To find empirical evidence for the factors claimed to have an influence on anatomical knowledge of students. Method: A literature search. Results: There is a lack of sufficient quantity and quality of information within the existing literature to support any of the claims, but the gathered literature did reveal some fascinating insights which are discussed. Conclusion: Anatomy education should be made as effective as possible, as nobody will deny that medical students cannot do without anatomical knowledge. Because of promising findings in the areas of teaching in context, vertical integration and assessment strategies, it is recommended that future research into anatomy education should focus on these factors. [ABSTRACT FROM AUTHOR]

Background: Publications in a variety of journals have described the problem of medical students'' decreasing anatomical knowledge. Interestingly, the number of people making this assertion is growing, despite a lack of empirical evidence that today's medical graduates actually know less about anatomy than medical students in the past. Nevertheless, many people are claiming that students'' anatomical knowledge is impaired due to negative effects from several factors, including teaching by non-medically qualified teachers, diminished use of cadaver dissection as a teaching tool and neglect of vertical integration of anatomy teaching. Aim: To find empirical evidence for the factors claimed to have an influence on anatomical knowledge of students. Method: A literature search. Results: There is a lack of sufficient quantity and quality of information within the existing literature to support any of the claims, but the gathered literature did reveal some fascinating insights which are discussed. Conclusion: Anatomy education should be made as effective as possible, as nobody will deny that medical students cannot do without anatomical knowledge. Because of promising findings in the areas of teaching in context, vertical integration and assessment strategies, it is recommended that future research into anatomy education should focus on these factors. [ABSTRACT FROM AUTHOR]

OBJECTIVES A mandatory remedial programme for students who repeat their first semester at medical school has resulted in large gains in academic performance and greatly reduced attrition. Here, we explore the students' views of this in order to clarify understanding of optimal remediation practice. METHODS Using a mixed-methods approach, quantitative and qualitative data were gathered from student surveys (n = 333) and three in-depth focus groups. Results were analysed for emergent themes. RESULTS Remedial programmes for at-risk medical students should be mandatory, but should respect students' identity as repeaters. Attitude and motivation are key, and working in stable groups provides essential emotional and cognitive support. The learning environment needs to foster changes in students' ways of thinking and their development as flexible, reflective learners. These endeavours require support from honest teachers with rigorous expectations and good facilitation skills. CONCLUSIONS Successful remediation needs to challenge students' conceptions of learning, works best in groups with skilled facilitators, and must take into account a blend of cognitive and affective factors and the complex interplay between learner and environment. Given a carefully designed programme, at-risk medical students can learn to make effective and lasting changes to their approach to study, and their views of learning can come to converge with influential ideas in the education literature. [ABSTRACT FROM AUTHOR]

Background: Many medical schools provide academic support programmes to aid increasing numbers of students from diverse backgrounds. There have been calls for research into successful intervention programmes, and for detailed descriptions of how they work. Aims: To explore the efficacy of a mandatory intervention programme for at-risk medical students. Method: Students who failed and then repeated first semester were required to participate in a cognitive skills programme, following a syllabus based on principles drawn from both educational experience and multi-disciplinary theory and practice. Performance of programme participants was compared to the performance of students who repeated prior to the mandatory programme. Results: Of the participants ( n = 216), 91% passed their repeat semester, compared to 58% ( n = 715) for controls ( p < 0.0001). This significant effect persisted for progression through the school for the subsequent three semesters ( p < 0.0005). Conclusion: A mandatory programme that draws on a blend of theories and research-proven techniques can make a positive difference to the outcomes for at-risk medical students. [ABSTRACT FROM AUTHOR]

Background: Teachers involved in test development usually prefer criterion-referenced standard setting methods using panels. Since expert panels are costly, standards are often set by a pre-fixed percentage of questions answered correctly or norm-referenced methods aimed at ranking examinees. Aim: To discuss the (dis)advantages of commonly used criterion and norm-referenced methods and present a new compromise method: standards based on a fixed cut-off score using the best scoring students as reference point. Methods: Historical data from 54 Maastricht (norm-referenced) and 52 Groningen (criterion-referenced) tests were used to demonstrate huge discrepancies and variability in cut-off scores and failure rates. Subsequently, the compromise model – known as Cohen's method – was applied to the Groningen tests. Results: The Maastricht norm-referenced method led to a large variation in required cut-off scores (15–46%), but a stable failure rate (about 17%). The Groningen method with a conventional, pre-fixed standard of 60% led to a large variation in failure rates (17–97%). The compromise method reduced variation in required cut-off scores as well as failure rates. Conclusion: Both the criterion and norm-referenced standards, used in practice, have disadvantages. The proposed compromise model reduces the disadvantages of both methods and is considered more acceptable. Last but not least, compared to standard setting methods using panels, this method is affordable. [ABSTRACT FROM AUTHOR]

Objective The drive towards valid and reliable assessment methods for health professions’ training is becoming increasingly focused towards authentic models of workplace performance assessment. This study investigates the validity of such a method, longitudinal evaluation of performance (LEP), which has been implemented in the assessment of postgraduate dental trainees in Scotland. Although it is similar in format to the mini-CEX (mini clinical evaluation exercise) and other tools that use global ratings for assessing performance in the workplace, a number of differences exist in the way in which the LEP has been implemented. These include the use of a reference point for evaluators’ judgement that represents the standard expected upon completion of the training, flexibility, a greater range of cases assessed and the use of frequency scores within feedback to identify trainees’ progress over time. Methods A range of qualitative and quantitative data were collected and analysed from 2 consecutive cohorts of trainees in Scotland (2002–03 and 2003–04). Results There is rich evidence supporting the validity, educational impact and feasibility of the LEP. In particular, a great deal of support was given by trainers for the use of a fixed reference point for judgements, despite initial concerns that this might be demotivating to trainees. Trainers were highly positive about this approach and considered it useful in identifying trainees’ progress and helping to drive learning. Conclusions The LEP has been successful in combining a strong formative approach to continuous assessment with the collection of evidence on performance within the workplace that (alongside other tools within an assessment system) can contribute towards a summative decision regarding competence. [ABSTRACT FROM AUTHOR]

We use a utility model to illustrate that, firstly, selecting an assessment method involves context-dependent compromises, and secondly, that assessment is not a measurement problem but an instructional design problem, comprising educational, implementation and resource aspects. In the model, assessment characteristics are differently weighted depending on the purpose and context of the assessment.Of the characteristics in the model, we focus on reliability, validity and educational impact and argue that they are not inherent qualities of any instrument. Reliability depends not on structuring or standardisation but on sampling. Key issues concerning validity are authenticity and integration of competencies. Assessment in medical education addresses complex competencies and thus requires quantitative and qualitative information from different sources as well as professional judgement. Adequate sampling across judges, instruments and contexts can ensure both validity and reliability. Despite recognition that assessment drives learning, this relationship has been little researched, possibly because of its strong context dependence.When assessment should stimulate learning and requires adequate sampling, in authentic contexts, of the performance of complex competencies that cannot be broken down into simple parts, we need to make a shift from individual methods to an integral programme, intertwined with the education programme. Therefore, we need an instructional design perspective.Programmatic instructional design hinges on a careful description and motivation of choices, whose effectiveness should be measured against the intended outcomes. We should not evaluate individual methods, but provide evidence of the utility of the assessment programme as a whole. [ABSTRACT FROM AUTHOR]

Written assessment techniques can be subdivided according to their stimulus format – what the questions asks – and their response format – how the answer is recorded. The former is more important in determining the type of competence being asked for than the latter. It is nevertheless important to consider both when selecting the most appropriate types. Some major elements to consider when making such a selection are cueing effect, reliability, validity, educational impact and resource-intensiveness. Open-ended questions should be used solely to test aspects that cannot be tested with multiple-choice questions. In all other cases the loss of reliability and the higher resource-intensiveness represent a significant downside. In such cases, multiple-choice questions are not less valid than open-ended questions. When making this distinction, it is important to consider whether the question is embedded within a relevant case or context and cannot be answered without the case, or not. This appears to be more or less essential according to what is being tested by the question. Context-rich questions test other cognitive skills than do context-free questions. If knowledge alone is the purpose of the test, context-free questions may be useful, but if it is the application of knowledge or knowledge as a part of problem solving that is being tested, then context is indispensable. Every format has its (dis)advantages and a combination of formats based on rational selection is more useful than trying to find or develop a panacea. The response format is less important in this respect than the stimulus. [ABSTRACT FROM AUTHOR]

In medical education, assessment of medical competence and performance, important changes have taken place in the last 5 decades. These changes have affected the basic concepts in all 3 domains. In education constructivism has provided a completely new view on how students learn best. In assessment the change from trait-orientated to competency- or role-orientated thinking has given rise to a whole range of new approaches. Certain methods of education, such as problem-based learning (PBL), and assessment, however, are often seen as almost synonymous with the underlying concepts, and one tends to forget that it is the concept that is important and that a particular method is but 1 way of using a concept. When doing this, one runs the risk of confusing means and ends, which may hamper or slow down new developments. A similar problem seems to occur often in research of medical education. Here too, methods – or, rather, methodologies – are confused with research questions. This may lead to an overemphasis on research that fits well known methodologies (e.g. the randomised controlled trial) and neglect of what are sometimes even more important research questions because they do not fit well known methodologies. In this paper we advocate a return to the underlying concepts and a careful reflection of their use in various situations. [ABSTRACT FROM AUTHOR]

The authors of this article, an assistant professor and a professor of medical education, share their perspectives on developments that need to take place in order for the field of medical education to advance. They address the necessities of effective practical training, accurate and consistent assessment, higher standards in research, and the cultivation of constructive perspectives on assessment.

Discusses general issues of written assessment examinations within medical education. Most commonly used types of assessments; Major advantages and disadvantages; Five criteria used to evaluated the advantages and disadvantages of question types.

AbstractIt is assumed that a programmatic approach to assessment supports students’ self-regulated learning (SRL). This study investigated students’ perceptions of this assumed support. Using a multiple-case study design, this study examined students’ perceptions in two distinct study programmes. In each case, first-year students were enrolled in an assessment programme that was designed in accordance with the principles of programmatic assessment, which integrate both assessment purposes of assessment for learning (AfL) and assessment of learning (AoL). The second-year students in each case were enrolled in an assessment programme with a traditional AoL assessment approach. The findings suggest that in both cases first-year students perceived their assessment programme as more positive for their learning than second-year students. Through a cross-case analysis of students’ perceptions of the two assessment programmes, three themes were identified that are essential to support students’ SRL: 1) assessment design; 2) assessment as a dialogue; and 3) assessment as an information source. Key aspects of the themes that support or hinder SRL are discussed. The findings of this study highlight the importance of programme-level assessment design to support students in their SRL. [ABSTRACT FROM AUTHOR]

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived dominance of the summative assessment paradigm throughout medical education reduces learners' willingness to seek feedback, and encourages supervisors to mix up feedback with provision of 'objective' grades or pass/fail marks. This eye-opener article argues that the provision and reception of effective feedback by clinical supervisors and their learners is dependent on both parties' awareness of the important distinction between feedback used in coaching towards growth and development (assessment for learning) and reaching a high-stakes judgement on the learner's competence and fitness for practice (assessment of learning). Using driving lessons and the driving test as a metaphor for feedback and assessment helps supervisors and learners to understand this crucial difference and to act upon it. It is the supervisor's responsibility to ensure that supervisor and learner achieve a clear mutual understanding of the purpose of each interaction (i.e. feedback or assessment). To allow supervisors to use the driving lesson—driving test metaphor for this purpose in their interactions with learners, it should be included in faculty development initiatives, along with a discussion of the key importance of separating feedback from assessment, to promote a feedback culture of growth and support programmatic assessment of competence. [ABSTRACT FROM AUTHOR]

In medical education, students are increasingly regarded as active seekers of feedback rather than passive recipients. Previous research showed that in the intentions of students to seek feedback, a learning and performance goal can be distinguished. In this study, we investigated the intentions (defined as level and orientation of motivation) of different performing students (low, average, and high performing students) to seek feedback in the clinical workplace using Self-Determination Theory. We conducted a quantitative study with students in their clinical clerkships and grouped them based on their performance. The level of motivation was measured by the number of Mini-CEXs each student collected. The orientation of motivation was measured by conducting the Academic Self-Regulation Questionnaire. We found that high performing students were more motivated and demonstrated higher self-determination compared to low performing students. [ABSTRACT FROM AUTHOR]

Background: Tutorial group effectiveness is essential for the success of learning in problem-based learning (PBL). Less effective and dysfunctional groups compromise the quality of students learning in PBL. Aims: This article aims to report on the reliability and validity of an instrument aimed at measuring tutorial group effectiveness in PBL. Method: The items within the instrument are clustered around motivational and cognitive factors based on Slavin's theoretical framework. A confirmatory factor analysis (CFA) was carried out to estimate the validity of the instrument. Furthermore, generalizability studies were conducted and alpha coefficients were computed to determine the reliability and homogeneity of each factor. Results: The CFA indicated that a three-factor model comprising 19 items showed a good fit with the data. Alpha coefficients per factor were high. The findings of the generalizability studies indicated that at least 9–10 student responses are needed in order to obtain reliable data at the tutorial group level. Conclusion: The instrument validated in this study has the potential to provide faculty and students with diagnostic information and feedback about student behaviors that enhance and hinder tutorial group effectiveness. [ABSTRACT FROM AUTHOR]

According to cognitive load theory, instructions can impose three types of cognitive load on the learner: intrinsic load, extraneous load, and germane load. Proper measurement of the different types of cognitive load can help us understand why the effectiveness and efficiency of learning environments may differ as a function of instructional formats and learner characteristics. In this article, we present a ten-item instrument for the measurement of the three types of cognitive load. Principal component analysis on data from a lecture in statistics for PhD students ( n = 56) in psychology and health sciences revealed a three-component solution, consistent with the types of load that the different items were intended to measure. This solution was confirmed by a confirmatory factor analysis of data from three lectures in statistics for different cohorts of bachelor students in the social and health sciences ( ns = 171, 136, and 148), and received further support from a randomized experiment with university freshmen in the health sciences ( n = 58). [ABSTRACT FROM AUTHOR]

Medical Education 2012: 46: 593-603 Objectives Regulatory focus theory posits the existence of two systems of self-regulation underlying human motivation: promotion focus, which is concerned with aspirations and accomplishments, and prevention focus, which is concerned with obligations and responsibilities. It has been proposed that regulatory focus theory may help to explain learners' variable responses to feedback, predicting that positive feedback is motivating under promotion focus, whereas negative feedback is motivating under prevention focus. We aimed to explore this link between regulatory focus theory and response to feedback using data collected in a naturalistic setting. Methods In a constructivist grounded theory study, we interviewed 22 early-career academic doctors about experiences they perceived as influential in their learning. Although feedback emerged as important, responses to feedback were highly variable. To better understand how feedback becomes (or fails to become) influential, we used the theoretical framework of regulatory focus to re-examine all descriptions of experiences of receiving and responding to feedback. Results Feedback could be influential or non-influential, regardless of its sign (positive or negative). In circumstances in which the individual's regulatory focus was readily determined, such as in choosing a career (promotion) or preparing for a high-stakes examination (prevention), the apparent influence of feedback was consistent with the prediction of regulatory focus theory. However, we encountered many challenges in applying regulatory focus theory to real feedback scenarios, including the frequent presence of a mixed regulatory focus, the potential for regulatory focus to change over time, and the competing influences of other factors, such as the perceived credibility of the source or content of the feedback. Conclusions Regulatory focus theory offers a useful, if limited, construct for exploring learners' responses to feedback in the clinical setting. The insights and predictions it offers must be considered in light of the motivational complexity of clinical learning tasks and of other factors influencing the impact of feedback. [ABSTRACT FROM AUTHOR]

Medical Education 2012: 46 : 192-200 Context: How learners interpret their clinical experiences to create meaningful learning has not been well studied. We explored experiences considered by doctors to be influential in their learning in order to better understand this process. Methods: Using a grounded theory approach, we interviewed 22 academic doctors who had been in practice for ≤ 5 years. Participants were asked to reflect on experiences they considered to have been influential during their training. Constant comparative analysis for emerging themes was conducted iteratively with data collection. Results: A model of clinical learning emerged in which the clinical work itself is central. As they observe and participate in clinical work, learners can attend to a variety of sources of information that facilitate the interpretation of the experience and the construction of knowledge from it. These 'learning cues' include feedback, role models, clinical outcomes, patient or family responses, and comparisons with peers. The integration of a cue depends on the learner's judgement of its credibility. Certain cues, such as clinical outcomes or feedback from patients, are seen as innately credible, whereas other cues, particularly feedback from supervisors, are subjected to critical judgement. Conclusions: Learners make complex judgements regarding the credibility of information about clinical performance. Credibility judgements influence the learning that arises from the clinical experience. Further understanding of how such judgements are made could guide educators in providing credible information to learners. [ABSTRACT FROM AUTHOR]

Context The clinical learning climate affects undergraduate medical students’ behaviour, satisfaction and success. Most studies predominantly describe aspects of the clinical learning climate using quantitative methodologies, such as questionnaires. This study aimed to illuminate medical students’ perceptions of the clinical learning climate, and which factors and their interactions explain differences in clinical learning climates. Methods We carried out a multi-method case study. Twelve departments of obstetrics and gynaecology distributed the Postgraduate Hospital Educational Environment Measure (PHEEM), a reliable questionnaire measuring the clinical learning environment, among medical students. After analysis (usinganova and post hoc tests), 14 medical students from the highest- and lowest-scoring departments participated in semi-structured interviews. We analysed the transcribed recordings using a content analysis approach. Researchers agreed on coding and an expert group reached consensus on the themes of the analysis. Results We found a significant difference between departments in PHEEM scores. The interviews indicated that department and medical student characteristics determine the clinical learning climate. For departments, ‘legitimacy’, ‘clerkship arrangements’ and ‘focus on personal development’ were the main themes. For medical students, ‘initial initiatives’, ‘continuing development’ and ‘clerkship fatigue’ were the principal themes. The amount and nature of participation played a central role in all themes. Conclusions Differences between clinical learning climates appear to be related to differing approaches to participation among departments. Participation depends on characteristics of both departments and students, and the interactions among them. The outcomes give valuable clues to how a favourable clinical learning climate is shaped. [ABSTRACT FROM AUTHOR]

Objective The aim of this study was to evaluate the applicability of a previously designed set of generic objectives for community-based education (CBE) emphasising community involvement. Methods The study was designed as a non-blinded, randomised trial. Experimental and conventional groups of students following CBE programmes either closely or weakly matching the set of generic objectives were compared. Student groups were subjected to passive participatory observation. Students evaluated their programmes through questionnaires. The impact of student interventions was assessed by community compliance. Community perception of the programmes was evaluated through structured interviews with community representatives. Results Students in experimental groups appreciated their programme more than students in conventional groups. High compliance and appreciation were recorded in communities hosting the modified programme. Most students in conventional groups judged their posting negatively, largely because of the high number of households to be visited. Health interventions performed by conventional groups lacked co-operation between students and the community. Communities hosting conventional groups felt their health needs were scarcely discussed and addressed. Conclusions The modification of an existing CBE programme to better match a set of generic CBE objectives emphasising community involvement had a positive effect on programme outcomes and levels of appreciation by both students and hosting communities. However, some confounding variables could not be controlled. Colleagues planning comparable studies may take advantage of the lessons we learned while performing this study. [ABSTRACT FROM AUTHOR]

There are data that suggest that medical students do not feel sufficiently prepared for clinical practice in the clerkships. The transition from pre-clinical to clinical training causes problems. To seek quantitative verification of qualitative findings from an earlier focus group study on problems medical students encounter when entering the clinical phase of undergraduate training. At the start of the clinical phase, all Year 4 students at Maastricht Medical School were surveyed on the transition from pre-clinical to clinical training and its effects on workload, knowledge, skills and learning. The response rate was 67%. Students were uncertain as to how to behave and act, mainly because they did not know what was expected of them. They experienced a drastic increase in workload and a lack of time for studying. They considered themselves to be moderately prepared with regard to knowledge and they regarded their physical examination skills as satisfactory. Students reported having difficulty applying theoretical knowledge in clinical practice and perceived shortcomings in basic science knowledge. In addition, they felt compelled to change their learning strategies. The results of this study confirm the findings of the focus group study. The students experienced problems related to professional socialisation and workload and deficiencies in knowledge and the organisation of knowledge. A good starting point for improvement may involve exploring students' suggestions of an extensive introduction into the clerkships, a more gradual transition with regard to workload and closer integration of pre-clinical and clinical education. [ABSTRACT FROM AUTHOR]

To explore learning outcomes from international traineeships for undergraduate medical students. In-depth interviews were conducted with 24 undergraduate medical students at Maastricht University Medical School, the Netherlands. The 24 subjects were selected by purposeful sampling. Students reported meaningful learning outcomes in 6 domains: medical knowledge; skills; international health care organisation; international medical education; society and culture, and personal growth. International traineeships appear to provide good opportunities for students to meet the requirements of globalisation as well as some of the generic objectives of undergraduate medical education. The tentative findings of this study need to be confirmed by further studies. [ABSTRACT FROM AUTHOR]

In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment. For fifteen programmes in health professions education affiliated with members of an expert group (n = 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research. Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes. [ABSTRACT FROM AUTHOR]

Introduction Much of undergraduate clinical teaching is provided by residents. An earlier study showed the attitude of residents towards teaching to be generally positive. Little is known, however, about attending doctors' views on their own and residents' roles as teachers of medical students. Objectives To examine attending doctors' perceptions of the (dis)advantages of resident teaching, their own teaching abilities and the need for a teacher training programme for residents. Method A questionnaire survey of 76 attending doctors was carried out in the Departments of Obstetrics & Gynaecology and Paediatrics at the teaching hospitals of the Universities of Maastricht and Amsterdam, the Netherlands. Results Attending doctors perceive teaching by residents to be beneficial for students and residents alike. Although they consider themselves to be better suited than residents to teach medical students, they see teaching as an integral part of residency training and feel it should be recognised as such by departments and medical schools. Attending doctors are in favour of a teacher training programme for residents, which should include communication, clinical and teaching skills as well as skills such as time management and (self-) assessment. Discussion Despite the uneven distribution of participants between the departments, no significant differences were found between departments. It is interesting that attending doctors perceive teacher training as beneficial to residents' teaching skills, but provide more feedback on residents' attitudes than on their teaching. The results show that, in general, attending doctors share residents' views that teaching is an important component of residency and that a teacher training programme for residents is to be recommended. [ABSTRACT FROM AUTHOR]

Introduction Problem-based learning (PBL) has gained a foothold within many schools in higher education as a response to the problems faced within traditional education. Discussion Working with PBL tutorial groups is assumed to have positive effects on student learning. Several studies provide empirical evidence that PBL stimulates cognitive effects and leads to restructuring of knowledge and enhanced intrinsic interest in the subject matter. However, staff members do not always experience the positive effects of group work which they had hoped for. When confronted with problems in group work, such as students who only maintain an appearance of being actively involved and students who let others do the work, teachers all too often implement solutions which can be characterized as teacher- directed rather than student-directed. Teachers tend to choose solutions which are familiar from their own experience during professional training, i.e. using the teacher-directed model. These solutions are not effective in improving group work and the negative experiences persist. Conclusion It is argued that teachers should hold on to the underlying educational philosophy when solving problems arising from group work in PBL, by choosing actions which are consistent with the student-directed view of education in PBL. [ABSTRACT FROM AUTHOR]

ObjectivesResearch on tutoring in problem-based learning has not focused so far on the variation in tutoring and how this variation can be interpreted by conceptions about effective tutoring. DesignThis study focuses on the profiles of tutors generated by means of an instrument, the so-called Tutor Intervention Profile (TIP), and tries to determine which profiles are more or less effective. The TIP contains four dimensions of tutor behaviour: (1) elaboration; (2) directing the learning process; (3) integration of knowledge; and (4) stimulating interaction and individual accountability. SettingThe medical school of the University of Maastricht, The Netherlands. SubjectsSixty-seven tutors who run 67 tutorial groups across three units (courses) in the academic year 1996–97. ResultsIt appeared that high, average and low performing tutors differ in their performance on each of the four dimensions of the TIP. Several different profiles of tutor performance could be distinguished, which were more or less effective. One group of tutors demonstrated a tutor intervention profile that was characterized as relying more on the use of expert knowledge, whereas another group of tutors was characterized as relying more on their abilities to stimulate the learning process in the tutorial group. The tutor intervention profile that was perceived by students as most effective showed high scores on each of the four dimensions, as expected. Notably, a tutor stressing the learning process in the tutorial group was perceived as more effective than a tutor stressing content (expert tutor). This is especially true for a relatively poor scoring tutor. ConclusionsThe results of this study are consistent with research on human tutoring and research on tutoring in problem-based learning. [ABSTRACT FROM AUTHOR]

Introduction: With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores. Methods: During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242 residents, 999 coworkers, and themselves in three MSF performance domains. Mixed-effect models quantified associations between the outcome variable "score changes" between first and second MSF evaluations, and the explanatory variable "negative discrepancy score" (number of items that physicians rated themselves higher compared with their assessors' scores) at the first MSF evaluation. Whether associations differed across assessor groups and across a physician's years of experience as a doctor was analyzed too. Results: Forty-nine percent of physicians improved their total MSF score at the second evaluation, as assessed by others. Number of negative discrepancies was negatively associated with score changes in domains "organization and (self)management" (b = 20.02; 95% confidence interval [CI], 20.03 to 20.02; SE = 0.004) and "patient-centeredness" (b = 20.03; 95% CI, 20.03 to 20.02; SE = 0.004). For "professional attitude," only negative associations between score changes and negative discrepancies existed for physicians with more than 6-year experience (b6-10yearsofexperience = 20.03; 95% CI, 20.05 to 20.003; SE = 0.01; b16-20yearsofexperience = 20.03; 95% CI, 20.06 to 20.004; SE = 0.01). Discussion: The extent of performance improvement was less for physicians confronted with negative discrepancies. Performance scores actually declined when physicians overrated themselves on more than half of the feedback items. PA score changes of more experienced physicians confronted with negative discrepancies and were affected more adversely. These physicians might have discounted feedback due to having more confidence in own performance. Future work should investigate how MSF could improve physicians' performance taking into account physicians' confidence. [ABSTRACT FROM AUTHOR]

This AMEE guide provides a framework and practical strategies for teachers, learners and institutions to promote meaningful feedback conversations that emphasise performance improvement and professional growth. Recommended strategies are based on recent feedback research and literature, which emphasise the sociocultural nature of these complex interactions. We use key concepts from three theories as the underpinnings of the recommended strategies: sociocultural, politeness and self-determination theories. We view the content and impact of feedback conversations through the perspective of learners, teachers and institutions, always focussing on learner growth. The guide emphasises the role of teachers in forming educational alliances with their learners, setting a safe learning climate, fostering self-awareness about their performance, engaging with learners in informed self-assessment and reflection, and co-creating the learning environment and learning opportunities with their learners. We highlight the role of institutions in enhancing the feedback culture by encouraging a growth mind-set and a learning goal-orientation. Practical advice is provided on techniques and strategies that can be used and applied by learners, teachers and institutions to effectively foster all these elements. Finally, we highlight throughout the critical importance of congruence between the three levels of culture: unwritten values, espoused values and day to day behaviours. [ABSTRACT FROM AUTHOR]

Purpose: According to the principles of programmatic assessment, a valid high-stakes assessment of the students' performance should amongst others, be based on a multiple data points, supposedly leading to saturation of information. Saturation of information is generated when a data point does not add important information to the assessor. In establishing saturation of information, institutions often set minimum requirements for the number of assessment data points to be included in the portfolio. Methods: In this study, we aimed to provide validity evidence for saturation of information by investigating the relationship between the number of data points exceeding the minimum requirements in a portfolio and the consensus between two independent assessors. Data were analyzed using a multiple logistic regression model. Results: The results showed no relation between the number of data points and the consensus. This suggests that either the consensus is predicted by other factors only, or, more likely, that assessors already reached saturation of information. This study took the first step in investigating saturation of information, further research is necessary to gain in-depth insights of this matter in relation to the complex process of decision-making. [ABSTRACT FROM AUTHOR]

Feedback in medical education has traditionally showcased techniques and skills of giving feedback, and models used in staff development have focused on feedback providers (teachers) not receivers (learners). More recent definitions have questioned this approach, arguing that the impact of feedback lies in learner acceptance and assimilation of feedback with improvement in practice and professional growth. Over the last decade, research findings have emphasized that feedback conversations are complex interpersonal interactions influenced by a multitude of sociocultural factors. However, feedback culture is a concept that is challenging to define, thus strategies to enhance culture are difficult to pin down. In this twelve tips paper, we have attempted to define elements that constitute a feedback culture from four different perspectives and describe distinct strategies that can be used to foster a learning culture with a growth mind-set. [ABSTRACT FROM AUTHOR]

Background: The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT).Methods: The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory.Results: The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases.Conclusions: This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice. [ABSTRACT FROM AUTHOR]

Context: Single‐best‐answer questions (SBAQs) have been widely used to test knowledge because they are easy to mark and demonstrate high reliability. However, SBAQs have been criticised for being subject to cueing. Objectives: We used a novel assessment tool that facilitates efficient marking of open‐ended very‐short‐answer questions (VSAQs). We compared VSAQs with SBAQs with regard to reliability, discrimination and student performance, and evaluated the acceptability of VSAQs. Methods: Medical students were randomised to sit a 60‐question assessment administered in either VSAQ and then SBAQ format (Group 1, n = 155) or the reverse (Group 2, n = 144). The VSAQs were delivered on a tablet; responses were computer‐marked and subsequently reviewed by two examiners. The standard error of measurement (SEM) across the ability spectrum was estimated using item response theory. Results: The review of machine‐marked questions took an average of 1 minute, 36 seconds per question for all students. The VSAQs had high reliability (alpha: 0.91), a significantly lower SEM than the SBAQs (p < 0.001) and higher mean item–total point biserial correlations (p < 0.001). The VSAQ scores were significantly lower than the SBAQ scores (p < 0.001). The difference in scores between VSAQs and SBAQs was attenuated in Group 2. Although 80.4% of students found the VSAQs more difficult, 69.2% found them more authentic. Conclusions: The VSAQ format demonstrated high reliability and discrimination and items were perceived as more authentic. The SBAQ format was associated with significant cueing. The present results suggest the VSAQ format has a higher degree of validity. [ABSTRACT FROM AUTHOR]

Where do we stand now? In the 30 years that have passed since The Edinburgh Declaration on Medical Education, we have made tremendous progress in research on fostering 'self-directed and independent study' as propagated in this declaration, of which one prime example is research carried out on problem-based learning. However, a large portion of medical education happens outside of classrooms, in authentic clinical contexts. Therefore, this article discusses recent developments in research regarding fostering active learning in clinical contexts. Self-regulated, lifelong learning in medical education Clinical contexts are much more complex and flexible than classrooms, and therefore require a modified approach when fostering active learning. Recent efforts have been increasingly focused on understanding the more complex subject of supporting active learning in clinical contexts. One way of doing this is by using theory regarding self-regulated learning ( SRL), as well as situated learning, workplace affordances, self-determination theory and achievement goal theory. Combining these different perspectives provides a holistic view of active learning in clinical contexts. Entry to practice, vocational training and continuing professional development Research on SRL in clinical contexts has mostly focused on the undergraduate setting, showing that active learning in clinical contexts requires not only proficiency in metacognition and SRL, but also in reactive, opportunistic learning. These studies have also made us aware of the large influence one's social environment has on SRL, the importance of professional relationships for learners, and the role of identity development in learning in clinical contexts. Additionally, research regarding postgraduate lifelong learning also highlights the importance of learners interacting about learning in clinical contexts, as well as the difficulties that clinical contexts may pose for lifelong learning. However, stimulating self-regulated learning in undergraduate medical education may also make postgraduate lifelong learning easier for learners in clinical contexts. [ABSTRACT FROM AUTHOR]

The successful remediation of clinicians demonstrating poor performance in the workplace is essential to ensure the provision of safe patient care. Clinicians may develop performance problems for numerous reasons, including health, personal factors, the workplace environment, or outdated knowledge/skills. Performance problems are often complex involving multifactorial issues, encompassing knowledge, skills, and professional behaviors. It is important that (where possible and appropriate) clinicians are supported through effective remediation to return them to safe clinical practice. A review of the literature demonstrated that research into remediation is in its infancy, with little known about the effectiveness of remediation programs currently. Current strategies for the development of remediation programs are mostly "intuitive"; a few draw upon established theories to inform their approach. Similarly, although it has been established that identification of the nature/scope of performance problems through assessment is an essential first step within remediation, the need for a more widespread "diagnosis" of why the problems exist is emerging. These reasons for poor performance, particularly in the context of experienced practicing clinicians, are likely to have an impact on the potential success of remediation and should be considered within the "diagnosis." A new model for diagnosing the performance problems of the clinicians has been developed, using behavioral change theories to explore known barriers to successful remediation, such as insight, motivation, attitude, self-efficacy and the working environment, in addition to addressing known deficits regarding knowledge and skills. This novel approach is described in this article. An initial feasibility study has demonstrated the acceptability and practical implementation of our model. [ABSTRACT FROM AUTHOR]

Background:It remains unclear which item format would best suit the assessment of clinical reasoning: context-rich single best answer questions (crSBAs) or key-feature problems (KFPs). This study compared KFPs and crSBAs with respect to students’ acceptance, their educational impact, and psychometric characteristics when used in a summative end-of-clinical-clerkship pediatric exam. Methods:Fifth-year medical students (n = 377) took a computer-based exam that included 6–9 KFPs and 9–20 crSBAs which assessed their clinical reasoning skills, in addition to an objective structured clinical exam (OSCE) that assessed their clinical skills. Each KFP consisted of a case vignette and three key features using a “long-menu” question format. We explored students’ perceptions of the KFPs and crSBAs in eight focus groups and analyzed statistical data of 11 exams. Results:Compared to crSBAs, KFPs were perceived as more realistic and difficult, providing a greater stimulus for the intense study of clinical reasoning, and were generally well accepted. The statistical analysis revealed no difference in difficulty, but KFPs resulted more reliable and efficient than crSBAs. The correlation between the two formats was high, while KFPs correlated more closely with the OSCE score. Conclusions:KFPs with long-menu exams seem to bring about a positive educational effect without psychometric drawbacks. [ABSTRACT FROM PUBLISHER]

Introduction: Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT. Methods: The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's a and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated. Results: For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's a $ 0.84 and item-total correlations $ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three tofour coworkers were sufficient. Discussion: The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor [ABSTRACT FROM AUTHOR]

Background: Medical education in Sub-Saharan Africa is in need of reform to promote the number and quality of physicians trained. Curriculum change and innovation in this region, however, face a challenging context that may affect curriculum outcomes. Research on outcomes of curriculum innovation in Sub-Saharan Africa is scarce. We investigated curriculum outcomes in a Sub-Saharan African context by comparing students' perceived preparedness for practice within three curricula in Mozambique: a conventional curriculum and two innovative curricula. Both innovative curricula used problem-based learning and community-based education. Methods: We conducted a comparative mixed methods study. We adapted a validated questionnaire on perceived professional competencies and administered it to 5th year students of the three curricula (n = 140). We conducted semi-structured interviews with 5th year students from these curricula (n = 12). Additional contextual information was collected. Statistical and thematic analyses were conducted. Results: Perceived preparedness for practice of students from the conventional curriculum was significantly lower than for students from one innovative curriculum, but significantly higher than for students from the other innovative curriculum. Major human and material resource issues and disorganization impeded the latter's sense of preparedness. Both innovative curricula, however, stimulated a more holistic approach among students toward patients, as well an inquiring and independent attitude, which is valuable preparation for Sub-Saharan African healthcare. Discussion: In Sub-Saharan Africa, risks and benefits of curriculum innovation are high. Positive outcomes add value to local healthcare in terms of doctors' meaningful preparedness for practice, but instead outcomes can be negative due to the implementation challenges sometimes found in Sub-Saharan African contexts. Before embarking on innovative curriculum reform, medical schools need to assess their capability and motivation for innovation. [ABSTRACT FROM AUTHOR]

Context: Narrative feedback documented in performance evaluations by the teacher, i.e. the clinical supervisor, is generally accepted to be essential for workplace learning. Many studies have examined factors of influence on the usage of mini-clinical evaluation exercise (mini-CEX) instruments and provision of feedback, but little is known about how these factors influence teachers’ feedback-giving behaviour. In this study, we investigated teachers’ use of mini-CEX in performance evaluations to provide narrative feedback in undergraduate clinical training. Methods: We designed an exploratory qualitative study using an interpretive approach. Focusing on the usage of mini-CEX instruments in clinical training, we conducted semi-structured interviews to explore teachers’ perceptions. Between February and June 2013, we conducted interviews with 14 clinicians participated as teachers during undergraduate clinical clerkships. Informed by concepts from the literature, we coded interview transcripts and iteratively reduced and displayed data using template analysis. Results: We identified three main themes of interrelated factors that influenced teachers’ practice with regard to mini-CEX instruments: teacher-related factors; teacher–student interaction-related factors, and teacher–context interaction-related factors. Four issues (direct observation, relationship between teacher and student, verbal versus written feedback, formative versus summative purposes) that are pertinent to workplace-based performance evaluations were presented to clarify how different factors interact with each other and influence teachers’ feedback-giving behaviour. Embedding performance observation in clinical practice and establishing trustworthy teacher–student relationships in more longitudinal clinical clerkships were considered important in creating a learning environment that supports and facilitates the feedback exchange. Conclusion: Teachers’ feedback-giving behaviour within the clinical context results from the interaction between personal, interpersonal and contextual factors. Increasing insight into how teachers use mini-CEX instruments in daily practice may offer strategies for creating a professional learning culture in which feedback giving and seeking would be enhanced. [ABSTRACT FROM PUBLISHER]

Background: Evidence tailored to sub-Saharan Africa on outcomes of innovations in medical education is needed to encourage and advance their implementation in this region. Aim: To investigate preparedness for practice of students and graduates from an innovative and a conventional medical curriculum in a sub-Saharan African context. Methods: Using mixed methods we compared junior doctors and fifth-year students from two Mozambican medical schools: one with an innovative problem- and community-based curriculum and one with a conventional lecture- and discipline-based curriculum. A questionnaire on professional competencies was administered, semi-structured interviews were conducted, and work diaries were collected. The findings were integrated in a conceptual model. Results: Six areas of tension between global health care ideals and local health care practice emerged from the data that challenged doctors' motivation and preparedness for practice. Four elements of the innovative curriculum equipped students and graduates with skills, attitudes and competencies to better cope with these tensions. Students and graduates from the innovative curriculum rated significantly higher levels on various competencies and expressed more satisfaction with the curriculum and its usefulness for their work. Conclusion: An innovative problem- and community-based curriculum can improve sub-Saharan African doctors' motivation and preparedness to tackle the challenges of health care practice in this region. [ABSTRACT FROM AUTHOR]

Background: Teamwork between clinical teachers is a challenge in postgraduate medical training. Although there are several instruments available for measuring teamwork in health care, none of them are appropriate for teaching teams. The aim of this study is to develop an instrument (TeamQ) for measuring teamwork, to investigate its psychometric properties and to explore how clinical teachers assess their teamwork. Method: To select the items to be included in the TeamQ questionnaire, we conducted a content validation in 2011, using a Delphi procedure in which 40 experts were invited. Next, for pilot testing the preliminary tool, 1446 clinical teachers from 116 teaching teams were requested to complete the TeamQ questionnaire. For data analyses we used statistical strategies: principal component analysis, internal consistency reliability coefficient, and the number of evaluations needed to obtain reliable estimates. Lastly, the median TeamQ scores were calculated for teams to explore the levels of teamwork. Results: In total, 31 experts participated in the Delphi study. In total, 114 teams participated in the TeamQ pilot. The median team response was 7 evaluations per team. The principal component analysis revealed 11 factors; 8 were included. The reliability coefficients of the TeamQ scales ranged from 0.75 to 0.93. The generalizability analysis revealed that 5 to 7 evaluations were needed to obtain internal reliability coefficients of 0.70. In terms of teamwork, the clinical teachers scored residents' empowerment as the highest TeamQ scale and feedback culture as the area that would most benefit from improvement. Conclusions: This study provides initial evidence of the validity of an instrument for measuring teamwork in teaching teams. The high response rates and the low number of evaluations needed for reliably measuring teamwork indicate that TeamQ is feasible for use by teaching teams. Future research could explore the effectiveness of feedback on teamwork in follow up measurements. [ABSTRACT FROM AUTHOR]

Objective--To determine the perceived importance of specific competencies in professional veterinary practice and education among veterinarians in several countries. Design--Survey-based prospective study. Sample-- 1,137 veterinarians in 10 countries. Procedures--Veterinarians were invited via email to participate in the study. A framework of 18 competencies grouped into 7 domains (veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development) was used. Respondents rated the importance of each competency for veterinary professional practice and for veterinary education by use of a 9-point Likert scale in an online questionnaire. Quantitative statistical analyses were performed to assess the data. Results--All described competencies were perceived as having importance (with overall mean ratings [all countries] > 6.45/9) for professional practice and education. Competencies related to veterinary expertise had the highest ratings (overall mean, 8.33/9 for both professional practice and education). For the veterinary expertise, entrepreneurship, and scholarship domains, substantial differences (determined on the basis of statistical significance and effect size) were found in importance ratings among veterinarians in different countries. Conclusions and Clinical Relevance--Results indicated a general consensus regarding the importance of specific types of competencies in veterinary professional practice and education. Further research into the definition of competencies essential for veterinary professionals is needed to help inform an international dialogue on the subject. [ABSTRACT FROM AUTHOR]

Contrary to classical problem-based learning, in guided problem-based learning, the learning goals are predetermined by the instructor—on the basis of a detailed decomposition of the subject matter to be studied—to activate prior knowledge and to structure self-study and subsequent group discussion. This study investigated the effects of classical problem-based learning and guided problem-based learning, for different prior knowledge levels, on perceived value and usefulness of the learning activity and on conceptual understanding of statistics. Participants randomly assigned 110 students to 10 problem-based learning groups, and subsequently, the 10 groups were allocated randomly to classical problem-based learning or guided problem-based learning. The results indicate that guided problem-based learning tends to enhance conceptual understanding of statistics more thanbreak; classical problem-based learning once students have some prior knowledge of the subject. Furthermore, guided problem-based learning tends to increase students’ awareness of the value and usefulness of the learning activity. [ABSTRACT FROM PUBLISHER]

Context: E-learning resources, such as virtual patients (VPs), can be more effective when they are integrated in the curriculum. To gain insights that can inform guidelines for the curricular integration of VPs, we explored students' perceptions of scenarios with integrated and non-integrated VPs aimed at promoting clinical reasoning skills. Methods: During their paediatric clerkship, 116 fifth-year medical students were given at least ten VPs embedded in eight integrated scenarios and as non-integrated add-ons. The scenarios differed in the sequencing and alignment of VPs and related educational activities, tutor involvement, number of VPs, relevance to assessment and involvement of real patients. We sought students' perceptions on the VP scenarios in focus group interviews with eight groups of 4-7 randomly selected students ( n = 39). The interviews were recorded, transcribed and analysed qualitatively. Results: The analysis resulted in six themes reflecting students' perceptions of important features for effective curricular integration of VPs: (i) continuous and stable online access, (ii) increasing complexity, adapted to students' knowledge, (iii) VP-related workload offset by elimination of other activities, (iv) optimal sequencing (e.g.: lecture - 1 to 2 VP(s) - tutor-led small group discussion - real patient) and (V) optimal alignment of VPs and educational activities, (vi) inclusion of VP topics in assessment. Conclusions: The themes appear to offer starting points for the development of a framework to guide the curricular integration of VPs. Their impact needs to be confirmed by studies using quantitative controlled designs. [ABSTRACT FROM AUTHOR]

Background: Despite acknowledgement that the Canadian Medical Educational Directives for Specialists (CanMEDS) framework covers the relevant competencies of physicians, many educators and medical professionals struggle to translate the CanMEDS roles into comprehensive training programmes for specific specialties. Aim: To gain insight into the applicability of the CanMEDS framework to guide the design of educational programmes for specific specialties by exploring stakeholders' perceptions of specialty specific competencies and examining differences between those competencies and the CanMEDS framework. Methods: This case study is a sequel to a study among ObsGyn specialists. It explores the perspectives of patients, midwives, nurses, general practitioners, and hospital boards on gynaecological competencies and compares these with the CanMEDS framework. Results: Clinical expertise, reflective practice, collaboration, a holistic view, and involvement in practice management were perceived to be important competencies for gynaecological practice. Although all the competencies were covered by the CanMEDS framework, there were some mismatches between stakeholders' perceptions of the importance of some competencies and their position in the framework. Conclusion: The CanMEDS framework appears to offer relevant building blocks for specialty specific postgraduate training, which should be combined with the results of an exploration of specialty specific competencies to arrive at a postgraduate curriculum that is in alignment with professional practice. [ABSTRACT FROM AUTHOR]

Medical Education 2012: 46: 1087-1098 OBJECTIVES We previously developed a model of the pre-assessment learning effects of consequential assessment and started to validate it. The model comprises assessment factors, mechanism factors and learning effects. The purpose of this study was to continue the validation process. For stringency, we focused on a subset of assessment factor-learning effect associations that featured least commonly in a baseline qualitative study. Our aims were to determine whether these uncommon associations were operational in a broader but similar population to that in which the model was initially derived. METHODS A cross-sectional survey of 361 senior medical students at one medical school was undertaken using a purpose-made questionnaire based on a grounded theory and comprising pairs of written situational tests. In each pair, the manifestation of an assessment factor was varied. The frequencies at which learning effects were selected were compared for each item pair, using an adjusted alpha to assign significance. The frequencies at which mechanism factors were selected were calculated. RESULTS There were significant differences in the learning effect selected between the two scenarios of an item pair for 13 of this subset of 21 uncommon associations, even when a p-value of < 0.00625 was considered to indicate significance. Three mechanism factors were operational in most scenarios: agency; response efficacy, and response value. CONCLUSIONS For a subset of uncommon associations in the model, the role of most assessment factor-learning effect associations and the mechanism factors involved were supported in a broader but similar population to that in which the model was derived. Although model validation is an ongoing process, these results move the model one step closer to the stage of usefully informing interventions. Results illustrate how factors not typically included in studies of the learning effects of assessment could confound the results of interventions aimed at using assessment to influence learning. [ABSTRACT FROM AUTHOR]