Search

Program directors in surgical disciplines need more tools from the ACGME to help them use Milestone ratings to improve trainees' performance. This is especially true in competencies that are notoriously difficult to measure, such as professionalism (PROF) and interpersonal and communication skills (ICS). It is now widely understood that skills in these two areas have direct impact on patient care outcomes. This study investigated the potential for generating early predictors of final Milestone ratings within the PROF and ICS competency categories.This retrospective cohort study utilized Milestone ratings from all ACGME-accredited vascular surgery training programs, covering residents and fellows who completed training in June 2019. The outcome measure studied was the rate of achieving the recommended graduation target of Milestone Level 4 (possible range: 1-5), while the predictors were the Milestone ratings attained at earlier stages of training. Predictive probability values (PPVs) were calculated for each of the 3 PROF and two ICS sub-competencies to estimate the probability of trainees not reaching the recommended graduation target based on their previous Milestone ratings.All ACGME-accredited vascular surgery training programs within the United States.All trainees completing a 2 year vascular surgery fellowship (VSF) in June 2019 (n = 119) or a 5 year integrated vascular surgery residency (IVSR) in June 2019 (n = 52) were included in the analyses.The overall rate of failing to achieve the recommended graduation target across all PROF and ICS sub-competencies ranged from 7.7% to 21.8% of all trainees. For trainees with a Milestone rating at ≤ 2.5 with 1 year remaining in their training program, the predictive probability of not achieving the recommended graduation target ranged from 37.0% to 71.5% across sub-competencies, with the highest risks observed under PROF for "Administrative Tasks" (71.5%) and under ICS for "Communication with the Healthcare Team" (56.7%).As many as 1 in 4 vascular surgery trainees did not achieve the ACGME vascular surgery Milestones targets for graduation in at least one of the PROF and ICS sub-competencies. Biannual ACGME Milestone assessment ratings of PROF and ICS during early training can be used to predict achievement of competency targets at time of graduation. Early clues to problems in PROF and ICS enable programs to address potential deficits early in training to ensure competency in these essential non-technical skills prior to entering unsupervised practice.

The quality and effectiveness of vascular surgery education should be evaluated based on patient care outcomes. To investigate predictive associations between trainee performance and subsequent patient outcomes, a critical first step is to determine the conceptual alignment of educational competencies with clinical outcomes in practice. We sought to generate expert consensus on the conceptual alignment of the Accreditation Council for Graduate Medical Education (ACGME) Vascular Surgery subcompetencies with patient care outcomes across different Vascular Quality Initiative (VQI) registries.A national panel of vascular surgeons with expertise in both clinical care and education were recruited to participate in a modified Delphi expert consensus building process to map ACGME Vascular Surgery subcompetencies (educational markers of resident performance) to VQI clinical modules (patient outcomes). A master list of items for rating was created, including the 31 ACGME Vascular Surgery subcompetencies and 8 VQI clinical registries (endovascular abdominal aortic aneurysm repair, open abdominal aortic aneurysm, thoracic endovascular aortic repair, carotid endarterectomy, carotid artery stent, infrainguinal, suprainguinal, and peripheral vascular intervention). These items were entered into an iterative Delphi process. Positive consensus was reached when 75% or more of the participants ranked an item as mandatory. Intraclass correlations (ICCs) were used to evaluate consistency between experts for each Delphi round.A total of 13 experts who contributed to the development of the Vascular Surgery Milestones participated; 12 experts (92%) participated in both rounds of the Delphi process. Two rounds of Delphi were conducted, as suggested by excellent expert agreement (round 1, ICC = 0.79 [95% confidence interval, 0.74-0.84]; round 2, ICC = 0.97 [95% confidence interval, 0.960-.98]). Using the predetermined consensus cutoff threshold, the Delphi process reduced the number of subcompetencies mapped to patient care outcomes from 31 to a range of 9 to 15 across the 8 VQI clinical registries. Practice-based learning and improvement, and professionalism subcompetencies were identified as less relevant to patient outcome variables captured by the VQI registries after the final round, and the only the systems-based practice subcompetency that was identified as relevant was radiation safety in two of the endovascular registries.A national panel of vascular surgeon experts reported a high degree of agreement on the relevance of ACGME subcompetencies to patient care outcomes as captured in the VQI clinical registry. Systems-based practice, practice-based learning and improvement, and professionalism competencies were identified as less relevant to patient outcomes after specific surgical procedures.

ImportanceEvaluation of trainees in graduate medical education training programs using Milestones has been in place since 2013. It is not known whether trainees who have lower ratings during the last year of training go on to have concerns related to interactions with patients in posttraining practice.ObjectiveTo investigate the association between resident Milestone ratings and posttraining patient complaints.Design, Setting, and ParticipantsThis retrospective cohort study included physicians who completed Accreditation Council for Graduate Medical Education (ACGME)–accredited programs between July 1, 2015, and June 30, 2019, and worked at a site that participated in the national Patient Advocacy Reporting System (PARS) program for at least 1 year. Milestone ratings from ACGME training programs and patient complaint data from PARS were collected. Data analysis was conducted from March 2022 to February 2023.ExposuresLowest professionalism (P) and interpersonal and communication skills (ICS) Milestones ratings 6 months prior to the end of training.Main Outcomes and MeasuresPARS year 1 index scores, based on recency and severity of complaints.ResultsThe cohort included 9340 physicians with median (IQR) age of 33 (31-35) years; 4516 (48.4%) were women physicians. Overall, 7001 (75.0%) had a PARS year 1 index score of 0, 2023 (21.7%) had a score of 1 to 20 (moderate), and 316 (3.4%) had a score of 21 or greater (high). Among physicians in the lowest Milestones group, 34 of 716 (4.7%) had high PARS year 1 index scores, while 105 of 3617 (2.9%) with Milestone ratings of 4.0 (proficient), had high PARS year 1 index scores. In a multivariable ordinal regression model, physicians in the 2 lowest Milestones rating groups (0-2.5 and 3.0-3.5) were statistically significantly more likely to have higher PARS year 1 index scores than the reference group with Milestones ratings of 4.0 (0-2.5 group: odds ratio, 1.2 [95% CI, 1.0-1.5]; 3.0-3.5 group: odds ratio, 1.2 [95% CI, 1.1-1.3]).Conclusions and RelevanceIn this study, trainees with low Milestone ratings in P and ICS near the end of residency were at increased risk for patient complaints in their early posttraining independent physician practice. Trainees with lower Milestone ratings in P and ICS may need more support during graduate medical education training or in the early part of their posttraining practice career.

Traditional teaching and assessment of clinical reasoning has focused on the individual clinician because of the preeminence of the information processing (IP) theory perspective. The clinician’s mind has been viewed as the main source of effective or ineffective reasoning, and other participants, the environment and their interactions have been largely ignored. A social cognitive theoretical lens could enhance our understanding of how reasoning and error and the environment are linked. Therefore, a new approach in which the clinical reasoning process is situated and examined within the context may be required. The theories of embodied cognition, ecological psychology, situated cognition (SitCog) and distributed cognition (DCog) offer new insights to help the teacher and assessor enhance the quality of clinical reasoning instruction and assessment. We describe the teaching and assessment implications of clinical reasoning and error through the lens of this family of theories. Direct observation in different contexts focused on individual and team performance, simulation (with or without enhancement of technology), stimulated recall, think-aloud, and modeling are examples of teaching and assessment strategies grounded in this family of social cognitive theories. Educators may consider the instructional design of learning environments and educational tools that promote a situated educational approach to the teaching and assessment of clinical reasoning.

Longitudinal Milestones data reported to the Accreditation Council for Graduate Medical Education (ACGME) can be used to measure the developmental and educational progression of learners. Learning trajectories illustrate the pattern and rate at which learners acquire competencies toward unsupervised practice.To investigate the reliability of learning trajectories and patterns of learning progression that can support meaningful intervention and remediation for residents.This national retrospective cohort study included Milestones data from residents in family medicine, representing 6 semi-annual reporting periods from July 2016 to June 2019.Longitudinal formative assessment using the Milestones assessment system reported to the ACGME.To estimate longitudinal consistency, growth rate reliability (GRR) and growth curve reliability (GCR) for 22 subcompetencies in the ACGME family medicine Milestones were used, incorporating clustering effects at the program level. Latent class growth curve models were used to examine longitudinal learning trajectories.This study included Milestones ratings from 3872 residents in 514 programs. The Milestones reporting system reliably differentiated individual longitudinal patterns for formative purposes (mean [SD] GRR, 0.63 [0.03]); there was also evidence of precision for model-based rates of change (mean [SD] GCR, 0.91 [0.02]). Milestones ratings increased significantly across training years and reporting periods (mean [SD] of 0.55 [0.04] Milestones units per reporting period; P .001); patterns of developmental progress varied by subcompetency. There were 3 or 4 distinct patterns of learning trajectories for each of the 22 subcompetencies. For example, for the professionalism subcompetency, residents were classified to 4 groups of learning trajectories; during the 3-year family medicine training period, trajectories diverged further after postgraduate year (PGY) 1, indicating a potential remediation point between the end of PGY 1 and the beginning of PGY 2 for struggling learners, who represented 16% of learners (620 residents). Similar inferences for learning trajectories were found for practice-based learning and improvement, systems-based practice, and interpersonal and communication skills. Subcompetencies in medical knowledge and patient care demonstrated more consistent patterns of upward growth.These findings suggest that the Milestones reporting system provides reliable longitudinal data for individualized tracking of progress in all subcompetencies. Learning trajectories with supporting reliability evidence could be used to understand residents' developmental progress and tailored for individualized learning plans and remediation.

ImportanceFinancial incentives and conflicts of interest may influence physician decision-making. It is important to understand financial interactions between the pharmaceutical and medical device industries and newly independent physicians who have recently completed their graduate medical education using a national transparency program.ObjectiveTo identify trends in industry payments to recent graduates of Accreditation Council for Graduate Medical Education–accredited residency or fellowship programs in orthopedic surgery, neurosurgery, and internal medicine.Design, Setting, and ParticipantsThis retrospective cohort study analyzed Open Payments reports of industry payments made between July 1, 2015, and June 30, 2021, to newly independent physicians from residency or fellowship programs in neurosurgery, orthopedic surgery, and internal medicine who graduated between January 1, 2015, and December 31, 2019.ExposuresSpecialties (neurosurgery and orthopedic surgery, with internal medicine as a comparison group).Main Outcomes and MeasuresIndustry payments to newly independent physicians, including any general payments (noninvestment or nonresearch) and at least $5000 of general payments in aggregate value per year, which are considered significant financial conflicts of interest. The percentage of newly independent physicians accepting general payments during the first 6 years after graduation was analyzed by specialty and sex using cumulative incidence curves and hazard ratios (HRs) in univariable and multivariable analyses.ResultsThere were 45 745 recent graduates (28 137 men [62%]; median age at graduation, 33.0 [IQR, 31.0-35.0 years]) in neurosurgery (n = 595), orthopedic surgery (n = 3481), and internal medicine (n = 41 669). In the first 2 years of independent practice, 95% (n = 3297), 92% (n = 546), and 59% (n = 24 522) of newly independent physicians in orthopedic surgery, neurosurgery, and internal medicine, respectively, accepted any general payments. A higher percentage of the newly independent physicians in orthopedic surgery and neurosurgery accepted any general payments (orthopedic surgery vs internal medicine: HR, 5.36 [95% CI, 4.42-6.51] for women and 7.01 [95% CI, 6.35-7.73] for men; neurosurgery vs internal medicine: HR, 3.25 [95% CI, 2.24-4.72] for women and 4.08 [95% CI, 3.37-4.94] for men; P = .03). A higher percentage of male physicians compared with female physicians accepted any general payments (orthopedic surgery, 2884 of 3026 [95%] vs 413 of 455 [91%]; P P = .01; and internal medicine, 15 462 of 24 609 [63%] vs 9043 of 17 034 [53%]; P P P P Conclusions and RelevanceIn this cohort study of newly independent physicians in orthopedic surgery, neurosurgery, and internal medicine, the financial relationship with potential conflicts of interest between newly independent physicians and industry began to develop soon after training programs and continued to expand in the early years of newly independent physician practice. Newly independent physicians in surgical specialties and male physicians accepted significantly higher industry payments. Further studies are needed to evaluate whether modifiable factors are associated with the future outcome of newly independent physicians accepting general payments.

Objectives Improving diagnosis-related education in the health professions has great potential to improve the quality and safety of diagnosis in practice. Twelve key diagnostic competencies have been delineated through a previous initiative. The objective of this project was to identify the next steps necessary for these to be incorporated broadly in education and training across the health professions. Methods We focused on medicine, nursing, and pharmacy as examples. A literature review was conducted to survey the state of diagnosis education in these fields, and a consensus group was convened to specify next steps, using formal approaches to rank suggestions. Results The literature review confirmed initial but insufficient progress towards addressing diagnosis-related education. By consensus, we identified the next steps necessary to advance diagnosis education, and five required elements relevant to every profession: 1) Developing a shared, common language for diagnosis, 2) developing the necessary content, 3) developing assessment tools, 4) promoting faculty development, and 5) spreading awareness of the need to improve education in regard to diagnosis. Conclusions The primary stakeholders, representing education, certification, accreditation, and licensure, in each profession must now take action in their own areas to encourage, promote, and enable improved diagnosis, and move these recommendations forward.

Introduction: Clinical reasoning is considered to be at the core of health practice. Here, we report on the diversity and inferred meanings of the terms used to refer to clinical reasoning and consider implications for teaching and assessment. Methods: In the context of a Best Evidence Medical Education (BEME) review of 625 papers drawn from 18 health professions, we identified 110 terms for clinical reasoning. We focus on iterative categorization of these terms across three phases of coding and considerations for how terminology influences educational practices. Results: Following iterative coding with 5 team members, consensus was possible for 74, majority coding was possible for 16, and full team disagreement existed for 20 terms. Categories of terms included: purpose/goal of reasoning, outcome of reasoning, reasoning performance, reasoning processes, reasoning skills, and context of reasoning. Discussion: Findings suggest that terms used in reference to clinical reasoning are non-synonymous, not uniformly understood, and the level of agreement differed across terms. If the language we use to describe, to teach, or to assess clinical reasoning is not similarly understood across clinical teachers, program directors, and learners, this could lead to confusion regarding what the educational or assessment targets are for "clinical reasoning."

An evidence-based approach to assessment is critical for ensuring the development of clinical reasoning (CR) competence. The wide array of CR assessment methods creates challenges for selecting assessments fit for the purpose; thus, a synthesis of the current evidence is needed to guide practice. A scoping review was performed to explore the existing menu of CR assessments.Multiple databases were searched from their inception to 2016 following PRISMA guidelines. Articles of all study design types were included if they studied a CR assessment method. The articles were sorted by assessment methods and reviewed by pairs of authors. Extracted data were used to construct descriptive appendixes, summarizing each method, including common stimuli, response formats, scoring, typical uses, validity considerations, feasibility issues, advantages, and disadvantages.A total of 377 articles were included in the final synthesis. The articles broadly fell into three categories: non-workplace-based assessments (e.g., multiple-choice questions, extended matching questions, key feature examinations, script concordance tests); assessments in simulated clinical environments (objective structured clinical examinations and technology-enhanced simulation); and workplace-based assessments (e.g., direct observations, global assessments, oral case presentations, written notes). Validity considerations, feasibility issues, advantages, and disadvantages differed by method.There are numerous assessment methods that align with different components of the complex construct of CR. Ensuring competency requires the development of programs of assessment that address all components of CR. Such programs are ideally constructed of complementary assessment methods to account for each method's validity and feasibility issues, advantages, and disadvantages.

Telemedicine and remote visits are becoming more and more popular in several medical disciplines, including oncology. The Covid-19 pandemic has enhanced the need to continue to meet patients' ambulatory care necessities ensuring social distancing and limiting the access to clinical facilities. The National Cancer Institute of Aviano, Italy, has recently launched a program called "Doctor @ Home" (D@H). The pillars of the program are the co-production of the oncological care and the co-learning approach, which sees the clinical staff "hand in hand" with patients to maximize the outcome of the care, trying to take advantage of the new tools offered by modern technologies.

Competency-based medical education (CBME) has developed strong roots in many professional education systems worldwide. The CBME movement is driven by concern and evidence of significant challenges in the quality and safety of healthcare. Medical education can be viewed as a translational science intervention because the competencies acquired by learners must transfer directly to the care of patients and families. Mastery learning specifically incorporates this critical outcomes-based philosophy into its design and application. Milestones and entrustable professional activities (EPAs) have emerged as developmental educational strategies and approaches to advance the implementation of CBME. However, implementation of competencies through milestones and EPAs has been challenging and is still in its infancy.

Background Clinical reasoning is at the core of health professionals’ practice. A mapping of what constitutes clinical reasoning could support the teaching, development, and assessment of clinical reasoning across the health professions. Methods We conducted a scoping study to map the literature on clinical reasoning across health professions literature in the context of a larger Best Evidence Medical Education (BEME) review on clinical reasoning assessment. Seven databases were searched using subheadings and terms relating to clinical reasoning, assessment, and Health Professions. Data analysis focused on a comprehensive analysis of bibliometric characteristics and the use of varied terminology to refer to clinical reasoning. Results Literature identified: 625 papers spanning 47 years (1968–2014), in 155 journals, from 544 first authors, across eighteen Health Professions. Thirty-seven percent of papers used the term clinical reasoning; and 110 other terms referring to the concept of clinical reasoning were identified. Consensus on the categorization of terms was reached for 65 terms across six different categories: reasoning skills, reasoning performance, reasoning process, outcome of reasoning, context of reasoning, and purpose/goal of reasoning. Categories of terminology used differed across Health Professions and publication types. Discussion Many diverse terms were present and were used differently across literature contexts. These terms likely reflect different operationalisations, or conceptualizations, of clinical reasoning as well as the complex, multi-dimensional nature of this concept. We advise authors to make the intended meaning of ‘clinical reasoning’ and associated terms in their work explicit in order to facilitate teaching, assessment, and research communication.

Purpose To develop and conduct feasibility testing of an evidence-based and theory-informed model for facilitating performance feedback for physicians so as to enhance their acceptance and use of the feedback. Method To develop the feedback model (2011-2013), the authors drew on earlier research which highlights not only the factors that influence giving, receiving, accepting, and using feedback but also the theoretical perspectives which enable the understanding of these influences. The authors undertook an iterative, multistage, qualitative study guided by two recognized research frameworks: the UK Medical Research Council guidelines for studying complex interventions and realist evaluation. Using these frameworks, they conducted the research in four stages: (1) modeling, (2) facilitator preparation, (3) model feasibility testing, and (4) model refinement. They analyzed data, using content and thematic analysis, and used the findings from each stage to inform the subsequent stage. Results Findings support the facilitated feedback model, its four phasesbuild relationship, explore reactions, explore content, coach for performance change (R2C2)and the theoretical perspectives informing them. The findings contribute to understanding elements that enhance recipients' engagement with, acceptance of, and productive use of feedback. Facilitators reported that the model made sense and the phases generally flowed logically. Recipients reported that the feedback process was helpful and that they appreciated the reflection stimulated by the model and the coaching. Conclusions The theory- and evidence-based reflective R2C2 Facilitated Feedback Model appears stable and helpful for physicians in facilitating their reflection on and use of formal performance assessment feedback.

Introduction Recent assessment innovations have included the development of resident-sensitive quality measures (RSQMs), which are characterized by: 1) their importance to care quality, and 2) likelihood they are performed by residents. This study explored how clinical competency committee (CCC) members interpret, use, and prioritize RSQM data added to their usual review processes. Methods In this constructivist grounded theory study, 19 members of the Cincinnati Children's pediatric residency CCC were purposively and theoretically sampled. Participants were provided a resident assessment portfolio comprised of performance ratings and narrative comments for five rotations, along with RSQM data for one of these rotations. They were asked to make a decision about the resident's ability to care for patients presenting with common, acute problems (a general pediatric entrustable professional activity). Data collection consisted of 1) observation and think aloud while participants reviewed performance data, and 2) semi-structured interviews to probe reviews. Results Five dimensions for how participants view and use RSQMs were identified: 1) Ability to orient to RSQMs: confusing to self-explanatory, 2) Propensity to use RSQMs: reluctant to enthusiastic, 3) RSQM interpretation: requires contextualization to self-evident, 4) RSQMs for assessment decisions: not sticky to sticky, and 5) Expectations for residents: potentially unfair to fair to use RSQMs. The interactions among dimensions generated three RSQM data user profiles, with the first two being most common: eager incorporation, willing incorporation, and disinclined incorporation. Conclusion Most participants used RSQMs to varying extents, demonstrating willingness to include them as resident assessment data for CCC review.

Self-assessment is a process of interpreting data about one's performance and comparing it to explicit or implicit standards.To examine the external data sources physicians used to monitor themselves.Focus groups were conducted with physicians who participated in three practice improvement activities: a multisource feedback program; a program providing patient and chart audit data; and practice-based learning groups. We used grounded theory strategies to understand the external sources that stimulated self-assessment and how they worked.Data from seven focus groups (49 physicians) were analyzed. Physicians used information from structured programs, other educational activities, professional colleagues, and patients. Data were of varying quality, often from non-formal sources with implicit (not explicit) standards. Mandatory programs elicited variable responses, whereas data and activities the physicians selected themselves were more likely to be accepted. Physicians used the information to create a reference point against which they could weigh their performance using it variably depending on their personal interpretation of its accuracy, application, and utility.Physicians use and interpret data and standards of varying quality to inform self-assessment. Physicians may benefit from regular and routine feedback and guidance on how to seek out data for self-assessment.

To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these.This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings.Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged.Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.

Self-assessment has been held out as an important mechanism for lifelong learning and self-improvement for health care professionals. However, there is growing concern that individual learners often interpret the results inaccurately. This idea has led to skepticism that self-assessment in its current form can ever be truly useful for lifelong professional development. We examine the proposal that self-assessment can and should be made more effective. First, relevance should be improved. The process should be tied more explicitly to the individual's actual practice profile, rather than being loosely relevant to broader constructs around the permitted scope of practice (eg, certification or licensure). In addition, self-assessment should include not only knowledge and reasoning but also what is done every day in practice, thereby broadening from competence in simulated settings to performance in real settings. Second, the impact of self-assessment should be substantially strengthened by periodic external validation of self-assessment results, together with goals set as a result and plans for further improvement. This offers to the individual the very tangible benefit of satisfying external mandates (eg, licensure and certification). In addition, impact should be reinforced by linking the results of self-assessment to subsequent learning activities including Continuing Medical Education (CME). Although these enhancements individually may not cure all of what ails self-assessment, they might ensure greater effectiveness for the purposes of lifelong learning.

This study was intended to develop a conceptual framework of the factors impacting on faculty members' judgements and ratings of resident doctors (residents) after direct observation with patients.In 2009, 44 general internal medicine faculty members responsible for out-patient resident teaching in 16 internal medicine residency programmes in a large urban area in the eastern USA watched four videotaped scenarios and two live scenarios of standardised residents engaged in clinical encounters with standardised patients. After each, faculty members rated the resident using a mini-clinical evaluation exercise and were individually interviewed using a semi-structured interview. Interviews were videotaped, transcribed and analysed using grounded theory methods.Four primary themes that provide insights into the variability of faculty assessments of residents' performance were identified: (i) the frames of reference used by faculty members when translating observations into judgements and ratings are variable; (ii) high levels of inference are used during the direct observation process; (iii) the methods by which judgements are synthesised into numerical ratings are variable, and (iv) factors external to resident performance influence ratings. From these themes, a conceptual model was developed to describe the process of observation, interpretation, synthesis and rating.It is likely that multiple factors account for the variability in faculty ratings of residents. Understanding these factors informs potential new approaches to faculty development to improve the accuracy, reliability and utility of clinical skills assessment.

Conceptualisations of self-assessment are changing as its role in professional development comes to be viewed more broadly as needing to be both externally and internally informed through activities that enable access to and the interpretation and integration of data from external sources. Education programmes use various activities to promote learners' reflection and self-direction, yet we know little about how effective these activities are in 'informing' learners' self-assessments.This study aimed to increase understanding of the specific ways in which undergraduate and postgraduate learners used learning and assessment activities to inform self-assessments of their clinical performance.We conducted an international qualitative study using focus groups and drawing on principles of grounded theory. We recruited volunteer participants from three undergraduate and two postgraduate programmes using structured self-assessment activities (e.g. portfolios). We asked learners to describe their perceptions of and experiences with formal and informal activities intended to inform self-assessment. We conducted analysis as a team using a constant comparative process.Eighty-five learners (53 undergraduate, 32 postgraduate) participated in 10 focus groups. Two main findings emerged. Firstly, the perceived effectiveness of formal and informal assessment activities in informing self-assessment appeared to be both person- and context-specific. No curricular activities were considered to be generally effective or ineffective. However, the availability of high-quality performance data and standards was thought to increase the effectiveness of an activity in informing self-assessment. Secondly, the fostering and informing of self-assessment was believed to require credible and engaged supervisors.Several contextual and personal conditions consistently influenced learners' perceptions of the extent to which assessment activities were useful in informing self-assessments of performance. Although learners are not guaranteed to be accurate in their perceptions of which factors influence their efforts to improve performance, their perceptions must be taken into account; assessment strategies that are perceived as providing untrustworthy information can be anticipated to have negligible impact.

The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications.To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents.Retrospective cohort study.One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005-2007, served as controls.A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations.We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities.After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p0.01).A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with improved PGY-3 IM-ITE performance.

As part of recertification, the American Board of Internal Medicine requires its diplomats to complete at least 1 practice improvement module (PIM). We assessed whether completing an asthma-specific PIM resulted in improved patient outcomes.Practices were the unit of randomization in this cluster randomized trial. Physicians in the intervention group were asked to complete the PIM through its planning phase. The primary outcome was the dispensing of an inhaled corticosteroid (ICS) after a postintervention visit for asthma. Secondary outcomes included patient reported processes of care, asthma-related heath care use, and asthma severity. Analyses were adjusted for baseline rates at the cluster-level as well as for individual sociodemographic characteristics.Eight practices (19 internists) were randomized to the intervention group and 8 practices (21 internists) to the control group. For the primary outcome, ICS fill rates, patients seen by intervention group physicians were not more likely to fill an ICS prescription in the postintervention period than patients seen by control group physicians (adjusted odd ratio [AOR], 1.00; 95% confidence interval [CI], 0.64-1.56). Patients seen for asthma by intervention group physicians were less likely to receive a written action plan than patients seen by control group physicians (AOR, 0.67; 95% CI, 0.48-0.93); however, they were more likely to discuss potential asthma triggers (AOR, 1.62; 95% CI, 1.08-2.42) and had lower self-reported asthma severity measures (unadjusted P = .03). Per-protocol analysis supported the latter 2 associations.A PIM designed to improve asthma care did not improve filling of ICS prescriptions but may have lessened asthma severity through an increased discussion of asthma triggers.

Medication mismanagement results in 10% of hospitalizations and 50% of therapeutic failures. Little is known about low-income patients' medication knowledge, a group disproportionately affected by chronic disease and often cared for by residents. This study assesses patients' medication knowledge, discrepancies between the patient and chart, and preferences for medication information in the training setting.This cross-sectional study evaluated adults receiving care in two teaching clinics. Patients were asked their medications' names, doses, frequencies, indications, and side effects (SE) and what information they needed to take medications safely. Charts served as a medication reference standard.Most patients were female, nonwhite, hador = one chronic condition, took a median of five drugs, and saw three different residents in the same clinic. Despite six clinic visits annually, patients could cite only 50% of their medication names, frequencies, and indications, 25% of doses, and 0% of SE, yet they paradoxically responded that SE knowledge was essential for safety. Forty-three percent reportedor = one medication that was undocumented in the chart. There were 84 total discrepant medications of which 63% were prescription only.Low-income patients had poor medication knowledge despite good clinic access. Discontinuity of resident providers may contribute to this phenomenon. Poor global knowledge, the large discrepancy between reported and charted medications, and lack of SE knowledge threaten medication adherence, achievement of treatment goals, and patient safety.

Self-appraisal has repeatedly been shown to be inadequate as a mechanism for performance improvement. This has placed greater emphasis on understanding the processes through which self-perception and external feedback interact to influence professional development. As feedback is inevitably interpreted through the lens of one’s self-perceptions it is important to understand how learners interpret, accept, and use feedback (or not) and the factors that influence those interpretations. 134 participants from 8 health professional training/continuing competence programs were recruited to participate in focus groups. Analyses were designed to (a) elicit understandings of the processes used by learners and physicians to interpret, accept and use (or not) data to inform their perceptions of their clinical performance, and (b) further understand the factors (internal and external) believed to influence interpretation of feedback. Multiple influences appear to impact upon the interpretation and uptake of feedback. These include confidence, experience, and fear of not appearing knowledgeable. Importantly, however, each could have a paradoxical effect of both increasing and decreasing receptivity. Less prevalent but nonetheless important themes suggested mechanisms through which cognitive reasoning processes might impede growth from formative feedback. Many studies have examined the effectiveness of feedback through variable interventions focused on feedback delivery. This study suggests that it is equally important to consider feedback from the perspective of how it is received. The interplay observed between fear, confidence, and reasoning processes reinforces the notion that there is no simple recipe for the delivery of effective feedback. These factors should be taken into account when trying to understand (a) why self-appraisal can be flawed, (b) why appropriate external feedback is vital (yet can be ineffective), and (c) why we may need to disentangle the goals of performance improvement from the goals of improving self-assessment.