Search

Qualitative researchers have underscored the value and importance of being reflexive in the research process, yet existing guidelines or checklists on how to practically address reflexivity are often scant and scattered across studies. In this scholarly perspective, we review, analyse, and present an overview of conceptions of reflexivity. Further, we offer practical guidelines for addressing and developing reflexivity statements in qualitative research. We describe reflexivity as both a concept and a deliberate ongoing process that requires a certain level of researcher consciousness, reflection, introspection, self-awareness, and an analytic attention to the researcher's role in the research process at all stages. We highlight the notion that reflexivity offers researchers an opportunity to examine potential assumptions, through the continuous process of questioning, examining, accepting, and articulating our attitudes, assumptions, perspectives, and roles. We present six recommendations to promote dialogue on the practice of reflexivity among researchers from various ontological and epistemological communities and encourage them to develop their own reflexivity practices.

Background: As competency-based medical education (CBME) curricula are introduced in residency programs across Canada, systematic evaluation efforts are needed to ensure fidelity of implementation. This study evaluated early outcomes of CBME implementation in one Canadian Physical Medicine and Rehabilitation program that was an early adopter of CBME, with an aim to inform continuous quality improvement initiatives and CBME implementation nationwide. Methods: Using Rapid Evaluation methodology, informed by the CBME Core Components Framework, the intended outcomes of CBME were compared to actual outcomes. Results: Results suggested that a culture of feedback and coaching already existed in this program prior to CBME implementation, yet faculty felt that CBME added a framework to support feedback. The small program size was valuable in fostering strong relationships and individualized learning. However, participants expressed concerns about CBME fostering a reductionist approach to the development of competence. Challenges existed with direct observation, clear expectations for off-service training experiences, and tracking trainee progress. There was trepidation surrounding national curricular change, yet the institution-wide approach to CBME implementation created shared experiences and a community of practice. Conclusions: Program evaluation can help understand gaps between planned versus enacted implementation of CBME, and foster adaptations to improve the fidelity of implementation.

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.

Background: While research suggests that manifestations of the hidden curriculum (HC) phenomenon have the potential to reinforce or undermine the values of an institution, very few studies have comprehensively measured its scope, impact, and the varied clinical teaching and learning contexts within which they occur. We explored the HC and examined the validity of newly developed constructs and determined the influence of context on the HC. Methods: We surveyed medical students (n =182), residents (n =148), and faculty (n = 140) from all disciplines at our institution between 2019 and 2020. Based on prior research and expertise, we measured participants’ experience with the HC including perceptions of respect and disrespect for different medical disciplines, settings in which the HC is experienced, impact of the HC, personal actions, efficacy, and their institutional perceptions. We examined the factor structure, reliability, and validity of the HC constructs using exploratory factor analysis Cronbach’s alpha, regression analysis and Pearson’s correlations. Results: Expert judges (physician faculty and medical learners) confirmed the content validity of the items used and the analysis revealed new HC constructs reflecting negative expressions, positive impacts and expressions, negative impacts, personal actions, and positive institutional perceptions of the HC. Evidence for criterion validity was found for the negative impacts and the personal actions constructs and were significantly associated with the stage of respondents’ career and gender. Support for convergent validity was obtained for HC constructs that were significantly correlated with certain contexts within which the HC occurs. Conclusion: More unique dimensions and contexts of the HC exist than have been previously documented. The findings demonstrate that specific clinical contexts can be targeted to improve negative expressions and impacts of the HC.

In September 2016, Queen’s University launched the first, fully online, 4-year Bachelor of Health Science degree program in Canada. This paper reports on the developmental structure, implementation philosophy, and challenges in the development of this competency-based program. All stakeholders directly involved in program development were invited to participate in this qualitative case study. Thirty-five interviews and three focus groups (n=14) were conducted. Interviews and focus groups were transcribed verbatim and data were analyzed using thematic design. Themes included: program vision; desired program outcomes; administrative processes for funding and recruitment; uniqueness of the program; local, regional and international impact of the program; communication and collaborations for program development; and uncertainty in long term outcomes. Findings suggest that during program development, an explicit vision of program goals encouraged buy-in at most levels of the university. There was consensus that the overarching outcome should be to provide a rigorous, high quality program with pathways to professional, basic science, global health and advocacy-based health professions. The online modality was expected to improve accessibility to degree programs, as well as address diverse student learning needs. Innovation played a vital role in the program’s development and was founded in educational theory and curriculum development practices. En septembre 2016, l’Université Queen’s a lancé le premier programme de 4 ans menant à un diplôme, le baccalauréat en sciences de la santé, entièrement en ligne. Cet article présente un rapport sur la structure du développement, la philosophie de mise en oeuvre et les défis qui se sont présentés lors du développement de ce programme basé sur la compétence. Toutes les parties prenantes qui avaient été directement impliquées dans le développement de ce programme ont été invitées à participer à cette étude de cas qualitative. Trente-cinq entrevues ont été menées et trois groupes de discussion (n=14) ont été organisés. Ce qui a été dit pendant les entrevues et les groupes de discussion a été transcrit verbatim et les données ont été analysées en utilisant la conception thématique. Les thèmes étaient les suivants : vision du programme, résultats désirés du programme, processus administratifs pour le financement et le recrutement, aspect unique du programme, impacts local, régional et international du programme, communications et collaborations pour le développement du programme, incertitudes concernant les résultats à long terme. Les résultats suggèrent qu’au cours de la phase de développement du programme, une vision explicite des objectifs du programme avait encouragé l’acceptation à tous les niveaux de l’université. Tout le monde était d’accord sur le fait que le résultat global devait être de fournir un programme rigoureux de haute qualité qui mènerait vers les professions de sciences de base, les professions de santé globale et les professions de santé basées sur la défense des intérêts. La modalité en ligne devait améliorer l’accessibilité à des programmes menant à un diplôme et devait également répondre à divers besoins d’apprentissage des étudiants et des étudiantes. L’innovation a joué un rôle primordial dans le développement du programme, elle était fondée sur la théorie de l’éducation et sur les pratiques de développement de programmes d’études.

The use of quantitative intercoder reliability measures in the analysis of qualitative research data has often generated acrimonious debates among researchers who view quantitative and qualitative research methodologies as incompatible due to their unique ontological and epistemological traditions. While these measures are invaluable in many contexts, critics point out that the use of such measures in qualitative analysis represents an attempt to import standards derived for positivist research. Guided by extant research and our experience in qualitative research, we argue that it is possible to develop a qualitative-based measure of intercoder reliability that is compatible with the interpretivist epistemological paradigm of qualitative research. We present eight qualitative research process-based guidelines for evaluating and reporting intercoder reliability in qualitative research and anticipate that these recommendations will particularly guide beginning researchers in the coding and analysis processes of qualitative data analysis.

Background: The direct observation and assessment of learners’ resuscitation skills by an attending physician is challenging due to the unpredictable and time-sensitive nature of these events. Multisource feedback (MSF) may address this challenge and improve the quality of assessments provided to learners. We aimed to describe the similarities and differences in the assessment rationale of attending physicians, registered nurses, and resident peers in the context of a simulation-based resuscitation curriculum. Methods: We conducted a qualitative content analysis of narrative MSF of medical residents in their first postgraduate year of training who were participating in a simulation-based resuscitation course at two Canadian institutions. Assessments included an entrustment score and narrative comments from attending physicians, registered nurses, and resident peers in addition to self-assessment. Narrative comments were transcribed and analyzed thematically using a constant comparative method. Results: All 87 residents (100%) participating in the 2017-2018 course provided consent. A total of 223 assessments were included in our analysis. Four themes emerged from the narrative data: 1) Communication, 2) Leadership, 3) Demeanor, and 4) Medical Expert. Relative to other assessor groups, feedback from nurses focused on patient-centred care and communication while attending physicians focused on the medical expert theme. Peer feedback was the most positive. Self-assessments included comments within each of the four themes. Conclusions: In the context of a simulation-based resuscitation curriculum, MSF provided learners with different perspectives in their narrative assessment rationale and may offer a more holistic assessment of resuscitation skills within a competency-based medical education (CBME) program of assessment.

Background: Residents’ accurate self-assessment and clinical judgment are essential for optimizing their clinical skills development. Evidence from the medical literature suggests that residents generally do poorly at self-assessing their performance, often due to factors relating to learners’ personal backgrounds, cultures, the specific contexts of the learning environment and rater bias or inaccuracies. We evaluated the accuracy of anesthesiology residents’ self-assessed Global Entrustment scores and determined whether differences between faculty and resident scores varied by resident seniority, faculty leniency, and/or year of assessment. Methods: We employed variance components modeling techniques and analyzed 329 pairs of faculty and self-assessed entrustment scores among 43 faculty assessors and 15 residents. Using faculty scores as the gold standard, we compared faculty scores with residents’ scores (xi(faculty)–xi(resident)), and determined residents’ accuracy, including over- and under-confidence. Results: The results indicate that residents were respectively over- and under-confident in 10.9% and 54.4% of the assessments but more consistent in their individual self-assessments (rho = 0.70) than faculty assessors. Faculty scores were significantly higher (α = 0.396; z = 4.39; p < 0.001) than residents’ self-assessed scores. Being a lenient/dovish (β = 0.121, z = 3.16, p < 0.01) and a neutral (β = 0.137, z = 3.57, p < 0.001) faculty assessor predicted a higher likelihood of resident under-confidence. Senior residents were significantly less likely to be under-confident compared to junior residents (β = -0.182, z =-2.45, p < 0.05). The accuracy of self-assessments did not significantly vary during the two years of the study period. Conclusions: The majority of residents’ self-assessments were inaccurate. Our findings may help identify the sources of such inaccuracies.

Background: Workplace-based assessment (WBA), foundational to competency-based medical education, relies on preceptors providing feedback to residents. Preceptors however get little timely, formative, specific, actionable feedback on the effectiveness of that feedback. Our study aimed to identify useful qualities of feedback for family medicine residents and to inform improving feedback-giving skills for preceptors in PGME training program. Methods: This study employed a two-phase exploratory design. Phase 1 collected qualitative data from preceptor feedback given to residents through Field Notes (FNs) and quantitative data from residents who provided feedback to preceptor about the quality of the feedback given. Phase 2 employed focus groups to explore ways in which residents are willing to provide preceptors with constructive feedback about the quality of the feedback they receive. Descriptive statistics and a thematic approach were used for data analysis. Findings: We collected 22 FNs identified by residents as being impactful to their learning; analysis of these FNs resulted in five themes. Functionality was then added to the electronic FNs allowing residents to indicate impactful feedback with a “Thumbs Up” icon. Over one year, 895 out of 8,496 FNs (11%) had a “Thumbs up” added, divided into reasons of: confirmation of learning (28.6%), practice improvement (21.2%), new learning (18.8%), motivation (17.7%), and evoking reflection (13.7%). Two focus groups (12 residents, convenience sampling) explored residents’ perception of constructive feedback and willingness to also provide constructive feedback to preceptors. Conclusion: Adding constructive feedback to existing positive feedback choices will provide preceptors with holistic information about the impact of their feedback on learners, which, in turn, should allow them to provide more effective feedback to learners. However, power differential, relationship impact, and institutional support were concerns for residents that would need to be addressed for this to be optimally operationalized.

Background: In North America, there is limited data to support deliberate application strategies for post-graduate residency training. There is significant interest in determining what factors play a role in Canadian medical graduate (CMG) matching to their first choice discipline and heightened concern about the number of students going unmatched altogether. Methods: We analyzed matching outcomes of CMGs based on seven years (2013-2019) of residency application data (n= 13,499) from the Canadian Residency Matching Service (CaRMS) database using descriptive and binary logistic regression modeling techniques. Results: The sample was 54% female, with 60% between the ages of 26 and 29, and 60% attended medical schools in Ontario. Applicants who received more rankings from residency programs were more likely (OR = 1.185, p < 0.001) to match. Higher research activities (OR = 0.985, p < 0.001) and number of applications submitted (OR = 0.920, p < 0.001) were associated with a reduced likelihood of matching. Number of volunteer activities and self-report publications did not significantly affect matching. Being male (OR = 0.799, p < 0.05) aged

Construct: Competence Based Medical Education (CBME) is designed to use workplace-based assessment (WBA) tools to provide observed assessment and feedback on resident competence. Moreover, WBAs are expected to provide evidence beyond that of more traditional mid- or end-of-rotation assessments [e.g., In Training Evaluation Records (ITERs)]. In this study we investigate competence in General Internal Medicine (GIM), by contrasting WBA and ITER assessment tools. Background: WBAs are hypothesized to improve and differentiate written and numerical feedback to support the development and documentation of competence. In this study we investigate residents’ and faculty members’ perceptions of WBA validity, usability, and reliability and the extent to which WBAs differentiate residents’ performance when compared to ITERs. Approach: We used a mixed methods approach over a three-year period, including perspectives gathered from focus groups, interviews, along with numerical and narrative comparisons between WBA and ITERs in one GIM program. Results: Residents indicated that the narrative component of feedback was more constructive and effective than numerical scores. They perceived the focus on specific workplace-based feedback was more effective than ITERs. However, quantitative analysis showed that overall rates of actionable feedback, including both ITERs and WBAs, were low (26%), with only 9% providing an improvement strategy. The provision of quality feedback was not statistically significantly different between tools; although WBAs provided more actionable feedback, ITERs provided more strategies. Statistical analyses showed that more than half of all assessments came from 11 core faculty. Conclusions: Participants in this study viewed narrative, actionable and specific feedback as essential, and an overall preference was found for written feedback over numerical assessments. However, quantitative analyses showed that specific actionable feedback was rarely documented, despite qualitative emphasis from both groups of its importance for developing competency. Neither formative WBAs or summative ITERs clearly provided better feedback, and both may still have a role in overall resident evaluation. Participant views differed in roles and responsibilities, with residents stating that faculty should be responsible for initiating assessments and vice-versa. These results reveal a disconnect between resident and faculty perceptions and practice around giving feedback and emphasize opportunities for programs adopting and implementing CBME to address how best to support residents and frontline clinical teachers.

p-Carboxylatocalix[n]arenes have emerged as useful building blocks for the construction of a diverse range of supramolecular assemblies. A convenient route to a p-carboxylatocalix[4]arene that is locked in a partial-cone conformation is presented. The conformation gives the molecule markedly different topological directionality relative to those previously used in self- and metal-directed assembly studies.