Search

Program evaluation is an "essential responsibility" but is often not seen as a scholarly pursuit. While Boyer expanded what qualifies as educational scholarship, many still need to engage in processes that are rigorous and of a requisite academic standard to be labelled as scholarly. Many medical educators may feel that scholarly program evaluation is a daunting task due to the competing interests of curricular change, remediation, and clinical care. This paper explores how educators can take their questions around outcomes and efficacy of our programs and efficiently engage in education scholarship. The authors outline how educators can examine whether training programs have a desired impact and outcomes, and then how they might leverage this process into education scholarship., Competing Interests: Dr. Shera Hosseini has received funding for her postdoctoral fellowship from the McMaster Institute for Research in Aging (MIRA). Dr. Yilmaz is the recipient of a 2019 TUBITAK Postdoctoral Fellowship grant. Dr. Shah—none; and no grants. Dr. Gottlieb holds grants for unrelated work with the Centers for Disease Control and Prevention, Council of Residency Directors in Emergency Medicine, Society for Academic Emergency Medicine, and eCampus Ontario. Dr. Stehman—none, Dr. Hall—holds grants for unrelated work from the Royal College of Physicians and Surgeons of Canada, Queen’s University Center for Teaching and Learning, and the Physician Services Incorporated Foundation. Dr. Chan holds grants for unrelated work from McMaster University, the PSI foundation, Society for Academic Emergency Medicine, eCampus Ontario, the University of Saskatchewan, and Royal College of Physicians and Surgeons of Canada., (© 2022 by the Society for Academic Emergency Medicine.)

Background: In the emergency department (ED), chest radiographs (CXRs) provide essential information for clinical diagnostic reasoning. Errors in interpretation by emergency physicians can lead to negative patient outcomes. To aid in teaching this important skill, an understanding of cognitive processes and cognitive load (CL) in CXR interpretation in emergency medicine (EM) personnel is warranted., Methods: This study adopted a concurrent mixed-methods research design. Participant groups included medical students (M), junior (J) and senior (S) EM residents, and attending emergency physicians (P) in the ED at an academic hospital. To elucidate cognitive processes, a real-time cognitive task analysis during CXR interpretation was performed. Interviews were audio recorded, transcribed verbatim, and analyzed thematically. The interview was followed by a questionnaire, where participants rated their CL, stress, and confidence level., Results: Levels of CL (M vs. S and M vs. P, p = 0.002; J vs. S, p   = 0.004; J vs. P, p = 0.005) and stress (J vs. P, p = 0.002) decreased, while confidence levels increased (M vs. S, p = 0.006; J vs. S, p ≤ 0.001; J vs. P, p   = 0.003) as experience level increased. Qualitative analysis of interviews revealed four themes: checking behavior, information reduction, pattern recognition versus systematic viewing, and recognizing scope of practice. Experts commonly utilized checking behavior (e.g., comparison to prior radiographs) and deprioritized task irrelevant data. Experts used a general overview technique as their initial approach as opposed to a systematic viewing approach, and they more readily recognized an EM physicians' scope of practice in this task., Conclusion: This study characterized differences in cognition that led to increased CL, stress, and lower level of confidence in EM learners during CXR interpretation and provided insight into expertise development in this important skill., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 Society for Academic Emergency Medicine.)

Background: Emergency physicians often experience a high cognitive load (CL) due to the inherent nature of working in acute care settings. CL has traditionally been measured in educational studies but has not been well studied in the clinical environment., Methods: Emergency medicine attending physicians and residents working in an academic urgent care center completed psychometric questionnaires while on shift to measure overall CL, intrinsic cognitive load (ICL), extraneous cognitive load (ECL), and acute stress. Data regarding the patient load, patient acuity, and the number of patients in the waiting room were also collected. Correlational analysis and simple linear regression were used to evaluate predictors of CL on shift., Results: Forty-two questionnaires were completed (26 by attending physicians, 16 by residents). Attending physicians carried a significantly higher patient load compared to residents ( p  < 0.001). No differences in mean overall CL, ICL, ECL, and acute stress were observed between attending physicians and residents. Bivariate analysis demonstrated associations between ICL, ECL, acute stress, and overall CL in attending physicians. In residents, acute stress was the only variable associated with overall CL and the number of high-acuity patients was associated with ICL., Conclusions: Factors influencing reported CL during clinical work are different between attending emergency physicians and residents. Further study to appreciate the impact of these differences is required and may help educators elucidate strategies to better manage CL, thereby improving clinical performance and potentially improving patient care., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 Society for Academic Emergency Medicine.)

Background: Free Open-Access Medical education (FOAM) use among residents continues to rise. However, it often lacks quality assurance processes and residents receive little guidance on quality assessment. The Academic Life in Emergency Medicine Approved Instructional Resources tool (AAT) was created for FOAM appraisal by and for expert educators and has demonstrated validity in this context. It has yet to be evaluated in other populations., Objectives: We assessed the AAT's usability in a diverse population of practicing emergency medicine (EM) physicians, residents, and medical students; solicited feedback; and developed a revised tool., Methods: As part of the Medical Education Translational Resources: Impact and Quality (METRIQ) study, we recruited medical students, EM residents, and EM attendings to evaluate five FOAM posts with the AAT and provide quantitative and qualitative feedback via an online survey. Two independent analysts performed a qualitative thematic analysis with discrepancies resolved through discussion and negotiated consensus. This analysis informed development of an initial revised AAT, which was then further refined after pilot testing among the author group. The final tool was reassessed for reliability., Results: Of 330 recruited international participants, 309 completed all ratings. The Best Evidence in Emergency Medicine (BEEM) score was the component most frequently reported as difficult to use. Several themes emerged from the qualitative analysis: for ease of use-understandable, logically structured, concise, and aligned with educational value. Limitations include deviation from questionnaire best practices, validity concerns, and challenges assessing evidence-based medicine. Themes supporting its use include evaluative utility and usability. The author group pilot tested the initial revised AAT, revealing a total score average measure intraclass correlation coefficient (ICC) of moderate reliability (ICC = 0.68, 95% confidence interval [CI] = 0 to 0.962). The final AAT's average measure ICC was 0.88 (95% CI = 0.77 to 0.95)., Conclusions: We developed the final revised AAT from usability feedback. The new score has significantly increased usability, but will need to be reassessed for reliability in a broad population., Competing Interests: The authors have no potential conflicts to disclose. All authors had full access to all the study data and had final responsibility for the decision to submit for publication., (© 2021 by the Society for Academic Emergency Medicine.)

Objectives: Trauma resuscitations require competence in both clinical and nonclinical skills but these can be difficult to observe and assess. Multisource feedback (MSF) is workplace-based, involves the direct observation of learners, and can provide feedback on nonclinical skills. We sought to compare and contrast the priorities of multidisciplinary trauma team members when assessing resident trauma team captain (TTC) performance. Additionally, we aimed to describe the nature of the assessment and perceived the utility of incorporating MSF into the trauma context., Methods: A convenience sample of 10 trauma team activations were observed. Following each activation, the attending physician trauma team leader (TTL), TTC, and a registered nurse (RN) participated in a semistructured interview. MSF was not provided to the TTC for the purpose of this study because MSF was not part of the assessment process of TTCs at the time of this study and maintaining anonymity may have encouraged more honest interview responses. Transcripts from each assessor group (TTL, TTC, RN) were coded and assigned to one of the five crisis resource management skills: leadership, communication, situational awareness, resource utilization, and problem-solving. Comments were also coded as positive, negative, or neutral as interpreted by the coder., Results: All assessor groups mentioned communication skills most frequently. After communication, the RN and TTC groups commented on situational awareness most frequently, comprising 15 and 29% of their total responses, respectively, whereas 31% of the TTL comments focused on leadership skills. The RN and TTL groups provided positive assessments, with 51 and 42% of their respective comments coded as positive. Forty-five percent of self-assessment comments in the TTC group were negative. All (100%) of the TTC and TTL respondents felt that incorporating MSF would add to the quality of feedback, only 66% of the RN group felt that way., Conclusions: We found that each assessor group brings a unique focus and perspective to the assessment of resident TTC performance. The future inclusion of MSF in the trauma team context has the potential to enhance the learning environment in a clinical arena that is difficult to directly observe and assess., (© 2020 by the Society for Academic Emergency Medicine.)

Objectives: Point-of-care ultrasound (POCUS) has become an integral diagnostic and interventional tool. Barriers to POCUS training persist, and it continues to remain heterogeneous across training programs. Structured POCUS assessment tools exist, but remain limited in their feasibility, acceptability, reliability, and validity; none of these tools are entrustment-based. The objective of this study was to derive a simple, entrustment-based POCUS competency assessment tool and pilot it in an assessment setting., Methods: This study was composed of two phases. First, a three-step modified Delphi design surveyed 60 members of the Canadian Association of Emergency Physicians Emergency Ultrasound Committee (EUC) to derive the anchors for the tool. Subsequently, the derived ultrasound competency assessment tool (UCAT) was used to assess trainee ( N  = 37) performance on a simulated FAST examination. The intraclass correlation (ICC) for inter-rater reliability and Cronbach's alpha for internal consistency were calculated. A statistical analysis was performed to compare the UCAT to other competency surrogates., Results: The three-round Delphi had 22, 26, and 26 responses from the EUC members. Consensus was reached, and anchors for the domains of preparation, image acquisition, image optimization, and clinical integration achieved approval rates between 92 and 96%. The UCAT pilot revealed excellent inter-rater reliability (with ICC values of 0.69-0.89; p   < 0.01) and high internal consistency (α = 0.91). While UCAT scores were not impacted by level of training, they were significantly impacted by the number of previous POCUS studies completed., Conclusions: We developed and successfully piloted the UCAT, an entrustment-based bedside POCUS competency assessment tool suitable for rapid deployment. The findings from this study indicate early validity evidence for the use of the UCAT as an assessment of trainee POCUS competence on FAST. The UCAT should be trialed in different populations performing several POCUS study types., (© 2020 by the Society for Academic Emergency Medicine.)

Purpose: To explore the social impact of, comfort with, and negative attitudes towards robots among young, middle-aged, and older adults in the United States., Design: Descriptive, cross-sectional. Conducted in 2014-2015 in an urban area of the western United States using a purposive sample of adults 18 years of age or older., Methods: Respondents completed a survey that included the Negative Attitudes Toward Robots Scale (NARS) and two questions taken or modified from the European Commission's Autonomous System 2015 Report. Analyses were conducted to compare perceptions and demographic factors by age groups (young adults:18-44, middle-aged adults: 45-64, and older adults: >65 years old)., Findings: Sample included 499 individuals (n = 322 age 18-44 years, n = 50 age 45-64 years, and n = 102 age 65-98 years). There were no significant differences between age groups for 9 of the 11 items regarding social impact of robots and comfort with robots. There were no significant differences by age groups for 9 of the 14 items in the NARS. Among those items with statistically significant differences, the mean scores indicate similar sentiments for each group., Conclusions: Older, middle-aged, and younger adults had similar attitudes regarding the social impact of and comfort with robots; they also had similar negative attitudes towards robots. Findings dispel current perceptions that older adults are not as receptive to robots as other adults. This has implications for nurses who integrate supportive robots in their practice., Clinical Relevance: Nurses working in clinical and community roles can use these findings when developing and implementing robotic solutions. Understanding attitudes towards robots can support how, where, and with whom robots can be used in nursing practice., (© 2018 Sigma Theta Tau International.)

Objective: Simulation stands to serve an important role in modern competency-based programs of assessment in postgraduate medical education. Our objective was to compare the performance of individual emergency medicine (EM) residents in a simulation-based resuscitation objective structured clinical examination (OSCE) using the Queen's Simulation Assessment Tool (QSAT), with portfolio assessment of clinical encounters using a modified in-training evaluation report (ITER) to understand in greater detail the inferences that may be drawn from a simulation-based OSCE assessment., Methods: A prospective observational study was employed to explore the use of a multicenter simulation-based OSCE for evaluation of resuscitation competence. EM residents from five Canadian academic sites participated in the OSCE. Video-recorded performances were scored by blinded raters using the scenario-specific QSATs with domain-specific anchored scores (primary assessment, diagnostic actions, therapeutic actions, communication) and a global assessment score (GAS). Residents' portfolios were evaluated using a modified ITER subdivided by CanMEDS roles (medical expert, communicator, collaborator, leader, health advocate, scholar, and professional) and a GAS. Correlational and regression analyses were performed comparing components of each of the assessment methods., Results: Portfolio review and ITER scoring was performed for 79 residents participating in the simulation-based OSCE. There was a significant positive correlation between total OSCE and ITER scores (r = 0.341). The strongest correlations were found between ITER medical expert score and each of the OSCE GAS (r = 0.420), communication (r = 0.443), and therapeutic action (r = 0.484) domains. ITER medical expert was a significant predictor of OSCE total (p = 0.002). OSCE therapeutic action was a significant predictor of ITER total (p = 0.02)., Conclusions: Simulation-based resuscitation OSCEs and portfolio assessment captured by ITERs appear to measure differing aspects of competence, with weak to moderate correlation between those measures of conceptually similar constructs. In a program of competency-based assessment of EM residents, a simulation-based OSCE using the QSAT shows promise as a tool for assessing medical expert and communicator roles.

Background: Alkaptonuria (AKU) is a serious genetic disease due to a defect in tyrosine metabolism, leading to increased serum levels of homogentisic acid (HGA). Nitisinone decreases HGA in AKU, but the concentration-response relationship has not been previously reported., Objectives: To determine the relationship between serum concentrations of nitisinone and the effect on both HGA and tyrosine; secondly to determine steady-state pharmacokinetics of nitisinone in AKU patients., Method: Thirty-two patients with AKU received either 1, 2, 4, or 8 mg nitisinone daily. Urine and serum HGA and serum tyrosine and nitisinone were measured during 24 h at baseline (before first dose) and after 4 weeks of treatment., Results: Nitisinone pharmacokinetics (area under the curve [AUC] and maximum concentrations [C max]) were dose proportional. The median oral clearance determined in all patients, irrespective of dose, was 3.18 mL/h·kg (range 1.6-6.7).Nitisinone decreased urinary excretion of HGA in a concentration-dependent manner, with a maximum effect seen at average nitisinone concentrations of 3 μmol/L. The association between nitisinone and tyrosine concentrations was less pronounced. Serum levels of HGA at Week 4 were below the limit of quantitation in 65% of samples, which prevented determination of the relationship with nitisinone concentrations., Conclusion: Nitisinone exhibits dose-proportional pharmacokinetics in the studied dosage interval. Urinary excretion of HGA decreases in a concentration-dependent manner, while the increase in tyrosine is less clearly related to nitisinone concentrations.

1. The effect of tachykinins on cholinergic neurotransmission was studied in an innervated tracheal tube preparation isolated from guinea-pigs anaesthetized with urethane. The tracheal tube was bathed in Krebs-Henseleit solution containing 5 microM indomethacin. 2. Neurokinin A (NKA), eledoisin (El) and substance P (SP) caused concentration-dependent increases in intraluminal pressure (ILP), with an order of potency NKA greater than El much greater than SP. 3. Low concentrations of tachykinins, that had little effect on ILP, caused an increase in the contractions elicited by stimulation of the preganglionic vagal nerve fibres and by postganglionic (transmural) stimulation. The order of potency was NKA greater than or equal to El greater than SP. Contractions induced by exogenous acetylcholine (ACh) were not increased by the tachykinins. 4. The magnitude of the tachykinin-induced augmentation of responses to nerve stimulation was inversely related to stimulation voltage and frequency. 5. These results suggest that tachykinins act on NK2 receptors, both on the trachealis muscle and on postganglionic pulmonary parasympathetic nerve terminals. Activation of the neuronal receptors may increase the probability of transmitter release from the nerve terminals.