Search

Abstract: Value‐based health care requires a balancing of medical outcomes with economic value. Administrators need to understand both the clinical and the economic effects of potentially expensive simulation programs to rationalize the costs. Given the often‐disparate priorities of clinical educators relative to health care administrators, justifying the value of simulation requires the use of economic analyses few physicians have been trained to conduct. Clinical educators need to be able to present thorough economic analyses demonstrating returns on investment and cost‐effectiveness to effectively communicate with administrators. At the 2017 Academic Emergency Medicine Consensus Conference “Catalyzing System Change through Health Care Simulation: Systems, Competency, and Outcomes,” our breakout session critically evaluated the cost‐benefit and return on investment of simulation. In this paper we provide an overview of some of the economic tools that a clinician may use to present the value of simulation training to financial officers and other administrators in the economic terms they understand. We also define three themes as a call to action for research related to cost–benefit analysis in simulation as well as four specific research questions that will help guide educators and hospital leadership to make decisions on the value of simulation for their system or program. [ABSTRACT FROM AUTHOR]

Abstract: Objectives: All residency programs in the United States are required to report their residents' progress on the milestones to the Accreditation Council for Graduate Medical Education (ACGME) biannually. Since the development and institution of this competency‐based assessment framework, residency programs have been attempting to ascertain the best ways to assess resident performance on these metrics. Simulation was recommended by the ACGME as one method of assessment for many of the milestone subcompetencies. We developed three simulation scenarios with scenario‐specific milestone‐based assessment tools. We aimed to gather validity evidence for this tool. Methods: We conducted a prospective observational study to investigate the validity evidence for three mannequin‐based simulation scenarios for assessing individual residents on emergency medicine (EM) milestones. The subcompetencies (i.e., patient care [PC]1, PC2, PC3) included were identified via a modified Delphi technique using a group of experienced EM simulationists. The scenario‐specific checklist (CL) items were designed based on the individual milestone items within each EM subcompetency chosen for assessment and reviewed by experienced EM simulationists. Two independent live raters who were EM faculty at the respective study sites scored each scenario following brief rater training. The inter‐rater reliability (IRR) of the assessment tool was determined by measuring intraclass correlation coefficient (ICC) for the sum of the CL items as well as the global rating scales (GRSs) for each scenario. Comparing GRS and CL scores between various postgraduate year (PGY) levels was performed with analysis of variance. Results: Eight subcompetencies were chosen to assess with three simulation cases, using 118 subjects. Evidence of test content, internal structure, response process, and relations with other variables were found. The ICCs for the sum of the CL items and the GRSs were >0.8 for all cases, with one exception (clinical management GRS = 0.74 in sepsis case). The sum of CL items and GRSs (p < 0.05) discriminated between PGY levels on all cases. However, when the specific CL items were mapped back to milestones in various proficiency levels, the milestones in the higher proficiency levels (level 3 [L3] and 4 [L4]) did not often discriminate between various PGY levels. L3 milestone items discriminated between PGY levels on five of 12 occasions they were assessed, and L4 items discriminated only two of 12 times they were assessed. Conclusion: Three simulation cases with scenario‐specific assessment tools allowed evaluation of EM residents on proficiency L1 to L4 within eight of the EM milestone subcompetencies. Evidence of test content, internal structure, response process, and relations with other variables were found. Good to excellent IRR and the ability to discriminate between various PGY levels was found for both the sum of CL items and the GRSs. However, there was a lack of a positive relationship between advancing PGY level and the completion of higher‐level milestone items (L3 and L4). [ABSTRACT FROM AUTHOR]

Participants in the 2008 Academic Emergency Medicine Consensus Conference “The Science of Simulation in Healthcare: Defining and Developing Clinical Expertise” morning workshop session on developing systems expertise were tasked with evaluating best applications of simulation techniques and technologies to small-scale systems in emergency medicine (EM). We collaborated to achieve several objectives: 1) describe relevant theories and terminology for discussion of health care systems and medical simulation, 2) review prior and ongoing efforts employing systems thinking and simulation programs in general medical sectors and acute care medicine, 3) develop a framework for discussion of systems thinking for EM, and 4) explore the rational application of advanced medical simulation methods to a defined framework of EM microsystems (EMMs) to promote a “quality-by-design ” approach. This article details the materials compiled and questions raised during the consensus process, and the resulting simulation application framework, with proposed solutions as well as their limitations for EM systems education and improvement. [ABSTRACT FROM AUTHOR]