Search

Purpose: To compare Manual Small Incision Cataract Surgery (MSICS) microsurgical performance in course participants who received virtual reality simulation‐based training by either a surgical expert or a non‐ophthalmologist instructor. Setting: Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark. Design: Randomized controlled trial. Methods: Residents and specialists in ophthalmology with no prior MSICS experience were included to receive virtual reality simulation training in MSICS using the HelpMeSee simulator. The participants were randomly allocated to receive training from either an experienced MSICS surgeon or a non‐ophthalmologist, also known as near‐peer teaching. The performances of the participants were evaluated at baseline and post‐training using a MSICS proficiency‐based test with evidence of validity. Results: Thirty participants were included in the study and 29 completed the course. There was no significant difference in final test score between the two groups (p = 0.13). The performance score of both groups of participants increased significantly after receiving the training (p < 0.001). All participants passed the proficiency‐based test after receiving the training. Conclusion: We found no significant difference in surgical proficiency‐level whether the participants were trained by a surgical expert or a non‐ophthalmologist instructor for MSICS in a virtual‐reality based setting. The findings of this study suggest that near‐peer teaching within microsurgical performance potentially could be applied with teaching outcomes comparable to a surgical expert‐instructor. [ABSTRACT FROM AUTHOR]

The objective of this study is to compare automated performance metrics (APM) and surgical gestures for technical skills assessment during simulated robot-assisted radical prostatectomy (RARP). Ten novices and six experienced RARP surgeons performed simulated RARPs on the RobotiX Mentor (Surgical Science, Sweden). Simulator APM were automatically recorded, and surgical videos were manually annotated with five types of surgical gestures. The consequences of the pass/fail levels, which were based on contrasting groups' methods, were compared for APM and surgical gestures. Intra-class correlation coefficient (ICC) analysis and a Bland–Altman plot were used to explore the correlation between APM and surgical gestures. Pass/fail levels for both APM and surgical gesture could fully distinguish between the skill levels of the surgeons with a specificity and sensitivity of 100%. The overall ICC (one-way, random) was 0.70 (95% CI: 0.34–0.88), showing moderate agreement between the methods. The Bland–Altman plot showed a high agreement between the two methods for assessing experienced surgeons but disagreed on the novice surgeons' skill level. APM and surgical gestures could both fully distinguish between novices and experienced surgeons in a simulated setting. Both methods of analyzing technical skills have their advantages and disadvantages and, as of now, those are only to a limited extent available in the clinical setting. The development of assessment methods in a simulated setting enables testing before implementing it in a clinical setting. [ABSTRACT FROM AUTHOR]

Objective: To develop appropriate content for high-stakes simulationbased assessments of operative competence in general surgery training through consensus. Background: Valid methods of summative operative competence assessment are required by competency-based training programs in surgery. Method: An online Delphi consensus study was conducted. Procedures were derived from the competency expectations outlined by the Joint Committee on Surgical Training Curriculum 2021, and subsequent brainstorming. Procedures were rated according to their perceived importance, perceived procedural risk, how frequently they are performed, and simualtion feasibility by a purposive sample of 30 surgical trainers and a 5-person steering group. A modified Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula was applied to the generated data to produce ranked procedural lists, which were returned to participants for re-prioritization. Results: Prioritized lists were generated for simulation-based operative competence assessments at 2 key stages of training; the end of ‘phase 2’ prior to the development of a sub-specialty interest, and the end of ‘phase 3’,that is, end-of-training certification. A total of 21 and 16 procedures were deemed suitable for assessments at each of these stages, respectively. Conclusions: This study describes a national needs assessment approach to content generation for simulation-based assessments of operative competence in general surgery using Delphi consensus methodology. The prioritized procedural lists generated by this study can be used to further develop operative skill assessments for use in high-stakes scenarios, such as trainee progression, entrustment, and end-of-training certification, before subsequent validity testing. [ABSTRACT FROM AUTHOR]

Aim: There is currently an increased focus on competency‐based training, in which training and assessment play a crucial role. The aim of this systematic review is to create an overview of hands‐on training methods and assessment tools for appendicectomy and colon and rectal surgery procedures using either an open, laparoscopic or robot‐assisted approach. Method: A systematic review of Medline, Embase, Cochrane and Scopus databases was conducted following the PRISMA guidelines. We conducted the last search on 9 March 2023. All published papers describing hands‐on training, evaluation of performance data and development of assessment tools were eligible. The quality of studies and the validity evidence of assessment tools are reported. Results: Fifty‐one studies were identified. Laparoscopic assessment tools are abundant, but the literature still lacks good‐quality assessment tools for open appendicectomy, robotic colectomy and open rectal surgery. Overall, there is a lack of discussion regarding the establishment of pass/fail standards and the consequences of assessment. Virtual reality simulation is used more for appendicectomy than colorectal procedures. Only a few of the studies investigating training were of acceptable quality. There is a need for high‐quality studies in open and robotic‐assisted colon surgery and all approaches to rectal surgery. Conclusion: This review provides an overview of current training methods and assessment tools and identifies where more research is needed based on the quality of the studies and the current validity evidence. [ABSTRACT FROM AUTHOR]

Navigating through the bronchial tree and visualizing all bronchial segments is the initial step toward learning flexible bronchoscopy. A novel bronchial segment identification system based on artificial intelligence (AI) has been developed to help guide trainees toward more effective training. Does feedback from an AI-based automatic bronchial segment identification system improve novice bronchoscopists' end-of-training performance? The study was conducted as a randomized controlled trial in a standardized simulated setting. Novices without former bronchoscopy experience practiced on a mannequin. The feedback group (n = 10) received feedback from the AI, and the control group (n = 10) trained according to written instructions. Each participant decided when to end training and proceed to performing a full bronchoscopy without any aids. The feedback group performed significantly better on all three outcome measures (median difference, P value): diagnostic completeness (3.5 segments, P <.001), structured progress (13.5 correct progressions, P <.001), and procedure time (−214 seconds, P =.002). Training guided by this novel AI makes novices perform more complete, more systematic, and faster bronchoscopies. Future studies should examine its use in a clinical setting and its effects on more advanced learners. [Display omitted] [ABSTRACT FROM AUTHOR]

Background: In the Difficult Airway Society's 2015 "cannot intubate, cannot oxygenate" guideline, the emergency cricothyroidotomy is the final option when managing an unanticipated difficult airway. How often training for maintenance of this skill is required for anesthesiologists remains unknown. We aimed to assess if specialist‐trained anesthesiologists' skills improved from a brush‐up intervention and if skills were retained after 3 months. Methods: In this multicenter, randomized, controlled trial, participants were randomized to either a simulation‐based brush‐up or no brush‐up. Both groups performed a mannequin‐based technical skills emergency cricothyroidotomy test twice and were assessed by a blinded rater using a structured assessment tool that included time, positioning, palpation, appropriate employment of instruments, and stepwise progression. After 3 months of non‐training, participants completed identical tests of retention. Results: A total of 54 anesthesiologists were included from three hospitals in the Region of Southern Denmark. Thirty‐seven percent of the participants had received skills training in emergency cricothyroidotomy in the prior 12 months. The intervention group (N = 27) performed better in the initial tests, with a mean time of 51.5 s (SD = 10.82), a total score per minute of 15.9 points (SD = 4.91), and 93% passing both initial tests compared to the control group (N = 27) with a mean time of 76.8 s (SD = 35.82), a total score per minute of 6.6 (SD = 4.68) and only 15% passing both initial tests. The intervention group managed to retain overall performance in retention tests in terms of performance time (48.9 s, p =.26), total score per minute (13.6 points, p =.094), and passing the tests (75%, p =.059). Conclusion: Exposure to simulation‐based brush‐up training in emergency cricothyroidotomy improved anesthesiologists' technical performance and was overall retained after 3 months. Some loss of skill concerning specific items was observed, highlighting the need for regular training in emergency cricothyroidotomy. Simulation‐based training should be prioritized to improve and maintain technical skills in infrequent high‐stakes procedures. [ABSTRACT FROM AUTHOR]

Objectives: This study aimed to gather validity evidence according to Messick's framework for a novel 3D‐printed simulator for myringotomy with ventilation tube insertion for use in technical skills training of otorhinolaryngology (ORL) residents. Methods: The study included 15 junior ORL residents (trainees) and 13 experienced teaching otolaryngologists (experts). Experts and trainees first received an identically structured introduction to the procedure, simulator, and simulation setup. Five procedures performed by each participant were video‐recorded and ordered randomly for blinded rating by two independent raters. The rating tools used were a global rating scale (GBRS) and a task‐specific checklist. Validity evidence was collected according to Messick's framework. Differences in time consumption and performance scores were analyzed. Finally, a pass/fail standard was established using the contrasting groups' method. Results: Trainees used significantly more time per procedure (109 s, 95% CI: 99–120) than experts (82 s, 95% CI: 71–93; p <.001). Adjusted for repetition and rater leniency, experts achieved an average GBRS score of 18.8 (95% CI: 18.3–19.2) out of 20 points, whereas trainees achieved an average of 17.1 points (95% CI: 16.6–17.5; p <.001). In contrast to the task‐specific checklist, the GBRS score discriminated between repetition number and participant experience. The pass/fail standard for the GBRS was established at 18.4 points. Conclusion: We established educational validity evidence for a novel 3D‐printed model for simulation‐based training of ventilation tube insertion and established a reliable pass/fail standard. Level of Evidence: 1b. [ABSTRACT FROM AUTHOR]

Background: Bronchoscopy is an essential procedure in the diagnosis and treatment of pulmonary diseases. However, the literature suggests that distractions affect the quality of bronchoscopy and affect inexperienced doctors more than experienced. Objectives: The objective of the study was as follows: does simulation-based bronchoscopy training with immersive virtual reality (iVR) improve the doctors' ability to handle distractions and thereby increase the quality, measured in procedure time, structured progression score, diagnostic completeness (%), and hand motor movements of a diagnostic bronchoscopy in a simulated scenario. Exploratory outcomes were heart rate variability and a cognitive load questionnaire (Surg-TLX). Methods: Participants were randomized. The intervention group practiced in an iVR environment with a head-mounted display (HMD) while using the bronchoscopy simulator, while the control group trained without the HMD. Both groups were tested in the iVR environment using a scenario with distractions. Results: 34 participants completed the trial. The intervention group scored significantly higher in diagnostic completeness (100 i.q.r. 100–100 vs. 94 i.q.r. 89–100, p value = 0.03) and structured progress (16 i.q.r. 15–18 vs. 12 i.q.r. 11–15, p value 0.03) but not in procedure time (367 s standard deviation [SD] 149 vs. 445 s SD 219, p value = 0.06) or hand motor movements (−1.02 i.q.r. −1.03–[−1.02] versus −0.98 i.q.r. −1.02–[–0.98], p value = 0.27). The control group had a tendency toward a lower heart rate variability (5.76 i.q.r. 3.77–9.06 vs. 4.12 i.q.r. 2.68–6.27, p = 0.25). There was no significant difference in total Surg-TLX points between the two groups. Conclusion: iVR simulation training increases the quality of diagnostic bronchoscopy in a simulated scenario with distractions compared with conventional simulation-based training. [ABSTRACT FROM AUTHOR]

Acupuncture is an efficient treatment modality for many different training, but the technique can be challenging to master. Identification of the different acupuncture points can now be practiced in a standardized and patient-safe simulation-based environment. We aim to gather additional evidence regarding simulation-based training of acupuncture skills and hope that the simulation-based test developed and will be used for simulation-based mastery learning in future develop. The aim of this study was to develop a test of acupuncture skills using the TCM (Traditional Chinese Medicine) Meridians & Acupoints Anatomy Teaching Platform and gather validity evidence using Messick's framework. Novice medical students without prior experience in acupuncture (n = 43), intermediate medical students who had finished the mandatory acupuncture course (n = 24), and skilled acupuncture doctors (n = 17) were included in the study. One-hundred and seven acupuncture points were presented to the participants in identical randomized order and they were asked to identify each point by touching the correct anatomical location on the simulator. The simulator automatically registered the accuracy of participants' answers. The reliability of the final test (66 acupuncture points) was at a high level with a Cronbach's alpha of 0.98 and only 16 acupuncture points were needed to ensure a reliability of 0.90. The test could discriminate significantly between all three groups and a credible pass/fail-score of 34 points was determined which resulted in eight out of 43 (19%) novices passing the test and only two out of 17 (12%) experienced failing the test. These consequences were statistically significant (P < 0.001). Based on Messick's framework we demonstrated strong validity evidence for a newly developed simulation-based test of acupuncture points localization skills. The test can be used as an aid to decide when trainees are ready to progress to more advanced training and clinical practice. Compared with the traditional point-point test, the point-point evaluation of the simulator can be more objective and reduce the effort of teachers. [ABSTRACT FROM AUTHOR]

Objective: This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures. Summary of background: Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods. Methods: Systematic literature search was done in 4 databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with the application of a standard setting were included. Quality of evidence was appraised using GRADE. Results: Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, setup, and expert level classification. The studies were appraised as having low and moderate evidence. Conclusion: Surgical education is shifting toward competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid, and reliable assessment. [ABSTRACT FROM AUTHOR]

Purpose: To investigate the learning curve of robot‐assisted vitreoretinal surgery compared to manual surgery in a simulated setting. Methods: The study was designed as a randomized controlled longitudinal study. Eight ophthalmic trainees in the 1st or 2nd year of their specialization were included. The participants were randomized to either manual or robot‐assisted surgery. Participants completed repetitions of a test consisting of three vitreoretinal modules on the Eyesi virtual reality simulator. The primary outcome measure was time to learning curve plateau (minutes) for total test score. The secondary outcome measures were instrument movement (mm), tissue treatment (mm2) and time with instruments inserted (seconds). Results: There was no significant difference in time to learning curve plateau for robot‐assisted vitreoretinal surgery compared to manual. Robot‐assisted vitreoretinal surgery was associated with less instrument movements (i.e. improved precision), −0.91 standard deviation (SD) units (p < 0.001). Furthermore, robot‐assisted vitreoretinal surgery was associated with less tissue damage when compared to manual surgery, −0.94 SD units (p = 0.002). Lastly, robot‐assisted vitreoretinal surgery was slower than manual surgery, 0.93 SD units (p < 0.001). Conclusions: There was no significant difference between the lengths of the learning curves for robot‐assisted vitreoretinal surgery compared to manual surgery. Robot‐assisted vitreoretinal surgery was more precise, associated with less tissue damage, and slower. [ABSTRACT FROM AUTHOR]

Background: Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists. Objectives: The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion. Methods: Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS). Results: Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson's correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson's correlation, r = -0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups' method, a passing score of -0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures. Conclusions: MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance. [ABSTRACT FROM AUTHOR]

The aim of this study was to systematically review the literature and give evidence based recommendations for future initiatives for simulation based training (SBT) and assessment in open vascular surgery. PubMed, Embase, and the Cochrane Library. A systematic review of PubMed, Embase, and the Cochrane Library was performed, with the last search on 31 March 2020, to identify studies describing SBT and assessment in open vascular surgery. Kirkpatrick's levels for efficacy of training were evaluated. Validity evidence for assessment tools was evaluated according to the recommended contemporary framework by Messick. Of 2 844 studies, 51 were included for data extraction. A high degree of heterogeneity in reporting standards and varying types of simulation was found. Vascular anastomosis was the most frequently simulated technical skill (43%). Assessment was mostly carried out using the Objective Structured Assessment of Technical Skills (55%). Validity evidence for assessment tools was found using outdated frameworks, and only one study used Messick's framework. Self directed training is valuable, the low trainer to trainee ratio is important to maximise efficiency, and experienced vascular surgeons are the most effective trainers. Carefully designed and structured SBT is effective and can improve technical skills, especially in less experienced trainees. However, the supporting evidence lacks homogeneity in the reporting standards and types of simulations. Pass/fail standards that support proficiency based learning and studies investigating skills transfer should be the focus in future studies. Validity evidence of assessment tools needs to be addressed using contemporary frameworks. [ABSTRACT FROM AUTHOR]

Introduction: As ultrasound becomes more accessible, the use of point-of-care ultrasound examinations performed by clinicians has increased. Sufficient theoretical and practical skills are prerequisites to integrate thoracic ultrasound into a clinical setting and to use it as supplement in the clinical decision-making. Recommendations on how to educate and train clinicians for these ultrasound examinations are debated, and simulation-based training may improve clinical performance. Objectives: The aim of this study was to explore the effect of simulation-based training in thoracic ultrasound compared to training on healthy volunteers. Method: A total of 66 physicians with no previous experience in thoracic ultrasound completed a training program and assessment of competences from November 2018 to May 2019. After a theoretical session in ultrasound physics, sonoanatomy, and thoracic ultrasound, the physicians were randomized into one of three groups for practical training: (1) simulation-based training, (2) training on a healthy volunteer, or (3) no training (control group). Primary outcome was difference in the clinical performance score after the training period. Results: Using a multiple comparison, ANOVA with Bonferroni correction for multiplicity, there was no statistical significant difference between the two trained groups' performance score: 45.1 points versus 41.9 points (minimum 17 points, maximum 68 points; p = 0.38). The simulation-based training group scored significantly higher than the control group without hands-on training, 36.7 points (p = 0.009). Conclusions: The use of simulation-based training in thoracic ultrasound does not improve the clinical performance score compared to conventional training on healthy volunteers. As focused, thoracic ultrasound is a relatively uncomplicated practical procedure when taught; focus should mainly be on the theoretical part and the supervised clinical training in a curriculum. However, simulation can be used instead or as an add-on to training on simulated patients. [ABSTRACT FROM AUTHOR]

Purpose: To develop and investigate an Eyesi simulator‐based test for the more experienced cataract surgeon for evidence of validity. Methods: The study was a prospective interventional cohort study and carried out at the Copenhagen Academy for Medical Education and Simulation. The Eyesi Simulator was used for the test which was developed by three expert cataract surgeons. Ten cataract surgeons (>250 surgeries performed) and ten ophthalmic residents performed two repetitions of the test. The test consisted of four modules: Iris Expansion Ring insertion – level 1, Iris Expansion Ring extraction – level 2, Capsulorhexis – level 3 and Anterior Vitrectomy – level 6. Results: Internal consistency reliability showed Cronbach's alpha of 0.63. Test–retest reliabilities were significant for Iris Expansion Ring extraction – level 2 (p = 0.012) and Capsulorhexis – level 3 (p = 0.018). Differences between the two groups were only significant in both repetitions for the Iris Expansion Ring extraction – level 2 (p < 0.001 and p = 0.041, respectively). Furthermore, we found a statistically significant difference between the mean module scores for novices and the more experienced surgeons for Iris Expansion Ring insertion – level 1 (p = 0.021) and Capsulorhexis – level 3 (p = 0.019) in the first repetition. Conclusion: The investigated modules show evidence of validity within several aspects of Messick's framework. However, the evidence is not strong enough to apply the test for certification purposes of cataract surgeons, but the modules may still be relevant in the training of advanced cataract surgical procedures. [ABSTRACT FROM AUTHOR]

The aims of this study were to develop a procedure specific assessment tool for open abdominal aortic aneurysm (AAA) repair, gather validity evidence for the tool and establish a pass/fail standard. Validity was studied based on the contemporary framework by Messick. Three vascular surgeons experienced in open AAA repair and an expert in assessment and validation within medical education developed the OPEn aortic aneurysm Repair Assessment of Technical Expertise (OPERATE) tool. Vascular surgeons with varying experiences performed open AAA repair in a standardised simulation based setting. All procedures were video recorded with the faces anonymised and scored independently by three experts in a mutual blinded setup. The Angoff standard setting method was used to establish a credible pass/fail score. Sixteen novices and nine experienced open vascular surgeons were enrolled. The OPERATE tool achieved high internal consistency (Cronbach's alpha.92) and inter-rater reliability (Cronbach's alpha.95) and was able to differentiate novices and experienced surgeons with mean scores (higher score is better) of 13.4 ± 12 and 25.6 ± 6, respectively (p =.01). The pass/fail score was set high (27.7). One novice passed the test while six experienced surgeons failed. Validity evidence was established for the newly developed OPERATE tool and was able to differentiate between novices and experienced surgeons providing a good argument that this tool can be used for both formative and summative assessment in a simulation based environment. The high pass/fail score emphasises the need for novices to train in a simulation based environment up to a certain level of competency before apprenticeship training in the clinical environment under the tutelage of a supervisor. Familiarisation with the simulation equipment must be ensured before performance is assessed as reflected by the low scores in the experienced group's first attempt. Image 1 [ABSTRACT FROM AUTHOR]

Purpose: Virtual reality (VR) training of mastoidectomy is effective in surgical training—particularly if organized as distributed practice. However, centralization of practice facilities is a barrier to implementation of distributed simulation training. Decentralized training could be a potential solution. Here, we aim to assess the feasibility, use, and barriers to decentralized VR mastoidectomy training using a freeware, high-fidelity temporal bone simulator. Methods: In a prospective, mixed-methods study, 20 otorhinolaryngology residents were given three months of local access to a VR mastoidectomy simulator. Additionally, trainees were provided a range of learning supports for directed, self-regulated learning. Questionnaire data were collected and focus group interviews conducted. The interviews were analyzed using thematic analysis and compared with quantitative findings. Results: Participants trained 48.5 h combined and mainly towards the end of the trial. Most participants used between two and four different learning supports. Qualitative analysis revealed five main themes regarding implementation of decentralized simulation training: convenience, time for training, ease of use, evidence for training, and testing. Conclusions: Decentralized VR training using a freeware, high-fidelity mastoidectomy simulator is feasible but did not lead to a high training volume or truly distributed practice. Evidence for training was found motivational. Access to training, educational designs, and the role of testing are important for participant motivation and require further evaluation. [ABSTRACT FROM AUTHOR]

Background: Trocar pigtail catheter thoracentesis (TPCT) is a common procedure often performed by junior physicians. Simulation-based training may effectively train physicians in the procedure prior to performing it on patients. An assessment tool with solid validity evidence is necessary to ensure sufficient procedural competence. Objectives: Our study objectives were (1) to collect evidence of validity for a newly developed pigtail catheter assessment tool (Thoracentesis Assessment Tool [ThorAT]) developed for the evaluation of TPCT performance and (2) to establish a pass/fail score for summative assessment. Methods: We assessed the validity evidence for the ThorAT using the recommended framework for validity by Messick. Thirty-four participants completed two consecutive procedures and their performance was assessed by two blinded, independent raters using the ThorAT. We compared performance scores to test whether the assessment tool was able to discern between the two groups, and a pass/fail score was established. Results: The assessment tool was able to discriminate between the two groups in terms of competence level. Experienced physicians received significantly higher test scores than novices in both the first and second procedure. A pass/fail score of 25.2 points was established, resulting in 4 (17%) passing novices and 1 (9%) failing experienced participant in the first procedure. In the second procedure 9 (39%) novices passed and 2 (18%) experienced participants failed. Conclusions: This study provides a tool for summative assessment of competence in TPCT. Strong validity evidence was gathered from five sources of evidence. A simulation-based training program using the ThorAT could ensure competence before performing thoracentesis on patients. [ABSTRACT FROM AUTHOR]

Objectives The aims of this study were to investigate the validity evidence for a novel procedure specific assessment tool of competence in endovascular aortic repair (EVAR) and to establish a pass/fail level for the assessment. Methods Computed tomography angiography data of a 55 mm in diameter infrarenal aortic aneurysm was implemented into an endovascular simulator. Twenty-three physicians with varying EVAR experiences were video-recorded when performing a standard EVAR procedure on the simulator. Two experienced EVAR operators assessed the participants using the novel rating scale, “EndoVascular Aortic Repair Assessment of Technical Expertise” (EVARATE). Validity was studied according to the framework endorsed by the American Educational Research Association. Results The EVARATE scale had a high internal consistency (Cronbach's alpha = .90). The inter-rater reliability was acceptable (Intraclass Correlation Coefficient = .68, p = .005). Specific EVAR experience correlated significantly with the EVARATE score (Spearman's rho = .62, p = .002), but general endovascular experience did not. Consequence analysis showed that the EVARATE assessment could distinguish novices from intermediates ( p < .01) and from experts ( p < .001). A pass/fail score was determined using the contrasting groups' method. Conclusion This paper presents the initial validity evidence for a novel procedure specific assessment tool, EVARATE, for operator competence in endovascular aortic repair investigated in a simulated setting. The assessment tool can be used to provide structured formative feedback to trainees. [ABSTRACT FROM AUTHOR]

Objective:Simulation-based training is well recognized in the transforming field of urological surgery; however, integration into the curriculum is often unstructured. Development of simulation-based curricula should follow a stepwise approach starting with a needs assessment. This study aimed to identify technical procedures in urology that should be included in a simulation-based curriculum for residency training. Materials and methods:A national needs assessment was performed using the Delphi method involving 56 experts with significant roles in the education of urologists. Round 1 identified technical procedures that newly qualified urologists should perform. Round 2 included a survey using an established needs assessment formula to explore: the frequency of procedures; the number of physicians who should be able to perform the procedure; the risk and/or discomfort to patients when a procedure is performed by an inexperienced physician; and the feasibility of simulation training. Round 3 involved elimination and reranking of procedures according to priority. Results:The response rates for the three Delphi rounds were 70%, 55% and 67%, respectively. The 34 procedures identified in Round 1 were reduced to a final prioritized list of 18 technical procedures for simulation-based training. The five procedures that reached the highest prioritization were cystoscopy, transrectal ultrasound-guided biopsy of the prostate, placement of ureteral stent, insertion of urethral and suprapubic catheter, and transurethral resection of the bladder. Conclusion:The prioritized list of technical procedures in urology that were identified as highly suitable for simulation can be used as an aid in the planning and development of simulation-based training programs. [ABSTRACT FROM AUTHOR]

Background: The use of simulators in a training programme for technically challenging procedures has the advantages of lowering the risk of patient complications while helping the trainees with the initial part of their learning curve. Objectives: The aim of this study was to perform a systematic review of simulation-based training in flexible bronchoscopy and endobronchial ultrasound (EBUS). Methods: We identified 1,006 publications in the PubMed database and included publications on flexible bronchoscopy below the carina and EBUS involving hands-on simulation-based training. Publications were excluded if they were written in languages other than English, if paediatric airways were involved or if they were not journal articles. The screening process was performed by 2 individuals, and a third reviewer made the final decision in case of disagreement. Results: We included 30 publications. The studies included participants of varying experience and most commonly used a virtual reality simulator as a training modality. Assessment of the participants' skills was based on simulator metrics or on an assessment tool. Some studies included performance on patients for assessment of the operator after training on a simulator. Conclusions: Simulation-based training was demonstrated to be more efficient than the traditional apprenticeship model. Physical models and virtual reality simulators complement each other. Simulation-based education should be based on a mastery learning approach and structured as directed self-regulated learning in a distributed training programme. [ABSTRACT FROM AUTHOR]

Background: Simulation training is a revolutionary addition to health care education. However, developing simulationbased training programs is often dictated by those simulators that are commercially available. Curriculum development requires deliberate planning and a standardized approach, including a 'general needs assessment'. Objectives: The aim of this study was to perform a national general needs assessment to identify technical procedures in pulmonary medicine that should be integrated in a simulationbased curriculum. Methods: A three-round Delphi process was initiated among 62 key opinion leaders. Round 1 was an open-ended question to identify technical procedures pulmonologists should learn. Round 2 was a survey using a newly developed needs assessment formula to explore the fre-quency of procedures, number of operators, risk or discomfort when performed by an inexperienced doctor, and feasibility of simulation-based training. In round 3, results were reviewed and ranked according to priority. Results: The response rates for the three rounds were 74, 63, and 60%, respectively. The Delphi process reduced the 30 procedures identified in round 1 to 11 prioritized technical procedures in round 3. These were: flexible bronchoscopy, pleurocentesis, endobronchial ultrasound, endoscopic ultrasoundguided fine-needle aspiration, noninvasive ventilation treatment, transthoracic biopsy of pleural or lung tumor, focused ultrasound scanning of the lungs, chest tube insertion, needle biopsy of visible lymph node/tumor of the skin, focused ultrasound scanning of the heart, and thoracoscopy. Conclusion: We performed a Delphi study using a needs assessment formula, which identified 11 technical procedures that are highly suitable for simulation-based training. Medical educators can use this list as a resource in planning simulation- based training programs for trainees in pulmonary medicine. [ABSTRACT FROM AUTHOR]

Background: The use of structured curricula for minimally invasive surgery training is becoming increasingly popular. However, many laparoscopic training programs still use basic skills and isolated task training, despite increasing evidence to support the use of training models with higher functional resemblance, such as whole procedural modules. In contrast to basic skills training, procedural training involves several cognitive skills such as elements of planning, movement integration, and how to avoid adverse events. The objective of this trial is to investigate the specificity of procedural practice in laparoscopic simulator training. Methods/Design: A randomised single-centre educational superiority trial. Participants are 96 surgical novices (medical students) without prior laparoscopic experience. Participants start by practicing a series of basic skills tasks to a predefined proficiency level on a virtual reality laparoscopy simulator. Upon reaching proficiency, the participants are randomised to either the intervention group, which practices two procedures (an appendectomy followed by a salpingectomy) or to the control group, practicing only one procedure (a salpingectomy) on the simulator. 1:1 central randomisation is used and participants are stratified by sex and time to complete the basic skills. Data collection is done at a surgical skills centre. The primary outcome is the number of repetitions required to reach a predefined proficiency level on the salpingectomy module. The secondary outcome is the total training time to proficiency. The improvement in motor skills and effect on cognitive load are also explored. Discussion: The results of this trial might provide new knowledge on how the technical part of surgical training curricula should be comprised in the future. To examine the specificity of practice in procedural simulator training is of great importance in order to develop more comprehensive surgical curricula. [ABSTRACT FROM AUTHOR]

Objective. A gold standard of skills required for nurse-administered propofol sedation (NAPS) for gastroenterological endoscopic procedures has been proposed but not established. Due to the potentially hazardous nature of NAPS, an assessment tool is needed to objectively judge the adequacy of training and for future certification. The aim of this study was to develop an assessment tool for measuring competency in propofol sedation and to explore the reliability and validity of the tool. Material and methods. The nurse-administered propofol assessment tool (NAPSAT) was developed in a Delphi-like fashion. Consensus was achieved on 17 items. Validity evidence was gathered in a case-control study in a full-scale simulation setting. Six experienced nurses and six novice nurses were filmed in two scenarios for assessment according to the assessment tool by three content expert raters. Results. A total of 72 NAPSAT assessment forms were analyzed. Inter-rater reliability, Cronbach's α = 0.54 and generalizability coefficient = 0.68. The experienced nurses scored higher than the novices, 52.8 versus 62.7, p = 0.009. The provided pass/borderline/fail assessment showed significant difference, p = < 0.001, Cronbach's α = 0.80, with the novices being more likely to fail and the experienced more likely to pass. Conclusion. Assessing sedation skills in a simulator is possible. Video assessment requires expert knowledge of the procedure and the rating matrix. Overall, NAPSAT showed fair inter-rater reliability and good construct validity. This makes NAPSAT fit for formative assessment and proficiency feedback; however, high stakes and summative assessment cannot be advised. [ABSTRACT FROM AUTHOR]

Introduction Virtual-reality ( VR) simulation provides a safe and effective learning environment prior to practicing on patients. However, existing bronchoscopy simulators are expensive and not easily portable. Objectives The aim of this study was to assess the effect of self-directed training on a new, portable, simple simulator measured by transfer of skills to performance of more complex bronchoscopy procedures on an advanced VR simulator. Methods Twenty medical students participated in the study. After a general introduction to bronchoscopy, they were randomised into two groups, receiving either self-directed bronchoscopy training using a portable, simple simulator or no manual training. Subsequently, all participants were tested on complex scenarios in an advanced VR simulator using a validated bronchoscopy quality test. Bronchoscopy quality scores were compared using independent samples t-test and correlated with a previously established pass-fail standard. Results The intervention group spent an average of 71-min training on the new simulator. The intervention group performed significantly better than the control group, mean bronchoscopy quality score 0.55 [standard deviation ( SD) 0.16] vs 0.36 ( SD 0.10), P = 0.005, effect size = 1.47. Eight out of 10 participants in the intervention group passed the test compared with only 1 out of 10 in the control group. Conclusion The effect of a brief, self-directed training session using a portable, simple simulator was substantial and transferred to performance of more complex skills. [ABSTRACT FROM AUTHOR]

Background: International guidelines suggest that trainees should perform at least 100 flexible bronchoscopies in a supervised setting, but this number is not evidence based. An objective assessment method could provide educational feedback to trainees and help supervisors decide when basic competency is established. No former assessment instrument has been able to distinguish between trainees and experts. Objectives: The aim of this study was to explore the validity and reliability of a new assessment procedure relating to testing operators across multiple tasks with increasing difficulty using a standardized scoring form. Methods: The test was administered on a virtual reality bronchoscopy simulator to a total of 42 test subjects (14 senior consultants, 14 trainees and 14 medical students). The inter-rater reliability of the test procedure was explored according to examination of test results from 10 subjects using 3 raters (2 blinded). Results: We found a high inter-rater reliability (Cronbach's α = 0.90). The assessment procedure sufficiently differentiated the performance of the 3 groups (p < 0.001). Assessment of 6 procedures was necessary to secure a generalizability coefficient >0.80. Conclusions: The new assessment procedure proved both valid and reliable. Copyright © 2011 S. Karger AG, Basel [ABSTRACT FROM AUTHOR]

Introduction: Simulation-based training has proven effective for learning flexible bronchoscopy. However, no studies have tested the efficacy of training toward established proficiency criteria, i.e., mastery learning (ML). We wish to test the effectiveness of ML compared to directed self-regulated learning (DSRL) on novice bronchoscopists’ end-of-training performance. Methods: In a standardized simulated setting, novices without prior bronchoscopy experience were trained using an artificial intelligence (AI) guidance system that automatically recognizes the bronchial segments. They were randomized into two groups: the ML group and the DSRL group. The ML group was trained until they completed two procedures meeting the proficiency targets: 18 inspected segments, 18 structured progressions, <120-s procedure time. The DSRL group was trained until they no longer perceived any additional benefits from training. Both groups then did a finalizing test, without the AI guidance enabled. Results: A total of 24 participants completed the study, with 12 in each group. Both groups had a high mean number of inspected segments (ML = 17.2 segments, DSRL = 17.3 segments, p = 0.85) and structured progressions (ML = 15.5 progressions, DSRL = 14.8 progressions, p = 0.58), but the ML group performed the test procedure significantly faster (ML = 107 s, DSRL = 180 s, p < 0.001). The ML did not spend significantly longer time training (ML = 114 min, DSRL = 109 min, p = 0.84). Conclusions: ML is a very efficient training form allowing novice trainees to learn how to perform a thorough, systematic, and quick flexible bronchoscopy. ML does not require longer time spent training compared to DSRL, and we therefore recommend training of future bronchoscopists by this method. [ABSTRACT FROM AUTHOR]

Patient-specific rehearsal (PsR) is a new concept whereby a procedure can be rehearsed virtually using the exact patient's anatomical data prior to performing the real procedure. The aim of this study was to ascertain if endovascular virtual-reality PsR simulation enhanced performance in real life. This was done by performing a systematic review following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines. A literature search was conducted in PubMed, Embase, The Cochrane Library and Web of Science concerning PsR in endovascular procedures. All publications were handled using Covidence. Reference lists were also screened. Data extracted from the studies were realism rating, procedure time, fluoroscopy time, contrast volume, number of angiograms and reduction of errors. Kirkpatrick's four-level model for measuring the efficiency of training was used for guidance of the relevance of studies. The search yielded 1155 results after the exclusion of duplicates, and 11 studies were included. Four studies had a control group, including one randomized trial; the remaining seven were feasibility studies. The realism was rated high, and overall, the studies showed improvements in procedure time, fluoroscopy time and contrast volume after PsR. One study assessed and confirmed the reduction in errors after PsR. Only two studies included more than 15 patients in their cohort. Kirkpatrick's model was applied to all studies, with one study reaching level 4. All studies found the concept of PsR to be feasible and realistic. The studies with a control group showed a reduction of overall procedure time, radiation exposure and potential errors in endovascular procedures following PsR. [ABSTRACT FROM AUTHOR]

BACKGROUND: Cochlear implantation requires excellent surgical skills; virtual reality simulation training is an effective method for acquiring basic competency in temporal bone surgery before progression to cadaver dissection. However, cochlear implantation virtual reality simulation training remains largely unexplored and only one simulator currently supports the training of the cochlear implantation electrode insertion. Here, we aim to evaluate the effect of cochlear implantation virtual reality simulation training on subsequent cadaver dissection performance and self-directedness. METHODS: This was a randomized, controlled trial. Eighteen otolaryngology residents were randomized to either mastoidectomy including cochlear implantation virtual reality simulation training (intervention) or mastoidectomy virtual reality simulation training alone (controls) before cadaver cochlear implantation surgery. Surgical performance was evaluated by two blinded expert raters using a validated, structured assessment tool. The need for supervision (reflecting self-directedness) was assessed via post-dissection questionnaires. RESULTS: The intervention group achieved a mean score of 22.9 points of a maximum of 44 points, which was 5.4% higher than the control group's 21.8 points (P = .51). On average, the intervention group required assistance 1.3 times during cadaver drilling; this was 41% more frequent in the control group who received assistance 1.9 times (P = .21). CONCLUSION: Cochlear implantation virtual reality simulation training is feasible in the context of a cadaver dissection course. The addition of cochlear implantation virtual reality training to basic mastoidectomy virtual reality simulation training did not lead to a significant improvement of performance or self-directedness in this study. Our findings suggest that learning an advanced temporal bone procedure such as cochlear implantation surgery requires much more training than learning mastoidectomy. [ABSTRACT FROM AUTHOR]

Ultrasound exams need skilled examiners, and simulation-based training could provide standardized and safe skill training. This study aims to give an overview of different types of virtual-reality simulators for use in abdominal diagnostic ultrasound training in radiology. Fifteen specialized radiologists and radiological residents were presented with two similar cases on four different simulators for abdominal ultrasound training. A feedback sheet for each individual simulator and for an overall comparison was filled out by all participants. All means of scores were compared, and simulators were ranked from least to most favorable. One simulator was ranked most favorable in seven out of nine questions, but none of the mean scores had statistically significant differences. All simulators were recommended for training radiologists, and all simulators were perceived to benefit trainees more than experienced ultra-sonographers. [ABSTRACT FROM AUTHOR]

Focused assessment with sonography for trauma (FAST) is a valuable ultrasound procedure in emergency settings, and there is a need for evidence-based education in FAST to ensure competencies. Immersive virtual reality (IVR) is a progressive training modality gaining traction in the field of ultrasound training. IVR holds several economic and practical advantages to the common instructor-led FAST courses using screen-based simulation (SBS). This prospective, interventional cohort study investigated whether training FAST using IVR unsupervised and out-of-hospital was non-inferior to a historical control group training at a 90 min SBS course in terms of developing FAST competencies in novices. Competencies were assessed in both groups using the same post-training simulation-based FAST test with validity evidence, and a non-inferiority margin of 2 points was chosen. A total of 27 medical students attended the IVR course, and 27 junior doctors attended the SBS course. The IVR group trained for a median time of 117 min and scored a mean 14.2 ± 2.0 points, compared with a mean 13.7 ± 2.5 points in the SBS group. As the lower bound of the 95% confidence interval at 13.6 was within the range of the non-inferiority margin (11.7–13.7 points), training FAST in IVR for a median of 117 min was found non-inferior to training at a 90 min SBS course. No significant correlation was found between time spent in IVR and test scores. Within the limitations of the use of a historical control group, the results suggest that IVR could be an alternative to SBS FAST training and suitable for unsupervised, out-of-hospital courses in basic FAST competencies. [ABSTRACT FROM AUTHOR]

• Training is required for all doctors new to REBOA regardless of existing vascular access skills and simpler catheter systems. • Doctors skilled in vascular access benefit from an initial inter-procedural transfer of skills effect when first learning REBOA. • However, novice doctors perform equally well after a focussed simulation-based training, indicating that either group can learn the technical aspects of inserting REBOA. • A time-fixed simulation-based training program does not ensure technical proficiency of doctors learning REBOA, despite vast experience. • Individual skills assessment should be used to ensure proficiency following training for all doctors regardless of previous experience. Resuscitative endovascular balloon occlusion of the aorta (REBOA) is an emerging and potentially life-saving procedure, necessitating qualified operators in an increasing number of centres. The procedure shares technical elements with other vascular access procedures using the Seldinger technique, which is mastered by doctors not only in endovascular specialties but also in trauma surgery, emergency medicine, and anaesthesiology. We hypothesised that doctors mastering the Seldinger technique (experienced anaesthesiologist) would learn the technical aspects of REBOA with limited training and remain technically superior to doctors unfamiliar with the Seldinger technique (novice residents) given similar training. This was a prospective trial of an educational intervention. Three groups of doctors were enroled: novice residents, experienced anaesthesiologists, and endovascular experts. The novices and the anaesthesiologists completed 2.5 h of simulation-based REBOA training. Their skills were tested before and 8–12 weeks after training using a standardised simulated scenario. The endovascular experts, constituting a reference group, were equivalently tested. All performances were video recorded and rated by three blinded experts using a validated assessment tool for REBOA (REBOA-RATE). Performances were compared between groups and with a previously published pass/fail cutoff. Sixteen novices, 13 board-certified specialists in anaesthesiology, and 13 endovascular experts participated. Before training, the anaesthesiologists outperformed the novices by 30 percentage points of the maximum REBOA-RATE score (56% (SD 14.0) vs 26% (SD 17%), p <0.01). After training, there was no difference in skills between the two groups (78% (SD 11%) vs 78 (SD 14%), p = 0.93). Neither group reached the endovascular experts' skill level (89% (SD 7%), p <0.05). For doctors mastering the Seldinger technique, there was an initial inter-procedural transfer of skills advantage when performing REBOA. However, after identical simulation-based training, novices performed equally well to anaesthesiologists, indicating that vascular access experience is not a prerequisite to learning the technical aspects of REBOA. Both groups would need more training to reach technical proficiency. [ABSTRACT FROM AUTHOR]

Focused lung ultrasound (FLUS) has high diagnostic accuracy in many common conditions seen in a variety of emergency settings. Competencies are essential for diagnostic success and patient safety but can be challenging to acquire in clinical environments. Immersive virtual reality (IVR) offers an interactive risk-free learning environment and is progressing as an educational tool. First, this study explored the educational impact of novice FLUS users participating in a gamified or non-gamified IVR training module in FLUS by comparing test scores using a test with proven validity evidence. Second, the learning effect was assessed by comparing scores of each group with known test scores of novices, intermediates and experienced users in FLUS. A total of 48 participants were included: 24 received gamified and 24 received non-gamified IVR training. No significant difference was found between gamified (mean = 15.5 points) and non-gamified (mean = 15.2 points), indicating that chosen gamification elements for our setup did not affect learning outcome (p = 0.66). The mean scores of both groups did not significantly differ from those of known intermediate users in FLUS (gamified p = 0.63, non-gamified p = 0.24), indicating that both IVR modules could be used as unsupervised out-of-hospital training for novice trainees in FLUS. [ABSTRACT FROM AUTHOR]

Background: Resuscitative endovascular balloon occlusion of the aorta (REBOA) is a potentially life-saving but high-risk emergency procedure in patients with haemorrhagic shock. Lack of physicians with competence in the procedure is a barrier to implementation of REBOA. It is currently unclear how training and assessment of competence should be done.Objectives: To report and evaluate research in training and assessment of competence in REBOA and femoral arterial access with the aim to investigate the effect of simulation-based training in the procedure and to provide suggestions for the future design of training programs and assessment tools.Methods: Following PRISMA guidelines, PubMed, Embase, and Cochrane Library databases were searched for studies on training or assessment of competence in REBOA and femoral arterial access. Bias assessment was done using the Medical Education Research Study Quality Instrument. Evidence level was assessed using GRADE.Results: Sixteen studies were included, six of them published as abstracts. Full-text studies included 189 trainees ranging in experience level from military medics to surgical specialists. Outcome measures were heterogenous; the most used were rater checklists, knowledge testing, and procedure time. All studies confirmed an effect of training of REBOA on procedural competence in a simulation setting but had a high degree of bias. No study developed or used an assessment tool supported by validity evidence and no study investigated mid and long-term outcomes.Conclusion: Simulation-based training of REBOA improves skills, however, the evidence level is very low and data cannot answer important questions on effect size, skill transfer and retention, and optimal course design. To advance research and training programmes, an assessment tool supported by validity evidence with broad applicability is needed. [ABSTRACT FROM AUTHOR]