Search

Background: Scrotal pain is a common presentation to the emergency department requiring prompt investigation and management to identify testicular torsion and increase the opportunity for testicular salvage. Using national data obtained from the NHS Improvement, we aim to identify key suboptimal areas in the acute diagnosis and management of testicular torsion with a view to learning from these and improving outcomes., Methods: A freedom of information request was made to all 134 hospital groups in England and Wales that manage testicular torsion. Serious incidents experienced by these hospitals were analyzed to see whether those with pediatric emergency departments fared better than those without. In addition, a qualitative thematic analysis of the NHS Improvement National Reporting and Learning System database over a 12-year period (2007-2019) was undertaken, to identify common themes associated with orchidectomies and "near-miss" events in the pediatric population., Results: Three hundred four patient safety incidents were returned and divided into 62 orchidectomies and 242 near-miss events. The most common factor in the orchidectomy group was individual factors (83.9%). Misdiagnosis of symptomatic testicle was not significant when comparing ED to specialist surgeons (odds ratio = 1.46, P = 0.3842). Atypical presentation resulting in orchidectomy was significant when comparing ED with specialists (odds ratio = 6, P = 0.0355). Near-miss events, not leading to orchidectomy, are often due to a wider variety of factors. A Mann-Whitney U test showed that there was no statistical significance in incidents when comparing general ED units caring for all ages with pediatric EDs in terms of incidents ( U = 807.5, z = -1.124, P = 0.261)., Conclusions: There is a need for education in ED about atypical presentation of testicular torsion and examination of scrotum in cases of lower abdominal pain. It is also vital that specialist teams are cognizant of the standard operating procedures relevant to scrotal pain and testicular within the organization and the wider clinical network. Finally, there should be a drive for annual collation of data in urology and ED units with regards to testicular torsion as patient safety incidents may be underreported., Competing Interests: Disclosure: The authors declare no conflict of interest., (Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.)

Objectives: To evaluate the economic case for nationwide scale-up of the World Health Organization (WHO) Surgical Safety Checklist using cost-effectiveness and benefit-cost analyses., Background: The Checklist improves surgical outcomes but the economic case for widespread use remains uncertain. For perioperative quality improvement interventions to compete successfully against other worthwhile health and nonhealth interventions for limited government resources they must demonstrate cost-effectiveness and positive societal benefit., Methods: Using data from 3 countries, we estimated the benefits as the total years of life lost (YLL) due to postoperative mortality averted over a 3 year period; converted the benefits to dollar equivalent values using estimates of the economic value of an additional year of life expectancy; estimated total implementation costs; and determined incremental cost-effectiveness ratio (ICER) and benefit-cost ratio (BCR). Costs are reported in international dollars using Word Bank purchasing power parity conversion factors at 2016 price-levels., Results: In Benin, Cameroon, and Madagascar ICERs were: $31, $138, and $118 per additional YLL averted; and BCRs were 62, 29, and 9, respectively. Sensitivity analysis demonstrated that the associated mortality reduction and increased usage due to Checklist scale-up would need to deviate approximately 10-fold from published data to change our main interpretations., Conclusions: According to WHO criteria, Checklist scale-up is considered "very cost-effective" and for every $ 1 spent the potential return on investment is $9 to $62. These results compare favorably with other health and nonhealth interventions and support the economic argument for investing in Checklist scale-up as part of a national strategy for improving surgical outcomes., Competing Interests: NS is the director of the London Safety and Training Solutions Ltd, which offers training in patient safety, implementation solutions and human factors to healthcare organizations. MW is on Board of Directors, Mercy Ships International, which is a nongovernmental organization providing free surgical services and training in coastal sub-Saharan African countries. The views expressed in this publication are those of the authors and not necessarily those of the NIHR, the NHS, the charities, the ESRC or the Department of Health and Social Care. The other authors report no conflicts of interest., (Copyright © 2020 The Author(s). Published by Wolters Kluwer Health, Inc.)

Objectives: To evaluate cost-effectiveness of the WHO Surgical Safety Checklist., Background: The clinical effectiveness of surgical checklists is largely understood. Few studies to-date have evaluated the cost-effectiveness of checklist use., Methods: An economic evaluation was carried out using data from the only available randomized controlled trial of the checklist. Analyses were based on 3702 procedures. Costs considered included checklist implementation costs and length and cost of hospital stay, costs of warming blanket use, blood transfusions and antibiotics used in the operating room, and the cost of clinical time in the operating room - all calculated for each procedure and its associated admission. Nonparametric bootstrapping was used to simulate an empirical distribution of the mean effect of the checklist on total admission costs and the probability of observing a complication-free admission and to quantify sampling uncertainty around mean cost estimates., Results: The overall cost of checklist implementation was calculated to be $900 per 100 admissions. Implementation of the WHO checklist resulted in an additional 5.9 complication-free admissions per 100 admissions and an average of 110 bed-days saved per 100 admissions. Accounting for all costs included in the analysis, for every 100 admissions, use of the WHO checklist was estimated to save $55,899., Conclusions: Implementation of the WHO checklist was a cost-effective strategy for improving surgical safety., Competing Interests: NS is the director of the London Safety and Training Solutions Ltd, which offers training in patient safety, implementation solutions and human factors to healthcare organizations. ASH is member of the European Society of Anesthesiologists’ Patient Safety & Quality Committee (ESA PSQC) as representative for the International Federation of Nurse Anaesthetists (IFNA). The other authors have no conflicts of interest to declare., (Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.)

Objective: Surgical site infection (SSI) prevention remains significant, particularly in the era of antimicrobial resistance. Feedback on practices and outcomes is known to be key to reduce SSI rates and optimize antibiotic usage. However, the optimal method, format and frequency of feedback for surgical teams remains unclear. The objective of the study is to understand how data from surveillance and audit are fed back in routine surgical practice., Methods: A systematic scoping review was conducted, using well-established implementation science frameworks to code the data. Two electronic health-oriented databases (MEDLINE, EMBASE) were searched to September 2019. We included studies that assessed the use of feedback as a strategy either in the prevention and management of SSI and/or in the use of antibiotics perioperatively., Results: We identified 21 studies: 17 focused on SSI rates and outcomes and 10 studies described antimicrobial stewardship for SSI (with some overlap in focus). Several interventions were reported, mostly multimodal with feedback as a component. Feedback was often provided in written format (62%), either individualized (38%) or in group (48%). Only 25% of the studies reported that feedback cascaded down to the frontline perioperative staff. In 65% of the studies, 1 to 5 implementation strategies were used while only 5% of the studies reported to have utilized more than 15 implementation strategies. Among studies reporting antibiotic usage in surgery, most (71%) discussed compliance with surgical antibiotic prophylaxis., Conclusions: Our findings highlight the need to provide feedback to all levels of perioperative care providers involved in patient care. Future research in this area should report implementation parameters in more detail., Competing Interests: NS is the director of London Safety and Training Solutions Ltd and MW is on Board of Directors of Mercy Ships International, which provides free surgical procedures. The other authors report no conflicts of interest., (Copyright © 2021 The Author(s). Published by Wolters Kluwer Health, Inc.)

Objective: The aim of this study was to review and appraise how quality improvement (QI) skills are taught to surgeons and surgical residents., Background: There is a global drive to deliver capacity in undertaking QI within surgical services. However, there are currently no specifications regarding optimal QI content or delivery., Methods: We reviewed QI educational intervention studies targeting surgeons or surgical trainees/residents published until 2017. Primary outcomes included teaching methods and training materials. Secondary outcomes were implementation frameworks and strategies used to deliver QI training successfully., Results: There were 20,590 hits across 10 databases, of which 11,563 were screened following de-duplication. Seventeen studies were included in the final synthesis. Variable QI techniques (eg, combined QI models, process mapping, and "lean" principles) and assessment methods were found. Delivery was more consistent, typically combining didactic teaching blended with QI project delivery. Implementation of QI training was poorly reported and appears supported by collaborative approaches (including building learning collaboratives, and coalitions). Study designs were typically pre-/post-training without controls. Studies generally lacked clarity on the underpinning framework (59%), setting description (59%), content (47%), and conclusions (47%), whereas 88% scored low on psychometrics reporting., Conclusions: The evidence suggests that surgical QI training can focus on any well-established QI technique, provided it is done through a combination of didactic teaching and practical application. True effectiveness and extent of impact of QI training remain unclear, due to methodological weaknesses and inconsistent reporting. Conduct of larger-scale educational QI studies across multiple institutions can advance the field., Competing Interests: The authors report no conflicts of interest., (Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.)

Objectives: To identify the implementation strategies used in World Health Organization Surgical Safety Checklist (SSC) uptake in low- and middle-income countries (LMICs); examine any association of implementation strategies with implementation effectiveness; and to assess the clinical impact., Background: The SSC is associated with improved surgical outcomes but effective implementation strategies are poorly understood., Methods: We searched the Cochrane library, MEDLINE, EMBASE and PsycINFO from June 2008 to February 2019 and included primary studies on SSC use in LMICs. Coprimary objectives were identification of implementation strategies used and evaluation of associations between strategies and implementation effectiveness. To assess the clinical impact of the SSC, we estimated overall pooled relative risks for mortality and morbidity. The study was registered on PROSPERO (CRD42018100034)., Results: We screened 1562 citations and included 47 papers. Median number of discrete implementation strategies used per study was 4 (IQR: 1-14, range 0-28). No strategies were identified in 12 studies. SSC implementation occurred with high penetration (81%, SD 20%) and fidelity (85%, SD 13%), but we did not detect an association between implementation strategies and implementation outcomes. SSC use was associated with a reduction in mortality (RR 0.77; 95% CI 0.67-0.89), all complications (RR 0.56; 95% CI 0.45-0.71) and infectious complications (RR 0.44; 95% CI 0.37-0.52)., Conclusions: The SSC is used with high fidelity and penetration is associated with improved clinical outcomes in LMICs. Implementation appears well supported by a small number of tailored strategies. Further application of implementation science methodology is required among the global surgical community., Competing Interests: The authors report no conflicts of interest., (Copyright © 2020 The Author(s). Published by Wolters Kluwer Health, Inc.)

Background: Surgical safety has advanced rapidly with evidence of improved patient outcomes through structural and process interventions. However, knowledge of how to apply these interventions successfully and sustainably at scale is often lacking. The 2019 Global Ministerial Patient Safety Summit called for a focus on implementation strategies to maintain momentum in patient safety improvements, especially in low- and middle-income settings. This study uses an implementation framework, knowledge to action, to examine a model of nationwide World Health Organization (WHO) Surgical Safety Checklist implementation in Cameroon. Cameroon is a lower-middle-income country, and based on data from high- and low-income countries, we hypothesized that more than 50% of participants would be using the checklist (penetration) in the correct manner (fidelity) 4 months postintervention., Methods: A collaboration of 3 stakeholders (Ministry of Health, academic institution, and nongovernmental organization) used a prospective observational design. Based on knowledge to action, there were 3 phases to the study implementation: problem identification (lack of routine checklist use in Cameroonian hospitals), multifaceted implementation strategy (3-day multidisciplinary training course, coaching, facilitated leadership engagement, and support networks), and outcome evaluation 4 months postintervention. Validated implementation outcomes were assessed. Primary outcomes were checklist use (penetration) and fidelity; secondary outcomes were perioperative teams' reactions, learning and behavior change; and tertiary outcomes were perioperative teams' acceptability of the checklist., Results: Three hundred and fifty-one operating room staff members from 25 hospitals received training. Median time to evaluation was 4.5 months (interquartile range [IQR]: 4.5-5.5, range 3-7); checklist use (penetration) increased from 20% (95% confidence interval [CI], 16-25) to 56% (95% CI, 49-63); fidelity for adherence to 6 basic safety processes was high: verification of patient identification was 91% (95% CI, 87-95); risk assessment for difficult intubation was 79% (95% CI, 73-85): risk assessment for blood loss was 88% (95% CI, 83-93) use of pulse oximetry was 93% (95% CI, 90-97); antibiotic administration was 95% (95% CI, 91-98); surgical counting was 89% (95% CI, 84-93); and fidelity for nontechnical skills measured by the WHO Behaviorally Anchored Rating Scale was 4.5 of 7 (95% CI, 3.5-5.4). Median scores for all secondary outcomes were 10/10, and 7 acceptability measures were consistently more than 70%., Conclusions: This study shows that a multifaceted implementation strategy is associated with successful checklist implementation in a lower-middle-income country such as Cameroon, and suggests that a theoretical framework can be used to practically drive nationwide scale-up of checklist use.

Objective: The aim of the study was to identify and evaluate scholarship in multi-institutional interventional surgical education trials., Summary Background Data: Most research on interventions in surgical education occurs at individual institutions. These studies typically involve a small number of learners in a unique environment, thereby limiting their generalizability. The status of multi-institutional studies in surgical education remains unknown., Methods: We searched the Pubmed, ERIC, PsycINFO, SCOPUS, and CINAHL databases for all English language articles published from January 1, 2000 to December 31, 2015 using the keywords "medical education," "surgical education," "multi-institutional," "multi-center," and related terms. Articles published in an English language peer-reviewed journal that described an educational intervention conducted at more than one institution and involving surgeons were included., Results: Of 3511 identified articles, 53 met criteria for full-text review and inclusion in this review. The median number of institutional sites was 4, with a range of 2 to 54. The 2 most common areas of focus were technical skills (43% of studies) and clinical knowledge (32% of studies). These were also the 2 most commonly measured outcomes (technical skills 32% of studies, clinical knowledge 21% of studies). Thirteen percentage of studies measured only learner attitudes and perceptions rather than learning outcomes., Conclusions: Multi-institutional surgical education studies do not uniformly incorporate characteristics of high quality research, particularly related to study design, measurable outcomes, and assessment tools used. Coordinated support, including grant funding, that addresses the challenging nature of multi-institutional surgical education research may improve the quality of these studies.

The incidence of surgical complications has remained largely unchanged over the past two decades. Inherent complexity in surgery, new technology possibilities, increasing age and comorbidity in patients may contribute to this. Surgical safety checklists may be used as some of the tools to prevent such complications. Use of checklists may reduce critical workload by eliminating issues that are already controlled for. The global introduction of the World Health Organization Surgical Safety Checklist aimed to improve safety in both anesthesia and surgery and to reduce complications and mortality by better teamwork, communication, and consistency of care. This review describes a literature synthesis on advantages and disadvantages in use of surgical safety checklists emphasizing checklist development, implementation, and possible clinical effects and using a theoretical framework for quality of provided healthcare (structure-process-outcome) to understand the checklists' possible impact on patient safety.

Objective: We hypothesize that high-quality implementation of the World Health Organization's Surgical Safety Checklist (SSC) will lead to improved care processes and subsequently reduction of peri- and postoperative complications., Background: Implementation of the SSC was associated with robust reduction in morbidity and length of in-hospital stay in a stepped wedge cluster randomized controlled trial conducted in 2 Norwegian hospitals. Further investigation of precisely how the SSC improves care processes and subsequently patient outcomes is needed to understand the causal mechanisms of improvement., Methods: Care process metrics are reported from one of our earlier trial hospitals. Primary outcomes were in-hospital complications and care process metrics, e.g., patient warming and antibiotics. Secondary outcome was quality of SSC implementation. Analyses include Pearson's exact χ test and binary logistic regression., Results: A total of 3702 procedures (1398 control vs. 2304 intervention procedures) were analyzed. High-quality SSC implementation (all 3 checklist parts) improved processes and outcomes of care. Use of forced air warming blankets increased from 35.3% to 42.4% (P < 0.001). Antibiotic administration postincision decreased from 12.5% to 9.8%, antibiotic administration preincision increased from 54.5% to 63.1%, and nonadministration of antibiotics decreased from 33.0% to 27.1%. Surgical infections decreased from 7.4% (104/1398) to 3.6% (P < 0.001). Adjusted SSC effect on surgical infections resulted in an odds ratio (OR) of 0.52 (95% confidence interval (CI): 0.38-0.72) for intervention procedures, 0.54 (95% CI: 0.37-0.79) for antibiotics provided before incision, and 0.24 (95% CI: 0.11-0.52) when using forced air warming blankets. Blood transfusion costs were reduced by 40% with the use of the SSC., Conclusions: When implemented well, the SSC improved operating room care processes; subsequently, high-quality SSC implementation and improved care processes led to better patient outcomes.

In the UK, treatment recommendations for patients with cancer are routinely made by multidisciplinary teams in weekly meetings. However, their performance is variable.The aim of this study was to explore the underlying structure of multidisciplinary decision-making process, and examine how it relates to team ability to reach a decision.This is a cross-sectional observational study consisting of 1045 patient reviews across 4 multidisciplinary cancer teams from teaching and community hospitals in London, UK, from 2010 to 2014. Meetings were chaired by surgeons.We used a validated observational instrument (Metric for the Observation of Decision-making in Cancer Multidisciplinary Meetings) consisting of 13 items to assess the decision-making process of each patient discussion. Rated on a 5-point scale, the items measured quality of presented patient information, and contributions to review by individual disciplines. A dichotomous outcome (yes/no) measured team ability to reach a decision. Ratings were submitted to Exploratory Factor Analysis and regression analysis.The exploratory factor analysis produced 4 factors, labeled "Holistic and Clinical inputs" (patient views, psychosocial aspects, patient history, comorbidities, oncologists', nurses', and surgeons' inputs), "Radiology" (radiology results, radiologists' inputs), "Pathology" (pathology results, pathologists' inputs), and "Meeting Management" (meeting chairs' and coordinators' inputs). A negative cross-loading was observed from surgeons' input on the fourth factor with a follow-up analysis showing negative correlation (r = -0.19, P < 0.001). In logistic regression, all 4 factors predicted team ability to reach a decision (P < 0.001).Hawthorne effect is the main limitation of the study.The decision-making process in cancer meetings is driven by 4 underlying factors representing the complete patient profile and contributions to case review by all core disciplines. Evidence of dual-task interference was observed in relation to the meeting chairs' input and their corresponding surgical input into case reviews., Competing Interests: The authors report no conflicts of interest.

Objective: To evaluate impact of WHO checklist compliance on risk-adjusted clinical outcomes, including the influence of checklist components (Sign-in, Time-out, Sign-out) on outcomes., Background: There remain unanswered questions surrounding surgical checklists as a quality and safety tool, such as the impact in cases of differing complexity and the extent of checklist implementation., Methods: Data were collected from surgical admissions (6714 patients) from March 2010 to June 2011 at 5 academic and community hospitals. The primary endpoint was any complication, including mortality, occurring before hospital discharge. Checklist usage was recorded as checklist completed in full/partly. Multilevel modeling was performed to investigate the association between complications/mortality and checklist completion., Results: Significant variability in checklist usage was found: although at least 1 of the 3 components was completed in 96.7% of cases, the entire checklist was only completed in 62.1% of cases. Checklist completion did not affect mortality reduction, but significantly lowered risk of postoperative complication (16.9% vs. 11.2%), and was largely noticed when all 3 components of the checklist had been completed (odds ratio = 0.57, 95% confidence interval: 0.37-0.87, P < 0.01). Calculated population-attributable fractions showed that 14% (95% confidence interval: 7%-21%) of the complications could be prevented if full completion of the checklist was implemented., Conclusions: Checklist implementation was associated with reduced case-mix-adjusted complications after surgery and was most significant when all 3 components of the checklist were completed. Full, as opposed to partial, checklist completion provides a health policy opportunity to improve checklist impact on surgical safety and quality of care.

Background: To date, there is a paucity of theory-driven research on the likely determinants of patient involvement in safety-relevant behaviors. In particular, very little work has focused on predictors of patient behaviors that do not involve direct interactions with health-care staff., Objective: To examine predictors of patients' intentions to engage in 2 safety behaviors: (1) reporting an error to a national reporting system and (2) bringing medicines into hospital., Design: Cross-sectional survey study., Participants: Eighty medical and surgical hospital inpatients aged 18 to 80 years (mean, 48 years) from one inner city London teaching hospital., Method: Survey that measured the utility of constructs of the health belief model and theory of planned behavior in predicting patients' intentions to report an error to a national reporting system or to bring their medications into hospital. Data were analyzed using multiple regression analysis., Results: Control beliefs were the strongest predictors of patients' intentions. Normative beliefs were also a strong predictor of intentions to report an error to a national reporting system. The regression model accounted for smaller percentage of the variance in patients' intentions to bring medications into hospital than to report an error to a national reporting system (37% and 48%, respectively)., Conclusions: Interventions aimed at encouraging the participation of patients in promoting their own safety should consider the extent to which patients feel in control and capable of performing the behavior in question; this will help support patients to work with health-care professionals in ensuring safe care.

Objective: To investigate whether distractions in the operating room (OR) are associated with higher mental workload and stress, and poorer teamwork among OR personnel., Background: Engaging in multiple tasks can affect performance. There is little research on the effect of distractions on surgical team members' behavior and cognitive processes., Methods: Ninety general surgery cases were observed in real time. Cases were assessed by a surgeon and a behavioral scientist using 4 validated tools: OR Distractions Assessment Form, the Observational Teamwork Assessment for Surgery tool, NASA-Task Load Index, and short form of the State Trait Anxiety Inventory. Analysis of variance was performed to evaluate significant differences between teamwork, workload, and stress level among team members. Correlations (Pearson r) were computed to evaluate associations between variables., Results: The most prevalent distractions were those initiated by external staff, followed by case-irrelevant conversations. Case-irrelevant conversations were associated with poorer team performance. Irrelevant conversations initiated by surgeons were associated with lower teamwork in surgeons (across team skills: r = -0.44 to -0.58, P < 0.05 to 0.01) and anesthesiologists (r = -0.38 and r = -0.40, for coordination and leadership; P < 0.05). Equipment-related distractions correlated with higher stress (r = 0.48, P < 0.05) and lower teamwork (across team skills: r = -0.42 to -0.50, P < 0.05) in nurses. Acoustic distractions correlated with higher stress in surgeons (r = 0.32, P < 0.05) and higher workload in anesthesiologists (r = 0.30, P < 0.05)., Conclusions: Although some distractions may be inevitable in the OR, they can also be detrimental to the team. A deeper understanding of the effect of distractions on teams and their outcomes can lead to targeted quality improvement.

Objectives: We hypothesized reduction of 30 days' in-hospital morbidity, mortality, and length of stay postimplementation of the World Health Organization's Surgical Safety Checklist (SSC)., Background: Reductions of morbidity and mortality have been reported after SSC implementation in pre-/postdesigned studies without controls. Here, we report a randomized controlled trial of the SSC., Methods: A stepped wedge cluster randomized controlled trial was conducted in 2 hospitals. We examined effects on in-hospital complications registered by International Classification of Diseases, Tenth Revision codes, length of stay, and mortality. The SSC intervention was sequentially rolled out in a random order until all 5 clusters-cardiothoracic, neurosurgery, orthopedic, general, and urologic surgery had received the Checklist. Data were prospectively recorded in control and intervention stages during a 10-month period in 2009-2010., Results: A total of 2212 control procedures were compared with 2263 SCC procedures. The complication rates decreased from 19.9% to 11.5% (P < 0.001), with absolute risk reduction 8.4 (95% confidence interval, 6.3-10.5) from the control to the SSC stages. Adjusted for possible confounding factors, the SSC effect on complications remained significant with odds ratio 1.95 (95% confidence interval, 1.59-2.40). Mean length of stay decreased by 0.8 days with SCC utilization (95% confidence interval, 0.11-1.43). In-hospital mortality decreased significantly from 1.9% to 0.2% in 1 of the 2 hospitals post-SSC implementation, but the overall reduction (1.6%-1.0%) across hospitals was not significant., Conclusions: Implementation of the WHO SSC was associated with robust reduction in morbidity and length of in-hospital stay and some reduction in mortality.

Objective: To review the current state of simulation use in surgery and to offer direction for future research and implementation of evidence-based findings., Background: Simulation-based training (SBT) in surgery has surged in recent years. Although several new simulators and curricula have become available, their optimization and implementation into surgical training has been lagging., Methods: Members of the Association for Surgical Education Simulation Committee with expertise in surgical simulation review and interpret the literature and describe the current status of the use of simulation in surgery, identify the challenges to its widespread adoption, and offer potential solutions to these challenges. The review focuses on simulation research and implementation of existing knowledge and explores possible future directions for the field., Results: Skill acquired on simulators has repeatedly and consistently been demonstrated to transfer to the operating room, and proficiency-based training maximizes this benefit. Several simulation-based curricula have been developed by national organizations to support resident training, but their implementation is lagging because of inadequate human resources, difficult integration of SBT into educational strategy, and logistical barriers. In research, lack of coordinated effort, flaws in study design, changes in simulator-validation concepts, limited attention to skill retention, and other areas are in need of improvement., Conclusions: Future research in surgical simulation should focus on demonstrating the cost-effectiveness of SBT and its impact on patient outcomes. Furthermore, to enable the more widespread incorporation of best practices and existing simulation curricula in surgery, effective implementation strategies need to be developed.

Objective: To establish the efficacy of simulation-based training for improving residents' management of postoperative complications on a surgical ward., Background: Effective postoperative care is a crucial determinant of patient outcome, yet trainees learn this through the Halstedian approach. Little evidence exists on the efficacy of simulation in this safety-critical environment., Methods: A pre-/postintervention design was employed with 185 residents from 5 hospitals. Residents participated in 2 simulated ward-based scenarios consisting of a deteriorating postoperative patient. A debriefing intervention was implemented between scenarios. Resident performance was evaluated by calibrated, blinded assessors using the validated Global Assessment Toolkit for Ward Care. This included an assessment of clinical skills (checklist of 35 tasks), team-working skills (score range 1-6 per skill), and physician-patient interaction skills., Results: Excellent interrater reliability was achieved in all assessments (reliability 0.89-0.99, P < 0.001). Clinically, improvements were obtained posttraining in residents' ability to recognize/respond to falling saturations (pre = 73.7% vs post = 94.8%, P < 0.01), check circulatory status (pre = 21.1% vs post = 84.2% P < 0.001), continuously reassess patient (pre = 42.1% vs post = 100%, P < 0.001), and call for help (pre = 36.8% vs post = 89.8%, P < 0.001). Regarding teamwork, there was a significant improvement in residents' communication (pre = 1.75 vs post = 3.43), leadership (pre = 2.43 vs post = 4.20), and decision-making skills (pre = 2.20 vs post = 3.81, P < 0.001). Finally, residents improved in all elements of interaction with patients: empathy, organization, and verbal and nonverbal expression (Ps < 0.001)., Conclusions: The study provides evidence for the efficacy of ward-based team training using simulation. Such exercises should be formally incorporated into training curricula to enhance patient safety in the high-risk surgical ward environment.

Objectives: To evaluate how the World Health Organization (WHO) surgical safety checklist was implemented across hospitals in England; to identify barriers and facilitators toward implementation; and to draw out lessons for implementing improvement initiatives in surgery/health care more generally., Background: The WHO checklist has been linked to improved surgical outcomes and teamwork, yet we know little about the factors affecting its successful uptake., Methods: A longitudinal interview study with operating room personnel was conducted across a representative sample of 10 hospitals in England between March 2010 and March 2011. Interviews were audio recorded over the phone. Interviewees were asked about their experience of how the checklist was introduced and the factors that hindered or aided this process. Transcripts were submitted to thematic analysis., Results: A total of 119 interviews were completed. Checklist implementation varied greatly between and within hospitals, ranging from preplanned/phased approaches to the checklist simply "appearing" in operating rooms, or staff feeling it had been imposed. Most barriers to implementation were specific to the checklist itself (eg, perceived design issues) but also included problematic integration into preexisting processes. The most common barrier was resistance from senior clinicians. The facilitators revealed some positive steps that can been taken to prevent/address these barriers, for example, modifying the checklist, providing education/training, feeding-back local data, fostering strong leadership (particularly at attending level), and instilling accountability., Conclusions: We identified common themes that have aided or hindered the introduction of the WHO checklist in England and have translated these into recommendations to guide the implementation of improvement initiatives in surgery and wider health care systems.

Objective: To evaluate the efficacy of an entire hospital simulation in imparting skills to expert healthcare providers, encompassing both retention and transfer to clinical practice., Background: Studies demonstrating the effectiveness of simulation do not concentrate upon expert multidisciplinary teams. Moreover, their focus is confined to a single clinical setting, thereby not considering the complex interactions across multiple hospital departments., Methods: A total of 288 participants (Attending surgeons, anesthesiologists, physicians, and nurses) completed this largest simulation study to date, set in the UK Defence Medical Services' Hospital Simulator and the conflict zone in Afghanistan. The simulator termed "Hospital Exercise" (HOSPEX) is a fully immersive live-in simulation experience that covers the entire environment of a military hospital with all departments. Participants undertook a 3-day training program within HOSPEX before deployment to war zones. Primary outcome measures were assessed with IMPAcT (the Imperial Military Personnel Assessment Tool). IMPAcT measures crisis management, trauma care, hospital environment, operational readiness, and transfer of skills to civilian practice. Reliability, skills learning, and retention in the conflict zone were assessed statistically., Results: Reliability in skills assessment was excellent (Cronbach α: nontechnical skills = 0.87-0.94; environment/patient skills = 0.83-0.95). Pre/post-HOSPEX comparisons revealed significant improvements in decision making (M = 4.98, SD = 1.20 to M = 5.39, SD = 0.91; P = 0.03), situational awareness (M = 5.44, SD = 1.04 to M = 5.74, SD = 0.92; P = 0.01), trauma care (M = 5.53, SD = 1.23 to M = 5.85, SD = 1.09; P = 0.05), and knowledge of hospital environment (M = 5.19, SD = 1.17 to M = 5.42, SD = 0.97; P = 0.04). No skills decayed over time when assessed several months later in the real conflict zone. All skills transferred to civilian clinical practice., Conclusions: This is the first study to describe the value of a full-hospital simulation across the entire patient pathway. Such macrosimulations may be the way forward for integrating the complex training needs of expert clinicians and testing organizational "fitness for purpose" of entire hospitals.

Objective: To develop a toolkit that covers the clinical, nontechnical, and empathic skills required for effective, safe surgical ward care., Background: Despite the explosion of interest in patient safety, little attention has been placed on the skill set required for safe and effective surgical ward care. Currently, there is a lack of a systematic approach to improving ward care via assessing and improving residents' ward care skills., Methods: A comprehensive evidence-based and expert-derived toolkit was developed, including a novel clinical checklist for ward care (Clinical Skills Assessment for Ward Care: C-SAW-C); a novel team assessment scale for wards rounds (Teamwork Skills Assessment for Ward Care: T-SAW-C); and a revised version of a physician-patient interaction scale (Physician-Patient Interaction Global Rating Scale: PP-GIS). Interassessor reliability (κ, intraclass correlation), internal consistency (Cronbach α), and convergent validity (Pearson r correlations) were evaluated statistically in 38 simulated scenarios (during which a patient rapidly deteriorated) involving 185 residents., Results: Excellent interassessor reliability was obtained for C-SAW-C [median κ = 0.89; median intraclass correlation coefficient (ICC) = 0.94], T-SAW-C (median ICC = 0.99), and the revised PP-GIs (κ = 1.00; ICC = 0.98 or higher). Internal consistency was also very high for all team skills assessed by T-SAW-C (Cronbach α range 0.87-0.94 across 6 skills) and the revised PP-GIS (Cronbach α = 0.96)-all P's < 0.001. Significant positive correlations were obtained between the 3 assessments (r = 0.73-0.92, P < 0.001) thus showing evidence for convergent validity., Conclusions: We developed a toolkit that captures comprehensively the skills that are required for safe and effective ward care, including the high-risk situation where a patient decompensates. The toolkit offers a systematic evaluation of the quality and safety of surgical ward care and can be used to train and debrief residents' skills and performance.

Objectives: The aim of this systematic review was to assess the impact of surgical safety checklists on the quality of teamwork and communication in the operating room (OR)., Background: Safety checklists have been shown to impact positively on patient morbidity and mortality following surgery, but it is unclear whether this clinical improvement is related to an improvement in OR teamwork and communication., Methods: A systematic search strategy of MEDLINE, EMBASE, PsycINFO, Google Scholar, and the Cochrane Database for Systematic Reviews was undertaken to obtain relevant articles. After de-duplication and the addition of limits, 315 articles were screened for inclusion by 2 researchers and all articles meeting a set of prespecified inclusion criteria were retained. Information regarding the type of checklist, study design, assessment tools used, outcomes, and study limitations was extracted., Results: Twenty articles formed the basis of this systematic review. All articles described an empirical study relating to a case-specific safety checklist for surgery as the primary intervention, with some measure of change/improvement in teamwork and/or communication relating to its use. The methods for assessing teamwork and communication varied greatly, including surveys, observations, interviews, and 360° assessments. The evidence suggests that safety checklists improve the perceived quality of OR teamwork and communication and reduce observable errors relating to poor team skills. This is likely to function through establishing an open platform for communication at the start of a procedure: encouraging the sharing of critical case-related information, promoting team coordination and decision making, flagging knowledge gaps, and enhancing team cohesion. However, the evidence would also suggest that when used suboptimally or when individuals have not bought in to the process, checklists may conversely have a negative impact on the function of the team., Conclusions: Safety checklists are beneficial for OR teamwork and communication and this may be one mechanism through which patient outcomes are improved. Future research should aim to further elucidate the relationship between how safety checklists are used and team skills in the OR using more consistent methodological approaches and utilizing validated measures of teamwork such that best practice guidelines can be established.

Objectives: To explore the current status of performance feedback (debriefing) in the operating room and to develop and evaluate an evidence-based, user-informed intervention termed "SHARP" to improve debriefing in surgery., Background: Effective debriefing is a key educational technique for optimizing learning in surgical settings. However, there is a lack of a debriefing culture within surgery. Few studies have prospectively evaluated educational interventions to improve the quality and quantity of performance feedback in surgery., Methods: This was a prospective pre- and post-study of 100 cases involving 22 trainers (attendings) and 30 surgical residents (postgraduate years 3-8). A trained researcher assessed the quality of debriefings provided to the trainee using the validated Objective Structured Assessment of Debriefing (OSAD) tool alongside ethnographic observation. Following the first 50 cases, an educational intervention termed "SHARP" was introduced and measures repeated for a further 50 cases. User satisfaction with SHARP was assessed via questionnaire. Twenty percent of the cases were observed independently by a second researcher to test interrater reliability., Results: Interrater reliability for OSAD was excellent (ICC = 0.994). Objective scores of debriefing (OSAD) improved significantly after the SHARP intervention: median pre = 19 (range, 8-31); median post = 33 (range, 26-40), P < 0.001. Strong correlations between observer (OSAD) and trainee rating of debriefing were obtained (median ρ = 0.566, P < 0.01). Ethnographic observations also supported a significant improvement in both quality and style of debriefings. Users reported high levels of satisfaction in terms of usefulness, feasibility, and comprehensiveness of the SHARP tool., Conclusions: SHARP is an effective and efficient means of improving performance feedback in the operating room. Its routine use should be promoted to optimize workplace-based learning and foster a positive culture of debriefing and performance improvement within surgery.

Objective: To develop guidelines for a faculty training program in nontechnical skill assessment in surgery., Background: Nontechnical skills in the operating room are critical for patient safety. The successful integration of these skills into workplace-based assessment is dependent upon the availability of faculty who are able to teach and assess them. At present, no guidelines exist regarding the training requirements for such faculty in surgical contexts., Methods: The development of the guidelines was carried out in several stages: stage 1-a detailed literature review on current training for nontechnical skill assessors; stage 2-semistructured interviews with a multidisciplinary panel (consisting of clinicians and psychologists/human factors specialists) of experts in surgical nontechnical skills; and stage 3-interview findings fed into an Expert Consensus Panel (ECP) Delphi approach to establish consensus regarding training requirements for faculty assessing nontechnical skills in surgery., Results: The ECP agreed that training in nontechnical skill assessment should be delivered by a multidisciplinary team consisting of clinicians and psychologists/human factors specialists. The ECP reached consensus regarding who should be targeted to be trained as faculty (including proficiency and revalidation requirements). Consensus was reached on 7 essential training program content elements (including training in providing feedback/debriefing) and 8 essential methods of evaluating the effectiveness of a "train-the-trainers" program., Conclusions: This study provides evidence-based guidelines that can be used to guide the development and evaluation of programs to educate faculty in the training and assessment of nontechnical skills. Uptake of these guidelines could accelerate the development of surgical expertise required for safe and high-quality patient care.

Background: Simultaneous task performance ("multitasking") is common in hospital physicians' work and is implicated as a major determinant for enhanced strain and detrimental performance., Objective: The aim was to determine the impact of multitasking by hospital physicians on their self reported strain and performance., Method: A prospective observational time-and-motion study in a Community Hospital was conducted. Twenty-seven hospital physicians (surgical and internal specialties) were observed in 40 full-shift observations. Observed physicians reported twice on their self-monitored strain and performance during the observation time. Associations of observed multitasking events and subsequent strain and performance appraisals were calculated., Results: About 21% of the working time physicians were engaged in simultaneous activities. The average time spent in multitasking activities correlated significantly with subsequently reported strain (r = 0.27, P = 0.018). The number of instances of multitasking activities correlated with self-monitored performance to a marginally significant level (r = 0.19, P = 0.098)., Conclusion: Physicians who engage in multitasking activities tend to self-report better performance but at the cost of enhanced psychophysical strain. Hence, physicians do not perceive their own multitasking activities as a source for deficient performance, for example, medical errors. Readjustment of workload, improved organization of work for hospital physicians, and training programs to improve physicians' skills in dealing with multiple clinical demands, prioritization, and efficient task allocation may be useful avenues to explore to reduce the potentially negative impact of simultaneous task performance in clinical settings.

Background: Despite increasing recognition that patients could play an important role in promoting the safety of their care, little is known on this issue regarding health-care professionals' (HCPs') attitudes toward patient involvement., Objectives: To investigate physicians' and nurses' attitudes toward patient involvement in safety-related behaviors, both through their eyes as a health-care professional and as a potential patient., Design: Cross-sectional exploratory study using 2 surveys. Survey 1 addressed HCPs' attitudes toward supporting patient participation in safety-related behaviors. Survey 2 addressed HCPs' reported willingness to participate in safety-related behaviors (as a patient)., Participants: Eighty health-care professionals (40 physicians and 40 nurses) from an inner city London teaching hospital., Findings: Attitudes were affected by the type of behavior, who the HCP is interacting with, and the participants own professional role. Overall, both professions held positive attitudes toward patient involvement, although in general, nurses versus physicians were more willing to both support patient involvement and participate themselves as a patient., Conclusion: Compared with other research on "lay" patients' attitudes, our data suggest that when HCPs are patients in hospital, they may be more willing to participate in safety-related behaviors. Promisingly, our data also suggest HCPs are willing to support patient involvement in safety-related behaviors, which may suggest they are happy to participate in interventions aimed at encouraging patient involvement in this area. Further in-depth research is needed to investigate the roles that HCPs (as both a patient and HCP) believe are appropriate for patients to participate in, under what circumstances and why.

Objective: : To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings., Introduction: : Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing-yet there is a paucity of research to guide best practice., Methods: : Phase 1-Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2-Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations., Results: : Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test-retest ICC = 0.90], and face and content valid (content validity index = 0.94)., Conclusions: : OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

Objectives: To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool., Background: Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated., Methods: Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session., Results: All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert., Conclusions: It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds.

Background: Patients can make valuable contributions to their health care safety. Little is known, however, about the factors that could affect patient participation in safety-related aspects of their health care management. Examining and understanding how patient involvement in safety-related behaviors can be conceptualized will allow greater insight into why patients may be more willing to participate in some behaviors more than others may., Objective: This study aimed to develop a new approach for understanding and conceptualizing patient involvement in safety with specific reference to a surgical patient cohort., Methods: The authors conducted a review of the key opportunities for patient involvement along the surgical care trajectory and examination and identification of the properties and characteristics of different safety-related behaviors and the barriers to patient involvement they entail., Results: Safety-related behaviors comprise 3 main properties including the type of error the behavior is trying to prevent (e.g., medication error), the action required by the patient (e.g., asking questions), and the characteristics of the action (e.g., whether the behavior involves interacting with a health care professional). Barriers to patient involvement that relate to patients and health care professionals can be broadly categorized as interpersonal, intrapersonal, and cultural., Conclusions: We believe that thinking of patient involvement in safety relating to properties and characteristics of the behavior together with the barriers to involvement could aid the design, implementation, and evaluation of interventions aimed at encouraging patient participation. It will also enable a greater understanding and assessment of not only what interventions may be effective (at encouraging patient involvement) but when they might be effective (i.e., what stage of the care pathway) and why.

Objectives: To explore face, content and construct validity of Distributed Simulation (DS), an innovative approach to low-cost, high-fidelity surgical simulation and compare technical performance in the DS with that on a standard surgical box trainer., Background: Immersive simulation is widely accepted as an important modality for surgical education. However, access and cost limit the uptake of full-scale simulation training. DS is a portable, simulated clinical environment aiming at widening access to immersive simulation., Methods: Ten novice and 10 expert surgeons performed a laparoscopic cholecystectomy on a porcine model in the DS and on a box trainer. Face and content validity were measured using 6-point Likert-type questionnaires. Construct validity was rated using the Objective Structured Assessment of Technical Skills (OSATS). Comparison of technical performance between DS and box trainer was measured using the Wilcoxon test., Results: Face validity was rated as 5.1 (SD = 0.54) by novices and 4.8 (SD = 0.64) by experts.Content validity was rated as 5.2 (SD = 0.40) by novices and 5.1 (SD = 0.56) by experts. Experts performed significantly better than novices in the DS (16.3 vs. 27.3, P < 0.001) demonstrating construct validity. Experts' technical performance did not differ between DS and box trainer (26.5 vs. 27.3, P > 0.84) whereas novices performed better on the box trainer (19.8 vs. 16.3, P < 0.01). Qualitative data analysis demonstrated that surgeons felt able to behave as if in a real operating room (OR)., Conclusion: DS offers a valid, low-cost, accessible environment for training and assessing surgeons. This approach has major implications for surgical education and for the widespread implementation of immersive simulation at a time of increasing financial austerity.

Objective: To assess the feasibility, validity, and reliability of a postoperative Handover Assessment Tool (PoHAT) and to evaluate the current practices of the postoperative handover at 2 large European hospitals., Background: Postoperative handover is one of the most critical phases in the care of a patient undergoing surgery. However, handovers are largely informal and variable. A thorough understanding of the problem is necessary before safety solutions can be considered., Methods: Postoperative Handover Assessment Tool (PoHAT) was developed through task analysis, semistructured interviews, literature review, and learned society guidelines. Subsequent validation was done by the Delphi technique. Feasibility and reliability were then evaluated by direct observation of handovers at 2 large European hospitals. Outcomes measures included information omissions, task errors, teamwork evaluation, duration of handover, and number of distractions., Results: The tool was feasible to use and inter-rater reliability was excellent (r = 0.96, P < 0.001). Evaluation of handover at the 2 study sites revealed a median of 8 information omissions per handover at both the centers (IQR 7-10). There were a median of 3 task errors per handover (IQR 2-4). Thirty-five percent of handovers had distractions, which included competing demands for nurse attention, bleeps, and case-irrelevant communication., Conclusion: This study has established the feasibility, validity, and reliability of a tool for evaluating postoperative handover. In addition to serving as an objective measure of postoperative handover, the tool can also be used to evaluate the efficacy of any intervention developed to improve this process. The study has also shown that postoperative handover is characterized by incomplete transfer of information and failures in the performance of key tasks.

Objective: To assess the effects of mental practice on surgical performance., Background: Increasing concerns for patient safety have highlighted a need for alternative training strategies outside the operating room. Mental practice (MP), "the cognitive rehearsal of a task before performance," has been successful in sport and music to enhance skill. This study investigates whether MP enhances performance in laparoscopic surgery., Methods: After baseline skills testing, 20 novice surgeons underwent training on an evidence-based virtual reality curriculum. After randomization using the closed envelope technique, all participants performed 5 Virtual Reality (VR) laparoscopic cholecystectomies (LC). Mental practice participants performed 30 minutes of MP before each LC; control participants viewed an online lecture. Technical performance was assessed using video Objective Structured Assessment of Technical Skills-based global ratings scale (scored from 7 to 35). Mental imagery was assessed using a previously validated Mental Imagery Questionnaire., Results: Eighteen participants completed the study. There were no intergroup differences in baseline technical ability. Learning curves were demonstrated for both MP and control groups. Mental practice was superior to control (global ratings) for the first LC (median 20 vs 15, P = 0.005), second LC (20.5 vs 13.5, P = 0.001), third LC (24 vs 15.5, P < 0.001), fourth LC (25.5 vs 15.5, P < 0.001) and the fifth LC (27.5 vs 19.5, P = 0.00). The imagery for the MP group was also significantly superior to the control group across all sessions (P < 0.05). Improved imagery significantly correlated with better quality of performance (ρ 0.51–0.62, Ps < 0.05)., Conclusions: This is the first randomized controlled study to show that MP enhances the quality of performance based on VR laparoscopic cholecystectomy. This may be a time- and cost-effective strategy to augment traditional training in the OR thus potentially improving patient care.

Objectives: We conducted a systematic review of published literature to gain a better understanding of interprofessional information transfer and communication (ITC) in hospital setting in the field of surgical and anesthetic care., Background: Communication breakdowns are a common cause of surgical errors and adverse events., Data Sources: Medline, Embase, PsycINFO, Cochrane Database of Systematic Reviews, and hand search of articles bibliography., Study Selection: Of the 4027 citations identified through the initial electronic search and screened for possible inclusion, 110 articles were retained following title and abstract reviews. Of these, 38 were accepted for this review., Data Extraction: Data were extracted from the studies about objectives, clinical domain, methodology including study design, sample population, tools for assessing communication, results, and limitations., Results: Information transfer failures are common in surgical care and are distributed across the continuum of care. They not only lead to errors in care provision but also lead to patient harm. Most of the articles have focused on ITC process in different phases especially in operating room. None of the studies have looked at whole of the surgical care process. No standard tool has been developed to capture the ITC process in different teams and to evaluate the effect of various communication interventions. Uses of standardized communication through checklist, proformas, and technology innovations have improved the ITC process, with an effect on clinical and patient outcomes., Conclusions: ITC deficits adversely affect patient care. There is a need for standard measures to evaluate this process. Effective and standardized communication among healthcare professionals during the perioperative process facilitates surgical safety.

Objective: To test the construct validity of the Observational Teamwork Assessment for Surgery (OTAS) tool., Summary Background Data: Poor teamwork in surgical teams has been implicated in adverse events to patients. The OTAS is a tool that assesses teamwork in real time for the entire surgical team. Existing empirical research on OTAS has yet to explore how expert versus novice tool users use the tool to assess teamwork in the operating room., Methods: Data were collected in 12 elective procedures by an expert/expert (N = 6) and an expert/novice (N = 6) pair of raters. Five teamwork behaviors (communication, coordination, leadership, monitoring, and cooperation) were scored via observation pre, intra, and postoperatively by blind raters., Results: Significant and sizeable correlations were obtained in 12 of 15 behaviors in the expert/expert pair, but only in 3 of 15 behaviors in the expert/novice pair. Significant differences in mean scores were obtained in 3 of 15 behaviors in the expert/expert pair, but in 11 of 15 behaviors in the expert/novice pair. Total OTAS scores exhibited strong correlations and no significant differences in ratings in the expert/expert pair. In the expert/novice pair no correlations were obtained and there were significant differences in mean scores. The overall size of inconsistency in the scoring was 2% for expert/expert versus 15% for expert/novice., Conclusions: OTAS exhibits adequate construct validity as assessed by consistency in the scoring by expert versus novices-ie, expert raters produce significantly more consistent scoring than novice raters. Further validation should assess the learning curve for novices in OTAS. Relationships between OTAS, measures of technical skill, and behavioral responses to surgical crises should also be quantified.