Search

Additional file 1. This additional file contains the following sequence of instructions received by each rater: 1- Explanatory email, 2- Information Table Subset*1, 3- Rating Instrument Subset*1*2, 4- Additional Information Table Subset* *1 – the table subset for each article varied according to the subset to which each rater was allocated; *2 each rater received their own rating instrument with entries relating to the corresponding article subsets.

The Job Demands-Resources model (JD-R model) was used as the theoretical basis of a tailormade questionnaire to measure the psychosocial work environment and personal resources of recently graduated veterinary professionals. According to the JD-R model, two broad categories of work characteristics that determine employee wellbeing can be distinguished: job demands and job resources. Recently, the JD-R model has been expanded by integrating personal resource measures into the model. Three semistructured group interviews with veterinarians active in different work domains were conducted to identify relevant job demands, job resources and personal resources. These demands and resources were organised in themes (constructs). For measurement purposes, a set of questions ('a priori scale') was selected from the literature for each theme. The full set of a priori scales was included in a questionnaire that was administered to 1760 veterinary professionals. Exploratory factor analysis and reliability analysis were conducted to arrive at the final set of validated scales (final scales). 860 veterinarians (73 per cent females) participated. The final set of scales consisted of seven job demands scales (32 items), nine job resources scales (41 items), and six personal resources scales (26 items) which were considered to represent the most relevant potential predictors of work-related wellbeing in this occupational group. The procedure resulted in a tailormade questionnaire: the Veterinary Job Demands and Resources Questionnaire (Vet-DRQ). The use of valid theory and validated scales enhances opportunities for comparative national and international research.

Background: One of the methods used for standard setting is the borderline regression method (BRM). This study aims to assess the reliability of BRM when the pass-fail standard in an objective structured clinical examination (OSCE) was calculated by averaging the BRM standards obtained for each station separately. Materials and Methods: In nine stations of the OSCE with direct observation the examiners gave each student a checklist score and a global score. Using a linear regression model for each station, we calculated the checklist score cut-off on the regression equation for the global scale cut-off set at 2. The OSCE pass-fail standard was defined as the average of all station's standard. To determine the reliability, the root mean square error (RMSE) was calculated. The R-2 coefficient and the inter-grade discrimination were calculated to assess the quality of OSCE. Results: The mean total test score was 60.78. The OSCE pass-fail standard and its RMSE were 47.37 and 0.55, respectively. The R-2 coefficients ranged from 0.44 to 0.79. The inter-grade discrimination score varied greatly among stations. Conclusion: The RMSE of the standard was very small indicating that BRM is a reliable method of setting standard for OSCE, which has the advantage of providing data for quality assurance.

What's known on the subject? and What does the study add? Although research in other fields of society demonstrate a negative effect of distraction on performance, little research is done within the surgical specialties. We aimed to establish what the effect of distraction is, on the performance of basic endourological tasks by medical students. OBJECTIVE center dot To establish the effect of distraction on the performance of cystoscopy and basic endourological tasks by using a virtual reality (VR) simulator. SUBJECTS AND METHODS center dot A total of 86 third-year medical students from Maastricht University, who had no previous experience in performing the tasks on a VR simulator, were randomly assigned to an intervention or control group. center dot All participants performed three endourological tasks on the VR simulator. Participants in the intervention group were distracted 1 min into the third task. The distraction consisted of being asked to answer questions about a medical case that had been presented to all the participants before the hands-on session. After two adequate verbal responses the conversation was terminated. center dot Number of traumata, number of missed lesions in the bladder and time to completion were measured by the VR simulator. RESULTS center dot Number of traumata and missed lesions, as well as time to completion were significantly higher in the intervention than in the control group with effect sizes (using Cohen's categorization) of 0.48, 0.41 and 0.50 respectively. center dot Nevertheless, only 9.5% of the participants in the intervention group reported feeling burdened by the distraction. CONCLUSIONS center dot Distraction during the performance of endourological skills results in significantly poorer performance by medical students on all the variables measured in a controlled learning environment. center dot Most students do not realize they are affected by distraction. center dot Further research is needed to determine the impact of distraction on more experienced participants and on patient safety.

OBJECTIVE To assess whether real-time cysto-urethroscopy (CUS) performance improves by simulator-based training (criterion or predictive validity), addressing the research question 'Does practical skills training on the URO Mentor (UM, Simbionix USA Corp., Cleveland, OH, USA) virtual-reality simulator improve the performance of flexible CUS in patients'. SUBJECTS AND METHODS Participants (71 interns from Catharina Hospital Eindhoven, CHE, and 29 from University Medical Centre Groningen, UMCG) were randomized to carry out CUS in a patient after training on the UM (UM-trained, 50) or without training on UM (control, 50). The assessment of real-time performance consisted of scoring on a Global Rating Scale (GRS) by supervisors unaware of training status. Data were analysed using stepwise multiple linear regression. The effect size (ES) indication for correlations was used to interpret the magnitude of a standard regression coefficient (beta); an ES of 0.10, 0.30 and 0.50 were considered small, moderate and large, respectively. The study was approved by the Medical Review Ethics Committees of the participating hospitals. RESULTS Overall, the group that received training performed significantly better than the controls (P

OBJECTIVE To determine the effectiveness of single-point benchmarking and longitudinal benchmarking for inter-school educational evaluation. METHODS We carried out a mixed, longitudinal, cross-sectional study using data from 24 annual measurement moments (4 tests x 6 year groups) over 4 years for 4 annual progress tests assessing the graduation-level knowledge of all students from 3 co-operating medical schools. Participants included undergraduate medical students (about 5000) from 3 medical schools. The main outcome measures involved between-school comparisons of progress test results based on different benchmarking methods. RESULTS Variations in relative school performance across different tests and year groups indicate instability and low reliability of single-point benchmarking, which is subject to distortions as a result of school-test and year group-test interaction effects. Deviations of school means from the overall mean follow an irregular, noisy pattern obscuring systematic between-school differences. The longitudinal benchmarking method results in suppression of noise and revelation of systematic differences. The pattern of a school's cumulative deviations per year group gives a credible reflection of the relative performance of year groups. CONCLUSIONS Even with highly comparable curricula, single-point benchmarking can result in distortion of the results of comparisons. If longitudinal data are available, the information contained in a school's cumulative deviations from the overall mean can be used. In such a case, the mean test score across schools is a useful benchmark for cross-institutional comparison.