Search

This letter, published in the Australasian Psychiatry journal, responds to a previous publication by Amos et al. The authors express surprise and concern over the publication's focus solely on the measurement aspects of assessment, which they believe oversimplifies the complexity of the matter. They argue that the publication contains misunderstandings and incorrect assumptions about assessment, such as confusing predictive validity with construct validity. The authors also criticize the comparison of pass rates, stating that it is not a meaningful way to evaluate assessment methods. They suggest that longitudinal assessment with feedback leads to better learning outcomes. [Extracted from the article]

The letter responds to Dr. Holmes' concerns about the removal of independent clinical assessment from the Fellowship training program of the Royal Australian and New Zealand College of Psychiatrists (RANZCP). The letter clarifies that the RANZCP is proposing two options for independent assessment: the Clinical Competency Portfolio Review and the Integrated Assessment Process. These options aim to assess trainees' ability to conduct clinical assessments of real patients. The letter acknowledges the challenges and burden associated with workplace-based assessment (WBA) but argues that the educational argument for WBA is solid, citing evidence from the literature on expertise development and medical education. The RANZCP is working on developing resources to support and train supervisors for an optimal use of supervision time. The letter concludes by stating that the College Assessments proposals and consultations are available on the Program of Assessments web portal. [Extracted from the article]

Aims: To identify, appraise and describe studies of cognitive interventions to improve diagnostic decision making (DDM) amongst medical professionals, assess their effectiveness and identify methodological limitations in existing studies. Methods: We systematically searched for studies (publication date 2000–2016) in multiple databases including Cochrane Controlled Trials, EMBASE, ERIC, Medline, PubMed and PsycINFO, and used additional strategies such as hand searching and snowballing. Included studies evaluated cognitive interventions to enhance DDM amongst medical professionals, using defined outcomes such as diagnostic accuracy. A meta-analysis assessed the impact of "reflection". Results: Forty-four studies out of 10,114 screened citations, involving 4380 medical professionals, were included. Studies evaluated reasoning workshops/curricula, de-biasing workshops, checklists, reflection, feedback, and instructions to induce analytical thinking. Guided reflection was demonstrated to improve DDM [effect size 0.38(95%CI 0.23–0.52), p < 0.001]. Immediate feedback and modeling reflection using contrasting examples also appeared to improve diagnostic accuracy, however underlying methodological issues prevented a quantitative assessment of any strategies other than reflection. Conclusions: Educational interventions incorporating practising deliberate reflection on a formulated diagnosis, modeled reflection on contrasting examples and immediate feedback are promising strategies for improving DDM. The effectiveness of other strategies is unknown, with more methodological refinements required in future research. [ABSTRACT FROM AUTHOR]

Programmatic assessment is both a philosophy and a method for assessment. It has been developed in medical education as a response to the limitation of the dominant testing or measurement approaches and to better align with changes in how medical competence was conceptualised. It is based on continual collection of assessment and feedback information which is collected in a portfolio and periodically discussed with a mentor or coach. Students produce reflections in which they analyse the assessment information triangulated across methods to meaningfully- rather than purely numericallyanalyse their strengths and weakness in the different competency domains. Decisions about students' progress are made proportionally, in that low-stakes decisions are made when only limited assessment information is available but high-stakes decisions will and can be made on rich information. Quality of the process and credibility of the outcomes is determined by a combination of psychometric methods, qualitative rigour, and through organisational due processes. The method is still young, preliminary results are promising but more empirical justification research is needed. [ABSTRACT FROM AUTHOR]

Introduction: Studies have shown that a physician’s clinical reasoning performance can be influenced by contextual factors. We explored how the clinical reasoning performance of medical students was impacted by contextual factors in order to expand upon previous findings in resident and board certified physicians. Using situated cognition as the theoretical framework, our aim was to evaluate the verbalized clinical reasoning processes of medical students in order to describe what impact the presence of contextual factors has on their reasoning performance.Methods: Seventeen medical student participants viewed three video recordings of clinical encounters portraying straightforward diagnostic cases in internal medicine with explicit contextual factors inserted. Participants completed a computerized post-encounter form as well as a think-aloud protocol. Three authors analyzed verbatim transcripts from the think-aloud protocols using a constant comparative approach. After iterative coding, utterances were analyzed and grouped into categories and themes.Results: Six categories and ten associated themes emerged, which demonstrated overlap with findings from previous studies in resident and attending physicians. Four overlapping categories included emotional disturbances, behavioural inferences about the patient, doctor-patient relationship, and difficulty with closure. Two new categories emerged to include anchoring and misinterpretation of data.Discussion: The presence of contextual factors appeared to impact clinical reasoning performance in medical students. The data suggest that a contextual factor can be innate to the clinical scenario, consistent with situated cognition theory. These findings build upon our understanding of clinical reasoning performance from both a theoretical and practical perspective. [ABSTRACT FROM AUTHOR]

Aim: Programmatic assessment for learning (PAL) is becoming more and more popular as a concept but its implementation is not without problems. In this paper we describe the design principles behind a PAL program in a general practice training context. Design principles: The PAL program was designed to optimise the meaningfulness of assessment information for the registrar and to make him/her use that information to self regulate their learning. The main principles in the program were cognitivist and transformative. The main cognitive principles we used were fostering the understanding of deep structures and stimulating transfer by making registrars constantly connect practice experiences with background knowledge. Ericsson's deliberate practice approach was built in with regard to the provision of feedback combined with Pintrich's model of self regulation. Mezirow's transformative learning and insights from social network theory on collaborative learning were used to support the registrars in their development to become GP professionals. Finally the principal of test enhanced learning was optimised. Epilogue: We have provided this example explain the design decisions behind our program, but not want to present our program as the solution to any given situation. [ABSTRACT FROM AUTHOR]