Search

We created a serious game to teach first year anesthesiology (CA-1) residents to perform general anesthesia for cesarean delivery. We aimed to investigate resident knowledge gains after playing the game and having received one of 2 modalities of debriefing. We report on the development and validation of scores from parallel test forms for criterion-referenced interpretations of resident knowledge. The test forms were intended for use as pre- and posttests for the experiment. Validation of instruments measuring the study's primary outcome was considered essential for adding rigor to the planned experiment, to be able to trust the study's results. Parallel, multiple-choice test forms development steps included: (1) assessment purpose and population specification; (2) content domain specification and writing/selection of items; (3) content validation by experts of paired items by topic and cognitive level; and (4) empirical validation of scores from the parallel test forms using Classical Test Theory (CTT) techniques. Field testing involved online administration of 52 shuffled items from both test forms to 24 CA-1's, 21 second-year anesthesiology (CA-2) residents, 2 fellows, 1 attending anesthesiologist, and 1 of unknown rank at 3 US institutions. Items from each form yielded near-normal score distributions, with similar medians, ranges, and standard deviations. Evaluations of CTT item difficulty (item p values) and discrimination (D) indices indicated that most items met assumptions of criterion-referenced test design, separating experienced from novice residents. Experienced residents performed better on overall domain scores than novices (P <.05). Kuder-Richardson Formula 20 (KR-20) reliability estimates of both test forms were above the acceptability cut of.70, and parallel forms reliability estimate was high at.86, indicating results were consistent with theoretical expectations. Total scores of parallel test forms demonstrated item-level validity, strong internal consistency and parallel forms reliability, suggesting sufficient robustness for knowledge outcomes assessments of CA-1 residents. [ABSTRACT FROM AUTHOR]

In this study, a novel method for generating multiple-choice tests is presented, which extracts the required number of tests of the same levels of difficulty in a single attempt and approximates the difficulty level requirement given by users. We propose an approach using parallelism and Pareto optimization for multi-swarm migration in a particle swarm optimization (PSO) algorithm. Multi-PSO is proposed for shortening the computing time. The proposed migration of PSOs increases the diversity of tests and controls the overlap of extracted tests. The experimental results show that the proposed method can generate many tests from question banks satisfying predefined levels of difficulty. Additionally, the developed method is shown to be effective in terms of many criteria when compared with other methods such as manually extracted tests, a simulated annealing algorithm (SA), random methods and PSO-based approaches in terms of the number of successful solutions, accuracy, standard deviation, search speed, and the number of questions overlapping between the exam questions, as well as for changing the search space, changing the number of individuals, changing the number of swarms, and changing the difficulty requirements.

As foreign language teachers, we have observed that reading comprehension is a complex phenomenon worth studying. Here, we address reading comprehension from the point of view of several disciplines: education (for assessment), logic (for inference), and linguistics (for foreign language), within an experimental framework. Our goal was to analyze and compare whether the properties of a test (say length, complexity, and inference) and the attributes of a test-taker (say test taking strategies, metacognitive awareness of reading strategies, and academic fields) have an effect on reading comprehension performance in native (Spanish) and foreign language (Italian). A quantitative cross-sectional study was performed on a sample of 35 students from a foreign languages centre; we used generalized linear mixed models to analyse the data. We found that the attributes of the test had an effect on performance in both languages, while the attributes of the test-takers had an effect only in foreign language.

Mastery learning assessments have been described in simulation-based educational interventions; however, studies applying mastery learning to multiple-choice tests (MCTs) are lacking. This study investigates an approach to item generation and standard setting for mastery learning MCTs and evaluates the consistency of learner performance across sequential tests. Item models, variables for question stems, and mastery standards were established using a consensus process. Two test forms were created using item models. Tests were administered at two training programs. The primary outcome, the test–retest consistency of pass–fail decisions across versions of the test, was 94% (κ =.54). Decision-consistency classification was.85. Item-level consistency was 90% (κ =.77, SE =.03). These findings support the use of automatic item generation to create mastery MCTs which produce consistent pass–fail decisions. This technique broadens the range of assessment methods available to educators that require serial MCT testing, including mastery learning curricula. [ABSTRACT FROM AUTHOR]

This paper proposes the use of multi-swarm method in particle swarm optimization (PSO) algorithm to generate multiple-choice tests based on assumed objective levels of difficulty. The method extracts an abundance of tests at the same time with the same levels of difficulty and approximates the difficulty-level requirement given by the users. The experimental results show that the proposed method can generate many tests from question banks satisfying predefined levels of difficulty. Additionally, the proposed method is also shown to be effective in terms of many criteria when compared with other methods such as manually extracted tests, random methods and PSO-based methods in terms of execution time, standard deviation, the number of particles per swarm and the number of swarms.

Multiple-choice tests do not assess examinees’ knowledge in accord with reality. In fact, the partial knowledge of examinees is not assessed. Providing a new approach to the assessment of reading comprehension in the framework of fuzzy logic, this study aims to measure this partial knowledge. In this approach, participants have to choose as many correct options as there are considering the stem. Therefore, the correct answer to each question can range from one option to all options. For the first session, an expository and an argumentative genre, and for the second session, the reading section of a TOEFL test was used. The results showed that the approach is fairer as it considers the partial knowledge of the examinees while in other common multiple-choice tests this is ignored. Also, the use of idea units as the units comprising a text gives us clues regarding the degree of difficulty of different parts of a text as well as clues about why misunderstanding may occur of the same text among different people.

Copyright of Ricerche di Pedagogia e Didattica is the property of Universita di Bologna, Dipartimento di Scienze dell'Educazione and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Introduction: Assessment of Educational Progress is an important part of the educational process, there are many ways to evaluate students, the most common type of Examination is multiple choice Questions and if they are properly designed, Become appropriate method for assessing learning. Objective: This study aimed to determine quantitative indicators multiple-choice tests, some faculty members of Alborz University of Medical Sciences and its relationship with demographic variables in the first semester 95-94 Methods: In this cross-sectional study (cross sectional) of 31 faculty members of Alborz University of Medical Sciences after obtaining permission from the Vice Chancellor for Research and the University of Education so that the researchers were examined by the software descriptive and analytical tests EDC unit test analysis software based in University studied anonymous. In order to analyze the data, descriptive statistics such as mean, standard deviation, frequency distribution and Pearson and Spearman correlation coefficient was used inferential. Results: Results showed that although the majority of tests have acceptable difficulty factor but their discrimination index is weak, the other had a small majority of test reliability. Conclusion: The results of this study can be argued that most of the questions used in the test was in need of fundamental review and be familiar with design workshops and assessment tests in order to meet the teachers and the inevitable creation of standard questions.

Recent findings have provided strong evidence that retrieval-based learning is an effective strategy for enhancing knowledge retention and long-term meaningful learning, but it is not a preferred learning strategy for the majority of students. The present research analyzes the application of learning gamification principles in online, open-book, multiple-choice tests in order to motivate students to engage in repeated retrieval-based learning activities. The results reveal a strong positive correlation between the number of successful retrieval attempts in these tests that cover content from the course textbook, and long-term knowledge retention as demonstrated in a live, final, closed-book, cumulative exam consisting of multiple-choice, labeling, definitions, and open-ended questions covering the content of both textbook readings and lectures. The presented results suggest that online, open-book tests designed using gamification principles, even when covering partial course content and one type of questions, are an effective strategy for using educational technology to motivate students to repeatedly engage in retrieval-based learning activities and improve long-term knowledge retention, regardless of the course delivery mode.

Automated summary evaluation is proposed as an alternative to rubrics and multiple-choice tests in knowledge assessment. Inbuilt rubric is a recent Latent Semantic Analysis (LSA) method that implements rubrics in an artificially-generated semantic space. It was compared with classical LSA's cosine-based methods assessing knowledge in a within-subjects design regarding two validation sources: a comparison with the results of rubric scores and multiple-choice tests, and the sensitivity of predicting the academic level of the test-taker. Results showed a higher reliability for inbuilt rubric (from Pearson correlation coefficient.81 to.49) over the classical LSA method (from.61 to.34), and a higher sensitivity using binary logistic regressions and effect sizes to predict academic level. It is concluded that inbuilt rubric has a qualitatively higher reliability and validity than classical LSA methods in a way that is complementary to models based on semantic networks. Thus, it is concluded that new automated summary evaluation approaches such as the inbuilt rubric method can be practical in terms of reliability and efficiency, and, thus, they can offer an affordable and valuable form of knowledge assessment in different educational levels. [ABSTRACT FROM AUTHOR]

This paper proposes the use of multi-swarm method in particle swarm optimization (PSO) algorithm to generate multiple-choice tests based on assumed objective levels of difficulty. The method extracts an abundance of tests at the same time with the same levels of difficulty and approximates the difficulty-level requirement given by the users. The experimental results show that the proposed method can generate many tests from question banks satisfying predefined levels of difficulty. Additionally, the proposed method is also shown to be effective in terms of many criteria when compared with other methods such as manually extracted tests, random methods and PSO-based methods in terms of execution time, standard deviation, the number of particles per swarm and the number of swarms. [ABSTRACT FROM AUTHOR]

Recent findings have provided strong evidence that retrieval-based learning is an effective strategy for enhancing knowledge retention and long-term meaningful learning, but it is not a preferred learning strategy for the majority of students. The present research analyzes the application of learning gamification principles in online, open-book, multiple-choice tests in order to motivate students to engage in repeated retrieval-based learning activities. The results reveal a strong positive correlation between the number of successful retrieval attempts in these tests that cover content from the course textbook, and long-term knowledge retention as demonstrated in a live, final, closed-book, cumulative exam consisting of multiplechoice, labeling, definitions, and open-ended questions covering the content of both textbook readings and lectures. The presented results suggest that online, open-book tests designed using gamification principles, even when covering partial course content and one type of questions, are an effective strategy for using educational technology to motivate students to repeatedly engage in retrieval-based learning activities and improve long-term knowledge retention, regardless of the course delivery mode. [ABSTRACT FROM AUTHOR]

Generating tests from question banks by using manually extracted items or involving random method consumes a great deal of time and effort. At the same time, the quality of the resulting tests is often not high. The generated tests may not entirely meet the requirements formulated in advance. Therefore, this study develops innovative ways to enhance this process by optimizing the execution time and generating results that closely meet the extraction requirements. The paper proposes the use of Particle Swarm Optimization (PSO) to generate multiple-choice tests based on assumed objective levels of difficulty. The experimental results reveal that PSO speed-ups the extraction process, and improves the quality of tests in comparison with the results produced by previously used methods such as Random or Genetic Algorithm (GA) optimized methods. In addition, PSO shows to be more efficient than GA and random selection in most criteria, such as execution time, search space, stability, and standard deviation. [ABSTRACT FROM AUTHOR]

Copyright of Dirasat: Educational Sciences is the property of University of Jordan and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Test-taking strategies are of two kinds: general and specific. General strategies are applicable to any test while specific ones can be employed by test-takers in special kinds of tests. Specific cognitive test-taking strategies, in turn, are divided into various types: structure, vocabulary, and reading comprehension. Based on their level of language proficiency, test-takers may show various degrees of tendency in making use of these strategies. The present study is an attempt to investigate whether there is any significant relationship between the subjects’ proficiency level and their tendency in using various types of strategies while taking a test of language proficiency.

We investigate the implications of penalizing incorrect answers to multiple-choice tests, from the perspective of both test-takers and test-makers. To do so, we use a model that combines a well-known item response theory model with prospect theory (Kahneman and Tversky, Prospect theory: An analysis of decision under risk, Econometrica 47:263-91, ). Our results reveal that when test-takers are fully informed of the scoring rule, the use of any penalty has detrimental effects for both test-takers (they are always penalized in excess, particularly those who are risk averse and loss averse) and test-makers (the bias of the estimated scores, as well as the variance and skewness of their distribution, increase as a function of the severity of the penalty). [ABSTRACT FROM AUTHOR]

The majority of multimedia learning studies focus on the use of graphics in learning process but very few of them examine the role of graphics in testing students' knowledge. This study investigates the use of static graphics versus animated graphics in a computer-based English achievement test from a cognitive load theory perspective. Three hundred and three 7th-grade students were randomly split into two groups and given the test questions either with static graphics or with animated graphics accompanied with text. Students' response time, response accuracy, self-reported ratings on cognitive load and secondary task approach were used to measure their cognitive load. Findings revealed that animating graphics increased the response time and secondary task scores of the students but did not have any significant effect on their test success. Furthermore, no difference was observed in self-reported cognitive loads. The relationship between the four different cognitive load measures was also examined in the study. No direct relation was found between self-reported ratings and secondary task scores. On the other hand, self-ratings and response accuracy were found to be more sensitive to intrinsic cognitive load, whereas response time and secondary task measures were found to be more sensitive to extraneous cognitive load in our testing environment. [ABSTRACT FROM AUTHOR]

Answering multiple-choice questions with competitive alternatives can enhance performance on a later test, not only on questions about the information previously tested, but also on questions about related information not previously tested-in particular, on questions about information pertaining to the previously incorrect alternatives. In the present research, we assessed a possible explanation for this pattern: When multiple-choice questions contain competitive incorrect alternatives, test-takers are led to retrieve previously studied information pertaining to all of the alternatives in order to discriminate among them and select an answer, with such processing strengthening later access to information associated with both the correct and incorrect alternatives. Supporting this hypothesis, we found enhanced performance on a later cued-recall test for previously nontested questions when their answers had previously appeared as competitive incorrect alternatives in the initial multiple-choice test, but not when they had previously appeared as noncompetitive alternatives. Importantly, however, competitive alternatives were not more likely than noncompetitive alternatives to be intruded as incorrect responses, indicating that a general increased accessibility for previously presented incorrect alternatives could not be the explanation for these results. The present findings, replicated across two experiments (one in which corrective feedback was provided during the initial multiple-choice testing, and one in which it was not), thus strongly suggest that competitive multiple-choice questions can trigger beneficial retrieval processes for both tested and related information, and the results have implications for the effective use of multiple-choice tests as tools for learning. [ABSTRACT FROM AUTHOR]

Copyright of Revista Educación en Ingeniería is the property of Asociacion Colombiana de Facultades de Ingenieria and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

La combinaison de la psychométrie et de la psychologie cognitive engendre de nouveaux modèles de mesure. Ceux-ci permettent d’établir le portrait de sujets ayant passé un test relativement à une liste d’attributs cognitifs. Cet article présente une comparaison de cinq modèles qui allient la vision psychométrique à la vision cognitive. Les comparaisons portent sur l’étude de la qualité des attributs cognitifs posés a priori, sur les caractérisations des sujets, sur la possibilité d’appliquer les modèles et d’interpréter les résultats et sur les notions de validité et de fidélité.

The generalized binomial test (GBT) and ω indices are the most recent methods suggested in the literature to detect answer copying behavior on multiple-choice tests. The ω index is one of the most studied indices, but there has not yet been a systematic simulation study for the GBT index. In addition, the effect of the ability levels of the examinees in answer copying pairs on the statistical properties of the GBT and ω indices have not been systematically addressed as yet. The current study simulated 500 answer copying pairs for each of 1,440 conditions (12 source ability level × 12 cheater ability level × 10 amount of copying) to study the empirical power and 10,000 pairs of independent response vectors for each of 144 conditions (12 source ability level × 12 cheater ability level) to study the empirical Type I error rates of the GBT and ω indices. Results indicate that neither GBT nor ω inflated the Type I error rates, and they are reliable to use in practice. The difference in statistical power of these two methods was very small, and GBT performs slightly better than does ω. The main effect for the amount of copying and the interaction effect between source ability level and the amount of copying are found to be very strong while all other main and interactions effects are negligible. [ABSTRACT FROM PUBLISHER]

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher-generated multiple-choice question (MCQ) exams in their courses. Multiple-choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined the influence student study approaches have on performance of shallow compared to deep, cognitive process outcomes measured by scores on factual versus application MCQ test bank questions respectively. Fifty-six introduction to psychology students, aged 18-52 years, completed a modified version of the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) to access deep or surface study approaches. Mid-term and final exam MCQ were equally divided between factual and application questions from the textbook publisher test bank. Overall, students performed significantly better on factual MCQ, with older students achieving higher factual and applied scores. Results suggest younger students tend to use a surface approach to study, with reproduction of what was taught to meet minimum requirements. With age statistically controlled, analyses demonstrated that the surface learning approach negatively impacts MCQ performance on questions categorised as factual and applied more than a deep learning approach benefits MCQ performance. [ABSTRACT FROM AUTHOR]

This article tests the assumption that state-mandated multiple-choice history exams are a cultural tool for disseminating an 'official' collective memory. Findings from a qualitative study of a collection of multiple-choice questions that relate to the history of the Soviet Union are presented. The 263 questions all come from New York State Regents exams that were administered between 1980-2010. Test questions were organized by decade, analysed for content, and then synthesized into historical narratives. These narratives were then analysed to see the extent to which they changed as the Soviet Union decayed, collapsed, and then broke up into new nation states. Particular attention was paid to the period that begins with the Cold War and ends with the present situation in post-Soviet Russia. The analysis shows that, despite great upheavals in the history of the Soviet Union, and changes in New York State standards and testing policies, the narrative that exists on these exam questions largely remains the same. This suggests that official narratives are slow to change. Further research should look at the extent to which test-taker knowledge and acceptance of official narratives affect performance on multiple-choice history items. [ABSTRACT FROM AUTHOR]

We present a new model of liberal (also known as free-choice) multiple-choice tests in which a candidate's test score is represented as a function of two other variables: their inclination to eliminate perceived distracters; and the accuracy with which they are able to do this. We propose a number of theoretical relationships between these three variables that appear to support the notion that good students ought to be better judges of their level of knowledge - and hence better able to benefit through the expression of partial knowledge - than weak students, as previous empirical studies have found. However, the results of our own small empirical study were not in line with our expectations. This work has given us a better understanding of the test format, but we now doubt whether it does in fact provide the benefits that have previously been claimed. [ABSTRACT FROM AUTHOR]

Summary: In multiple-choice tests, four-option items are the standard in nursing education. There are few evidence-based reasons, however, for MCQs to have four or more options as studies have shown that three-option items perform equally as well and the additional options most often do not improve test reliability and validity. The aim of this study was to examine and compare the psychometric properties of four-option items with the same items rewritten as three-option items. Using item-analysis data to eliminate the distractor with the lowest response rate, we compared three- and four-option versions of 41 multiple-choice items administered to two student cohorts over two subsequent academic years. Removing the non-functioning distractor resulted in minimal changes in item difficulty and discrimination. Three-option items contained more functioning distractors despite having fewer distractors overall. Existing distractors became more discriminating when infrequently selected distractors were removed from items. Overall, three-option items perform equally as well as four-option items. Since three-option items require less time to develop and administer and additional options provide no psychometric advantage, teachers are encouraged to adopt three-option items as the standard on multiple-choice tests. [Copyright &y& Elsevier]

The article presents a research paper addressing the suitability of conventional multiple-choice tests for assessing student learning outcomes in business education. It presents and illustrates design guidelines to help business faculty recognize and revise poorly designed questions. Finally, the importance of test reliability, test validity and test-item analysis is focused on to shed light on how well learning is being measured.

Both professional certification and academic tests rely heavily on multiple-choice questions, despite the widespread belief that alternate, constructed-response questions are superior measures of a test taker's understanding of the underlying material. Empirically, the search for a link between these two assessment metrics has met with limited success, leading some researchers to conclude that the relationship is close and others to conclude that no relationship exists at all. The authors suggest that “knowledge level” may play a key role in explaining this disparity in findings. This article outlines the theory for such a concept, and investigates the possibility using 172 carefully constructed tests in several entry-level programming classes. The article also discusses several caveats that suggest the usefulness of yet further research in the area. [ABSTRACT FROM AUTHOR]

This research attempted to determine the effect of redundant and consistent stems in multiple-choice vocabulary tests on pre-university students' performance. To this purpose, a sample English language proficiency tests was administered to a total of 130 pre-university students. Sixty homogeneous students were randomly selected and divided into two groups, one consistent group that took the test consisting of consistent stems and the other redundant group that took the test containing redundant stems. Next, three versions of the same test were provided in which the items had the same alternatives but different stems. The first and second versions of the consistent test were assigned to the consistent group, and the first two versions of the redundant tests to the redundant group. In the last versions of the two tests, the groups were reversed. This time, the consistent group took the test containing redundant stems while the redundant group took the test consisting of consistent stems. All the tests were teacher-made summative achievement tests and their reliability coefficients, the statistical independent t-test, paired samples t-test, and Pearson correlation formula were calculated. The results revealed that there was no significant difference between the students' performance on two kinds of tests (p<0.05). The empirical findings of the present study suggest that pre-university students' competence in vocabulary can be measured through both tests with redundant and tests with consistent stems. [ABSTRACT FROM AUTHOR]

Dyer’s woad, Isatis tinctoria, a plant of Eurasian origin is a problematic weed in western North America against which a classical biological weed control programme was initiated in 2004. Three European insect species were selected as candidate agents to control this invasive species, including the root-mining weevil Aulacobaris fallax. To determine its suitability as an agent, the biology and host specificity of A. fallax were studied in outdoor plots and in the field between 2004 and 2006 in its native European range. Aulacobaris fallax is a univoltine species that lays its eggs from March to August into leaf stalks and roots of dyer’s woad. Larvae mine and pupate in the roots and adults emerge from August to October. Up to 62% of the dyer’s woad plants at the field sites investigated were attacked by this weevil. In no-choice host-specificity tests, A. fallax attacked 16 out of 39 species and varieties within the Family Brassicaceae. Twelve of these are native to North America. In subsequent multiple-choice tests, seven species, all native to North America, suffered a similar level of attack as dyer’s woad, while none of the European species were attacked. Our results demonstrate the importance of including test plant species that have not co-evolved with the respective candidate agent. In sum, we conclude that the risk of non-target effects is too high for A. fallax to be considered as a biological control agent for dyer’s woad in the United States. [ABSTRACT FROM AUTHOR]

Abstract: The authors explored the credibility of using informal reading inventories and writing samples for 138 students (K–4) to evaluate the effectiveness of a summer literacy program. Running Records (a measure of a child''s reading level) and teacher experience during daily reading instruction were used to estimate the reliability of the more formal Developmental Reading Assessment scores. Training of scorers was used to increase the reliability of writing scores; a second scoring was used to estimate the reliability of the scores. The results suggested that with minimal modifications to administration and scoring procedures, scores from both reading inventories and writing samples can be a dependable source of data for teachers, administrators, and policy makers. This result is significant because it suggests that formative literacy assessments can be reliably used instead of standardized multiple-choice tests to make more credible summative decisions without taking time away from instruction, and can truly match curriculum, instruction, and assessment. [Copyright &y& Elsevier]

Multiple-choice exams are often the standard in large, introductory college courses. Although students sometimes report that multiple-choice exams are easier than essay exams, the multiple-choice format often proves to be more difficult. This may be true because multiple-choice exams in college are often composed predominantly of application questions. They ask students to grapple with scenarios and recognize concepts in context, which proves to be difficult for many students. The author details the changes she has made in her introductory sociology curriculum and discusses some of the indicators of success. [ABSTRACT FROM AUTHOR]

This study investigates the effects of student teachers' hands-on experience with evaluation on their preferences for assessment methods. A course on child development within the 1st year of the elementary teacher education program provides the quasi-experimental learning/teaching setting. Five research conditions are linked to four assessment modes, namely, portfolio, case-based, peer assessment, and multiple-choice evaluations. Data collection was obtained by questionnaires, adopting a longitudinal design that involves three moments of measurement. Results show initially negative responses to unknown assessment methods. As familiarity with the assessment tool grew, student teachers' preferences changed positively. Although the extent differs, every assessment method benefited significantly from the teacher candidates' experience with the format during the course. Moreover, student teachers' perceptions of the appropriateness of the assessment method for evaluation purposes are congruent with their preferences. Consequently, to change student teachers' preferences for unknown assessment methods, hands-on experiences are fundamental and need to be positive. Results are particularly encouraging for teacher education instructors who use a variety of assessment modes. [ABSTRACT FROM AUTHOR]

Contrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions generally show less consistency with the responses of other questions, and the reliability of the test will be decreased. The surprising implication is that adding questions to a multiple-choice test may lower its reliability when the test is speeded. This article develops the mathematical derivations and shows the effects of speededness on reliability in simulations. [ABSTRACT FROM AUTHOR]

We offer a case-study in irrationality, showing that even in a high stakes context, intelligent and well trained professionals may adopt dominated practices. In multiple-choice tests one cannot distinguish lucky guesses from answers based on knowledge. Test-makers have dealt with this problem by lowering the incentive to guess, through penalizing errors (called formula scoring), and by eliminating various cues for outperforming random guessing (e.g., a preponderance of correct answers in middle positions), through key balancing. These policies, though widespread and intuitively appealing, are in fact "irrational", and are dominated by alternative solutions. Number-right scoring is superior to formula scoring, and key randomization is superior to key balancing. We suggest that these policies have persisted since all stake-holders -- test-makers, test-takers and test-coaches -- share the same faulty intuitions. [ABSTRACT FROM AUTHOR]

Instructors can use both“multiple-choice” (MC) and“constructed response” (CR) questions (such as short answer, essay, or problem-solving questions) to evaluate student understanding of course materials and principles. This article begins by discussing the advantages and concerns of using these alternate test formats and reviews the studies conducted to test the hypothesis (or perhaps better described as the hope) that MC tests, by themselves, perform an adequate job of evaluating student understanding of course materials. Despite research from educational psychology demonstrating the potential for MC tests to measure the same levels of student mastery as CR tests, recent studies in specific educational domains find imperfect relationships between these two performance measures. We suggest that a significant confound in prior experiments has been the treatment of MC questions as homogeneous entities when in fact MC questions may test widely varying levels of student understanding. The primary contribution of the article is a modified research model for CR/MC research based on knowledge-level analyses of MC test banks and CR question sets from basic computer language programming. The analyses are based on an operationalization of Bloom's Taxonomy of Learning Goals for the domain, which is used to develop a skills-focused taxonomy of MC questions. However, we propose that their analyses readily generalize to similar teaching domains of interest to decision sciences educators such as modeling and simulation programming. [ABSTRACT FROM AUTHOR]

This study compares the parametric multiple-choice model and the nonparametric kernel smoothing approach to estimating option characteristic functions (OCCs) using an empirical criterion, the stability of curve estimates over occasions that represents random error. The potential utility of graphical OCCs in item analysis was illustrated with selected items. The effect of increasing the smoothing parameter on the nonparametric model and the effect of small sample on both approaches were investigated. Differences between estimated curve values for between-model within-occasion, within-model between-occasion, and between-model between-occasion were evaluated. The between-model differences were minor in relation to the within-model stabilities, and the incremental difference attributable to model was smaller than that attributable to occasion. Either model leads to the same choice in item analysis. [ABSTRACT FROM AUTHOR]

This study examines the effect of three scoring methods (number-correct, discouraging guessing, and the partial knowledge award) on the psychometric indices (reliability and validity) of a test, given examinees' risk-taking level. One hundred and twenty undergraduate students in a psychology research methodology class served as the sample. A 40-item multiple-choice test with 4 responses per item was used to assess the effect of different scoring methods on test reliability and validity, and a test of 10 nonsense items was used to classify the examinees into high risk-taking and low risk-taking groups. The results showed that the 3 methods produce different reliability and validity coefficients, with the partial knowledge method choice. [ABSTRACT FROM AUTHOR]

This paper reports the development of indices reflecting the probability that the observed correspondence between the multiple-choice test responses of two examinees was due to chance. Applications of the indices are presented both with respect to apprehending persons who cheat by copying answers and with respect to monitoring the prevalence of this form of cheating in order to evaluate methods of preventing it. [ABSTRACT FROM PUBLISHER]

The article presents information on an R package, CopyDetect, a programming language that is developed to detect answer copying on multiple-choice examination through computing statistical analysis. It informs that identifying answer copying is an essential part of maintaining the integrity of test scores. It also offers information on a report which shows that 35 percent of high school students are engaged in some type of est fraud.

Multiple-choice tests are the most used method of assessment in medical education. However, there is limited literature in medical education and psychiatry to inform the best practices in writing good-quality multiple-choice questions. Moreover, few physicians and psychiatrists have received training and have experience in writing them. This article highlights the strategies in writing high-quality multiple-choice items and discusses some common flaws that can impact validity and reliability of the assessment examinations., Competing Interests: Disclosure The authors have no financial disclosures., (Copyright © 2021 Elsevier Inc. All rights reserved.)