Search

Scores from noncognitive measures are increasingly valued for their utility in helping to inform postsecondary admissions decisions. However, their use has presented challenges because of faking, response biases, or subjectivity, which standardized third‐party evaluations (TPEs) can help minimize. Analysts and researchers using TPEs, however, need to be mindful of the potential for construct‐irrelevant differences that may arise in TPEs due to differences in evaluators' rating approaches, which introduces measurement error. Research on sources of construct‐irrelevant variance in TPEs is scarce. We address this paucity by conducting generalizability theory (G theory) analyses using TPE data that informs postsecondary admissions decisions. We also demonstrate an approach to assess the size of interevaluator variability and conduct a decision study to determine the number of evaluators necessary to achieve the desired generalizability coefficient. We illustrate these approaches using a TPE whereby applicants select their evaluators, leading to a situation where most evaluators solely rate one applicant. We conclude by presenting strategies to improve the design of TPEs to help increase confidence in their use. Report Number: ETS RR–18‐37 [ABSTRACT FROM AUTHOR]

Within this study the author examines factors commonly employed as master of business administration applicant evaluation criteria to see if these criteria are important in determining an applicant's potential for success. The findings indicate that the Graduate Management Admissions Test (GMAT) is not a significant predictor of student success when considering factors such as undergraduate grade point average and work experience. Furthermore, the results suggest that prior findings in support of the GMAT are the result of missing variables in the model specification. Our results show that undergraduate grade point average alone can be employed as an admission criterion and indicator of potential success in lieu of the GMAT; adopting this criterion instead can streamline the admission process while minimizing student expenses. Within the discussion section the author offers suggestions for reducing the need for the GMAT score information in the admissions process. [ABSTRACT FROM AUTHOR]

Scores from the GRE® revised General Test provide important information regarding the verbal and quantitative reasoning abilities and analytical writing skills of applicants to graduate programs. The validity and utility of these scores depend upon the degree to which the scores predict success in graduate and business school in specific contexts. To assess the predictive validity of the GRE test for graduate business programs, we collaborated with a number of universities and obtained data from each university's admissions office and registrar. We focused specifically on part-time and full-time students in master's of business administration ( MBA) degree programs. Given the nested structure of the data, we used a 2-level (representing students and institutions) hierarchical linear model ( HLM) to estimate regression models with first-semester MBA grade point average ( GPA) or cumulative MBA GPA as the dependent variable and GRE scores and undergraduate GPA ( UGPA) as independent variables. For predicting cumulative MBA GPA, the pseudo R-squared ( R2) value was .04 using UGPA, cohort year, and program type as the only predictors; this value increased substantially to .19 with the addition of GRE scores. The HLM results show that both GRE-Quantitative and GRE-Verbal scores were statistically significant predictors of both first-semester MBA GPA and cumulative MBA GPA. [ABSTRACT FROM AUTHOR]

In this research, we investigated the feasibility of implementing the e-rater® scoring engine as a check score in place of all-human scoring for the Graduate Record Examinations® ( GRE®) revised General Test (rGRE) Analytical Writing measure. This report provides the scientific basis for the use of e-rater as a check score in operational practice. We proceeded with the investigation in four phases. In phase I, for both argument and issue prompts, we investigated the quality of human scoring consistency across individual prompts, as well as two groups of prompts organized into sets. The sets were composed of prompts with separate focused questions (i.e., variants) that must be addressed by the writer in the process of responding to the topic of the prompt. There are also groups of variants of prompts (i.e., grouped for scoring purposes by similar variants). Results showed adequate human scoring quality for model building and evaluation. In phase II, we investigated eight different e-rater model variations each for argument and issue essays including prompt-specific; variant-specific; variant-group-specific; and generic models both with and without content features at the rating level, at the task score level, and at the writing score level. Results showed the generic model was a valued alternative to the prompt-specific, variant-specific, and variant-group-specific models, with and without the content features. In phase III, we evaluated the e-rater models on a recently tested group from the spring of 2012 (between March 18, 2012, to June 18, 2012) following the introduction of scoring benchmarks. Results confirmed the feasibility of using a generic model at the rating and task score level and at the writing score level, demonstrating reliable cross-task correlations, as well as divergent and convergent validity. In phase IV of the study, we purposely introduced a bias to simulate the effects of training the model on a potentially less able group of test takers in the spring of 2012. Results showed that use of the check-score model increased the need for adjudications between 5% and 8%, yet the increase in bias actually increased the agreement of the scores at the analytical writing score level with all-human scoring. [ABSTRACT FROM AUTHOR]

The computer-based Graduate Record Examinations® ( GRE®) revised General Test includes interactive item types and testing environment tools (e.g., test navigation, on-screen calculator, and help). How well do test takers understand these innovations? If test takers do not understand the new item types, these innovations may introduce construct-irrelevant variance, with test takers performing differently than they would with more familiar item types. Similarly, the navigational and other test environment tools are another potential source of variance, if some test takers understand how to use them and others do not. In this study, we examined the reactions, engagement, and difficulties encountered as 20 potential test takers completed Verbal and Quantitative Reasoning sections of a practice GRE test. Participants were sophomores and juniors from colleges and universities in the local area. Their reactions were captured through the use of cognitive laboratory sessions that incorporated interviews that required test takers to think aloud, as well as researcher observations as test takers worked quietly. Results of the analysis of this data revealed that some participants needed time to figure out what was being asked of them when they encountered the new item types, although most were able to answer each item eventually. On the other hand, most participants stated that they did not even notice the test environment tools, and few were observed actually using the tools. Several participants provided suggestions about improving the usability of the new item types and test environment tools. [ABSTRACT FROM AUTHOR]

Although many articles have been written about thesis assessment, none provide a comprehensive, general picture of what examiners do as they assess a thesis. To synthesise this diverse literature, we reviewed 30 articles, triangulated their conclusions and identified 11 examiner practices. Thesis examiners tend to be broadly consistent in their practices and recommendations; they expect and want a thesis to pass, but first impressions are also very important. They read with academic expectations and the expectations of a normal reader. Like any reader, thesis examiners get annoyed and distracted by presentation errors, and they want to read a work that is a coherent whole. As academic readers, examiners favour a thesis with a convincing approach that engages with the literature and the findings, but they require a thesis to be publishable research. Finally, examiners give not only a final evaluation of a thesis, but also instruction and advice to improve the thesis and further publications and research. We hope that these generalisations will demystify the often secret process of assessing a thesis, and reassure, guide and encourage students as they write their theses. [ABSTRACT FROM AUTHOR]

This article describes the yearlong process of systematically developing an MPA Major Field Test with a team of five schools in various regions of the United States. Each school professor on the test team interviewed numerous practitioners to determine current issues in the profession. The practitioner information was filtered through at least one academic team member who used the information to create exam questions and submitted them for peer review. The test is based on NASPAA's five competencies for the MPA degree. Four schools list the data gleaned from the test results and its impact on their MPA program. Though the data set is currently not large enough to determine validity or reliability, the article describes the process and the utility of the test to offer the first collaborative measurement for schools to determine the strength of their MPA program [ABSTRACT FROM AUTHOR]

Master of Public Administration (MPA) programs are offered in more than 100 universities in China. All require applicants to pass a competitive national entrance exam in five subject areas. This requirement significantly discourages senior-level public administrators, many of whom graduated from college several years ago and have work schedules that do not allow time to prepare for an academically focused examination. These senior-level administrators need an Executive MPA (EMPA) or similar program with more realistic entrance requirements and a curriculum designed for people with significant professional experience. Drawing on the data collected from surveys of MPA students enrolled in six MPA programs in China and students enrolled in an EMPA program in Singapore specifically designed for Chinese public servants, this article reports how the demographic differences between MPA students and EMPA students affect their preference for professional public administration education in terms of knowledge components, managerial competencies, teaching modalities, and examination methods. The findings of the study significantly challenge not only Chinese universities that would like to adopt the existing MPA education model for EMPA education but also universities outside of China that are establishing EMPA or joint EMPA programs with Chinese governments or universities. [ABSTRACT FROM AUTHOR]

Aim To assess the effects of binge drinking on students' next-day academic test-taking performance. Design A placebo-controlled cross-over design with randomly assigned order of conditions. Participants were randomized to either alcoholic beverage [mean = 0.12 g% breath alcohol concentration (BrAC)] or placebo on the first night and then received the other beverage a week later. The next day, participants were assessed on test-taking, neurocognitive performance and mood state. Participants A total of 196 college students (≥21 years) recruited from greater Boston. Setting The trial was conducted at the General Clinical Research Center at the Boston Medical Center. Measurements The Graduate Record Examinations© (GREs) and a quiz on a lecture presented the previous day measured test-taking performance; the Neurobehavioral Evaluation System (NES3) and the Psychomotor Vigilance Test (PVT) measured neurocognitive performance; and the Profile of Mood States (POMS) measured mood. Findings Test-taking performance was not affected on the morning after alcohol administration, but mood state and attention/reaction-time were affected. Conclusion Drinking to a level of 0.12 g% BrAC does not affect next-day test-taking performance, but does affect some neurocognitive measures and mood state. [ABSTRACT FROM AUTHOR]

The validity of Graduate Management Admission Test (GMAT) scores is examined by summarizing 273 studies conducted between 1997 and 2004. Each of the studies was conducted through the Validity Study Service of the test sponsor and contained identical variables and statistical methods. Validity coefficients from each of the studies were corrected for restriction of range on the predictors. The interquartile range of the validity coefficients for the combination of GMAT scores and undergraduate grades in predicting early graduate school performance was .45 to .63. Further examination of study findings showed that higher validity coefficients were observed for executive MBA programs compared to either full-time or part-time programs. There appeared to be no measurable differences in results for public versus private institutions. For programs not located at U.S. institutions, quantitative scores showed lower average validity coefficients compared to programs located in the United States. [ABSTRACT FROM AUTHOR]

Recent assessment research joining cognitive psychology and psychometric theory has introduced a new technology, item generation. In algorithmic item generation, items are systematically created based on specific combinations of features that underlie the processing required to correctly solve a problem. Reading comprehension items have been more difficult to model than other item types due to the complexities of quantifying text. However, recent developments in artificial intelligence for text analysis permit quantitative indices to represent cognitive sources of difficulty. The current study attempts to identify generative components for the Graduate Record Examination paragraph comprehension items through the cognitive decomposition of item difficulty. Text comprehension and decision processes accounted for a significant amount of the variance in item difficulties. The decision model variables contributed significantly to variance in item difficulties, whereas the text representation variables did not. Implications for score interpretation and future possibilities for item generation are discussed [ABSTRACT FROM AUTHOR]

This article describes the rationale, implementation, and assessment of a summer-study format, part-time doctoral program that was designed to address the shortcomings of traditional part-time doctoral education in social welfare. Findings of a comparative analysis between 67 full-time and 52 part-time doctoral students suggest that the program offers a number of significant advantages over traditional part-time education. There were no statistically significant differences between summer-study and full-time students in Graduate Record Examination and Miller Analogies Test scores, in rates of graduation, or in length of time to complete the degree. Educational limitations of the program for summer-study students as compared to full-time students are presented. [ABSTRACT FROM AUTHOR]

The aim of the present research was to develop a difficulty model for logical reasoning problems involving complex ordered arrays used in the Graduate Record Examination. The approach used involved breaking down the problems into their basic cognitive elements such as the complexity of the rules used, the number of mental models required to represent the problem, and question type. Weightings for these different elements were derived from two experimental studies and from the reasoning literature. Based on these weights, difficulty models were developed which were then tested against new data. The models had excellent predictive validity and showed the relative influence of rule based factors and factors relating to the number of underlying models. Different difficulty models were needed for different question types, suggesting that people used a variety of approaches and, at a wider level, that both mental models and mental rules may be used in reasoning. [ABSTRACT FROM AUTHOR]

In this study, the author examined the validity of the Graduate Management Admissions Test (GMAT) for non-U.S. students (N = 75) through a predictive validation procedure in which applicants were given the predictor test but the test results were not used to admit students. The author's business school admitted students to three overseas MBA programs. The author then gathered academic performance data (MBA grades) and correlated the predictor (GMAT) with the criterion (MBA GPA). The findings showed that the correlation between the two was .64 (p < .001), meaning that 41% of the variance in MBA academic performance was explained by the GMAT. This number is higher than the corresponding one for U.S. students. The findings indicate that the GMAT was a valid predictor of academic performance in an MBA program for these non-U.S. students. [ABSTRACT FROM AUTHOR]

With the advent of modern computer technology, there have been growing efforts in recent years to computerize standardized tests, including the popular Graduate Record Examination (GRE), the Graduate Management Admission Test (GMAT) and the Test of English as a Foreign Language (TOEFL). Many of such computer-based tests are known as the computerized adaptive tests, a major feature of which is that, depending on their performance in the course of testing, different examinees may be given with different sets of items (questions). In doing so, items can be efficiently utilized to yield maximum accuracy for estimation of examinees’ ability traits. We consider, in this article, one type of such tests where test lengths vary with examinees to yield approximately same predetermined accuracy for all ability traits. A comprehensive large sample theory is developed for the expected test length and the sequential point and interval estimates of the latent trait. Extensive simulations are conducted with results showing that the large sample approximations are adequate for realistic sample sizes. [Copyright &y& Elsevier]

Presents an experimental administration of both the verbal and quantitative section of the Graduate Record Examination General Test (GRE) with standard time limits and with 1.5 times the standard time limit. Aim to test the assumption that GRE is a measure of academic reasoning abilities that is intended to be a power test in which speed of responding plays at most a minor role; Results indicating that extra time added about 7 points to verbal scores and 7 points to quantitative scores; Comparable results across gender and ethnic groups; Finding that quantitative scores were slightly higher for lower ability examinees.

Opposes the Graduate Record Exam (GRE), which, despite its well-documented predictive limitations, is still widely used in the academic sorting process in the U.S. Patterns, insights and evidence about the uses and misuses of testing; Harsh effects of pre-determined test score cut-offs on students of color and those from modest economic backgrounds; Model of academic merit.

The article discusses the predictors of the job placement prospects of economics graduate students in the United States. The study focused on 344 students who applied for admission to the economics department of the Princeton University in 1989. The job placement of the students were monitored in 1996 and in 1999. It was found that mathematics Graduate Record Examination (GRE) scores, economics GRE scores and ratings of the admissions committee are effective predictors of graduate students' job placement. The GRE score can be an effective resource for college admission decisions.

Investigates the predictive relationship between Graduate Record Examination (GRE) scores and subsequent grade performance in graduate chemistry courses by evaluating records of students in a chemistry program. Cases where GRE scores predicts the graduate performance of chemistry students; Chart showing the validity coefficients for GRE scores and graduate course grades.

Presents a practical guide to running preparation courses for postgraduate examinations. Objectives of preparation classes; Funding aspects of preparation classes; Resources; Preparation and evaluation.

Reports on the use of meta-analytic review to examine the relationship between performance on the quantitative and verbal components of the Graduate Record Examination and graduate grade point average. Validity of the examination in the graduate admissions process.

A questionnaire study was undertaken to determine the extent to which the GRE Biology Test reflects the content of undergraduate biological sciences curricula currently recommended as preparation for graduate work in the biological sciences, the agricultural sciences, and the health sciences. The results indicate that the test generally reflects the content of the undergraduate biology curricula at many of the respondents' institutions, (Accepted for publication 4 November 1982) [ABSTRACT FROM AUTHOR]

The article presents information on the Graduate Record Examinations (GRE). It was first administered in 1937 and represented a joint venture of the Carnegie Foundation for the Advancement of Teaching and the graduate school deans of four eastern United States universities. Educational Testing Service (ETS) assumed responsibility for the GRE on 1 January 1948. ETS appoints the committee of examiners responsible for the Advanced GRE Biology Test, with the cooperation of the American Institute of Biological Sciences Inc., the Botanical Society of America, and the American Society of Zoologists.

The Graduate Record Examination is probably the most celebrated and controversial element of all entrance requirements to graduate school. Yet little is known about how test scores are used. This study reports the findings from a national survey of Master of Public Administration programs. Following a discussion dealing with the evolution of and literature on standardized testing, the extent, rationale, and role of the test in admissions procedures are explored. The analysis closes with a brief commentary on the part that objective examinations can play in public administration programs. [ABSTRACT FROM AUTHOR]

A postgraduate quiz on the case of a 24-year-old man with juvenile-onset systemic lupus erythematosus (SLE) is presented.

The article views on educational programs designed for college teachers. The programs provide for preparation in the candidate's instructional field. Graduates of these programs have the authority to identify the field of specialization in connection with the degree title. The requirements needed for the Doctor of Education degree in college teaching include a minimum of three academic years. It is equivalent to ninety semester points of graduate course work. On the other hand, a candidate for admission to a program for college teaching must have completed a master's degree, must have achieved high scores on the Graduate Record Examination, and must have completed at least one semester of course work in his Teachers College department during his master's degree program.

Reports on the Graduate Record Examination (GRE) at Denison University in Granville, Ohio. Information on the GRE in speech; Disparity between what the GRE covers and what the small college actually offers; Limitations of the GRE.

The article focuses on the prediction of graduate performance in economics course at the University of Wisconsin, Madison, Wisconsin. It employs a comprehensive set of predictors and attempts to develop a fuller and richer set of results with the help of a multiple regression model. In addition, the use of a measure of undergraduate school quality, specifically one reflecting the intellectualism of the student body, is found to be highly useful and to explain why undergraduate grade-point averages and Graduate Record Examinations scores are usually found to be poor predictors when used either alone or together. It takes a measure of graduate student success Graduate Grade Point Average during the first and then the first two years of graduate study. It is by no means certain that this measure of performance is the most appropriate one to reflect either the success of graduate work or of career success; indeed, there are some grounds for skepticism about the usefulness of grades in evaluating post-school success, as reflected by income.

The article offers information about an Objective Structured Clinical Examination (OSCE) assessment at the Warren Alpert Medical School of Brown University, Rhode Island which enables the medical educators to evaluate the clinical skills and professionalism of the medical students. It states that the University conducts several OCSEs in the first two years of the Doctoring course including clinical skills clerkship, internal medicine and obstetrics. It mentions that the fourth-year OSCE is administered at the start of the final year of medical students and the students who do not pass the examination are required to remediate. It also presents several tables which depicts student's performance in the fourth-year OSCE.

We surveyed graduate programs to see how they use the Graduate Record Examination Analytic Writing (GRE-AW) Test. Only 35% of the graduate programs that responded use the GRE-AW test in their admission policy; of the programs not using it, most do not plan to do so. The programs using the GRE-AW rated it as medium or low in importance in their admission decisions. Few programs have cutoff scores. Approximately 8% reported using an initial validation study; about half reported considering or running an ongoing validation study. We urge program coordinators to evaluate the potential benefits of the GRE-AW as part of admissions decisions and the Educational Testing Service to help coordinators do so. [ABSTRACT FROM AUTHOR]

Reports on the issues related to the establishment of an independent educational equivalence for graduates of nonaccredited veterinary colleges in the U.S. Meeting held which focused on resolving the certification of veterinary graduates; Reasons cited by the working group for the advantages of having a single licensing body; Ways of evaluating educational equivalence.

Details the Getting Everyone to Excel (Project GEE), a tutorial plan developed and utilized by Mandeville High School, part of the Saint Tammany Parish School District in New Orleans, Louisiana, to assist students in their preparation for the Louisiana Graduate Exit Exam 21 and the Iowa Test of Educational Development. Steps in establishing the program; Most significant modification to Project GEE; Impact of the program on the students.

Opinion. Comments on the article `Does the Graduate Record Examination (GRE) Predict Meaningful Success in the Graduate Training of Psychologists? A Case Study,' by R.J. Sternberg and W.M. Williams, in the American Psychologist journal. Indication that the GRE is not valid for important criteria; Indepth look at the prediction and measurement of professionalism; What factors constitute the final graduate school validation.