Search

Critical thinking has become an essential skill required by both higher education and the workforce. Research to date has reported moderate cross-sectional learning differences in critical thinking as students progress through a 4-year college. However, retention, differential participation, and students' low test-taking motivation possibly confounded conclusions from prior studies. Controlling for such factors, we found a cross-sectional difference of 0.24 SDs after 4 years in college (n = 2,381 students, 46 institutions), considerably smaller than what's reported in prior studies. Natural science majors performed the highest and business majors performed the lowest. Minority students achieved only half of the cross-sectional difference (0.18 SDs) of white students (0.39 SDs), and only 6% of them scored at the Advanced level, compared to 24% of white students.

Economic globalization and interdisciplinary advancements have increased the demand for college graduates to possess transferable skills that would allow them to contribute effectively to the modern workforce. In particular, transferable competencies such as civic competency and intercultural competency are critical for colleges to prepare responsible citizens and productive workers. Despite the recognized importance, the choice and quality of assessment for such competencies have been fairly limited due to the challenges in defining such complex, multidimensional constructs and identifying item types that can adequately assess them. In this report, we describe the principles we followed to operationalize definitions for civic competency and intercultural competency and the process we followed to design assessments for these 2 competencies. Findings from a large-scale pilot test are reported. Results showed that these multidimensional constructs can be adequately assessed and that there is room for students to improve in these areas. Implications for higher education institutions on how to promote these critical competencies are discussed.

We examine the effects of computer-based versus paper-based assessment of critical thinking skills, adapted from English (in the U.S.) to Chinese. Using data collected based on a random assignment between the two modes in multiple Chinese colleges, we investigate mode effects from multiple perspectives: mean scores, measurement precision, item functioning (i.e. item difficulty and discrimination), response behavior (i.e. test completion and item omission), and user perceptions. Our findings shed light on assessment and item properties that could be the sources of mode effects. At the test level, we find that the computer-based test is more difficult and more speeded than the paper-based test. We speculate that these differences are attributable to the test's structure, its high demands on reading, and test-taking flexibility afforded under the paper testing mode. Item-level evaluation allows us to identify item characteristics that are prone to mode effects, including targeted cognitive skill, response type, and the amount of adaptation between modes. Implications for test design are discussed, and actionable design suggestions are offered with the goal of minimizing mode effect.

Chinese higher education is experiencing rapid development and growth. With tremendous resources invested in higher education, policy makers have requested more direct evidence of student learning. However, assessment tools that can be used to measure college-level learning are scarce in China. To mitigate this situation, we translated the critical thinking test from the ETS® Proficiency Profile (EPP) into Chinese. EPP has been widely used in the United States to assess general college learning outcomes. We pilot tested the EPP--Chinese test with students from a university in China. Results suggest that (a) the test is unidimensional and therefore is sufficient to report a total score from a practical standpoint; (b) the total score reliability is satisfactory; (c) most items showed moderate correlations with the total score, but the translation of one item needs additional revision; (d) the test is correlated with related constructs (e.g., the Chinese college entrance examination and a national English test); and (e) no item showed differential item functioning or was found to be biased toward any subgroup. In summary, the Chinese version of the critical thinking test showed potential as a suitable assessment tool for Chinese college students.

Critical thinking has been identified as a crucial general skill contributing to academic and career success in the twenty-first century. With the increasing demands of the modern workplace and a global trend of accountability in higher education, educators and employers pay great attention to the development of students' critical thinking skills throughout their training. Therefore, there is an urgent need worldwide for an updated and comprehensive assessment tool of college-level critical thinking. This paper reports on the preliminary validation for the Russian version of the "HEIghten"® Critical Thinking assessment developed by Educational Testing Service (ETS). Based on a large Russian college student sample (N = 1060), we evaluated the psychometric quality of the items, individual and institution-level reliability, external validity, and student perceptions. Overall, the results suggested good psychometric quality, except that a few items showed low discriminating power and should be further examined with a second wave of data collection. IRT analyses revealed testlet effects and supported the essentially unidimensional structure of the measure. Appropriate correlations with external criteria provided support for the measure's convergent validity. Implications of the preliminary validation study results and the future research agenda, especially the need to collect longitudinal data, are discussed.

Critical thinking is one of the most important skills deemed necessary for college graduates to become effective contributors in the global workforce. The first part of this article provides a comprehensive review of its definitions by major frameworks in higher education and the workforce, existing assessments and their psychometric qualities, and challenges surrounding the design, implementation, and use of critical thinking assessment. In the second part, we offer an operational definition that is aligned with the dimensions of critical thinking identified from the reviewed frameworks and discuss the key assessment considerations when designing a next-generation critical thinking assessment. This article has important implications for institutions that are currently using, planning to adopt, or designing an assessment of critical thinking.

Community colleges currently enroll about 44% of the undergraduate students in the United States and are rapidly expanding. It is of critical importance to obtain direct evidence of student learning to see if students receive adequate training at community colleges. This study investigated the 10-year trends of community college students' (n = 46,403) performance in reading, writing, mathematics, and critical thinking, as assessed by the ETS[TM] Proficiency Profile (EPP), an assessment of college-level learning outcomes. Results showed that community college students caught up with and significantly outperformed students from liberal arts colleges by the end of the 10-year period and made significant improvement in critical-thinking skills. An increasing gender gap was observed in mathematics at community colleges. Prevalent ethnic minority and English as a second language (ESL) gaps were noted but gaps between ESL and non-ESL students and between Hispanic and White students were decreasing. Additionally, Asian students at community colleges showed an overall decline in performance. Findings from this study provide significant implications for community college leaders, researchers, and policymakers.

Assessing student learning outcomes has become a global trend in higher education. In this paper, we report on the validation of the Chinese HEIghten® Critical Thinking assessment with a nationally representative sample of Electrical Engineering and Computer Science students from 35 institutions in China. Key findings suggest that there was a test delivery mode effect favoring the paper tests over the online tests. In general, the psychometric quality of the items was satisfactory for low-stakes, group-level uses but there were a few items with low discrimination that awaits further investigation. The relationships between test scores and various external variables such as college entrance examination scores, university elite status and student perceptions of the test were as expected. We conclude with speculations on the key findings and discussion of directions for future research.

This study examines learning gains of college students' performance in critical thinking, reading, writing, and mathematics as assessed by the ETS Proficiency Profile (EPP). In this study, students' college learning gain was estimated by calculating the score differences between their first and last test administrations. Results revealed that (a) after being in college for one or two years, students did not demonstrate significant learning gains, (b) after three or more years, students made small to moderate gains on the EPP total score, and reading and mathematics subscales, (c) after four or five years, students made small to moderate learning gains on EPP total score and all four subscales, and (d) among various demographic and college-level variables, college experience was the largest significant predictor of students' learning gain, followed by first-year GPA. Implications of these results are discussed.

Critical thinking is a learning outcome highly valued by higher education institutions and the workforce. The Educational Testing Service (ETS) has designed a next generation assessment, the HEIghten™ critical thinking assessment, to measure students' critical thinking skills in analytical and synthetic dimensions. This paper introduces the theoretical framework that guided the assessment design, and also reports on the preliminary validity evidence of the pilot data from over 3000 students from 35 two and four-year institutions. The critical thinking scores demonstrated satisfactory total and subscale reliabilities, were reasonably correlated with SAT scores, high school grade point average (GPA), and college GPA, and were able to detect cross-sectional performance difference between freshmen and seniors. In addition, most examinees reported having tried their best when taking the test. Results show that test-taking motivation has a significant impact on performance. We encourage institutions to pay attention to motivational issues in implementing low-stakes learning outcomes assessment such as the HEIghten™ critical thinking assessment.