Search

The aim of this study is to develop a theoretically sound, valid, and reliable scale to examine seventh and eighth grade students' attitudes towards so-called skill-based mathematics questions (SBMQ). The sample of the study was 820 seventh and eighth grade students at three different middle schools in Etimesgut District of Ankara Province. A draft item pool was crafted after a detailed literature review and the scale was finalized for the implementation after receiving expert opinions. This version of scale was administered shortly before the national examination which included skill-based questions in the 2022-2023 academic year. Exploratory and confirmatory factor analysis methods were used to examine the construct validity of the scale. The analysis resulted in 16 items in three factors (affective, cognitive, and motivational) which explained 46% of the total variance. Cronbach's alpha and composite reliability scores were above 0.70 for each factor and whole scale. The results indicated that the Attitudes Towards SBMQ Scale developed for seventh and eighth grade students is valid and reliable.

This study was designed to determine the psychometric properties of the School Readiness Assessment Tool (SRAT), a new measurement tool for assessing school readiness. The measure aims to assess preschool children's school readiness by estimating their academic, motor, social, emotional, and self regulation skills. Both academic and motor skills are assessed through direct child observation while teacher ratings evaluate social-emotional and self-regulation skills. 228 children attending preschool participated in the study's first phase and 185 children in the second phase. The results show that the items of the new measurement tool had sufficient internal consistency. Both study phases also showed strong evidence of face (content) and construct validity (i.e., convergent and concurrent validity). Overall, the results show that SRAT is a promising new measure for school readiness among preschoolers in Türkiye. In addition to the evidence from randomized control trials, studies should be conducted to determine its psychometric properties with diverse samples, such as rural children and children not enrolled in preschool education.

GCSE examinations (taken by students aged 16 years in England) are not intended to be speeded (i.e. to be partly a test of how quickly students can answer questions). However, there has been little research exploring this. The aim of this research was to explore the speededness of past GCSE written examinations, using only the data from scored responses to items from a sample of 340 GCSE components. Speededness was calculated as the average (mean) percentage marks lost from the longest string of unanswered items at the end of each student's examination paper. The potential impact of student ability on examination completion patterns was taken into account. The data suggested that most GCSEs analysed were unlikely to have been speeded. This method of exploring the speededness of exams using only scored responses has potential (although there are limitations), and it can flag potentially problematic components for further investigation.

Metacognition is one of the key learning skills in the 21st century, with a strong potential to help students succeed in science learning. Until now, this metacognitive awareness is less empowered by lecturers in learning. This study aimed to analyze the problem-based learning (PBL) reading-questioning-answering (PBLRQA) model's effect on metacognitive awareness in pre-service teachers. This research is a quasi-experiment with a pretest-posttest nonequivalent control group design. Determination of the research sample class was carried out by random sampling. Each learning model was represented by PBLRQA and PBL class. The number of classes used was 2, totaling 57 students. The instrument used was the metacognitive awareness inventory (MAI). This inventory comprised 52 statement items divided into planning, monitoring, evaluating, and revising skills. An important finding in this research is that students' metacognitive awareness in learning with PBL and PBLRQA is not significantly different. Thus, the PBLRQA and PBL models can still be used in students' science learning to encourage metacognitive awareness, so that academic success can be achieved.

This study aimed to develop a valid and reliable multiple-choice achievement test for the subject area of ecology. The study was conducted within the framework of exploratory sequential design based on mixed research methods, and the study group consisted of a total of 250 middle school students studying at the sixth and seventh grade level. In the process of preparing the question pool, the level and scope of the questions in the pool were shaped by taking into account the achievements of science, environmental education and climate change courses. A question pool consisting of questions obtained as a result of the literature review and prepared by the researcher was created. The pool of 41 questions was reduced to 29 questions in line with expert opinions. The 29-question multiple-choice test, whose language comprehensibility was examined and necessary arrangements were made, was made ready for implementation. The draft form of the achievement test, which was piloted, was applied to 250 students. Item difficulty indices and item discrimination indices of the test answers obtained from the students were calculated. After the item analysis, a total of 4 questions were removed from the test. As a result of the analyses, the average difficulty value of the test was calculated as 0.58 and the average discrimination value was calculated as 0.60. The KR-20 reliability coefficient of the developed test was 0.85 and Cronbach alpha reliability coefficient was 0.88. As a result of the research, a valid and reliable ecology achievement test was developed.

This study focuses on developing and validating a questionnaire to assess the integration of ICT by EFL teachers in higher education settings in Vietnam. Twenty EFL teachers from three language institutes participated in semi-structured interviews, generating 20 questionnaire items. Qualitative content analysis of the interview data provided insights into EFL teachers' perspectives on ICT integration. Additionally, a document analysis and literature review informed the questionnaire development. A total of 150 EFL teachers were selected to validate the questionnaire by completing the items. Exploratory Factor Analysis identified the underlying constructs of ICT integration. Grounded in the SAMR model, the investigation explored the perceived effectiveness of ICT and identified obstacles to optimal utilization in the EFL classroom. The findings contribute to understanding ICT integration in EFL teaching, informing improvements in language education practices at Vietnamese higher education institutions.

Given the significant global influence of the Common European Framework of Reference for Languages: Teaching, Learning, and Assessment (CEFR) on English language education, this study deals with aligning a university's academic reading tests to the CEFR. It aimed at validating the test construct of the academic reading tests in relation to the proficiency levels defined by the CEFR. The study employs two standard setting procedures outlined in the CEFR Manual: the Familiarization procedure and the Specification procedure, to explore the CEFR level of the academic reading tests as well as the prominent characteristics of the reading texts and the test items in terms of their level and key features. Three academic reading tests were randomly selected. The CEFR Content Analysis Grid for Reading was employed to characterize the content of test items and test tasks. The results indicated that 9 out of 18 reading texts were estimated to correspond to the B2 CEFR level. Texts estimated as B1 and C1 levels were evenly distributed, and none of the reading texts were classified as A1 or C2 levels. Moreover, the findings demonstrated a significant prevalence of B2 and B1 level items. Specifically, B2 items represented the largest proportion at 31.88%, closely followed by B1 items at 25.12%.

The purpose of this research is to determine whether PISA 2018 mathematical literacy test items show a differential item functioning across countries. For this purpose, only the items in booklet number three were examined using the MIMIC method with Latent Class Analysis (LCA) approach. PISA 2018 tests are mostly developed in English. Therefore, in DIF analyses, the reference group is the UK, while the focal groups consist of the other countries examined in the research (Türkiye, Finland, Japan, and the USA). According to the results, of the 23 test items, statistically significant DIF was observed in eight items in the UK-Türkiye sample, in seven items in the UK-Finland sample, in eleven items in the UK-Japan sample, and in three items in the UK-USA sample. It is seen that the effect and size of DIF in non-homogeneous groups differ between groups and these effects can be examined in more detail with the LCA method.

This study aimed to develop a valid and reliable measurement tool to determine the anxiety levels of parents with gifted children. The participants were selected using a convenient sampling method. The sample of this study consisted of 550 parents of gifted middle school students attending 12 Science and Art Centers throughout Turkey in the 2022-2023 academic year. This study utilized the "Anxiety Scale for Parents of Gifted Children," developed by the researchers in this study, which consisted of 30 items, to obtain quantitative data. A Personal Information Form was used for demographic information. The scale's validity was assessed through face, content, and construct validity, and its reliability was examined using Cronbach's alpha coefficient, item discrimination, and test-retest analyses. Expert opinion and a pilot study were conducted for face and content validity, whereas EFA and CFA were performed for construct validity. The data of 300 randomly selected participants (229 females and 71 males) were used for EFA, while the data of the remaining 250 participants (189 females and 61 males) were used for CFA. As a result of the EFA, eleven items were excluded, and CFA was conducted. The analysis yielded the following fit indices for the scale: RMSEA= 0.054, GFI= 0.912, CFI= 0.937, and IFI= 0.939. The Cronbach's alpha reliability coefficient was .890. Based on the analyses, a valid and reliable scale comprising 19 items and five factors was developed to assess the anxiety levels of parents with gifted children.

ChatGPT has surged interest to cause people to look for its use in different tasks. However, before allowing it to replace humans, its capabilities should be investigated. As ChatGPT has potential for use in testing and assessment, this study aims to investigate the questions generated by ChatGPT by comparing them to those written by a course instructor. To investigate this issue, this study involved 36 junior students who took a practice test including 20 multiple-choice items generated by ChatGPT and 20 others by the course instructor, resulting in a 40-item test. Results indicate that there was an acceptable degree of consistency between the ChatGPT and the course instructor. Post-hoc analyses point to consistency between the instructor and the chatbot in item difficulty, yet the chatbot's results were weaker in item discrimination power and distractor analysis. This indicates that ChatGPT can potentially generate multiple-choice exams similar to those of the course instructor.

This research aims to compare the ability and item parameter estimations of Item Response Theory according to Maximum likelihood and Bayesian approaches in different Monte Carlo simulation conditions. For this purpose, depending on the changes in the priori distribution type, sample size, test length, and logistics model, the ability and item parameters estimated according to the maximum likelihood and Bayesian method and the differences in the RMSE of these parameters were examined. The priori distribution (normal, left-skewed, right-skewed, leptokurtic, and platykurtic), test length (10, 20, 40), sample size (100, 500, 1000), logistics model (2PL, 3PL). The simulation conditions were performed with 100 replications. Mixed model ANOVA was performed to determine RMSE differentiations. The prior distribution type, test length, and estimation method in the differentiation of ability parameter and RMSE were estimated in 2PL models; the priori distribution type and test length were significant in the differences in the ability parameter and RMSE estimated in the 3PL model. While prior distribution type, sample size, and estimation method created a significant difference in the RMSE of the item discrimination parameter estimated in the 2PL model, none of the conditions created a significant difference in the RMSE of the item difficulty parameter. The priori distribution type, sample size, and estimation method in the item discrimination RMSE were estimated in the 3PL model; the a priori distribution and estimation method created significant differentiation in the RMSE of the lower asymptote parameter. However, none of the conditions significantly changed the RMSE of item difficulty parameters.

In an upcoming coverage-comprehension study, we plan to assess learners' meaning-recall knowledge of words as they occur in the study's reading passage. As several meaning-recall test formats exist, the purpose of this small-scale study (N = 10) was to determine which of three formats was most similar to a criterion interview regarding mean score and the consistency of correct/incorrect classifications (match rate, k = 30). In Test 1, the prompt consisted of only the target item, and a written translation of its meaning was elicited. In Test 2, the prompt was a short sentence in which a target item was highlighted, and a written translation of only that target item was requested. In Test 3, the prompt was the same sentence as in Test 2, but the target item was unhighlighted, and participants were requested to translate the entire sentence. Finally, in the criterion interview, participants were asked to demonstrate their understanding of the target items in the same prompt sentences as in Tests 2-3. The results indicated that Test 3 produced a mean score and match rate most similar to the interview, followed by Test 2, with Test 1 being the least similar. The paper discusses several factors explaining differences in test performance that were explored during the interview.

The Vocabulary Size Test (VST) measures English learners' decontextualised receptive vocabulary knowledge of written English and has nine bilingual versions with multiple-choice options written in other languages. This study used the English-Japanese version of the VST to investigate the extent to which loanword items were answered correctly by Japanese first language (L1) university students compared to non-loanword items, and whether it was easier to answer these loanword items when the correct answer option was written in loanwords rather than Japanese-words. Paired t-tests showed a significant difference in correct response rates between the loanword and non-loanword items, and the loanword options and Japanese-word options, with a large effect size. The results suggest the relative ease of learning English loanwords compared to non-loanwords for L1 Japanese users, and the need to consider the use of loanwords in vocabulary tests to measure test-takers' vocabulary size more accurately.

The present study has aimed to develop and validate a protein concept inventory (PCI) consisting of 25 multiple-choice (MC) questions to assess students' understanding of protein, which is a fundamental concept across different biology disciplines. The development process of the PCI involved a literature review to identify protein-related content, validation interviews to iteratively validate and refine the created items (n = 26), and data collection from a large sample (n = 291) for statistical analysis. An expert interview was held with two different field experts regarding the content validity of the draft PCI tool, the suitability of the options, and the clarity of the items. Free choice format (multiple marking) was used to answer the developed MC items. In scoring these items, positive points were given to correct options, and negative points were given to incorrect options. Evidence regarding the psychometric properties of the PCI trial form was collected through factor analysis, group differentiation, internal consistency, and item analysis using quantitative data. The evidence collected demonstrates that the validity and reliability of the PCI as a measurement tool have been confirmed. PCI's scoring approach and the use of response patterns created by multiple markings in teaching are discussed.

In this study, it was aimed to develop an achievement test taking into account the subject acquisitions of the sound and properties unit in the sixth-grade science course. In the test development phase, firstly, literature review for the study was conducted. Then, 30 multiple choice questions in align with the subject acquisition in the 2018 science curriculum. This 30-question test was presented to opinion of three academicians and a science teacher for the validity of the test. Necessary adjustments were made in line with the opinions of the experts. Subsequently, the draft test, modified according to the experts' feedback, was applied to a total of 300 seventh grade students. After the analysis, the number of questions in the test was reduced to 27. As a result of item analysis, mean difficulty index value of the test was 0.41, and item discrimination index was 0.49. The reliability analysis for the developed test was found 0.92 by calculating the KR-20 reliability coefficient value. By considering, the validity and reliability analysis results, it was concluded that the final version of the developed test, grounded in Bloom's Taxonomy, is a valid and reliable test with different difficulty levels.

Engineering education aims to equip children with the skills to solve and apply complex problems. Problem-solving processes in engineering require high-level thinking and mind habits. Habit is a term used to describe various aspects of intelligence. Engineering habits of mind are the values, attitudes, and thinking skills associated with engineering. This research aimed to develop a scale to assess the engineering habits of mind of children ages 5-8. The study involved 417 children in two provinces in the southwest and northwest of Türkiye. We performed Explanatory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) for the scale's construct validity. The scale consisted of 35 items and six factors and explained 59.2% of the total variance. We called the factors "system thinking," "creativity," "optimism," "collaboration," "communication," and "attention to ethical considerations." According to the CFA result, the construct we obtain is reasonable (X[superscript 2]/sd= 1.97, RMSEA= 0.068, CFI= 0.93, TLI= 0.91, SRMR= 0.062). The item total correlations range from 0.43 to 0.66. We determined the Cronbach Alpha coefficient of the scale to be 0.94 and the test re-test reliability to be 0.87. According to the results, it can be said that the scale obtained can be used validly and reliably to determine the engineering habits of mind children according to teacher reports. The study contributes to the ever-increasing engineering habits of mind literature.

This study focuses on developing a five-tier chemical diagnostic test based on a computer-based test with 11 assessment categories with an assessment score from 0 to 10. A total of 20 items produced were validated by education experts, material experts, measurement experts, and media experts, and an average index of the Aiken test > 0.70 was obtained. The validation results were tested on 580 respondents and analyzed using the Generalized Partial Credit Model (GPCM) Item Response Theory (IRT) type. The results of the analysis show that all of the items meet the requirements to be said to be valid for the model; the evidence of the value this: RMSEA < 0.08, CFI > 0.87, SRMR < 0.10, GFI > 0.90, NFI > 0.90, NNFI > 0.90, IFI > 0.90, TLI > 0.90, and RFI > 0.90, and all items were obtained has a p.S_X2 value greater than 0.05 which indicates that all items developed are fit and by the GPCM model. The construct reliability (CR) value is 0.99, which suggests the construct is reliable. The most challenging item is item 9, and the most accessible item is item 4. [Note: The page range (91-105) shown on the PDF citation is incorrect. The correct page range is 92-106.]

Biology education plays a vital role in nurturing the understanding of learners about the intricacy of life. Various efforts have emerged to strengthen learning biological concepts but there were still studies that showed that learners have low mastery in some aspects. To determine how well students understood various biological topics, including human physiology, Concept inventory tests (CIT) were used. The concept inventory test may be able to spot students' misconceptions and ultimately lead to improved comprehension. The crafted CIT developed with the aid of a table of specifications based on Bloom's taxonomy of cognitive domain was assessed according to its validity and reliability. In validation, content validity and item analysis were considered while reliability test was employed through Cronbach's alpha. Distractor analysis was also performed to determine possible source of misconception per item. The CIT was administered to 120 senior high school STEM students (50.8% from the private schools, 37.5% in regular public schools and 11.7% from public schools with special programs in science). The results displayed high content validity with a mean of 4.83 for content validity and an average Aiken's validity coefficient of 0.98. It also highlighted that the test is moderately difficult with the test difficulty of 0.58, as well as, discriminatory with a discriminating level of 0.46. After item classification, 63 items were retained (39 accepted, 24 for revisions) and Chronbach's alpha ([alpha]=0.74) indicated good internal consistency. The concept inventory test propounds to be a good classroom test science with minor items to be revised.

The purpose of this study was to examine gender and discipline-based Differential Item Functioning (DIF) and Differential Distractor Functioning (DDF) on the Islamic Azad University English Proficiency Test (IAUEPT). The study evaluated DIF and DDF across genders and disciplines using the Rasch model. To conduct DIF and DDF analysis, the examinees were divided into two groups: Humanities and Social Sciences (HSS) and Non-Humanities and Social Sciences (N-HSS). The results of the DIF analysis showed that four out of 100 items had DIF across gender, and two items had discipline DIF. Additionally, gender DDF analysis identified one item each for Options A, B, and C, and four items for Option D. Similarly, the discipline DDF analysis revealed one item for Option A, three items for Option B, four items for Option C, and three items for Option D. The findings of this study have significant implications for test developers. The identification of potential biases in high-stakes proficiency tests can help ensure fairness and equity for all examinees. Furthermore, identifying gender DIF can shed light on potential gender-based gaps in the curriculum, highlighting areas where male or female learners may be disadvantaged or underrepresented in terms of knowledge or skills.

Language proficiency diagnosis is essential to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Different from comprehensive standardized tests, many language learning apps often revolve around word-level questions. Therefore, knowledge concepts and linguistic skills are hard to define, and diagnosis must be well-designed. Traditional approaches are widely applied for modeling knowledge in science or mathematics, where skills or knowledge concepts are easy to associate with each item. However, only a few works focus on defining knowledge concepts and skills using linguistic characteristics for language knowledge proficiency diagnosis. In addressing this, we propose a framework for language proficiency diagnosis based on neural networks. Specifically, we propose a series of methods based on our framework that uses different linguistic features to define skills and knowledge concepts in the context of the language learning task. Experimental results on a real-world second-language learning dataset demonstrate the effectiveness and interpretability of our framework. We also provide empirical evidence with comprehensive experiments and analysis to prove that our knowledge concept and skill definitions are reasonable and critical to the performance of our model.

The link between a student's math fluency and their success in general chemistry has been thoroughly documented in the literature. One diagnostic instrument that can be used to assess a student's arithmetic skills is the Math-Up Skills Test (MUST), a 20-question, free-response math test completed in 15 min. The MUST instrument assesses the student's ability to conduct mathematical operations, including multiplication, division, fraction simplification, logarithms/exponents, and symbol manipulation without the aid of a calculator. In our study, we looked at how score changes per MUST question are affected by an early online math review implemented in a general chemistry class versus the control group that had an early online chemistry review across students of varying logical ability as measured by the online Test of Logical Thinking (TOLT). The Math Group significantly improved on ten questions involving all five types of mathematical operations, including one question in symbol manipulation involving balancing equations. The control Chemistry Group only increased on three questions involving faction simplification and logarithms/exponents beyond the two symbol manipulation questions on balancing equations, which were covered in their review of matter. As a function of logic ability, middle TOLT students benefited the most from the math intervention, as evidenced by a greater number of improved MUST questions post-treatment for that group compared to those for the low and high TOLT groups. Identification of item types affected by the math treatment could help instructors identify key arithmetic topics crucial to a student's performance, among many other factors, in first-semester general chemistry.

Mixed-mode surveys are popular as they can save costs and maintain (or improve) response rates relative to single-mode surveys. Nevertheless, it is not yet clear how design decisions like survey mode or questionnaire length impact measurement quality. In this study, we compare measurement quality in an experiment of three distinct survey designs implemented in the German sample of the European Values Study: a single-mode face-to-face design, a mixed-mode mail/web design, and a shorter (matrix) questionnaire in the mixed-mode design. We compare measurement quality in different ways, including differences in distributions across several data quality indicators as well as equivalence testing over 140 items in 25 attitudinal scales. We find similar data quality across the survey designs, although the mixed-mode survey shows more item nonresponse compared to the single-mode survey. Using equivalence testing we find that most scales achieve metric equivalence and, to a lesser extent, scalar equivalence across the designs.

Topic models are mathematical and statistical models used to analyze textual data. The objective of topic models is to gain information about the latent semantic space of a set of related textual data. The semantic space of a set of textual data contains the relationship between documents and words and how they are used. Topic models are becoming more common in educational measurement research as a method for analyzing students' responses to constructed-response items. Two popular topic models are latent semantic analysis (LSA) and latent Dirichlet allocation (LDA). LSA uses linear algebra techniques, whereas LDA uses an assumed statistical model and generative process. In educational measurement, LSA is often used in algorithmic scoring of essays due to its high reliability and agreement with human raters. LDA is often used as a supplemental analysis to gain additional information about students, such as their thinking and reasoning. This article reviews and compares the LSA and LDA topic models. This article also introduces a methodology for comparing the semantic spaces obtained by the two models and uses a simulation study to investigate their similarities.

Conceptualizing two-variable disturbances preventing good model fit in confirmatory factor analysis as item-level method effects instead of correlated residuals avoids violating the principle that residual variation is unique for each item. The possibility of representing such a disturbance by a method factor of a bifactor measurement model was investigated with respect to model identification. It turned out that a suitable way of realizing the method factor is its integration into a fixed-links, parallel-measurement or tau-equivalent measurement submodel that is part of the bifactor model. A simulation study comparing these submodels revealed similar degrees of efficiency in controlling the influence of two-variable disturbances on model fit. Perfect correspondence characterized the fit results of the model assuming correlated residuals and the fixed-links model, and virtually also the tau-equivalent model.

Exceptionally talented refugee students are often underrepresented in allocating to gifted programs because of inadequate identification methods in Arab countries. This study investigates the Arabic version of the Having Opportunities Promotes Excellence (HOPE) Scale for identifying gifted refugee students. Students (n = 13,598) from refugee camp schools in Jordan were surveyed in the 2020/2021 academic year. Elementary, middle, and high school teachers (n = 423) completed the translated Arabic version of the HOPE Scale, measuring 11 items on the academic and social construct of giftedness. Confirmatory factor analysis indicated an excellent fit; however, multigroup confirmatory factor analysis results revealed that the scale needed more invariance across groups. The results suggest that although the HOPE Scale--Arabic version has sufficient validation evidence, careful consideration is required when applying it to different subgroups of refugee students. Constructs of cultural relevance may need to be added to better assess the item's validity.

This study aimed to develop a parent rating scale (PRSG) for screening children for further identification process in terms of giftedness. The participants of the study were 255 parents of gifted and non-gifted students. The PRSG, consisting of 30 items, was created by consulting parents and reviewing instruments existent in the literature. As part of the validity testing, the content, construct, and criterion-related validities were examined. Expert opinion was sought for content validity. Construct validity was achieved as the findings of the confirmatory factor analysis confirmed the three-factor model in the 27-item instrument. The parents rated their own children after the researchers showed them how to rate their children. One hundred and sixty parents had a gifted child. Finally, the scores given by the parents of gifted children and those of the non-gifted were compared, which yielded a statistically significant difference between the mean scores in favor of the scores given by the parents of the gifted. The Cronbach alpha value was found to be .95 for the whole instrument.

The purpose of this study was to (a) examine the underlying assessment structure of the Derivational Morphology Task (DMORPH) and (b) investigate the relation of the DMORPH to vocabulary and reading comprehension outcomes with a sample of struggling adult readers. Specifically, participants included 218 struggling adult readers enrolled in adult literacy classes. We used item-level analyses to evaluate the underlying structure of the DMORPH. Items with phonological (e.g., "music" to "musician") and non-phonological transformations ("teach" to "teacher") were examined in relation to adult literacy students' vocabulary and reading comprehension skills. A bifactor model was the best fit to the data, suggesting that the DMORPH measured a single factor of derivational morphological awareness with some variation due to phonological and non-phonological change items. Follow-up analyses revealed that the DMORPH can essentially be considered unidimensional, which justified the use of a single scoring system for the DMORPH with adult literacy students. However, after controlling for word reading and phonological awareness, the phonological change items uniquely predicted vocabulary knowledge and reading comprehension, whereas the non-phonological change items were not significant. The results support the structural validity of the DMORPH and the need to use both phonological and non-phonological change items with adult literacy students. The present findings also provide insight into potential intervention targets for instructors in adult literacy programs who are interested in improving students' vocabulary and reading comprehension skills.

Addressing the participation rates in senior secondary mathematics courses in Australian schools remains a critically important issue. In this paper, the authors report on quantitative findings from a study in which all year 11 and year 12 (aged 17-18 years) Australian Tertiary Admissions Ranking (ATAR) students in Western Australia were invited to participate. The aim was to explore the perceptions of these students regarding their enrolment in higher-level mathematics courses. Data from 1633 students were collected using a survey instrument comprised of 12, 5-point, Likert-scale items. Data were analysed by applying two statistical procedures: calculating frequencies of the 12 items in the scale and examining associations with demographic characteristics and individual items through generalised linear modelling. Analyses indicated most students agreed that other courses of study were more attractive, with almost half indicating that they did not like mathematics. In addition, approximately half of the students said that they did not need to enrol in any mathematics course for ATAR or for university entrance. Significant findings were also identified for gender, school type, and school gender. The knowledge gained from this research is valuable in understanding students' reasons for choosing not to enrol in higher-level mathematics courses when they have the option to do so and, more broadly, to address persistently low or declining participation rates in these areas of study.

Background: Valid and reliable instruments for measuring emotional development are critical for a proper diagnostic assignment in individuals with intellectual disabilities. This exploratory study examined the psychometric properties of the items on the Scale of Emotional Development--Short (SED-S). Method: The sample included 612 adults with intellectual disabilities (M[subscript age = 37.35, SD[subscript age] = 13.27; 59.8% males). Item validity analysis comprising sensitivity and specificity rates and discriminatory power were determined. Results: The relative mean frequency of 'yes' answers to all 200 items was 29.5%. The mean sensitivity rate was 67.5% and the mean specificity rate was 79.3%. Most items (85.0%) showed good discriminatory power with the adjacent stage(s), especially between SED-1, SED-2, SED-3 and SED-4. Particularly in SED-4 some items showed weaknesses in the differentiation between these stages. Discussion: This study adds to previous validation studies by showing that most SED-S items have psychometrically sound properties.

Assessing students' (mis)conceptions is a challenging task for teachers as well as for researchers. While individual assessment, for example through interviews, can provide deep insights into students' thinking, this is very time-consuming and therefore not feasible for whole classes or even larger settings. For those settings, automatically evaluated multiple-choice (MC) items could be a solution. However, it is a challenge to design those items and to adapt them for other countries in a way that they adequately reveal students' (mis)conceptions. In this article, we investigate the question whether it is valid to use a German adaption of a multiple-choice test developed in Australia for formative assessment of the "letter-as-object" misconception in Germany. For this, first semi-structured interviews with five German Year 8 students were conducted, and second, 616 students were asked for short written explanations. These data were analysed with regards to the students' (mis)conceptions and compared with their automatic online diagnosis. In general, a high concordance between online SMART test results and students' explanations was observed, confirming that useful diagnoses of student misconceptions can be obtained from such a short well-designed MC test.

Quality early childhood care and education (ECCE) is important for young children's holistic healthy development. As ECCE scales, contextually relevant and feasible measurement is needed to inform policy and programs on strengths and areas for improvement. However, few measures have been designed for use across diverse contexts. Drawing on principles of mixed methods design, this study reports on a new approach to ECCE quality measurement: the Brief Early Childhood Quality Inventory. Using data from the USA, Liberia, and Colombia, results indicate variation in the items perceived as highly relevant to each setting and in the characteristics of classrooms including the degree of child autonomy, the types of activities, and in child/educator interactions and dialogue. However, despite this variation, a small set of items indicate potential functionality as cross-country anchor items. Findings lend support to the idea that quality measures can have some common elements with room for adaptation within and across settings. Future work in this area should address the possibility that the significance of these practices for child development also varies across settings.

This study examined the preliminary feasibility, acceptability, and efficacy of an autism-adapted cognitive behavioral therapy for depression in autistic youth, CBT-DAY. Twenty-four autistic youth (11-17 years old) participated in the pilot non-randomized trial including 5 cisgender females, 14 cisgender males, and 5 non-binary youth. Youth participated in 12 weeks of, CBT-DAY and youth depressive symptoms (i.e., primary clinical outcome) and emotional reactivity and self-esteem (i.e., intervention mechanisms) were assessed through self-report and caregiver report at four timepoints: baseline (week 0), midpoint (week 6), post-treatment (week 12), and follow-up (week 24). Results suggested that CBT-DAY may be feasible (16.67% attrition) in an outpatient setting and acceptable to adolescents and their caregivers. Bayesian linear mixed-effects models showed that CBT-DAY may be efficacious in targeting emotional reactivity [[beta][subscript T1-T3] = -2.53, CrI[subscript 95%] (-4.62, -0.58), P[subscript d] = 0.995, d = -0.35] and self-esteem [[beta][subscript T1-T3] = -3.57, CrI[subscript 95%] (-5.17, -2.00), P[subscript d] > 0.999, d = -0.47], as well as youth depressive symptom severity [[beta] = -2.72, CrI[subscript 95%] (-3.85, -1.63), P[subscript d] > 0.999]. Treatment gains were maintained at follow-up. A cognitive behavioral group therapy designed for and with autistic people demonstrates promise in targeting emotional reactivity and self-esteem to improve depressive symptom severity in youth. Findings can be leveraged to implement larger, more controlled trials of CBT-DAY. The trial was registered at Clinicaltrials.gov (Identifier: NCT05430022; https://beta.clinicaltrials.gov/study/NCT05430022).

Particularly for adolescent students, tools for measuring their experience of negatively consequential aspects of student-teacher relationships are limited even though evidence shows that keeping this group motivated in the classroom is especially challenging. Therefore, using basic psychological needs theory, this paper proposes a conceptualization of adolescent classroom psychological need thwarting by advancing a set of items to measure and empirically test the phenomenon. The outcome of this process is a new scale with psychometric properties that support its validity and usefulness for understanding an important dimension of adolescent classroom relational experiences. Specifically, empirical findings confirm the reliability and a priori hypothesized specification for the scale. Further, the new scale has the expected correlations with an external theoretically-related construct and demonstrates good measurement invariance. Together, the development of this type of instrument opens up the opportunity for more research, especially focusing on how the relational structure of the learning environment affects adolescent students.

Descriptive fit indices that do not require a formal statistical basis and do not specifically depend on a given estimation criterion are useful as auxiliary devices for judging the appropriateness of unrestricted or exploratory factor analytical (UFA) solutions, when the problem is to decide the most appropriate number of common factors. While overall indices of this type are well known in UFA applications, especially those intended for item analysis, difference indices are much more scarce. Recently, Raykov and collaborators proposed a family of effect-size-type descriptive difference indices that are promising for UFA applications. As a starting point, we considered the simplest measure of this family, which (a) can be viewed as absolute and (b) from which only tentative cutoffs and reference values have been provided so far. In this situation, this article has three aims. The first is to propose a relative version of Raykov's effect-size measure, intended to be used as a complement of the original measure, in which the increase in explained common variance is related to the overall prior estimated amount of common factor variance. The second is to establish reference values for both indices in item-analysis scenarios using simulation. And the third aim (instrumental) is to implement the proposal in both R language and a well-known non-commercial factor analysis program. The functioning and usefulness of the proposal is illustrated using an existing empirical dataset.

Thurstonian forced-choice modeling is considered to be a powerful new tool to estimate item and person parameters while simultaneously testing the model fit. This assessment approach is associated with the aim of reducing faking and other response tendencies that plague traditional self-report trait assessments. As a result of major recent methodological developments, the estimation of normative trait scores has become possible in addition to the computation of only ipsative scores. This opened up the important possibility of comparisons between individuals with forced-choice assessment procedures. With item response theory (IRT) methods, a multidimensional forced-choice (MFC) format has also been proposed to estimate individual scores. Customarily, items to assess different traits are presented in blocks, often triplets, in applications of the MFC, which is an efficient form of item presentation but also a simplification of the original models. The present study provides a comprehensive review of the present status of Thurstonian forced-choice models and their variants. Critical features of the current models, especially the block models, are identified and discussed. It is concluded that MFC modeling with item blocks is highly problematic and yields biased results. In particular, the often-recommended presentation of blocks with items that are keyed in different directions of a trait proves to be counterproductive considering the goal to reduce response tendencies. The consequences and implications of the highlighted issues are further discussed.

While several test concordance tables have been published, the research underpinning such tables has rarely been examined in detail. This study aimed to survey the publically available studies or documentation underpinning the test concordance tables of the providers of four major international language tests, all accepted by the Australian Department of Home Affairs for Australian visa purposes. To evaluate the concordance studies, we first identified the good practice principles in concordance research through a review of both the relevant literature and leading professional standards in the field of educational measurement and language assessment. Next, we reviewed the concordance studies against the identified good practice principles. Our findings revealed that the information supplied by test providers varied, with some making the full research papers available, whereas others providing little information about their underpinning research. None of the concordance studies fulfilled all the good practice principles. Based on the findings of this study, we offer recommendations for future concordance research in the field of language testing as well as suggestions for practice.

Researchers and practitioners in the field of second language acquisition have come to realize the importance of non-cognitive skills such as self-efficacy and self-regulation in students' learning of a second language. However, there has been limited systematic research on such measures in the second language context and the validity and reliability of their intended use and interpretation. The Questionnaire of English Self-Efficacy (QESE) was designed to measure English language learners' self-efficacy beliefs. This study applied the Rasch model to evaluate the item-level psychometric properties of QESE and its adequacy for use in the Vietnamese context. The results demonstrated that the QESE items predominantly measured a unidimensional construct (i.e. self-efficacy beliefs in learning English as a second language). The scale was highly reliable, and its rating categories functioned effectively, reflecting varying levels of self-efficacy beliefs. Although the item difficulty hierarchical order was generally consistent with the anticipated hierarchy within the Vietnamese context, there were a few surprising results, contradicting what was previously found with Chinese students. For example, item 28 (writing diaries in English) was a very difficult task for Vietnamese students, but a previous study showed that it was a moderately difficult task for Chinese students. Item 5 (writing blogs on the Internet) was an easy task for Vietnamese students, but it was a difficult task for Chinese students. The findings provide directions for future research and have implications for helping professionals provide best practices that meet the unique educational needs of English language learners.

Quality of life (QOL) and life satisfaction are important research priorities for autistic adults. As such, we saw a need to evaluate individual items of commonly used subjective QOL scales to understand how they are interpreted and perceived by autistic adults. This study used cognitive interviews and repeated sampling to evaluate the accessibility, test-retest reliability and internal consistency of several common QOL measures in a sample of young autistic adults (n = 20; aged 19-32). Cognitive interviews suggested that the Satisfaction with Life Scale was well understood and demonstrated excellent internal consistency and test-retest reliability. While the WHOQoL-BREF and WHOQoL Disability Modules had adequate reliability, cognitive interviews suggested that additional instructions and examples would further enhance their accessibility for use with autistic adults.

To revise the Parental Autonomy Support Scale (PASS) and to validate its psychometric properties in Chinese college students, this study recruited 738 Chinese college students, with 72 of them retested four weeks later. The Parental Psychological Control Scale (PPCS) and the Self-Esteem Scale (RSES) were used to evaluate the validity of the PASS. Results showed that the revised PASS contained a total of nine items measuring three dimensions: autonomous decision-making, transpositional thinking, and willingness to respect. Results showed that the three-factor ESEM model fitted the data well (TLI = 0.981, CFI = 0.994, RMSEA = 0.046). The PASS showed a significant negative correlation between the total score of the PASS and the total score of the PPCS (r = -0.344, p < 0.001), and a significant positive correlation of the PASS with the total score of the RSES (r = 0.297, p < 0.001). The composite reliability coefficient was 0.913 for the total score. The test-retest reliability coefficients were 0.827 for the total score and ranged from 0.626 to 0.744 for the three dimensions. The revised PASS is shown to have sound psychometric properties and thus can be used as a tool to measure the level of parental autonomy support among Chinese college students.

A modified and improved inductive inferential approach to evaluate item discriminations in a conditional maximum likelihood and Rasch modeling framework is suggested. The new approach involves the derivation of four hypothesis tests. It implies a linear restriction of the assumed set of probability distributions in the classical approach that represents scenarios of different item discriminations in a straightforward and efficient manner. Its improvement is discussed, compared to classical procedures (tests and information criteria), and illustrated in Monte Carlo experiments as well as real data examples from educational research. The results show an improvement of power of the modified tests of up to 0.3.

Given the frequent presence of slipping and guessing in item responses, models for the inclusion of their effects are highly important. Unfortunately, the most common model for their inclusion, the four-parameter item response theory model, potentially has severe deficiencies related to its possible unidentifiability. With this issue in mind, the dyad four-parameter normal ogive (Dyad-4PNO) model was developed. This model allows for slipping and guessing effects by including binary augmented variables--each indicated by two items whose probabilities are determined by slipping and guessing parameters--which are subsequently related to a continuous latent trait through a two-parameter model. Furthermore, the Dyad-4PNO assumes uncertainty as to which items are paired on each augmented variable. In this way, the model is inherently exploratory. In the current article, the new model, called the Set-4PNO model, is an extension of the Dyad-4PNO in two ways. First, the new model allows for more than two items per augmented variable. Second, these item sets are assumed to be fixed, that is, the model is confirmatory. This article discusses this extension and introduces a Gibbs sampling algorithm to estimate the model. A Monte Carlo simulation study shows the efficacy of the algorithm at estimating the model parameters. A real data example shows that this extension may be viable in practice, with the data fitting a more general Set-4PNO model (i.e., more than two items per augmented variable) better than the Dyad-4PNO, 2PNO, 3PNO, and 4PNO models.

Engineering lacks quantitative measures of racism and sexism in engineering graduate education. Racial and gendered disparities in participation, persistence, and experience combined with qualitative evidence indicate that discrimination interferes with marginalized students' well-being and degree completion. The Discrimination in Engineering Graduate Education (DEGrE) scale helps researchers identify environmental gender and race/ethnicity-based discrimination in engineering graduate education. The scale development and initial assessments reported here demonstrate the validity, internal reliability, and utility of the DEGrE subscales. The DEGrE scale will assist engineering colleges and departments evaluate their graduate students' experiences to guide decision makers in identifying changes needed in their graduate programs to improve equity and inclusion of marginalized students.

Identifying items with differential item functioning (DIF) in an assessment is a crucial step for achieving equitable measurement. One critical issue that has not been fully addressed with existing studies is how DIF items can be detected when data are multilevel. In the present study, we introduced a Lord's Wald X[superscript 2] test-based procedure for detecting both uniform and non-uniform DIF with polytomous items in the presence of the ubiquitous multilevel data structure. The proposed approach is a multilevel extension of a two-stage procedure, which identifies anchor items in its first stage and formally evaluates candidate items in the second stage. We applied the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm to estimate multilevel polytomous item response theory (IRT) models and to obtain accurate covariance matrices. To evaluate the performance of the proposed approach, we conducted a preliminary simulation study that considered various conditions to mimic real-world scenarios. The simulation results indicated that the proposed approach has great power for identifying DIF items and well controls the Type I error rate. Limitations and future research directions were also discussed.

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's information criterion (DIC), sample size adjusted BIC (SABIC), relative entropy, the integrated classification likelihood criterion (ICL-BIC), the adjusted Lo-Mendell-Rubin (LMR), and Vuong-Lo-Mendell-Rubin (VLMR). The accuracy of the fit indices was assessed for correct detection of the number of latent classes for different simulation conditions including sample size (2,500 and 5,000), test length (15, 30, and 45), mixture proportions (equal and unequal), number of latent classes (2, 3, and 4), and latent class separation (no-separation and small separation). Simulation study results indicated that as the number of examinees or number of items increased, correct identification rates also increased for most of the indices. Correct identification rates by the different fit indices, however, decreased as the number of estimated latent classes or parameters (i.e., model complexity) increased. Results were good for BIC, CAIC, DIC, SABIC, ICL-BIC, LMR, and VLMR, and the relative entropy index tended to select correct models most of the time. Consistent with previous studies, AIC and AICc showed poor performance. Most of these indices had limited utility for three-class and four-class mixture 3PL model conditions.

According to faking models, personality variables and faking are related. Most prominently, people's tendency to try to make an appropriate impression (impression management; IM) and their tendency to adjust the impression they make (self-monitoring; SM) have been suggested to be associated with faking. Nevertheless, empirical findings connecting these personality variables to faking have been contradictory, partly because different studies have given individuals different tests to fake and different faking directions (to fake low vs. high scores). Importantly, whereas past research has focused on faking by examining test scores, recent advances have suggested that the faking process could be better understood by analyzing individuals' responses at the item level (response pattern). Using machine learning (elastic net and random forest regression), we reanalyzed a data set (N = 260) to investigate whether individuals' faked response patterns on extraversion (features; i.e., input variables) could reveal their IM and SM scores. We found that individuals had similar response patterns when they faked, irrespective of their IM scores (excluding the faking of high scores when random forest regression was used). Elastic net and random forest regression converged in revealing that individuals higher on SM differed from individuals lower on SM in how they faked. Thus, response patterns were able to reveal individuals' SM, but not IM. Feature importance analyses showed that whereas some items were faked differently by individuals with higher versus lower SM scores, others were faked similarly. Our results imply that analyses of response patterns offer valuable new insights into the faking process.

In recent years, some models for binary and graded format responses have been proposed to assess unipolar variables or "quasi-traits." These studies have mainly focused on clinical variables that have traditionally been treated as bipolar traits. In the present study, we have made a proposal for unipolar traits measured with continuous response items. The proposed log-logistic continuous unipolar model (LL-C) is remarkably simple and is more similar to the original binary formulation than the graded extensions, which is an advantage. Furthermore, considering that irrational, extreme, or polarizing beliefs could be another domain of unipolar variables, we have applied this proposal to an empirical example of superstitious beliefs. The results suggest that, in certain cases, the standard linear model can be a good approximation to the LL-C model in terms of parameter estimation and goodness of fit, but not trait estimates and their accuracy. The results also show the importance of considering the unipolar nature of this kind of trait when predicting criterion variables, since the validity results were clearly different.

The forced-choice response format is often considered superior to the standard Likert-type format for controlling social desirability in personality inventories. We performed simulations and found that the trait information based on the two formats converges when the number of items is high and forced-choice items are mixed with regard to positively and negatively keyed items. Given that forced-choice items extract the same personality information as Likert-type items do, including socially desirable responding, other means are needed to counteract social desirability. We propose using evaluatively neutralized items in personality measurement, as they can counteract social desirability regardless of response format.

The Big Three Perfectionism Scale (BTPS) was created to integrate different aspects of perfectionism, including the newly conceptualized concept of narcissistic perfectionism. The goal of our two studies (N = 1341) was to examine the psychometric properties of the Polish adaptation of the BTPS, supporting the validity and portability of the measure. The studies were conducted among people who had a child, thus contributing to a better understanding of parental perfectionism, one of the key factors influencing parental well-being and a child's functioning. Our analyses included investigating the structure of the scale, intercorrelations between subscales, reliability, and convergent validity by correlating BTPS scores with other measures of perfectionism and correlates of psychopathology (borderline symptoms) and parental difficulties (parental stress and parental burnout). Results supported the structure of the original BTPS. As predicted, confirmatory factor analysis indicated that items comprising the Polish adaptation of the questionnaire, like the original version, measure three related but specific aspects of perfectionism: rigid perfectionism, self-critical perfectionism, and narcissistic perfectionism. The three dimensions were also found to be specifically related to the difficulties experienced by parents. Further, the Polish version of the BTPS was found to have good internal reliability and validity. Our results from two independent Polish samples suggest that the Polish version of the BTPS is a psychometrically robust measure of perfectionism for assessing the three perfectionism factors.

Computerized assessments and interactive simulation tasks are increasingly popular and afford the collection of process data, i.e., an examinee's sequence of actions (e.g., clickstreams, keystrokes) that arises from interactions with each task. Action sequence data contain rich information on the problem-solving process but are in a nonstandard, variable-length discrete sequence format. Two methods that directly extract features from the raw action sequences, namely multidimensional scaling and sequence-to-sequence autoencoders, produce multidimensional numerical features that summarize original sequence information. This study explores the utility of action sequence features in understanding how problem-solving behavior relates to cognitive proficiencies and demographic characteristics. This is empirically illustrated with the process data from the 2012 PIAAC PSTRE digital assessment. Regularized regression results showed that action sequence features are more predictive of examinees' demographic and cognitive characteristics compared to final outcomes. Partial least squares analysis further aided the identification of behavioral patterns systematically associated with demographic/cognitive characteristics.