Search

The aim of the current research was to provide recommendations to facilitate the development and use of anchoring vignettes (AVs) for cross-cultural comparisons in education. Study 1 identified six factors leading to order violations and ties in AV responses based on cognitive interviews with 15-year-old students. The factors were categorized into three domains: varying levels of AV format familiarity, differential interpretations of content, and individual differences in processing information. To inform the most appropriate approach to treat order violations and re-scaling method, Study 2 conducted Monte Carlo simulations with the manipulation of three factors and incorporation of five responses styles. Study 2 found that the AV approach improved accuracy in score estimation by successfully controlling for response styles. The results also revealed the reordering approach to treat order violations produced the most accurate estimation combined with the re-scaling method assigning the middle value among possible scores. Along with strategies to develop AVs, strengths and limitations of the implemented nonparametric AV approach were discussed in comparison to item response theory modeling for response styles.

Cultural mismatch theory of inequality and associated research indicates that a mismatch between independent cultural norms of institutions of higher education and students' interdependent backgrounds account for disparities in academic achievement for students from historically underrepresented groups. Despite growing interest in interventions to reduce cultural mismatch, there was not yet a comprehensive measure of cultural match. We developed and validated the Cultural Match Scale (CMS) composed of 24 Likert-scale items. The CMS measures Navigation of Institutional Culture, Connection to Community, and Connection to Institutional Values. Data from 858 participants were utilized to obtain validity and reliability evidence. Three factors with a general method factor for all negatively worded items showed the best fit. Consistent with the prior findings, relationships with four criterion measures showed medium to large magnitudes of correlations. Reliability was acceptable to excellent levels. The use of the CSM was discussed along with the needs for future research. [ABSTRACT FROM AUTHOR]

Based on the framework of testlet models, the current study suggests the Bayesian random block item response theory (BRB IRT) model to fit forced-choice formats where an item block is composed of three or more items. To account for local dependence among items within a block, the BRB IRT model incorporated a random block effect into the response function and used a Markov Chain Monte Carlo procedure for simultaneous estimation of item and trait parameters. The simulation results demonstrated that the BRB IRT model performed well for the estimation of item and trait parameters and for screening those with relatively low scores on target traits. As found in the literature, the composition of item blocks was crucial for model performance; negatively keyed items were required for item blocks. The empirical application showed the performance of the BRB IRT model was equivalent to that of the Thurstonian IRT model. The potential advantage of the BRB IRT model as a base for more complex measurement models was also demonstrated by incorporating gender as a covariate into the BRB IRT model to explain response probabilities. Recommendations for the adoption of forced-choice formats were provided along with the discussion about using negatively keyed items.

The current simulation study examined the effects of Item Parameter Drift (IPD) occurring in a short scale on parameter estimates in multilevel models where scores from a scale were employed as a time-varying predictor to account for outcome scores. Five factors, including three decisions about IPD, were considered for simulation conditions. It was revealed that IPD occurring in a relatively shorter scale led to a substantial increase in the amount of relative bias in parameter estimates. The bias was more prominent in the estimates of level-2 time-varying predictors relative to those of level-1 time-varying predictors. Regarding the decisions about IPD, keeping items exhibiting IPD was more appropriate than removing them based on the results from relative bias of standard errors of estimates. Based on the findings, it can be concluded that removing items exhibiting IPD may lead to an increase of Type II errors due to the underestimation of parameter estimates and overestimation of standard errors. The applied example showed findings consistent with those in the simulation study.

The current study investigated the impact of matching criterion purification on the accuracy of differential item functioning (DIF) detection in large-scale assessments. The three matching approaches for DIF analyses (block-level matching, pooled booklet matching, and equated pooled booklet matching) were employed with the Mantel-Haenszel procedure. Five factors--the length of a test, the proportion of items exhibiting DIF, a sample size, a ratio of a reference and focal group, and the existence of an average ability difference between two groups--were manipulated. The three matching approaches were used with and without purification. Also, a systematic test form difference was considered. The results indicated that overall, matching criterion purification in the three approaches contributed to the improvement of power in the detection of DIF. Depending on the psychometric characteristics of items exhibiting DIF and the existence of an average ability difference, the amount of power improvement due to matching criterion purification was different across the three approaches. The purification of a matching criterion contributed to the slight reduction of Type I error rates in the three approaches when no mean ability difference existed between the two groups. Considering power improvement with the control of Type I error rates, the purification of a matching criterion in the pooled booklet matching and the equated pooled booklet matching approaches can be recommended for DIF analyses in large-scale assessments.

Differential item functioning (DIF) analysis is important in terms of test fairness. While DIF analyses have mainly been conducted with manifest grouping variables, such as gender or race/ethnicity, it has been recently claimed that not only the grouping variables but also contextual variables pertaining to examinees should be considered in DIF analyses. This study adopted propensity scores to incorporate the contextual variables into the gender DIF analysis. In this study, propensity scores were used to control for the contextual variables that potentially affect the gender DIF. Subsequent DIF analyses with the Mantel-Haenszel (MH) procedure and the Logistic Regression (LR) model were run with the propensity score applied reference (males) and focal groups (females) through propensity score matching. The propensity score embedded MH model and LR model detected fewer number of gender DIF than the conventional MH and LR models. The propensity score embedded models, as a confirmatory approach in DIF analysis, could contribute to hypothesizing an inference on the potential cause of DIF. Also, salient advantages of propensity score embedded DIF analysis models are discussed.

We adapted three practice College Scholastic Ability Tests (CSAT) of English listening, each with five-option items, to create four- and three-option versions by asking 73 Korean speakers or learners of English to eliminate the least plausible options in two rounds. Two hundred and sixty-four Korean high school English-language learners formed three groups. Each took three of the nine tests, one with five-option items, one with four-, and one with three-, with administrations counterbalanced to control for order and practice effects. Mean test scores of the three-option tests were significantly higher than those of four- and five-option tests. While no difference was found in mean item discriminations across the three different test formats, reliability coefficients showed inconsistent patterns depending on the number of options and test versions. One possible interpretation of the low correlations among the scores of three test formats is that items with different numbers of options tap into skills other than listening. The findings suggest that statistically, three options may or may not be optimal depending on the point of view taken--from that of the test score users, or from that of the test stakeholders. Test developers must consider multiple statistical, affective, and contextual factors in determining the optimal number of options. (Contains 5 tables, 2 figures and 2 notes.)

Forced‐choice format tests have been suggested as an alternative to Likert‐scale measures for personnel selection due to robustness to faking and response styles. This study compared degrees of faking occurring in Likert‐scale and forced‐choice five‐factor personality tests between South Korea and the United States. Also, it was examined whether the forced‐choice format was effective at reducing faking in both countries. Data were collected from 396 incumbents participating in both honest and applicant conditions (NSK = 179, NUS = 217). Cohen's d values for within‐subjects designs (dswithin) for between the two conditions were utilized to measure magnitudes of faking occurring in each format and country. In both countries, the degrees of faking occurring in the Likert‐scale were larger than those from the forced‐choice format, and the magnitudes of faking across five personality traits were larger in South Korea by from 0.07 to 0.12 in dswithin. The forced‐choice format appeared to successfully reduce faking for both countries as the average dswithin decreased by 0.06 in both countries. However, the patterns of faking occurring in the forced‐choice format varied between the two countries. In South Korea, degrees of faking in Openness and Conscientiousness increased, whereas those in Extraversion and Agreeableness were substantially decreased. Potential factors leading to trait‐specific faking under the forced‐choice format were discussed in relation to cultural influence on the perception of personality traits and score estimation in Thurstonian item response theory (IRT) models. Finally, the adverse impact of using forced‐choice formats on multicultural selection settings was elaborated. Practitioner Points: The benefit of using forced‐choice formats for cross‐cultural selection settings is not yet clear because there is a lack of scholarly evidence on the performance of forced‐choice formats with respect to faking occurring in different cultures.With the use of a forced‐choice format personality test, the magnitudes of faking decreased in the United States for all five personality traits, whereas the magnitudes of faking occurring in Openness and Conscientiousness increased in South Korea.Potential factors leading to trait‐specific faking under the forced‐choice format can be related to cultural influences on the perception of personality traits and score estimation in Thurstonian item response theory (IRT) models.Practitioners should consider cultural differences in how applicants view target constructs if a forced‐choice format is considered for cross‐cultural/international personnel selection settings. [ABSTRACT FROM AUTHOR]

The unparalleled speed of COVID-19 vaccine development has necessitated an expansion of existing knowledge on vaccination decision-making. The current study explored (1) how cognitions and emotions shaped college students' COVID-19 vaccination decisions, and (2) where vaccination-inclined and vaccination-hesitant students converged and diverged in their decision-making process. Seventy-seven students participated in 26 focus groups to discuss their complex thoughts and feelings regarding COVID-19 vaccination, offering a more nuanced understanding of COVID-19 vaccination decision-making that has not been fully captured by quantitative studies. Thematic analysis found that vaccination-inclined participants and their hesitant counterparts reported differential patterns of positive and negative emotions, systematic appraisals, and heuristics in decision-making. Future research should investigate the roles of hope and relief, non-health-related benefits of vaccination, social trust, and interpersonal influence in vaccination decision-making. [ABSTRACT FROM AUTHOR]

Abstract This work demonstrates the synthesis and characterization of novel nanoporous silicified phospholipid bilayers assembled inorganic powders. The materials are obtained by silicification process with silica precursor at the hydrophilic region of phospholipid bilayers. This process involves the co-assembly of a chemically active phospholipids bilayer within the ordered porosity of a silica matrix and holds promise as a novel application for controlled drug release or drug containers with a high level of specificity and throughput. The controlled release application of the synthesized materials was achieved to glycolic acid, and obtained a zero-order release pattern due to the nanoporosity.