Search

Despite the growing interest in incorporating response time data into item response models, there has been a lack of research investigating how the effect of speed on the probability of a correct response varies across different groups (e.g., experimental conditions) for various items (i.e., differential response time item analysis). Furthermore, previous research has shown a complex relationship between response time and accuracy, necessitating a functional analysis to understand the patterns that manifest from this relationship. In this study, response time data are incorporated into an item response model for two purposes: (a) to examine how individuals' speed within an experimental condition affects their response accuracy on an item, and (b) to detect the differences in individuals' speed between conditions in the presence of within-condition effects. For these two purposes, by-variable smooth functions are employed to model differential and functional response time effects by experimental condition for each item. This model is illustrated using an empirical data set to describe the effect of individuals' speed on their reading comprehension ability in two experimental conditions of reading medium (paper vs. digital) by item. A simulation study showed that the recovery of parameters and by-variable smooth functions of response time was satisfactory, and that the type I error rate and power of the test for the by-variable smooth function of response time were acceptable in conditions similar to the empirical data set. In addition, the proposed method correctly identified the range of response time where between-condition differences in the effect of response time on the probability of a correct response were accurate.

A modified and improved inductive inferential approach to evaluate item discriminations in a conditional maximum likelihood and Rasch modeling framework is suggested. The new approach involves the derivation of four hypothesis tests. It implies a linear restriction of the assumed set of probability distributions in the classical approach that represents scenarios of different item discriminations in a straightforward and efficient manner. Its improvement is discussed, compared to classical procedures (tests and information criteria), and illustrated in Monte Carlo experiments as well as real data examples from educational research. The results show an improvement of power of the modified tests of up to 0.3.

Are there some differences so small that we cannot detect them? Are some quantities so similar (e.g., the number of spots on two speckled hens) that they simply look the same to us? Although modern psychophysical theories such as Signal Detection Theory would predict that, with enough trials, even minute differences would be perceptible at an above-chance rate, this prediction has rarely been empirically tested for any psychological dimension, and never for the domain of number perception. In an experiment with over 400 adults, we find that observers can distinguish which of two collections has more dots from a brief glance. Impressively, observers performed above chance on every numerical comparison tested, even when discriminating a comparison as difficult as 50 versus 51 dots. Thus, we present empirical evidence that numerical discrimination abilities, consistent with SDT, are remarkably fine-grained.

This study has two main purposes. Firstly, to compare the different item selection methods and stopping rules used in Computerized Adaptive Testing (CAT) applications with simulative data generated based on the item parameters of the Vocational Maturity Scale. Secondly, to test the validity of CAT application scores. For the first purpose, simulative data produced based on Vocational Maturity Scale item parameters were analyzed under different item selection methods (Maximum Fisher Information [MFI], Maximum Likelihood Weighted Information [MLWI] Maximum Posterior Weighted Information [MPWI] Maximum Expected Information [MEI] Minimum Expected Posterior Variance [MEPV] Maximum Expected Posterior Weighted Information [MEPWI]) and stopping rules (Standard Error [SE]<0.30, SE<0.50, SE <0.70, Number of Item [NI]=10, NI=20) by calculating the average number of items, standard error averages, correlation coefficients, bias, and RMSE statistics. For all the conditions of the item selection methods, standard error averages, correlation coefficients, bias, and RMSE statistics showed similar results. When the average number of items is considered, MFI and SE<0.30 were found as most appropriate methods to be used in CAT application. For the second purpose of the study, the paper-pencil form of the Vocational Maturity scale and CAT version were administered to 33 students. A moderate, positive, and statistically significant relationship was found between the CAT application scores and the paper-pencil form scores on the vocational maturity scale. As a result, it can be said that the vocational maturity scale can be applied as a computerized adaptive test and can be used in career guidance processes.

We consider the statistical inference for noisy incomplete binary (or 1-bit) matrix. Despite the importance of uncertainty quantification to matrix completion, most of the categorical matrix completion literature focuses on point estimation and prediction. This paper moves one step further toward the statistical inference for binary matrix completion. Under a popular nonlinear factor analysis model, we obtain a point estimator and derive its asymptotic normality. Moreover, our analysis adopts a flexible missing-entry design that does not require a random sampling scheme as required by most of the existing asymptotic results for matrix completion. Under reasonable conditions, the proposed estimator is statistically efficient and optimal in the sense that the Cramer-Rao lower bound is achieved asymptotically for the model parameters. Two applications are considered, including (1) linking two forms of an educational test and (2) linking the roll call voting records from multiple years in the United States Senate. The first application enables the comparison between examinees who took different test forms, and the second application allows us to compare the liberal-conservativeness of senators who did not serve in the Senate at the same time.

The current study sought out to advance the Social Self-Efficacy and Social Outcome Expectations scale using multiple approaches to scale development. Data from 583 undergraduate students were used in two scale development approaches: Classic Test Theory (CTT) and Item Response Theory (IRT). Confirmatory factor analysis suggested a 2-factor structure that aligns with the theoretically based domains for SEOES items and supports previously proposed models of this scale from CTT and psychometric analyses. The IRT analysis indicated that the SEOES items have greater measurement precision at measuring lower levels of the latent constructs. Future research directions are provided and practice implications are discussed.

Machine learning (ML) methods for causal inference have gained popularity due to their flexibility to predict the outcome model and the propensity score. In this article, we provide a within-group approach for ML-based causal inference methods in order to robustly estimate average treatment effects in multilevel studies when there is cluster-level unmeasured confounding. We focus on one particular ML-based causal inference method based on the targeted maximum likelihood estimation (TMLE) with an ensemble learner called SuperLearner. Through our simulation studies, we observe that training TMLE within groups of similar clusters helps remove bias from cluster-level unmeasured confounders. Also, using within-group propensity scores estimated from fixed effects logistic regression increases the robustness of the proposed within-group TMLE method. Even if the propensity scores are partially misspecified, the within-group TMLE still produces robust ATE estimates due to double robustness with flexible modeling, unlike parametric-based inverse propensity weighting methods. We demonstrate our proposed methods and conduct sensitivity analyses against the number of groups and individual-level unmeasured confounding to evaluate the effect of taking an eighth-grade algebra course on math achievement in the Early Childhood Longitudinal Study.

Checking for possible inconsistency between direct and indirect evidence is an important task in network meta-analysis. Recently, an evidence-splitting (ES) model has been proposed, that allows separating direct and indirect evidence in a network and hence assessing inconsistency. A salient feature of this model is that the variance for heterogeneity appears in both the mean and the variance structure. Thus, full maximum likelihood (ML) has been proposed for estimating the parameters of this model. Maximum likelihood is known to yield biased variance component estimates in linear mixed models, and this problem is expected to also affect the ES model. The purpose of the present paper, therefore, is to propose a method based on residual (or restricted) maximum likelihood (REML). Our simulation shows that this new method is quite competitive to methods based on full ML in terms of bias and mean squared error. In addition, some limitations of the ES model are discussed. While this model splits direct and indirect evidence, it is not a plausible model for the cause of inconsistency.

The present work focuses on the performance of two types of shrinkage priors--the horseshoe prior and the recently developed regularized horseshoe prior--in the context of inducing sparsity in path analysis and growth curve models. Prior research has shown that these horseshoe priors induce sparsity by at least as much as the "gold standard" spike-and-slab prior. The horseshoe priors are compared to the ridge prior and lasso prior, as well as default non-informative priors, in terms of the percent shrinkage in the model parameters and out-of-sample predictive performance. Empirical studies using data from two large-scale educational assessments reveal the clear advantages of the horseshoe priors in terms of both shrinkage and predictive performance. Simulation studies reveal clear advantages in terms of shrinkage, but less obvious advantages in terms of predictive performance, except in the small sample size condition where both horseshoe priors provide noticeably improved predictive performance.

In confirmatory factor analysis (CFA), model parameters are usually estimated by iteratively minimizing the Maximum Likelihood (ML) fit function. In optimal circumstances, the ML estimator yields the desirable statistical properties of asymptotic unbiasedness, efficiency, normality, and consistency. In practice, however, real-life data tend to be far from optimal, making the algorithm prone to convergence failure, inadmissible solutions, and bias. In this study, we revisited some old, yet largely neglected non-iterative alternatives and compared their performance to more recently proposed procedures in an extensive simulation study. We conclude that closed-form expressions may serve as viable alternatives for ML, with the Multiple Group Method -- the oldest method under consideration -- showing favorable results across all settings.

Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such as discriminant analysis or logistic regression imputation, for handling missing data in asymmetric items may result in significant bias. It is also recommended to exercise caution when employing alternative strategies, such as listwise deletion or mean imputation, because these methods rely on assumptions that are often unrealistic in surveys and rating scales. This article explores the potential of implementing a deep learning-based imputation method. Additionally, we provide access to deep learning-based imputation to a broader group of researchers without requiring advanced machine learning training. We apply the methodology to the Wilmington Street Participatory Action Research Health Project.

In this article, we extend Liao's test for across-group comparisons of the fixed effects from the generalized linear model to the fixed and random effects of the generalized linear mixed model (GLMM). Using as our basis the Wald statistic, we developed an asymptotic test statistic for across-group comparisons of these effects. The test can be applied when the fixed and random effects are multivariate normally distributed, and it works well for any link function and conditional distribution of the dependent variable of the GLMM. We also derived the asymptotic properties of this test, and because power information does not exist for either our new test statistic or Liao's test, we implemented a power study to demonstrate the superiority of these tests over the alternatively proposed F test. Using an example, we show the application of the test and then discuss its possible restrictions with respect to the distribution of the random effects.

Any time examinees have had access to items and/or answers prior to taking a test, the fairness of the test and validity of test score interpretations are threatened. Therefore, there is a high demand for procedures to detect both compromised items (CI) and examinees with preknowledge (EWP). In this article, we develop a procedure that uses item scores and distractors to simultaneously detect CI and EWP. The false positive rate and true positive rate are evaluated for both items and examinees using detailed simulations. A real data example is also provided using data from an information technology certification exam.

Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from the unequal probability selection. However, the difficulty of applying PWA in multilevel models (MLM) has been generally underestimated and practical guidance is scarce. This study utilizes an empirical as well as a Monte Carlo simulation investigation to examine the performance of the multilevel pseudo maximum likelihood (MPML) estimation based on information derived from the Early Childhood Longitudinal Study Kindergarten cohort of 2010-2011 (ECLS-K:2011). Variance components and fixed effects estimators across four estimation methods including three MPML estimators (i.e., weighted without scaling, weighted size-scaled and weighted effective-scaled) and the unweighted estimator are provided. Practical guidance about the use of sampling weights in MLM analyses of LSAS data is also offered.

Applied educational researchers may be interested in exploring random slope effects in multilevel models, such as when examining individual growth trajectories with longitudinal data. Random slopes are effects for which the slope of an individual-level coefficient varies depending on group membership, however these effects can be difficult to interpret. The change in variance accounted for is often used as an effect size measure and could be appropriate for helping to interpret a random slope effect. Two methods for computing variance accounted for include likelihood-based methods and variance partition methods. It is unclear how results from these two methods compare to each other when used to compute a measure of change in variance accounted for with a random slope effect. The present study fills this gap through a simulation study comparing these two methods under various conditions. Results indicate that the value of variance accounted for may differ depending on the type of measure used, and that applied researchers should consider reporting values for both measures.

In order to evaluate the effect of a policy or treatment with pre- and post-treatment outcomes, we propose an approach based on a transition model, which may be applied with multivariate outcomes and accounts for unobserved heterogeneity. This model is based on potential versions of discrete latent variables representing the individual characteristic of interest and may be cast in the hidden (latent) Markov literature for panel data. Therefore, it can be estimated by maximum likelihood in a relatively simple way. The approach extends the difference-in-difference method as it is possible to deal with multivariate outcomes. Moreover, causal effects may be expressed with respect to transition probabilities. The proposal is validated through a simulation study, and it is applied to evaluate educational programs administered to pupils in the sixth and seventh grades during their middle school period. These programs are carried out in an Italian region to improve non-cognitive skills (CSs). We study if they impact also on students' CSs in Italian and Mathematics in the eighth grade, exploiting the pretreatment test scores available in the fifth grade. The main conclusion is that the educational programs aimed to develop noncognitive abilities help the best students to maintain their higher cognitive abilities over time.

Factor score regression (FSR) is widely used as a convenient alternative to traditional structural equation modeling (SEM) for assessing structural relations between latent variables. But when latent variables are simply replaced by factor scores, biases in the structural parameter estimates often have to be corrected, due to the measurement error in the factor scores. The method of Croon (MOC) is a well-known bias correction technique. However, its standard implementation can render poor quality estimates in small samples (e.g. less than 100). This article aims to develop a small sample correction (SSC) that integrates two different modifications to the standard MOC. We conducted a simulation study to compare the empirical performance of (a) standard SEM, (b) the standard MOC, (c) naive FSR, and (d) the MOC with the proposed SSC. In addition, we assessed the robustness of the performance of the SSC in various models with a different number of predictors and indicators. The results showed that the MOC with the proposed SSC yielded smaller mean squared errors than SEM and the standard MOC in small samples and performed similarly to naive FSR. However, naive FSR yielded more biased estimates than the proposed MOC with SSC, by failing to account for measurement error in the factor scores.

A Monte Carlo simulation was performed to compare methods for handling missing data in growth mixture models. The methods considered in the current study were (a) a fully Bayesian approach using a Gibbs sampler, (b) full information maximum likelihood using the expectation-maximization algorithm, (c) multiple imputation, (d) a two-stage multiple imputation method, and (e) listwise deletion. Of the five methods, it was found that the Bayesian approach and two-stage multiple imputation methods generally produce less biased parameter estimates compared to maximum likelihood or single imputation methods, although key differences were observed. Similarities and disparities among methods are highlighted and general recommendations articulated.

The purpose of this study was to examine the effects of different data conditions on item parameter recovery and classification accuracy of three dichotomous mixture item response theory (IRT) models: the Mix1PL, Mix2PL, and Mix3PL. Manipulated factors in the simulation included the sample size (11 different sample sizes from 100 to 5000), test length (10, 30, and 50), number of classes (2 and 3), the degree of latent class separation (normal/no separation, small, medium, and large), and class sizes (equal vs. nonequal). Effects were assessed using root mean square error (RMSE) and classification accuracy percentage computed between true parameters and estimated parameters. The results of this simulation study showed that more precise estimates of item parameters were obtained with larger sample sizes and longer test lengths. Recovery of item parameters decreased as the number of classes increased with the decrease in sample size. Recovery of classification accuracy for the conditions with two-class solutions was also better than that of three-class solutions. Results of both item parameter estimates and classification accuracy differed by model type. More complex models and models with larger class separations produced less accurate results. The effect of the mixture proportions also differentially affected RMSE and classification accuracy results. Groups of equal size produced more precise item parameter estimates, but the reverse was the case for classification accuracy results. Results suggested that dichotomous mixture IRT models required more than 2,000 examinees to be able to obtain stable results as even shorter tests required such large sample sizes for more precise estimates. This number increased as the number of latent classes, the degree of separation, and model complexity increased.

The instrument that assesses teachers' competence on AR methodology is limited. Thus, it is one of the issues concerning evaluating the effectiveness of a professional development program on designing AR projects. It is difficult to determine how much and what teachers have learned in a course or training. Thus, this cross-sectional study aimed to evaluate further the validity and reliability of the Teacher's Competence in Action Research Questionnaire, a seven-factor instrument previously proposed by Cortes, Pineda, and Geverola (2020). This self-report scale was not subjected to confirmatory factor analysis, had less sample size, and had homogenous participants. In the present study, 450 participants, both pre- and in-service teachers and from different teaching specializations, answered the survey. The data were analyzed using the confirmatory factor analysis method through the Maximum Likelihood approach. Four model fit indices recorded satisfactory results (CFI = 0.890; TLI = 0.884; RMSEA = 0.072; SRMR = 0.039), thus, supporting the seven-factor scale. The standardized factor loading, composite reliability, average variance extracted, and Cronbach's alpha coefficient/s of the entire scale and within subscales also provide evidence of the convergent validity and reliability of the scale. There may be an issue in the discriminant validity of the scale, but the conceptual distinctions of each factor as supported by theoretical foundation and arguments provide a principal reason for retaining all the items and factors.

In the current study, the appropriateness of the Mathematics Attitude Questionnaire administered to middle school 8th grade students in the TIMSS 2015 application to the exploratory structural equation and confirmatory factor analysis models was examined. The study was conducted on 6079 students making up the sample of Turkey. In the TIMSS 2015 application, the attitude items are presented under four headings called students' interest in mathematics, students' views on engaging teaching in mathematics lessons, students' self-confidence in mathematics, and students' value mathematics. As a result of the investigation of the factor structure of these items, the attitude questionnaire with its 5 factors and 35 items was accepted to be suitable for the Exploratory Structural Equation Model (ESEM). Moreover, invariance of the TIMSS 8th grade mathematics attitude questionnaire depending on gender was investigated at six stages as configural, weak (metric), strong (scalar), strict, variance-covariance, and latent mean invariance through ESEM. It was concluded that the questionnaire satisfied all the invariance conditions.

Weighted least squares (WLS), weighted least squares mean-and-variance-adjusted (WLSMV), unweighted least squares mean-and-variance-adjusted (ULSMV), maximum likelihood (ML), robust maximum likelihood (MLR) and Bayesian estimation methods were compared in mixed item response type data via Monte Carlo simulation. The percentage of polytomous items, distribution of polytomous items, categories of polytomous items, average factor loading, sample size and test length conditions were manipulated. ULSMV and WLSMV were found to be the more accurate methods under all simulation conditions. All methods except WLS had acceptable relative bias and relative standard error bias. No method gives accurate results with small sample sizes and low factor loading, however, the ULSMV method can be recommended to researchers because it gives more appropriate results in all conditions.

This study aims to compare the results of the factor analysis performed with Frequentist and Bayesian approaches. The number of sub-dimensions of the measurement tool obtained from different methods, the variation of the items in the sub-dimensions, and the fit statistics' differentiation were examined. 778 students constitute the study sample. "Buss and Perry Aggression Questionnaire"-short form was used in the study. The results obtained from Bayesian factor analysis and maximum likelihood, minimum residuals, principal axis factoring, ordinary least squares, weighted least squares, generalized least squares factor extraction methods, which are frequentist factor analysis methods, were compared. As a result of the study, the number of factors obtained from different factor extraction methods was the same, and the measurement tool had a four-dimensional structure. The analyses were performed in R, and JASP the factor loading cut-off was set as 0.40. As a result of study, Bayesian approach also yields valid results in terms of factor analysis, similar to the frequentist approach. Bayesian factor analysis method allowed to keep the measurement tool's structure, and there is no need to remove items from the measurement tool. So the researchers were recommended to include the Bayesian factor analysis in their studies.

When meta-analyzing heterogeneous bodies of literature, meta-regression can be used to account for potentially relevant between-studies differences. A key challenge is that the number of candidate moderators is often high relative to the number of studies. This introduces risks of overfitting, spurious results, and model non-convergence. To overcome these challenges, we introduce Bayesian Regularized Meta-Analysis (BRMA), which selects relevant moderators from a larger set of candidates by shrinking small regression coefficients towards zero with regularizing (LASSO or horseshoe) priors. This method is suitable when there are many potential moderators, but it is not known beforehand which of them are relevant. A simulation study compared BRMA against state-of-the-art random effects meta-regression using restricted maximum likelihood (RMA). Results indicated that BRMA outperformed RMA on three metrics: BRMA had superior predictive performance, which means that the results generalized better; BRMA was better at rejecting irrelevant moderators, and worse at detecting true effects of relevant moderators, while the overall proportion of Type I and Type II errors was equivalent to RMA. BRMA regression coefficients were slightly biased towards zero (by design), but its residual heterogeneity estimates were less biased than those of RMA. BRMA performed well with as few as 20 studies, suggesting its suitability as a small sample solution. We present free open source software implementations in the R-package pema (for penalized meta-analysis) and in the stand-alone statistical program JASP. An applied example demonstrates the use of the R-package.

This article is concerned with a subset of numerically stable and scalable algorithms useful to support computationally complex psychometric models in the era of machine learning and massive data. The subset selected here is a core set of numerical methods that should be familiar to computational psychometricians and considers whitening transforms for dealing with correlated data, computational concepts for linear models, multivariable integration, and optimization techniques.

Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

Recent research has provided evidence that performance change during a student's test event can indicate the presence of test-taking disengagement. Meaningful performance change implies that some portions of the test event reflect assumed maximum performance better than others and, because disengagement tends to diminish performance, lower-performing portions are less likely to reflect maximum performance than higher-performing portions. This empirical study explored the use of differential weighting of item responses during scoring, with weighting schemes representing either declining or increasing performance. Results indicated that weighted scoring could substantially decrease the score distortion due to disengagement factors and thereby improve test score validity. The study findings support the use of scoring procedures that manage disengagement by adapting to student test-taking behavior.

We examined the effect of estimation methods, maximum likelihood (ML), unweighted least squares (ULS), and diagonally weighted least squares (DWLS), on three population SEM (structural equation modeling) fit indices: the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the standardized root mean square residual (SRMR). We considered different types and levels of misspecification in factor analysis models: misspecified dimensionality, omitting cross-loadings, and ignoring residual correlations. Estimation methods had substantial impacts on the RMSEA and CFI so that different cutoff values need to be employed for different estimators. In contrast, SRMR is robust to the method used to estimate the model parameters. The same criterion can be applied at the population level when using the SRMR to evaluate model fit, regardless of the choice of estimation method.

One of the aims of the current study is to specify the model providing the best fit to the data among the exploratory, the bifactor exploratory and the confirmatory structural equation models. The study compares the three models based on the model data fit statistics and item parameter estimations (factor loadings, cross-loadings, factor correlations) provided by the models. The second aim of the study is to examine correlations among the factor scores, the academic achievement and the goal orientations to provide criterion validity for the person parameter estimations of the best-fitting model. The Academic Motivation Scale was conducted on 1,858 junior and senior students. The three measurement models were compared based on the model data fit and parameter estimations. All estimations were done on the Mplus 6.0 statistical program using the Maximum Likelihood method. It was found that the bifactor exploratory structural equation model provided only trivial improvement on the model data fit relative to the exploratory structural equation model. However, the results of the study revealed that including a general factor in the model achieved a decrease in item cross-loadings. In addition, items could lie along the relative autonomy continuum in a consistent way with the Self-Determination Theory according to their general factor loadings estimated by the bifactor model. The model data fit statistics, parameter estimations and correlation coefficients indicated that the bifactor exploratory structural equation better fit to data than the other two models both theoretically and statistically.

This Monte Carlo simulation study aimed to investigate confirmatory factor analysis (CFA) estimation methods under different conditions, such as sample size, distribution of indicators, test length, average factor loading, and factor structure. Binary data were generated to compare the performance of maximum likelihood (ML), mean and variance adjusted unweighted least squares (ULSMV), mean and variance adjusted weighted least squares (WLSMV), and Bayesian estimators. As a result of the study, it was revealed that increased average factor loading and sample size had a positive effect on the performance of the estimation methods. According to the research findings, it can be said that the methods are sufficient to estimate average factor loading and interfactor correlations, regardless of the estimation methods, in most of the conditions where the average factor loading is 0.7. In small sample sizes particularly, the interfactor correlation was underestimated for skewed indicator conditions. According to the findings of the study, although there is not the most accurate method in all conditions, it can be recommended to use ULSMV method because it performs adequately in more conditions.

Heterogeneity is commonplace in meta-analysis. When heterogeneity is found, researchers often aim to identify predictors that account for at least part of such heterogeneity by using mixed-effects meta-regression models. Another potentially relevant goal is to focus on the amount of heterogeneity as a function of one or more predictors, but this cannot be examined with standard random- and mixed-effects models, which assume a constant (i.e., homoscedastic) value for the heterogeneity variance component across studies. In this paper, we describe a location-scale model for meta-analysis as an extension of the standard random- and mixed-effects models that not only allows an examination of whether predictors are related to the size of the outcomes (i.e., their location), but also the amount of heterogeneity (i.e., their scale). We present estimation methods for such a location-scale model through maximum and restricted maximum likelihood approaches, as well as methods for inference and suggestions for visualization. We also provide an implementation via the "metafor" package for R that makes this model readily available to researchers. Location-scale models can provide a useful tool to researchers interested in heterogeneity in meta-analysis, with the potential to enhance the scope of research questions in the field of evidence synthesis.

Instrument measurement conducted with Rasch analysis is a common process in language assessment research. A recent systematic review of 215 studies involving Rasch analysis in language testing and applied linguistics research reported that 23 different software packages had been utilized. However, none of the analyses were conducted with one of the numerous R-based Rasch analysis software packages, which generally employ one of the three estimation methods: conditional maximum likelihood estimation (CMLE), joint maximum likelihood estimation (JMLE), or marginal maximum likelihood estimation (MMLE). For this study, eRm, a CMLE-based R package, was utilized to conduct a dichotomous Rasch analysis of a Yes/No vocabulary test based on the academic word list. The resulting parameters and diagnostic statistics were compared with the equivalent results from four other R-based Rasch measurement software packages and Winsteps. Finally, all of the packages were utilized in the analysis of 1000 simulated datasets to investigate the extent to which results generated from the contrasting estimation methods converged or diverged. Overall, the differences between the results produced with the three estimation methods were negligible, and the discrepancies observed between datasets were attributable to the software choice as opposed to the estimation method.

Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals' proficiency estimates relative to a conventional IRT model.

Prior studies investigating the effects of non-normality in structural equation modeling typically induced non-normality in the indicator variables. This procedure neglects the factor analytic structure of the data, which is defined as the sum of latent variables and errors, so it is unclear whether previous results hold if the source of non-normality is considered. We conducted a Monte Carlo simulation manipulating the underlying multivariate distribution to assess the effect of the source of non-normality (latent, error, and marginal conditions with either multivariate normal or non-normal marginal distributions) on different measures of fit (empirical rejection rates for the likelihood-ratio model test statistic, the root mean square error of approximation, the standardized root mean square residual, and the comparative fit index). We considered different estimation methods (maximum likelihood, generalized least squares, and (un)modified asymptotically distribution-free), sample sizes, and the extent of non-normality in correctly specified and misspecified models to investigate their performance. The results show that all measures of fit were affected by the source of non-normality but with varying patterns for the analyzed estimation methods.

A maximum marginal likelihood estimation with an expectation-maximization algorithm has been developed for estimating multigroup or mixture multidimensional item response theory models using the generalized partial credit function, graded response function, and 3-parameter logistic function. The procedure includes the estimation of item parameters, attribute population distribution parameters, and test takers' attributes. All estimation functions and derivatives are provided. This procedure has been implemented in an R program. A simulation study has been conducted using this R program on various models related to the generalized partial credit function, and the result shows reasonable parameter recovery.

Logistic regression is one of the most fundamental tools in predictive analytics. Graduate business analytics students are often familiarized with implementation of logistic regression using Python, R, SPSS, or other software packages. However, an understanding of the underlying maximum likelihood model and the mechanics of estimation are often lacking. This paper describes two Excel workbooks that can be used to enhance conceptual understanding of logistic regression in several respects: (i) by providing a clear formulation and solution of the maximum likelihood estimation problem; (ii) by showing the process for testing the significance of logistic regression coefficients; (iii) by demonstrating different methods for model selection to avoid overfitting, specifically, all possible subsets ordinary least squares regression and l[subscript 1]-regularized logistic regression (lasso); and (iv) by illustrating the measurement of relative predictor importance using all possible subsets.

In the current study, we aimed to develop a reliable and valid scale to address individual cognitive load types. Existing scale development studies involved limited number of items without adequate convergent, discriminant and criterion validity checks. Through a multistep correlational study, we proposed a three-factor scale with 13 items to address intrinsic, extraneous and germane cognitive load in computer-based learning environments. A thorough literature search of cognitive load indicators in the literature was followed by expert panels for content and construct validity. The initial item pool was administered to 236 undergraduate students who watched an instructional video on IP address classes. A multiple-choice achievement test was also implemented after the intervention. The exploratory factor analysis with maximum likelihood explained 58 percent of the variance with factor loads above 0.49, and internal consistency coefficients above 0.81. Convergent and discriminant validity indices were acceptable. Besides, achievement was related positively with the germane load and negatively with intrinsic and extraneous load. Fifteen percent of the achievement was explained by the three sources of the cognitive load. Then, the developed factor structure was validated with 193 undergraduate students immediately after they participated in online webinars, and with 99 undergraduate students after they participated in face to face classes. The proposed structure was confirmed in both settings so the scale was considered a reliable and valid indicator of cognitive load in both online and face-to-face learning environments.

The Schwarz or Bayesian information criterion (BIC) is one of the most widely used tools for model comparison in social science research. The BIC, however, is not suitable for evaluating models with order constraints on the parameters of interest. This article explores two extensions of the BIC for evaluating order-constrained models, one where a truncated unit information prior is used under the order-constrained model and the other where a truncated local unit information prior is used. The first prior is centered on the maximum likelihood estimate, and the latter prior is centered on a null value. Several analyses show that the order-constrained BIC based on the local unit information prior works better as an Occam's razor for evaluating order-constrained models and results in lower error probabilities. The methodology based on the local unit information prior is implemented in the R package "BICpack" which allows researchers to easily apply the method for order-constrained model selection. The usefulness of the methodology is illustrated using data from the European Values Study.

New statistical methods are being added to the literature as a result of scientific developments each and every day. This study aims at investigating one of these, Maximum Likelihood Score Estimation with Fences (MLEF) method, in ca-MST. The results obtained from this study will contribute to both national and international literature since there is no such study on the applicability of MLEF method in ca-MST. In line with the aim of this study, 48 conditions (4 module lengths (5-10-15-20) x 2 panel designs (1-3; 1-3-3) x 2 ability distribution (normal-uniform) x 3 ability estimation methods (MLEFMLE-EAP) were simulated and the data obtained from the simulation were interpreted with correlation, RMSE and AAD as an implication of measurement precision; and with conditional bias calculation in order to show the changes in each ability level. This study is a post-hoc simulation study using the data from TIMSS 2015 at the 8th grade in mathematics. "xxIRT" R package program and MSTGen simulation software tool were used in the study. As a result, it can be said that MLEF, as a new ability estimation method, is superior to MLE method in all conditions. EAP estimation method gives the best results in terms of the measurement precision based on correlation, RMSE and AAD values, whereas the results gained via MLEF estimation method are pretty close to those in EAP estimation method. MLE proves to be less biased in ability estimation, especially in extreme ability levels, when compared to EAP ability estimation method.