28,981 results on '"MAXIMUM likelihood statistics"'
Search Results
2. Differential and Functional Response Time Item Analysis: An Application to Understanding Paper versus Digital Reading Processes
- Author
-
Sun-Joo Cho, Amanda Goodwin, Matthew Naveiras, and Jorge Salas
- Abstract
Despite the growing interest in incorporating response time data into item response models, there has been a lack of research investigating how the effect of speed on the probability of a correct response varies across different groups (e.g., experimental conditions) for various items (i.e., differential response time item analysis). Furthermore, previous research has shown a complex relationship between response time and accuracy, necessitating a functional analysis to understand the patterns that manifest from this relationship. In this study, response time data are incorporated into an item response model for two purposes: (a) to examine how individuals' speed within an experimental condition affects their response accuracy on an item, and (b) to detect the differences in individuals' speed between conditions in the presence of within-condition effects. For these two purposes, by-variable smooth functions are employed to model differential and functional response time effects by experimental condition for each item. This model is illustrated using an empirical data set to describe the effect of individuals' speed on their reading comprehension ability in two experimental conditions of reading medium (paper vs. digital) by item. A simulation study showed that the recovery of parameters and by-variable smooth functions of response time was satisfactory, and that the type I error rate and power of the test for the by-variable smooth function of response time were acceptable in conditions similar to the empirical data set. In addition, the proposed method correctly identified the range of response time where between-condition differences in the effect of response time on the probability of a correct response were accurate.
- Published
- 2024
- Full Text
- View/download PDF
3. An Improved Inferential Procedure to Evaluate Item Discriminations in a Conditional Maximum Likelihood Framework
- Author
-
Clemens Draxler, Andreas Kurz, Can Gürer, and Jan Philipp Nolte
- Abstract
A modified and improved inductive inferential approach to evaluate item discriminations in a conditional maximum likelihood and Rasch modeling framework is suggested. The new approach involves the derivation of four hypothesis tests. It implies a linear restriction of the assumed set of probability distributions in the classical approach that represents scenarios of different item discriminations in a straightforward and efficient manner. Its improvement is discussed, compared to classical procedures (tests and information criteria), and illustrated in Monte Carlo experiments as well as real data examples from educational research. The results show an improvement of power of the modified tests of up to 0.3.
- Published
- 2024
- Full Text
- View/download PDF
4. Differential and Functional Response Time Item Analysis: An Application to Understanding Paper versus Digital Reading Processes
- Author
-
Sun-Joo Cho, Amanda Goodwin, Matthew Naveiras, and Jorge Salas
- Abstract
Despite the growing interest in incorporating response time data into item response models, there has been a lack of research investigating how the effect of speed on the probability of a correct response varies across different groups (e.g., experimental conditions) for various items (i.e., differential response time item analysis). Furthermore, previous research has shown a complex relationship between response time and accuracy, necessitating a functional analysis to understand the patterns that manifest from this relationship. In this study, response time data are incorporated into an item response model for two purposes: (a) to examine how individuals' speed within an experimental condition affects their response accuracy on an item, and (b) to detect the differences in individuals' speed between conditions in the presence of within-condition effects. For these two purposes, by-variable smooth functions are employed to model differential and functional response time effects by experimental condition for each item. This model is illustrated using an empirical data set to describe the effect of individuals' speed on their reading comprehension ability in two experimental conditions of reading medium (paper vs. digital) by item. A simulation study showed that the recovery of parameters and by-variable smooth functions of response time was satisfactory, and that the type I error rate and power of the test for the by-variable smooth function of response time were acceptable in conditions similar to the empirical data set. In addition, the proposed method correctly identified the range of response time where between-condition differences in the effect of response time on the probability of a correct response were accurate. [This is the online version of an article published in "Journal of Educational Measurement."]
- Published
- 2024
- Full Text
- View/download PDF
5. Successful Discrimination of Tiny Numerical Differences
- Author
-
Sanford, Emily M. and Halberda, Justin
- Abstract
Are there some differences so small that we cannot detect them? Are some quantities so similar (e.g., the number of spots on two speckled hens) that they simply look the same to us? Although modern psychophysical theories such as Signal Detection Theory would predict that, with enough trials, even minute differences would be perceptible at an above-chance rate, this prediction has rarely been empirically tested for any psychological dimension, and never for the domain of number perception. In an experiment with over 400 adults, we find that observers can distinguish which of two collections has more dots from a brief glance. Impressively, observers performed above chance on every numerical comparison tested, even when discriminating a comparison as difficult as 50 versus 51 dots. Thus, we present empirical evidence that numerical discrimination abilities, consistent with SDT, are remarkably fine-grained.
- Published
- 2023
6. Application of the Professional Maturity Scale as a Computerized Adaptive Testing
- Author
-
Süleyman Demir, Derya Çobanoglu Aktan, and Nese Güler
- Abstract
This study has two main purposes. Firstly, to compare the different item selection methods and stopping rules used in Computerized Adaptive Testing (CAT) applications with simulative data generated based on the item parameters of the Vocational Maturity Scale. Secondly, to test the validity of CAT application scores. For the first purpose, simulative data produced based on Vocational Maturity Scale item parameters were analyzed under different item selection methods (Maximum Fisher Information [MFI], Maximum Likelihood Weighted Information [MLWI] Maximum Posterior Weighted Information [MPWI] Maximum Expected Information [MEI] Minimum Expected Posterior Variance [MEPV] Maximum Expected Posterior Weighted Information [MEPWI]) and stopping rules (Standard Error [SE]<0.30, SE<0.50, SE <0.70, Number of Item [NI]=10, NI=20) by calculating the average number of items, standard error averages, correlation coefficients, bias, and RMSE statistics. For all the conditions of the item selection methods, standard error averages, correlation coefficients, bias, and RMSE statistics showed similar results. When the average number of items is considered, MFI and SE<0.30 were found as most appropriate methods to be used in CAT application. For the second purpose of the study, the paper-pencil form of the Vocational Maturity scale and CAT version were administered to 33 students. A moderate, positive, and statistically significant relationship was found between the CAT application scores and the paper-pencil form scores on the vocational maturity scale. As a result, it can be said that the vocational maturity scale can be applied as a computerized adaptive test and can be used in career guidance processes.
- Published
- 2023
7. Statistical Inference for Noisy Incomplete Binary Matrix
- Author
-
Yunxiao Chen, Chengcheng Li, Jing Ouyang, and Gongjun Xu
- Abstract
We consider the statistical inference for noisy incomplete binary (or 1-bit) matrix. Despite the importance of uncertainty quantification to matrix completion, most of the categorical matrix completion literature focuses on point estimation and prediction. This paper moves one step further toward the statistical inference for binary matrix completion. Under a popular nonlinear factor analysis model, we obtain a point estimator and derive its asymptotic normality. Moreover, our analysis adopts a flexible missing-entry design that does not require a random sampling scheme as required by most of the existing asymptotic results for matrix completion. Under reasonable conditions, the proposed estimator is statistically efficient and optimal in the sense that the Cramer-Rao lower bound is achieved asymptotically for the model parameters. Two applications are considered, including (1) linking two forms of an educational test and (2) linking the roll call voting records from multiple years in the United States Senate. The first application enables the comparison between examinees who took different test forms, and the second application allows us to compare the liberal-conservativeness of senators who did not serve in the Senate at the same time.
- Published
- 2023
8. Variational Estimation for Multidimensional Generalized Partial Credit Model
- Author
-
Chengyu Cui, Chun Wang, and Gongjun Xu
- Abstract
Multidimensional item response theory (MIRT) models have generated increasing interest in the psychometrics literature. Efficient approaches for estimating MIRT models with dichotomous responses have been developed, but constructing an equally efficient and robust algorithm for polytomous models has received limited attention. To address this gap, this paper presents a novel Gaussian variational estimation algorithm for the multidimensional generalized partial credit model (MGPCM). The proposed algorithm demonstrates both fast and accurate performance, as illustrated through a series of simulation studies and two real data analyses. [This is the online version of an article published in "Psychometrika."]
- Published
- 2024
- Full Text
- View/download PDF
9. Further Validation of the Social Efficacy and Social Outcome Expectations Scale
- Author
-
Stephen L. Wright and Michael A. Jenkins-Guarnieri
- Abstract
The current study sought out to advance the Social Self-Efficacy and Social Outcome Expectations scale using multiple approaches to scale development. Data from 583 undergraduate students were used in two scale development approaches: Classic Test Theory (CTT) and Item Response Theory (IRT). Confirmatory factor analysis suggested a 2-factor structure that aligns with the theoretically based domains for SEOES items and supports previously proposed models of this scale from CTT and psychometric analyses. The IRT analysis indicated that the SEOES items have greater measurement precision at measuring lower levels of the latent constructs. Future research directions are provided and practice implications are discussed.
- Published
- 2024
- Full Text
- View/download PDF
10. A Within-Group Approach to Ensemble Machine Learning Methods for Causal Inference in Multilevel Studies
- Author
-
Youmi Suk
- Abstract
Machine learning (ML) methods for causal inference have gained popularity due to their flexibility to predict the outcome model and the propensity score. In this article, we provide a within-group approach for ML-based causal inference methods in order to robustly estimate average treatment effects in multilevel studies when there is cluster-level unmeasured confounding. We focus on one particular ML-based causal inference method based on the targeted maximum likelihood estimation (TMLE) with an ensemble learner called SuperLearner. Through our simulation studies, we observe that training TMLE within groups of similar clusters helps remove bias from cluster-level unmeasured confounders. Also, using within-group propensity scores estimated from fixed effects logistic regression increases the robustness of the proposed within-group TMLE method. Even if the propensity scores are partially misspecified, the within-group TMLE still produces robust ATE estimates due to double robustness with flexible modeling, unlike parametric-based inverse propensity weighting methods. We demonstrate our proposed methods and conduct sensitivity analyses against the number of groups and individual-level unmeasured confounding to evaluate the effect of taking an eighth-grade algebra course on math achievement in the Early Childhood Longitudinal Study.
- Published
- 2024
- Full Text
- View/download PDF
11. A REML Method for the Evidence-Splitting Model in Network Meta-Analysis
- Author
-
Hans-Peter Piepho, Johannes Forkman, and Waqas Ahmed Malik
- Abstract
Checking for possible inconsistency between direct and indirect evidence is an important task in network meta-analysis. Recently, an evidence-splitting (ES) model has been proposed, that allows separating direct and indirect evidence in a network and hence assessing inconsistency. A salient feature of this model is that the variance for heterogeneity appears in both the mean and the variance structure. Thus, full maximum likelihood (ML) has been proposed for estimating the parameters of this model. Maximum likelihood is known to yield biased variance component estimates in linear mixed models, and this problem is expected to also affect the ES model. The purpose of the present paper, therefore, is to propose a method based on residual (or restricted) maximum likelihood (REML). Our simulation shows that this new method is quite competitive to methods based on full ML in terms of bias and mean squared error. In addition, some limitations of the ES model are discussed. While this model splits direct and indirect evidence, it is not a plausible model for the cause of inconsistency.
- Published
- 2024
- Full Text
- View/download PDF
12. On the Performance of Horseshoe Priors for Inducing Sparsity in Structural Equation Models
- Author
-
Kjorte Harra and David Kaplan
- Abstract
The present work focuses on the performance of two types of shrinkage priors--the horseshoe prior and the recently developed regularized horseshoe prior--in the context of inducing sparsity in path analysis and growth curve models. Prior research has shown that these horseshoe priors induce sparsity by at least as much as the "gold standard" spike-and-slab prior. The horseshoe priors are compared to the ridge prior and lasso prior, as well as default non-informative priors, in terms of the percent shrinkage in the model parameters and out-of-sample predictive performance. Empirical studies using data from two large-scale educational assessments reveal the clear advantages of the horseshoe priors in terms of both shrinkage and predictive performance. Simulation studies reveal clear advantages in terms of shrinkage, but less obvious advantages in terms of predictive performance, except in the small sample size condition where both horseshoe priors provide noticeably improved predictive performance.
- Published
- 2024
- Full Text
- View/download PDF
13. An Evaluation of Non-Iterative Estimators in Confirmatory Factor Analysis
- Author
-
Sara Dhaene and Yves Rosseel
- Abstract
In confirmatory factor analysis (CFA), model parameters are usually estimated by iteratively minimizing the Maximum Likelihood (ML) fit function. In optimal circumstances, the ML estimator yields the desirable statistical properties of asymptotic unbiasedness, efficiency, normality, and consistency. In practice, however, real-life data tend to be far from optimal, making the algorithm prone to convergence failure, inadmissible solutions, and bias. In this study, we revisited some old, yet largely neglected non-iterative alternatives and compared their performance to more recently proposed procedures in an extensive simulation study. We conclude that closed-form expressions may serve as viable alternatives for ML, with the Multiple Group Method -- the oldest method under consideration -- showing favorable results across all settings.
- Published
- 2024
- Full Text
- View/download PDF
14. Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items
- Author
-
Zachary K. Collier, Minji Kong, Olushola Soyoye, Kamal Chawla, Ann M. Aviles, and Yasser Payne
- Abstract
Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such as discriminant analysis or logistic regression imputation, for handling missing data in asymmetric items may result in significant bias. It is also recommended to exercise caution when employing alternative strategies, such as listwise deletion or mean imputation, because these methods rely on assumptions that are often unrealistic in surveys and rating scales. This article explores the potential of implementing a deep learning-based imputation method. Additionally, we provide access to deep learning-based imputation to a broader group of researchers without requiring advanced machine learning training. We apply the methodology to the Wilmington Street Participatory Action Research Health Project.
- Published
- 2024
- Full Text
- View/download PDF
15. Multiple Group Comparisons of the Fixed and Random Effects from the Generalized Linear Mixed Model
- Author
-
Daniel Kasper, Katrin Schulz-Heidorf, and Knut Schwippert
- Abstract
In this article, we extend Liao's test for across-group comparisons of the fixed effects from the generalized linear model to the fixed and random effects of the generalized linear mixed model (GLMM). Using as our basis the Wald statistic, we developed an asymptotic test statistic for across-group comparisons of these effects. The test can be applied when the fixed and random effects are multivariate normally distributed, and it works well for any link function and conditional distribution of the dependent variable of the GLMM. We also derived the asymptotic properties of this test, and because power information does not exist for either our new test statistic or Liao's test, we implemented a power study to demonstrate the superiority of these tests over the alternatively proposed F test. Using an example, we show the application of the test and then discuss its possible restrictions with respect to the distribution of the random effects.
- Published
- 2024
- Full Text
- View/download PDF
16. Likelihood-Based Estimation of Model-Derived Oral Reading Fluency
- Author
-
Cornelis Potgieter, Xin Qiao, Akihito Kamata, and Yusuf Kara
- Abstract
As part of the effort to develop an improved oral reading fluency (ORF) assessment system, Kara et al. (2020) estimated the ORF scores based on a latent variable psychometric model of accuracy and speed for ORF data via a fully Bayesian approach. This study further investigates likelihood-based estimators for the model-derived ORF scores, including maximum likelihood estimator (MLE), maximum a posteriori (MAP), and expected a posteriori (EAP), as well as their standard errors. The proposed estimators were demonstrated with a real ORF assessment dataset. Also, the estimation of model-derived ORF scores and their standard errors by the proposed estimators were evaluated through a simulation study. The fully Bayesian approach was included as a comparison in the real data analysis and the simulation study. Results demonstrated that the three likelihood-based approaches for the model-derived ORF scores and their standard error estimation performed satisfactorily. [This paper was published in the "Journal of Educational Measurement."]
- Published
- 2024
- Full Text
- View/download PDF
17. Using Item Scores and Distractors to Detect Item Compromise and Preknowledge
- Author
-
Gorney, Kylie, Wollack, James A., Sinharay, Sandip, and Eckerly, Carol
- Abstract
Any time examinees have had access to items and/or answers prior to taking a test, the fairness of the test and validity of test score interpretations are threatened. Therefore, there is a high demand for procedures to detect both compromised items (CI) and examinees with preknowledge (EWP). In this article, we develop a procedure that uses item scores and distractors to simultaneously detect CI and EWP. The false positive rate and true positive rate are evaluated for both items and examinees using detailed simulations. A real data example is also provided using data from an information technology certification exam.
- Published
- 2023
- Full Text
- View/download PDF
18. Incorporating Complex Sampling Weights in Multilevel Analyses of Education Data
- Author
-
Shen, Ting and Konstantopoulos, Spyros
- Abstract
Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from the unequal probability selection. However, the difficulty of applying PWA in multilevel models (MLM) has been generally underestimated and practical guidance is scarce. This study utilizes an empirical as well as a Monte Carlo simulation investigation to examine the performance of the multilevel pseudo maximum likelihood (MPML) estimation based on information derived from the Early Childhood Longitudinal Study Kindergarten cohort of 2010-2011 (ECLS-K:2011). Variance components and fixed effects estimators across four estimation methods including three MPML estimators (i.e., weighted without scaling, weighted size-scaled and weighted effective-scaled) and the unweighted estimator are provided. Practical guidance about the use of sampling weights in MLM analyses of LSAS data is also offered.
- Published
- 2022
19. Effect Size for a Multilevel Model Random Slope Effect: Change in Variance Accounted for with Likelihood-Based versus Variance Partition Measures
- Author
-
Lorah, Julie
- Abstract
Applied educational researchers may be interested in exploring random slope effects in multilevel models, such as when examining individual growth trajectories with longitudinal data. Random slopes are effects for which the slope of an individual-level coefficient varies depending on group membership, however these effects can be difficult to interpret. The change in variance accounted for is often used as an effect size measure and could be appropriate for helping to interpret a random slope effect. Two methods for computing variance accounted for include likelihood-based methods and variance partition methods. It is unclear how results from these two methods compare to each other when used to compute a measure of change in variance accounted for with a random slope effect. The present study fills this gap through a simulation study comparing these two methods under various conditions. Results indicate that the value of variance accounted for may differ depending on the type of measure used, and that applied researchers should consider reporting values for both measures.
- Published
- 2022
20. A Causal Latent Transition Model with Multivariate Outcomes and Unobserved Heterogeneity: Application to Human Capital Development
- Author
-
Bartolucci, Francesco, Pennoni, Fulvia, and Vittadini, Giorgio
- Abstract
In order to evaluate the effect of a policy or treatment with pre- and post-treatment outcomes, we propose an approach based on a transition model, which may be applied with multivariate outcomes and accounts for unobserved heterogeneity. This model is based on potential versions of discrete latent variables representing the individual characteristic of interest and may be cast in the hidden (latent) Markov literature for panel data. Therefore, it can be estimated by maximum likelihood in a relatively simple way. The approach extends the difference-in-difference method as it is possible to deal with multivariate outcomes. Moreover, causal effects may be expressed with respect to transition probabilities. The proposal is validated through a simulation study, and it is applied to evaluate educational programs administered to pupils in the sixth and seventh grades during their middle school period. These programs are carried out in an Italian region to improve non-cognitive skills (CSs). We study if they impact also on students' CSs in Italian and Mathematics in the eighth grade, exploiting the pretreatment test scores available in the fifth grade. The main conclusion is that the educational programs aimed to develop noncognitive abilities help the best students to maintain their higher cognitive abilities over time.
- Published
- 2023
- Full Text
- View/download PDF
21. Maximum Likelihood Estimation of Hierarchical Linear Models from Incomplete Data: Random Coefficients, Statistical Interactions, and Measurement Error
- Author
-
Shin, Yongyun and Raudenbush, Stephen W.
- Abstract
We consider two-level models where a continuous response R and continuous covariates C are assumed missing at random. Inferences based on maximum likelihood or Bayes are routinely made by estimating their joint normal distribution from observed data R[subscript obs] and C[subscript obs]. However, if the model for R given C includes random coefficients, interactions, or polynomial terms, their joint distribution will be nonstandard. We propose a family of unique factorizations involving selected "provisionally known random effects'' u such that h(R[subscript obs], C[subscript obs] |u) is normally distributed and u is a low-dimensional normal random vector; we approximate h(R[subscript obs], C[subscript obs]) =[integral]h(R[subscript obs], C[subscript obs]¦u)g(u)du via adaptive Gauss-Hermite quadrature. For polynomial models, the approximation is exact but, in any case, can be made as accurate as required given sufficient computation time. The model incorporates random effects as explanatory variables, reducing bias due to measurement error. By construction, our factorizations solve problems of compatibility among fully conditional distributions that have arisen in Bayesian imputation based on the Gibbs Sampler. We spell out general rules for selecting u, and show that our factorizations can support fully compatible Bayesian methods of imputation using the Gibbs Sampler. [This paper will be published in "Journal of Computational and Graphical Statistics."]
- Published
- 2023
- Full Text
- View/download PDF
22. A Small Sample Correction for Factor Score Regression
- Author
-
Bogaert, Jasper, Loh, Wen Wei, and Rosseel, Yves
- Abstract
Factor score regression (FSR) is widely used as a convenient alternative to traditional structural equation modeling (SEM) for assessing structural relations between latent variables. But when latent variables are simply replaced by factor scores, biases in the structural parameter estimates often have to be corrected, due to the measurement error in the factor scores. The method of Croon (MOC) is a well-known bias correction technique. However, its standard implementation can render poor quality estimates in small samples (e.g. less than 100). This article aims to develop a small sample correction (SSC) that integrates two different modifications to the standard MOC. We conducted a simulation study to compare the empirical performance of (a) standard SEM, (b) the standard MOC, (c) naive FSR, and (d) the MOC with the proposed SSC. In addition, we assessed the robustness of the performance of the SSC in various models with a different number of predictors and indicators. The results showed that the MOC with the proposed SSC yielded smaller mean squared errors than SEM and the standard MOC in small samples and performed similarly to naive FSR. However, naive FSR yielded more biased estimates than the proposed MOC with SSC, by failing to account for measurement error in the factor scores.
- Published
- 2023
- Full Text
- View/download PDF
23. Handling Missing Data in Growth Mixture Models
- Author
-
Lee, Daniel Y. and Harring, Jeffrey R.
- Abstract
A Monte Carlo simulation was performed to compare methods for handling missing data in growth mixture models. The methods considered in the current study were (a) a fully Bayesian approach using a Gibbs sampler, (b) full information maximum likelihood using the expectation-maximization algorithm, (c) multiple imputation, (d) a two-stage multiple imputation method, and (e) listwise deletion. Of the five methods, it was found that the Bayesian approach and two-stage multiple imputation methods generally produce less biased parameter estimates compared to maximum likelihood or single imputation methods, although key differences were observed. Similarities and disparities among methods are highlighted and general recommendations articulated.
- Published
- 2023
- Full Text
- View/download PDF
24. The Impact of Sample Size and Various Other Factors on Estimation of Dichotomous Mixture IRT Models
- Author
-
Sen, Sedat and Cohen, Allan S.
- Abstract
The purpose of this study was to examine the effects of different data conditions on item parameter recovery and classification accuracy of three dichotomous mixture item response theory (IRT) models: the Mix1PL, Mix2PL, and Mix3PL. Manipulated factors in the simulation included the sample size (11 different sample sizes from 100 to 5000), test length (10, 30, and 50), number of classes (2 and 3), the degree of latent class separation (normal/no separation, small, medium, and large), and class sizes (equal vs. nonequal). Effects were assessed using root mean square error (RMSE) and classification accuracy percentage computed between true parameters and estimated parameters. The results of this simulation study showed that more precise estimates of item parameters were obtained with larger sample sizes and longer test lengths. Recovery of item parameters decreased as the number of classes increased with the decrease in sample size. Recovery of classification accuracy for the conditions with two-class solutions was also better than that of three-class solutions. Results of both item parameter estimates and classification accuracy differed by model type. More complex models and models with larger class separations produced less accurate results. The effect of the mixture proportions also differentially affected RMSE and classification accuracy results. Groups of equal size produced more precise item parameter estimates, but the reverse was the case for classification accuracy results. Results suggested that dichotomous mixture IRT models required more than 2,000 examinees to be able to obtain stable results as even shorter tests required such large sample sizes for more precise estimates. This number increased as the number of latent classes, the degree of separation, and model complexity increased.
- Published
- 2023
- Full Text
- View/download PDF
25. A Confirmatory Factor Analysis of Teacher's Competence in Action Research (TCAR) Questionnaire
- Author
-
Cortes, Sylvester T., Pineda, Hedeliza A., and Geverola, Immar Jun R.
- Abstract
The instrument that assesses teachers' competence on AR methodology is limited. Thus, it is one of the issues concerning evaluating the effectiveness of a professional development program on designing AR projects. It is difficult to determine how much and what teachers have learned in a course or training. Thus, this cross-sectional study aimed to evaluate further the validity and reliability of the Teacher's Competence in Action Research Questionnaire, a seven-factor instrument previously proposed by Cortes, Pineda, and Geverola (2020). This self-report scale was not subjected to confirmatory factor analysis, had less sample size, and had homogenous participants. In the present study, 450 participants, both pre- and in-service teachers and from different teaching specializations, answered the survey. The data were analyzed using the confirmatory factor analysis method through the Maximum Likelihood approach. Four model fit indices recorded satisfactory results (CFI = 0.890; TLI = 0.884; RMSEA = 0.072; SRMR = 0.039), thus, supporting the seven-factor scale. The standardized factor loading, composite reliability, average variance extracted, and Cronbach's alpha coefficient/s of the entire scale and within subscales also provide evidence of the convergent validity and reliability of the scale. There may be an issue in the discriminant validity of the scale, but the conceptual distinctions of each factor as supported by theoretical foundation and arguments provide a principal reason for retaining all the items and factors.
- Published
- 2021
26. Factor Structure and Measurement Invariance of the TIMSS 2015 Mathematics Attitude Questionnaire: Exploratory Structural Equation Modelling Approach
- Author
-
Uyar, Seyma
- Abstract
In the current study, the appropriateness of the Mathematics Attitude Questionnaire administered to middle school 8th grade students in the TIMSS 2015 application to the exploratory structural equation and confirmatory factor analysis models was examined. The study was conducted on 6079 students making up the sample of Turkey. In the TIMSS 2015 application, the attitude items are presented under four headings called students' interest in mathematics, students' views on engaging teaching in mathematics lessons, students' self-confidence in mathematics, and students' value mathematics. As a result of the investigation of the factor structure of these items, the attitude questionnaire with its 5 factors and 35 items was accepted to be suitable for the Exploratory Structural Equation Model (ESEM). Moreover, invariance of the TIMSS 8th grade mathematics attitude questionnaire depending on gender was investigated at six stages as configural, weak (metric), strong (scalar), strict, variance-covariance, and latent mean invariance through ESEM. It was concluded that the questionnaire satisfied all the invariance conditions.
- Published
- 2021
27. Comparison of Confirmatory Factor Analysis Estimation Methods on Mixed-Format Data
- Author
-
Kilic, Abdullah Faruk and Dogan, Nuri
- Abstract
Weighted least squares (WLS), weighted least squares mean-and-variance-adjusted (WLSMV), unweighted least squares mean-and-variance-adjusted (ULSMV), maximum likelihood (ML), robust maximum likelihood (MLR) and Bayesian estimation methods were compared in mixed item response type data via Monte Carlo simulation. The percentage of polytomous items, distribution of polytomous items, categories of polytomous items, average factor loading, sample size and test length conditions were manipulated. ULSMV and WLSMV were found to be the more accurate methods under all simulation conditions. All methods except WLS had acceptable relative bias and relative standard error bias. No method gives accurate results with small sample sizes and low factor loading, however, the ULSMV method can be recommended to researchers because it gives more appropriate results in all conditions.
- Published
- 2021
28. Comparison of Bayesian and Frequentist Factor Analysis Methods: Buss and Perry Aggression Questionnaire Example
- Author
-
Eser, Mehmet Taha
- Abstract
This study aims to compare the results of the factor analysis performed with Frequentist and Bayesian approaches. The number of sub-dimensions of the measurement tool obtained from different methods, the variation of the items in the sub-dimensions, and the fit statistics' differentiation were examined. 778 students constitute the study sample. "Buss and Perry Aggression Questionnaire"-short form was used in the study. The results obtained from Bayesian factor analysis and maximum likelihood, minimum residuals, principal axis factoring, ordinary least squares, weighted least squares, generalized least squares factor extraction methods, which are frequentist factor analysis methods, were compared. As a result of the study, the number of factors obtained from different factor extraction methods was the same, and the measurement tool had a four-dimensional structure. The analyses were performed in R, and JASP the factor loading cut-off was set as 0.40. As a result of study, Bayesian approach also yields valid results in terms of factor analysis, similar to the frequentist approach. Bayesian factor analysis method allowed to keep the measurement tool's structure, and there is no need to remove items from the measurement tool. So the researchers were recommended to include the Bayesian factor analysis in their studies.
- Published
- 2021
29. Missing Data: An Update on the State of the Art
- Author
-
Enders, Craig K.
- Abstract
The year 2022 is the 20th anniversary of Joseph Schafer and John Graham's paper titled "Missing data: Our view of the state of the art," currently the most highly cited paper in the history of "Psychological Methods." Much has changed since 2002, as missing data methodologies have continually evolved and improved; the range of applications that are possible with modern missing data techniques has increased dramatically, and software options are light years ahead of where they were. This article provides an update on the state of the art that catalogs important innovations from the past two decades of missing data research. The paper addresses topics described in the original paper, including developments related to missing data theory, full information maximum likelihood, Bayesian estimation, multiple imputation, and models for missing not at random processes. The paper also describes newer factored regression specifications and missing data handling for multilevel models, both of which have been a focus of recent research. The paper concludes with a summary of the current software landscape and a discussion of several practical issues. [This paper will be published in "Psychological Methods."]
- Published
- 2023
- Full Text
- View/download PDF
30. Selecting Relevant Moderators with Bayesian Regularized Meta-Regression
- Author
-
Van Lissa, Caspar J., van Erp, Sara, and Clapper, Eli-Boaz
- Abstract
When meta-analyzing heterogeneous bodies of literature, meta-regression can be used to account for potentially relevant between-studies differences. A key challenge is that the number of candidate moderators is often high relative to the number of studies. This introduces risks of overfitting, spurious results, and model non-convergence. To overcome these challenges, we introduce Bayesian Regularized Meta-Analysis (BRMA), which selects relevant moderators from a larger set of candidates by shrinking small regression coefficients towards zero with regularizing (LASSO or horseshoe) priors. This method is suitable when there are many potential moderators, but it is not known beforehand which of them are relevant. A simulation study compared BRMA against state-of-the-art random effects meta-regression using restricted maximum likelihood (RMA). Results indicated that BRMA outperformed RMA on three metrics: BRMA had superior predictive performance, which means that the results generalized better; BRMA was better at rejecting irrelevant moderators, and worse at detecting true effects of relevant moderators, while the overall proportion of Type I and Type II errors was equivalent to RMA. BRMA regression coefficients were slightly biased towards zero (by design), but its residual heterogeneity estimates were less biased than those of RMA. BRMA performed well with as few as 20 studies, suggesting its suitability as a small sample solution. We present free open source software implementations in the R-package pema (for penalized meta-analysis) and in the stand-alone statistical program JASP. An applied example demonstrates the use of the R-package.
- Published
- 2023
- Full Text
- View/download PDF
31. A Collection of Numerical Recipes Useful for Building Scalable Psychometric Applications
- Author
-
Doran, Harold
- Abstract
This article is concerned with a subset of numerically stable and scalable algorithms useful to support computationally complex psychometric models in the era of machine learning and massive data. The subset selected here is a core set of numerical methods that should be familiar to computational psychometricians and considers whitening transforms for dealing with correlated data, computational concepts for linear models, multivariable integration, and optimization techniques.
- Published
- 2023
- Full Text
- View/download PDF
32. Evaluation of Statistical Methods Used to Meta-Analyse Results from Interrupted Time Series Studies: A Simulation Study
- Author
-
Korevaar, Elizabeth, Turner, Simon L., Forbes, Andrew B., Karahalios, Amalia, Taljaard, Monica, and McKenzie, Joanne E.
- Abstract
Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.
- Published
- 2023
- Full Text
- View/download PDF
33. Variability in Causal Effects on a Binary Outcome and Noncompliance in a Multisite Randomized Trial
- Author
-
Sun, Xinxin
- Abstract
Noncompliance to treatment assignment is widespread in randomized trials and presents challenges in causal inference. In the presence of noncompliance, the most commonly estimated effect of treatment assignment, also known as the intent-to-treat (ITT) effect, is biased. Of interest in this setting is the complier average causal effect (CACE), the ITT effect among compliers. A further complication arises when the outcome variable is partially observed. My research focuses on estimating the distribution of a site-specific CACE in a multisite randomized controlled trial (MRCT) by maximum likelihood (ML). Assuming compliance missing at random (MAR). We express the likelihood as an integral with respect to random effects that cannot be evaluated analytically. We derive ML estimators by a combination of the EM algorithm and Newton Raphson method conditional on random effects and, then, integrate each estimator and the likelihood with respect to the random effects by adaptive Gauss Hermite quadrature (AGHQ). Next, we extend this approach to both outcome and imperfect compliance MAR. A distinctive feature of the approach is to estimate a site-specific CACE and its variance over sites efficiently. We applied our method to data sets from two MRCTs with imperfect compliance: the e-assist intervention trial to assess the effectiveness of the e-assist intervention in promoting colorectal cancer screening outcome fully observed; and the National Study of Learning Mindsets trial to evaluate the effectiveness of a growth mindset program in improving academic achievement outcomes MAR among ninth-graders.
- Published
- 2023
34. Comparing DIC and WAIC for Multilevel Models with Missing Data
- Author
-
Du, Han, Keller, Brian, Alacam, Egamaria, and Enders, Craig
- Abstract
In Bayesian statistics, the most widely used criteria of Bayesian model assessment and comparison are Deviance Information Criterion (DIC) and Watanabe-Akaike Information Criterion (WAIC). A multilevel mediation model is used as an illustrative example to compare different types of DIC and WAIC. More specifically, the study compares the performance of conditional and marginal DICs and WAICs, and investigates their performance with missing data. The study focuses on two versions of DIC (DIC[subscript 1] and DIC[subscript 2]) and one version of W AIC. In addition, the study explores whether it is necessary to include the nuisance models of incomplete exogenous variables in likelihood. Based on the simulation results, whether DIC[subscript 2] is better than DIC[subscript 1] and WAIC and whether including the nuisance models of exogenous variables in likelihood functions depends on whether using marginal or conditional likelihoods. Overall, the marginal likelihood based DIC[subscript 2] that excludes the likelihood of covariate models generally had the highest true model selection rates. [This paper will be published in "Behavior Research Methods."]
- Published
- 2023
35. Change in Engagement during Test Events: An Argument for Weighted Scoring?
- Author
-
Steven L. Wise, G. Gage Kingsbury, and Meredith L. Langi
- Abstract
Recent research has provided evidence that performance change during a student's test event can indicate the presence of test-taking disengagement. Meaningful performance change implies that some portions of the test event reflect assumed maximum performance better than others and, because disengagement tends to diminish performance, lower-performing portions are less likely to reflect maximum performance than higher-performing portions. This empirical study explored the use of differential weighting of item responses during scoring, with weighting schemes representing either declining or increasing performance. Results indicated that weighted scoring could substantially decrease the score distortion due to disengagement factors and thereby improve test score validity. The study findings support the use of scoring procedures that manage disengagement by adapting to student test-taking behavior.
- Published
- 2023
- Full Text
- View/download PDF
36. The Effect of Estimation Methods on SEM Fit Indices
- Author
-
Shi, Dexin and Maydeu-Olivares, Alberto
- Abstract
We examined the effect of estimation methods, maximum likelihood (ML), unweighted least squares (ULS), and diagonally weighted least squares (DWLS), on three population SEM (structural equation modeling) fit indices: the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the standardized root mean square residual (SRMR). We considered different types and levels of misspecification in factor analysis models: misspecified dimensionality, omitting cross-loadings, and ignoring residual correlations. Estimation methods had substantial impacts on the RMSEA and CFI so that different cutoff values need to be employed for different estimators. In contrast, SRMR is robust to the method used to estimate the model parameters. The same criterion can be applied at the population level when using the SRMR to evaluate model fit, regardless of the choice of estimation method.
- Published
- 2020
- Full Text
- View/download PDF
37. Examining Factors for the Academic Motivation Based on the Confirmatory, the Exploratory and the Bifactor Exploratory Structural Equation Models
- Author
-
Kartal, Seval Kula
- Abstract
One of the aims of the current study is to specify the model providing the best fit to the data among the exploratory, the bifactor exploratory and the confirmatory structural equation models. The study compares the three models based on the model data fit statistics and item parameter estimations (factor loadings, cross-loadings, factor correlations) provided by the models. The second aim of the study is to examine correlations among the factor scores, the academic achievement and the goal orientations to provide criterion validity for the person parameter estimations of the best-fitting model. The Academic Motivation Scale was conducted on 1,858 junior and senior students. The three measurement models were compared based on the model data fit and parameter estimations. All estimations were done on the Mplus 6.0 statistical program using the Maximum Likelihood method. It was found that the bifactor exploratory structural equation model provided only trivial improvement on the model data fit relative to the exploratory structural equation model. However, the results of the study revealed that including a general factor in the model achieved a decrease in item cross-loadings. In addition, items could lie along the relative autonomy continuum in a consistent way with the Self-Determination Theory according to their general factor loadings estimated by the bifactor model. The model data fit statistics, parameter estimations and correlation coefficients indicated that the bifactor exploratory structural equation better fit to data than the other two models both theoretically and statistically.
- Published
- 2020
38. Gaussian Variational Estimation for Multidimensional Item Response Theory
- Author
-
Cho, April E., Wang, Chun, Zhang, Xue, and Xu, Gongjun
- Abstract
Multidimensional Item Response Theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional integrals due to latent variable structure. Various methods have been proposed that either involve direct numerical approximations to the integrals or Monte Carlo simulations. However, these methods are known to be computationally demanding in high dimensions and rely on sampling data points from a posterior distribution. We propose a new Gaussian Variational EM (GVEM) algorithm which adopts a variational inference to approximate the intractable marginal likelihood by a computationally feasible lower bound. In addition, the proposed algorithm can be applied to assess the dimensionality of the latent traits in an exploratory analysis. Simulation studies are conducted to demonstrate the computational efficiency and estimation precision of the new GVEM algorithm in comparison to the popular alternative Metropolis-Hastings Robbins-Monro (MHRM) algorithm. In addition, theoretical results are also presented to establish the consistency of the estimator from the new GVEM algorithm. [This paper will be published in "British Journal of Mathematical and Statistical Psychology."]
- Published
- 2020
- Full Text
- View/download PDF
39. A Multiple Imputation Score Test for Model Modification in Structural Equation Models
- Author
-
Mansolf, Maxwell, Jorgensen, Terrence D., and Enders, Craig K.
- Abstract
Structural equation modeling (SEM) applications routinely employ a trilogy of significance tests that includes the likelihood ratio test, Wald test, and score test or modification index. Researchers use these tests to assess global model fit, evaluate whether individual estimates differ from zero, and identify potential sources of local misfit, respectively. This full cadre of significance testing options is not yet available for multiply imputed data sets, as methodologists have yet to develop a general score test for this context. Thus, the goal of this paper is to outline a new score test for multiply imputed data. Consistent with its complete-data counterpart, this imputation-based score test provides an estimate of the familiar expected parameter change statistic. The new procedure is available in the R package semTools and naturally suited for identifying local misfit in SEM applications (i.e., a model modification index). The article uses a simulation study to assess the performance (Type I error rate, power) of the proposed score test relative to the score test produced by full information maximum likelihood (FIML) estimation. Owing to the two-stage nature of multiple imputation, the score test exhibited slightly lower power than the corresponding FIML statistic in some situations but was generally well-calibrated. [The paper was published in "Psychological Methods."]
- Published
- 2020
40. Comparison of Confirmatory Factor Analysis Estimation Methods on Binary Data
- Author
-
Kilic, Abdullah Faruk, Uysal, Ibrahim, and Atar, Burcu
- Abstract
This Monte Carlo simulation study aimed to investigate confirmatory factor analysis (CFA) estimation methods under different conditions, such as sample size, distribution of indicators, test length, average factor loading, and factor structure. Binary data were generated to compare the performance of maximum likelihood (ML), mean and variance adjusted unweighted least squares (ULSMV), mean and variance adjusted weighted least squares (WLSMV), and Bayesian estimators. As a result of the study, it was revealed that increased average factor loading and sample size had a positive effect on the performance of the estimation methods. According to the research findings, it can be said that the methods are sufficient to estimate average factor loading and interfactor correlations, regardless of the estimation methods, in most of the conditions where the average factor loading is 0.7. In small sample sizes particularly, the interfactor correlation was underestimated for skewed indicator conditions. According to the findings of the study, although there is not the most accurate method in all conditions, it can be recommended to use ULSMV method because it performs adequately in more conditions.
- Published
- 2020
41. Location-Scale Models for Meta-Analysis
- Author
-
Viechtbauer, Wolfgang and López-López, José Antonio
- Abstract
Heterogeneity is commonplace in meta-analysis. When heterogeneity is found, researchers often aim to identify predictors that account for at least part of such heterogeneity by using mixed-effects meta-regression models. Another potentially relevant goal is to focus on the amount of heterogeneity as a function of one or more predictors, but this cannot be examined with standard random- and mixed-effects models, which assume a constant (i.e., homoscedastic) value for the heterogeneity variance component across studies. In this paper, we describe a location-scale model for meta-analysis as an extension of the standard random- and mixed-effects models that not only allows an examination of whether predictors are related to the size of the outcomes (i.e., their location), but also the amount of heterogeneity (i.e., their scale). We present estimation methods for such a location-scale model through maximum and restricted maximum likelihood approaches, as well as methods for inference and suggestions for visualization. We also provide an implementation via the "metafor" package for R that makes this model readily available to researchers. Location-scale models can provide a useful tool to researchers interested in heterogeneity in meta-analysis, with the potential to enhance the scope of research questions in the field of evidence synthesis.
- Published
- 2022
- Full Text
- View/download PDF
42. Assessing Rasch Measurement Estimation Methods across R Packages with Yes/No Vocabulary Test Data
- Author
-
Nicklin, Christopher and Vitta, Joseph P.
- Abstract
Instrument measurement conducted with Rasch analysis is a common process in language assessment research. A recent systematic review of 215 studies involving Rasch analysis in language testing and applied linguistics research reported that 23 different software packages had been utilized. However, none of the analyses were conducted with one of the numerous R-based Rasch analysis software packages, which generally employ one of the three estimation methods: conditional maximum likelihood estimation (CMLE), joint maximum likelihood estimation (JMLE), or marginal maximum likelihood estimation (MMLE). For this study, eRm, a CMLE-based R package, was utilized to conduct a dichotomous Rasch analysis of a Yes/No vocabulary test based on the academic word list. The resulting parameters and diagnostic statistics were compared with the equivalent results from four other R-based Rasch measurement software packages and Winsteps. Finally, all of the packages were utilized in the analysis of 1000 simulated datasets to investigate the extent to which results generated from the contrasting estimation methods converged or diverged. Overall, the differences between the results produced with the three estimation methods were negligible, and the discrepancies observed between datasets were attributable to the software choice as opposed to the estimation method.
- Published
- 2022
- Full Text
- View/download PDF
43. A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
- Author
-
Nagy, Gabriel and Ulitzsch, Esther
- Abstract
Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals' proficiency estimates relative to a conventional IRT model.
- Published
- 2022
- Full Text
- View/download PDF
44. The Effect of Latent and Error Non-Normality on Measures of Fit in Structural Equation Modeling
- Author
-
Jobst, Lisa J., Auerswald, Max, and Moshagen, Morten
- Abstract
Prior studies investigating the effects of non-normality in structural equation modeling typically induced non-normality in the indicator variables. This procedure neglects the factor analytic structure of the data, which is defined as the sum of latent variables and errors, so it is unclear whether previous results hold if the source of non-normality is considered. We conducted a Monte Carlo simulation manipulating the underlying multivariate distribution to assess the effect of the source of non-normality (latent, error, and marginal conditions with either multivariate normal or non-normal marginal distributions) on different measures of fit (empirical rejection rates for the likelihood-ratio model test statistic, the root mean square error of approximation, the standardized root mean square residual, and the comparative fit index). We considered different estimation methods (maximum likelihood, generalized least squares, and (un)modified asymptotically distribution-free), sample sizes, and the extent of non-normality in correctly specified and misspecified models to investigate their performance. The results show that all measures of fit were affected by the source of non-normality but with varying patterns for the analyzed estimation methods.
- Published
- 2022
- Full Text
- View/download PDF
45. Maximum Marginal Likelihood Estimation with an Expectation-Maximization Algorithm for Multigroup/Mixture Multidimensional Item Response Theory Models. Research Report. ETS RR-19-35
- Author
-
Fu, Jianbin
- Abstract
A maximum marginal likelihood estimation with an expectation-maximization algorithm has been developed for estimating multigroup or mixture multidimensional item response theory models using the generalized partial credit function, graded response function, and 3-parameter logistic function. The procedure includes the estimation of item parameters, attribute population distribution parameters, and test takers' attributes. All estimation functions and derivatives are provided. This procedure has been implemented in an R program. A simulation study has been conducted using this R program on various models related to the generalized partial credit function, and the result shows reasonable parameter recovery.
- Published
- 2019
46. Logistic Regression via Excel Spreadsheets: Mechanics, Model Selection, and Relative Predictor Importance
- Author
-
Brusco, Michael
- Abstract
Logistic regression is one of the most fundamental tools in predictive analytics. Graduate business analytics students are often familiarized with implementation of logistic regression using Python, R, SPSS, or other software packages. However, an understanding of the underlying maximum likelihood model and the mechanics of estimation are often lacking. This paper describes two Excel workbooks that can be used to enhance conceptual understanding of logistic regression in several respects: (i) by providing a clear formulation and solution of the maximum likelihood estimation problem; (ii) by showing the process for testing the significance of logistic regression coefficients; (iii) by demonstrating different methods for model selection to avoid overfitting, specifically, all possible subsets ordinary least squares regression and l[subscript 1]-regularized logistic regression (lasso); and (iv) by illustrating the measurement of relative predictor importance using all possible subsets.
- Published
- 2022
- Full Text
- View/download PDF
47. In Search of a Measure to Address Different Sources of Cognitive Load in Computer-Based Learning Environments
- Author
-
Dönmez, Onur, Akbulut, Yavuz, Telli, Esra, Kaptan, Miray, Özdemir, Ibrahim H., and Erdem, Mukaddes
- Abstract
In the current study, we aimed to develop a reliable and valid scale to address individual cognitive load types. Existing scale development studies involved limited number of items without adequate convergent, discriminant and criterion validity checks. Through a multistep correlational study, we proposed a three-factor scale with 13 items to address intrinsic, extraneous and germane cognitive load in computer-based learning environments. A thorough literature search of cognitive load indicators in the literature was followed by expert panels for content and construct validity. The initial item pool was administered to 236 undergraduate students who watched an instructional video on IP address classes. A multiple-choice achievement test was also implemented after the intervention. The exploratory factor analysis with maximum likelihood explained 58 percent of the variance with factor loads above 0.49, and internal consistency coefficients above 0.81. Convergent and discriminant validity indices were acceptable. Besides, achievement was related positively with the germane load and negatively with intrinsic and extraneous load. Fifteen percent of the achievement was explained by the three sources of the cognitive load. Then, the developed factor structure was validated with 193 undergraduate students immediately after they participated in online webinars, and with 99 undergraduate students after they participated in face to face classes. The proposed structure was confirmed in both settings so the scale was considered a reliable and valid indicator of cognitive load in both online and face-to-face learning environments.
- Published
- 2022
- Full Text
- View/download PDF
48. BIC Extensions for Order-Constrained Model Selection
- Author
-
Mulder, J. and Raftery, A. E.
- Abstract
The Schwarz or Bayesian information criterion (BIC) is one of the most widely used tools for model comparison in social science research. The BIC, however, is not suitable for evaluating models with order constraints on the parameters of interest. This article explores two extensions of the BIC for evaluating order-constrained models, one where a truncated unit information prior is used under the order-constrained model and the other where a truncated local unit information prior is used. The first prior is centered on the maximum likelihood estimate, and the latter prior is centered on a null value. Several analyses show that the order-constrained BIC based on the local unit information prior works better as an Occam's razor for evaluating order-constrained models and results in lower error probabilities. The methodology based on the local unit information prior is implemented in the R package "BICpack" which allows researchers to easily apply the method for order-constrained model selection. The usefulness of the methodology is illustrated using data from the European Values Study.
- Published
- 2022
- Full Text
- View/download PDF
49. Weak Separation in Mixture Models and Implications for Principal Stratification
- Author
-
Feller, Avi, Greif, Evan, Ho, Nhat, Miratrix, Luke, and Pillai, Natesh
- Abstract
Principal stratification is a widely used framework for addressing post-randomization complications. After using principal stratification to define causal effects of interest, researchers are increasingly turning to finite mixture models to estimate these quantities. Unfortunately, standard estimators of mixture parameters, like the MLE, are known to exhibit pathological behavior. We study this behavior in a simple but fundamental example, a two-component Gaussian mixture model in which only the component means and variances are unknown, and focus on the setting in which the components are weakly separated. In this case, we show that the asymptotic convergence rate of the MLE is quite poor, such as O(n[superscript -1/6]) or even O(n[superscript -1/8]). We then demonstrate via theoretical arguments as well as extensive simulations that, in finite samples, the MLE behaves like a threshold estimator, in the sense that the MLE can give strong evidence that the means are equal when the truth is otherwise. We also explore the behavior of the MLE when the MLE is non-zero, showing that it is difficult to estimate both the sign and magnitude of the means in this case. We provide diagnostics for all of these pathologies and apply these ideas to re-analyzing two randomized evaluations of job training programs, JOBS II and Job Corps. Our results suggest that the corresponding maximum likelihood estimates should be interpreted with caution in these cases.
- Published
- 2019
50. Analyzing the Maximum Likelihood Score Estimation Method with Fences in Ca-MST
- Author
-
Sahin, Melek Gülsah and Öztürk, Nagihan Boztunç
- Abstract
New statistical methods are being added to the literature as a result of scientific developments each and every day. This study aims at investigating one of these, Maximum Likelihood Score Estimation with Fences (MLEF) method, in ca-MST. The results obtained from this study will contribute to both national and international literature since there is no such study on the applicability of MLEF method in ca-MST. In line with the aim of this study, 48 conditions (4 module lengths (5-10-15-20) x 2 panel designs (1-3; 1-3-3) x 2 ability distribution (normal-uniform) x 3 ability estimation methods (MLEFMLE-EAP) were simulated and the data obtained from the simulation were interpreted with correlation, RMSE and AAD as an implication of measurement precision; and with conditional bias calculation in order to show the changes in each ability level. This study is a post-hoc simulation study using the data from TIMSS 2015 at the 8th grade in mathematics. "xxIRT" R package program and MSTGen simulation software tool were used in the study. As a result, it can be said that MLEF, as a new ability estimation method, is superior to MLE method in all conditions. EAP estimation method gives the best results in terms of the measurement precision based on correlation, RMSE and AAD values, whereas the results gained via MLEF estimation method are pretty close to those in EAP estimation method. MLE proves to be less biased in ability estimation, especially in extreme ability levels, when compared to EAP ability estimation method.
- Published
- 2019
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.