1. Indirect Effects in Multilevel Structural Equation Models: The Impact of Design Configuration and Cluster Size Imbalance
- Author
-
Nichols, Robert
- Subjects
- Educational Tests and Measurements, Quantitative Psychology, Statistics, Multilevel Structural Equation Modeling, Indirect Effects, Multilevel Mediation, Monte Carlo Simulation
- Abstract
Mediation analysis occupies a unique place in the social science literature. It allows researchers to go beyond testing whether an independent variable has an effect on a dependent variable; rather, it allows the researcher to examine the process or mechanism that enables that effect to occur. In its simplest form, a mediation model tests the effect that an independent variable, X, has on a dependent variable, Y, through its effect on an intervening variable, M. In this way, it provides a more complete and nuanced description of the relationship between X and Y. Researchers working within the field of education are often required to work with data that are not independent due to how the data are typically structured. The lack of independence inherent in the majority of educational data is caused by the multilevel structure of observations (e.g., students nested within classrooms, teachers nested within schools, and/or longitudinal observations, to name a few). This requires researchers to adjust their methods of analysis in order to properly model this lack of independence at the lowest level of the hierarchy. The estimation of mediated relationships becomes complicated when working in a multilevel context. Traditional approaches to multilevel modeling (MLM) can be used to model some mediated relationships, but it carries multiple limitations. A latent variable approach using multilevel structural equation modeling (MSEM) overcomes many of the limitations found in the MLM approach. This Monte Carlo simulation study examined the performance of MSEM when estimating two specific types of multilevel mediated relationships (the 2-1-1 and 2-1-2 mediation design configurations). Simulations were performed to investigate how well MSEM was able to estimate the indirect effect of X on Y through M in these two configurations under various ICCs, indirect effect sizes, number of clusters, and levels of cluster size imbalance. Model performance was assessed by model convergence rates, levels of bias in the parameter estimate or its standard error, estimate efficiency, confidence interval coverage, Type-I error rate, and statistical power. Results indicated that the design configuration had an effect on model performance through its interaction with the other design factors. Specifically, the 2-1-1 model tended to perform worse than the 2-1-2 model in terms of model convergence rates and bias in the standard error of the indirect effect under conditions of low ICC and low number clusters. In line with previous studies of multilevel mediation and of MSEM in general, low ICC was the most common design factor that led to poor performance on almost all outcome measures and was especially problematic when combined with a small number of clusters. The magnitude of the indirect effect, and often how that indirect effect was formed, also impacted model convergence, the amount of bias in the estimate of the indirect effect, estimate efficiency, confidence interval coverage, Type-I error rate, and statistical power. Additionally, results indicated that cluster size imbalance did not impact model performance under the selected study conditions.
- Published
- 2021