Analyzing Transitions in Sequential Data with Marginal Models

Various areas of educational research are interested in the transitions between different states--or events--in sequential data, with the goal of understanding the significance of these transitions; one notable example is affect dynamics, which aims to identify important transitions between affective states. Unfortunately, several works have uncovered issues with the metrics and procedures commonly used to analyze these transitions. As such, our goal in this work is to address these issues by outlining an alternative procedure that is based on the use of marginal models. We begin by looking at the specific mechanisms responsible for a recently discovered statistical bias with several metrics used in sequential data analysis. After giving a theoretical explanation for the issue, we show that the marginal model procedure appears to adjust for this bias. Next, a related problem is that the common practice of removing transitions to repeated states has been shown to have unintended side-effects--to account for this issue, we develop a method for extending the marginal model procedure to this specific type of analysis. Finally, in a recent study evaluating the problem of multiple comparisons and sequential data analysis, the Benjamini-Hochberg (BH) procedure, a commonly used approach to control for false discoveries, did not perform as expected. By applying a technique from the biostatistics and epidemiology literature, we show that the performance of the BH procedure, when used with the marginal model method, can be brought back to its expected level. In all of our analyses, we evaluate the proposed method by both running simulations and using actual student data. The results indicate that the marginal model procedure seemingly compensates for the problems observed with other transition metrics, thus resulting in more accurate estimates of the importance of transitions between states.