Search

We propose a novel model, Wide & Deep Item Response Theory (Wide & Deep IRT), to predict the correctness of students' responses to questions using historical clickstream data. This model combines the strengths of conventional Item Response Theory (IRT) models and Wide & Deep Learning for Recommender Systems. By leveraging clickstream data, Wide & Deep IRT provides precise predictions of answer correctness while enabling the exploration of behavioral patterns among different ability groups. Our experimental results based on a real-world dataset (EDM Cup 2023) demonstrate that Wide & Deep IRT outperforms conventional IRT models and state-of-the-art knowledge tracing models while maintaining the ease of interpretation associated with IRT models. Our model performed very well in the EDM Cup 2023 competition, placing second on the public leaderboard and third on the private leaderboard. Additionally, Wide & Deep IRT identifies distinct behavioral patterns across ability groups. In the EDMCup2023dataset, low-ability students were more likely to directly request an answer to a question before attempting to respond, which can negatively impact their learning outcomes and potentially indicates attempts to game the system. Lastly, the Wide & Deep IRT model consists of significantly fewer parameters compared to traditional IRT models and deep knowledge tracing models, making it easier to deploy in practice. The source code is available via Open Science Framework.

Historically, organizational researchers have fully embraced frequentist statistics and null hypothesis significance testing (NHST). Bayesian statistics is an underused alternative paradigm offering numerous benefits for organizational researchers and practitioners: e.g., accumulating direct evidence for the null hypothesis (vs. 'fail to reject the null'), capturing uncertainty across a distribution of population parameters (vs. a 95% confidence interval on a single point estimate) -- and through these benefits, communicating statistical findings more clearly. Although organizational methodologists in the past have promoted Bayesian methods, only now is easy-to-use JASP statistical software available for more widespread implementation. Moreover, the software is free to download and use, is menu-driven, and is supported by an active multidisciplinary user community. Using JASP, our tutorial compares and contrasts frequentist and Bayesian approaches for two analyses: a multiple linear regression analysis and a linear mixed regression analysis.

The procedures of carrying out factorial invariance to validate a construct were well developed to ensure the reliability of the construct that can be used across groups for comparison and analysis, yet mainly restricted to the frequentist approach. This motivates an update to incorporate the growing Bayesian approach for carrying out the Bayesian factorial invariance, as well as the frequentist approach, using the recent add-on R packages to show the procedures systematically for testing measurement equivalence via multigroup confirmatory factor analysis. The practical procedure and guidelines for carrying out factorial invariance under MCFA using a classic empirical example are demonstrated. Comparison between the frequentist and the Bayesian procedures and demonstration using priors are another two nuclei of the paper.

Learning a second language (L2) is dependent upon numerous external and internal factors, among which motivation plays a relevant role. In fact, motivation has been recognized as crucial in the L2 learning process (Ushioda, 2012). Such has been its importance that interest in L2 motivation has led to the development of theories such as the L2 motivational construct, and the L2 motivational self system (Dörnyei, 2005, 2009). Nevertheless, despite the academic focus on L2 learning motivation (Dörnyei & Ushioda, 2013), the impact of additional L2 lessons on students already engaged in formal L2 instruction at an official educational level (e.g., Higher Education) remaines vastly underexplored. Thus, this study aims to bridge this gap by analyzing the differences between 118 undergraduate EFL students who attended extra L2 lessons and those who did not. Considering the complex nature of the motivational construct, a Bayesian network analysis was used, categorizing motivations into two modules based on attendance of additional L2 lessons. This allowed us to observe the different factors of motivation as a whole construct, and not individually. The findings revealed that students who attended extra lessons are internally motivated toward self-improvement, whereas those who do not attend extra L2 lessons are influenced by external pressures and career aspirations.

Multimodal learning analytics (MMLA) research has shown the feasibility of building automated models of collaboration quality using artificial intelligence (AI) techniques (e.g., supervised machine learning (ML)), thus enabling the development of monitoring and guiding tools for computer-supported collaborative learning (CSCL). However, the practical applicability and performance of these automated models in authentic settings remains largely an under-researched area. In such settings, the quality of data features or attributes is often affected by noise, which is referred to as attribute noise. This paper undertakes a systematic exploration of the impact of attribute noise on the performance of different collaboration-quality estimation models. Moreover, we also perform a comparative analysis of different ML algorithms in terms of their capability of dealing with attribute noise. We employ four ML algorithms that have often been used for collaboration-quality estimation tasks due to their high performance: random forest, naive Bayes, decision tree, and AdaBoost. Our results show that random forest and decision tree outperformed other algorithms for collaboration-quality estimation tasks in the presence of attribute noise. The study contributes to the MMLA (and learning analytics (LA) in general) and CSCL fields by illustrating how attribute noise impacts collaboration-quality model performance and which ML algorithms seem to be more robust to noise and thus more likely to perform well in authentic settings. Our research outcomes offer guidance to fellow researchers and developers of (MM)LA systems employing AI techniques with multimodal data to model collaboration-related constructs in authentic classroom settings.

This research aims to compare the ability and item parameter estimations of Item Response Theory according to Maximum likelihood and Bayesian approaches in different Monte Carlo simulation conditions. For this purpose, depending on the changes in the priori distribution type, sample size, test length, and logistics model, the ability and item parameters estimated according to the maximum likelihood and Bayesian method and the differences in the RMSE of these parameters were examined. The priori distribution (normal, left-skewed, right-skewed, leptokurtic, and platykurtic), test length (10, 20, 40), sample size (100, 500, 1000), logistics model (2PL, 3PL). The simulation conditions were performed with 100 replications. Mixed model ANOVA was performed to determine RMSE differentiations. The prior distribution type, test length, and estimation method in the differentiation of ability parameter and RMSE were estimated in 2PL models; the priori distribution type and test length were significant in the differences in the ability parameter and RMSE estimated in the 3PL model. While prior distribution type, sample size, and estimation method created a significant difference in the RMSE of the item discrimination parameter estimated in the 2PL model, none of the conditions created a significant difference in the RMSE of the item difficulty parameter. The priori distribution type, sample size, and estimation method in the item discrimination RMSE were estimated in the 3PL model; the a priori distribution and estimation method created significant differentiation in the RMSE of the lower asymptote parameter. However, none of the conditions significantly changed the RMSE of item difficulty parameters.

Bayesian Knowledge Tracing (BKT) is a popular interpretable computational model in the educational mining community that can infer a student's knowledge state and predict future performance based on practice history, enabling tutoring systems to adaptively select exercises to match the student's competency level. Existing BKT implementations do not scale to large datasets and are difficult to extend and improve in terms of prediction accuracy. On the other hand, uninterpretable neural network (NN) student models, such as Deep Knowledge Tracing, enjoy the speed and modeling flexibility of popular computational frameworks (e.g., PyTorch, Tensorflow, etc.), making them easy to develop and extend. To bridge this gap, we develop a collection of BKT recurrent neural network (RNN) cells that are much faster than brute-force implementations and are within an order of magnitude of a fast, fine-tuned but inflexible C++ implementation. We leverage our implementation's modeling flexibility to create two novel extensions of BKT that significantly boost its performance. The first merges item response theory (IRT) and BKT by modeling multidimensional problem difficulties and student abilities without fitting student-specific parameters, allowing the model to easily generalize to new students in a principled way. The second extension discovers the discrete assignment matrix of problems to knowledge components (KCs) via stochastic neural network techniques and supports further guidance via problem input features and an auxiliary loss objective. Both extensions are learned in an end-to-end fashion; that is, problem difficulties, student abilities, and assignments to knowledge components are jointly learned with BKT parameters. In synthetic experiments, the skill discovery model can partially recover the true generating problem-KC assignment matrix while achieving high accuracy, even in some cases where the true KCs are structured unfavorably (interleaving sequences). On a real dataset where problem content is available, the skill discovery model matches BKT with expert-provided skills, despite using fewer KCs. On seven out of eight real-world datasets, our novel extensions achieve prediction performance that is within 0.04 AUC-ROC points of state-of-the-art models. We conclude by showing visualizations of the parameters and inferences to demonstrate the interpretability of our BKT RNN models on a real-life dataset.

Teachers routinely observe and interpret student behavior to make judgements about whether and how to support their students' learning. Simulated cases can help pre-service teachers to gain this skill of diagnostic reasoning. With 118 pre-service teachers, we tested whether participants rate simulated cases presented in a serial-cue case format as more authentic and become more involved with the materials compared to cases presented in a whole case format. We further investigated whether participants with varying prior conceptual knowledge (what are symptoms of ADHD and dyslexia) gain more strategic knowledge (how to detect ADHD and dyslexia) with a serial-cue versus whole case format. We found that the case format did not impact authenticity ratings but that learners reported higher involvement in the serial-cue case format condition. Bayes factors provide moderate evidence for the absence of a case format effect on strategic knowledge and strong evidence for the absence of an interaction of case format and prior knowledge. We recommend using serial-cue case formats in simulations as they are a more authentic representation of the diagnostic reasoning process and cognitively involve learners. We call for replications to gather more evidence for the impact of case format on knowledge acquisition. We suggest a further inquiry into the relationship of case format, involvement, and authenticity but think that a productive way forward for designing authentic simulations is attention to aspects that make serial-cue cases effective for diverse learners. For example, adaptive feedback or targeted practice of specific parts of diagnostic reasoning such as weighing evidence.

This analysis employs a Bayesian framework to estimate the impact of a Cognitive-Behavioral Therapy (CBT) intervention on the recidivism of high-risk people under community supervision. The study relies on the reanalysis of experimental datal using a Bayesian logistic regression model. In doing so, new estimates of programmatic impact were produced using weakly informative Cauchy priors and the Hamiltonian Monte Carlo method. The Bayesian analysis indicated that CBT reduced the prevalence of new charges for total, non-violent, property, and drug crimes. However, the effectiveness of the CBT program varied meaningfully depending on the participant's age. The probability of the successful reduction of drug offenses was high only for younger individuals (<26 years old), while there was an impact on property offenses only for older individuals (>26 years old). In general, the probability of the successful reduction of new charges was higher for the older group of people on probation. Generally, this study demonstrates that Bayesian analysis can complement the more commonplace Null Hypothesis Significance Test (NHST) analysis in experimental research by providing practically useful probability information. Additionally, the specific findings of the reestimation support the principles of risk-needs responsivity and risk-stratified community supervision and align with related findings, though important differences emerge. In this case, the Bayesian estimations suggest that the effect of the intervention may vary for different types of crime depending on the age of the participants. This is informative for the development of evidence-based correctional policy and effective community supervision programming.

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to corroborate process tracing claims. In particular, three steps are prescribed for an interventionist investigation, and each step in turn is shown to make the causal analysis more robust, amongst others by disambiguating causal claims and clarifying or strengthening the existing methodological advice on counterfactual analysis. The article's claims are then illustrated using a concrete example, Haggard and Kaufman's analysis of the Argentinian transition to democracy. It is shown that interventionism could have strengthened the authors' conclusions. The article concludes with a short Bayesian analysis of its key methodological proposals.

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed- or random effects meta-analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and measures used. PBF shines as a solution for small sample meta-analyses, where the number of between-study differences is often large relative to the number of studies, precluding the use of meta-regression to account for these differences. Users should be mindful of the fact that the PBF answers a qualitatively different research question than other evidence synthesis methods. For example, whereas fixed-effect meta-analysis estimates the size of a population effect, the PBF quantifies to what extent an informative hypothesis is supported in all included studies. This tutorial paper showcases the user-friendly PBF functionality within the bain R-package. This new implementation of an existing method was validated using a simulation study, available in an Online Supplement. Results showed that PBF had a high overall accuracy, due to greater sensitivity and lower specificity, compared to random-effects meta-analysis, individual participant data meta-analysis, and vote counting. Tutorials demonstrate applications of the method on meta-analytic and individual participant data. The example datasets, based on published research, are included in bain so readers can reproduce the examples and apply the code to their own data. The PBF is a promising method for synthesizing evidence for informative hypotheses across conceptual replications that are not suitable for conventional meta-analysis.

The application of network meta-analysis is becoming increasingly widespread, and for a successful implementation, it requires that the direct comparison result and the indirect comparison result should be consistent. Because of this, a proper detection of inconsistency is often a key issue in network meta-analysis as whether the results can be reliably used as a clinical guidance. Among the existing methods for detecting inconsistency, two commonly used models are the design-by-treatment interaction model and the side-splitting models. While the original side-splitting model was initially estimated using a Bayesian approach, in this context, we employ the frequentist approach. In this paper, we review these two types of models comprehensively as well as explore their relationship by treating the data structure of network meta-analysis as missing data and parameterizing the potential complete data for each model. Through both analytical and numerical studies, we verify that the side-splitting models are specific instances of the design-by-treatment interaction model, incorporating additional assumptions or under certain data structure. Moreover, the design-by-treatment interaction model exhibits robust performance across different data structures on inconsistency detection compared to the side-splitting models. Finally, as a practical guidance for inconsistency detection, we recommend utilizing the design-by-treatment interaction model when there is a lack of information about the potential location of inconsistency. By contrast, the side-splitting models can serve as a supplementary method especially when the number of studies in each design is small, enabling a comprehensive assessment of inconsistency from both global and local perspectives.

Self-monitoring is essential for effectively regulating learning, but difficult in visual diagnostic tasks such as radiograph interpretation. Eye-tracking technology can visualize viewing behavior in gaze displays, thereby providing information about visual search and decision-making. We hypothesized that individually adaptive gaze-display feedback improves posttest performance and self-monitoring of medical students who learn to detect nodules in radiographs. We investigated the effects of: (1) Search displays, showing which part of the image was searched by the participant; and (2) Decision displays, showing which parts of the image received prolonged attention in 78 medical students. After a pretest and instruction, participants practiced identifying nodules in 16 cases under search-display, decision-display, or no feedback conditions (n = 26 per condition). A 10-case posttest, without feedback, was administered to assess learning outcomes. After each case, participants provided self-monitoring and confidence judgments. Afterward, participants reported on self-efficacy, perceived competence, feedback use, and perceived usefulness of the feedback. Bayesian analyses showed no benefits of gaze displays for post-test performance, monitoring accuracy (absolute difference between participants' estimated and their actual test performance), completeness of viewing behavior, self-efficacy, and perceived competence. Participants receiving search-displays reported greater feedback utilization than participants receiving decision-displays, and also found the feedback more useful when the gaze data displayed was precise and accurate. As the completeness of search was not related to posttest performance, search displays might not have been sufficiently informative to improve self-monitoring. Information from decision displays was rarely used to inform self-monitoring. Further research should address if and when gaze displays can support learning.

Background: Traditionally, meta-analysis of time-to-event outcomes reports a single pooled hazard ratio assuming proportional hazards (PH). For health technology assessment evaluations, hazard ratios are frequently extrapolated across a lifetime horizon. However, when treatment effects vary over time, an assumption of PH is not always valid. The Royston-Parmar (RP), piecewise exponential (PE), and fractional polynomial (FP) models can accommodate non-PH and provide plausible extrapolations of survival curves beyond observed data. Methods: Simulation study to assess and compare the performance of RP, PE, and FP models in a Bayesian framework estimating restricted mean survival time difference (RMSTD) at 50 years from a pairwise meta-analysis with evidence of non-PH. Individual patient data were generated from a mixture Weibull distribution. Twelve scenarios were considered varying the amount of follow-up data, number of trials in a meta-analysis, non-PH interaction coefficient, and prior distributions. Performance was assessed through bias and mean squared error. Models were applied to a metastatic breast cancer example. Results: FP models performed best when the non-PH interaction coefficient was 0.2. RP models performed best in scenarios with complete follow-up data. PE models performed well on average across all scenarios. In the metastatic breast cancer example, RMSTD at 50-years ranged from -14.6 to 8.48 months. Conclusions: Synthesis of time-to-event outcomes and estimation of RMSTD in the presence of non-PH can be challenging and computationally intensive. Different approaches make different assumptions regarding extrapolation and sensitivity analyses varying key assumptions are essential to check the robustness of conclusions to different assumptions for the underlying survival function.

In relational event networks, the tendency for actors to interact with each other depends greatly on the past interactions between the actors in a social network. Both the volume of past interactions and the time that has elapsed since the past interactions affect the actors' decision-making to interact with other actors in the network. Recently occurred events may have a stronger influence on current interaction behavior than past events that occurred a long time ago--a phenomenon known as "memory decay". Previous studies either predefined a short-run and long-run memory or fixed a parametric exponential memory decay using a predefined half-life period. In real-life relational event networks, however, it is generally unknown how the influence of past events fades as time goes by. For this reason, it is not recommendable to fix memory decay in an ad-hoc manner, but instead we should learn the shape of memory decay from the observed data. In this paper, a novel semi-parametric approach based on Bayesian Model Averaging is proposed for learning the shape of the memory decay without requiring any parametric assumptions. The method is applied to relational event history data among socio-political actors in India and a comparison with other relational event models based on predefined memory decays is provided.

This study examined the preliminary feasibility, acceptability, and efficacy of an autism-adapted cognitive behavioral therapy for depression in autistic youth, CBT-DAY. Twenty-four autistic youth (11-17 years old) participated in the pilot non-randomized trial including 5 cisgender females, 14 cisgender males, and 5 non-binary youth. Youth participated in 12 weeks of, CBT-DAY and youth depressive symptoms (i.e., primary clinical outcome) and emotional reactivity and self-esteem (i.e., intervention mechanisms) were assessed through self-report and caregiver report at four timepoints: baseline (week 0), midpoint (week 6), post-treatment (week 12), and follow-up (week 24). Results suggested that CBT-DAY may be feasible (16.67% attrition) in an outpatient setting and acceptable to adolescents and their caregivers. Bayesian linear mixed-effects models showed that CBT-DAY may be efficacious in targeting emotional reactivity [[beta][subscript T1-T3] = -2.53, CrI[subscript 95%] (-4.62, -0.58), P[subscript d] = 0.995, d = -0.35] and self-esteem [[beta][subscript T1-T3] = -3.57, CrI[subscript 95%] (-5.17, -2.00), P[subscript d] > 0.999, d = -0.47], as well as youth depressive symptom severity [[beta] = -2.72, CrI[subscript 95%] (-3.85, -1.63), P[subscript d] > 0.999]. Treatment gains were maintained at follow-up. A cognitive behavioral group therapy designed for and with autistic people demonstrates promise in targeting emotional reactivity and self-esteem to improve depressive symptom severity in youth. Findings can be leveraged to implement larger, more controlled trials of CBT-DAY. The trial was registered at Clinicaltrials.gov (Identifier: NCT05430022; https://beta.clinicaltrials.gov/study/NCT05430022).

Marginal maximum likelihood, a common estimation method for item response theory models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for marginal maximum estimation. In this study, using sample sizes of 1,000 or smaller, not using priors often led to extreme, implausible parameter estimates. Applying prior distributions to the c-parameters alleviated the estimation problems with samples of 500 or more; for the samples of 100, priors on both the a-parameters and c-parameters were needed. Estimates were biased when the mode of the prior did not match the true parameter value, but the degree of the bias did not depend on the strength of the prior unless it was extremely informative. The root mean squared error (RMSE) of the a-parameters and b-parameters did not depend greatly on either the mode or the strength of the prior unless it was extremely informative. The RMSE of the c-parameters, like the bias, depended on the mode of the prior for c.

There is extensive literature focusing on students' misconceptions in various subject domains. Several conceptual change approaches have been trying to understand how conceptual change occurs to help learners handle these misconceptions. This meta-analysis aims to integrate studies investigating the effectiveness of three types of conceptual change strategy: cognitive conflict, cognitive bridging, and ontological category shift in science learning. We conducted a random-effects meta-analysis to calculate an overall effect size in Hedges' g with a sample of 218 primary studies, including 18,051 students. Our analyses resulted in a large overall effect size (g = 1.10, 95% CI [1.01, 1.19], k = 218, p < 0.001). We also performed a robust Bayesian meta-analysis to calculate an adjusted effect size, which specified a large effect (adjusted g = 0.93, 95% CI [0.68, 1.07], k = 218). Results are also consistent across the conceptual change strategies of cognitive conflict (g = 1.10, 95% CI [0.99, 1.21], k = 150, p < 0.001), cognitive bridging (g = 1.06, 95% CI [0.84, 1.28], k = 30, p < 0.001), and ontological category shift (g = 0.88, 95% CI [0.50, 1.26], k = 9, p < 0.001). However, a wide-ranging prediction interval [0.19, 2.38] points out a high level of heterogeneity in the distribution of effect sizes. Thus, we investigated the moderating effects of several variables using simple and multiple meta-regression. The final meta-regression model we created explained 35% of overall heterogeneity. This meta-analysis provides robust evidence that conceptual change strategies significantly enhance students' learning in science.

Extant literature suggests a positive correlation between social trust (also called generalized trust) and life satisfaction. However, the psychological pathways underlying this relationship can be complex. Using the Bayesian Mindsponge Framework (BMF), we examined the influence of social trust in a high-violence environment. Employing Bayesian analysis on a sample of 1,237 adults in Cali, Colombia, we found that in a linear relationship, generalized trust is positively associated with life satisfaction. However, in a model including the interactions between trust and education level as well as between trust and socioeconomic status, generalized trust is found to be negatively associated with life satisfaction. In this non-linear relationship, both education level and socioeconomic status have moderating effects against the negative association between generalized trust and life satisfaction. In other words, less educated people living in worse socioeconomic conditions are more likely to have lower life satisfaction when they have higher levels of social trust. In contrast, highly educated people living in better socioeconomic conditions are more likely to have higher life satisfaction when they have higher levels of social trust. Due to the facilitating function of trust in information processing, lowering the rigor of the filtering system in a high-violence social environment will likely put an individual at risk. Based on our findings, we suggest that policymakers should be more meticulous and consider many socioeconomic factors when advocating for increasing social trust. We also recommend that researchers should investigate deeper the complexity of human psychology and the information-processing mechanisms of social trust.

Free time in a working memory task often improves the recall performances of the to-be-remembered items. It is still debated whether the free-time effect in working memory is purely proactive, purely retroactive, or both proactive and retroactive. In the present study, we used the single-gap paradigm to explore this question. In Experiment 1, we measured the gap-length effect (i.e., the difference in memory performance elicited by the gap-length difference) under three long-short-gap combinations (i.e., 2,500 ms/100 ms, 2,500 ms/500 ms, 2,500 ms/1,000 ms). Proactive effects have been observed in all the three combinations whereas retroactive effects have only been found in two of them (i.e., 2,500 ms/100 ms, 2,500 ms/500 ms). To rule out the possibility that the retroactive effects found in Experiment 1 were simply due to the temporal grouping caused by the gap, in Experiment 2, the 2,500 ms/500 ms combination was retested, with the memory materials being changed from letters (the material used in Experiment 1) to words. The results showed that the range of the retroactive effect (i.e., the number of affected memory items prior to the gap) increased when the memory material changed from letters to words, which cannot be explained by temporal grouping. Taken together, the two experiments provided solid evidence that free time in working memory could produce both retroactive and proactive effects that cannot be explained by temporal grouping. These findings also provide insight into the underlying mechanism of working memory, for example, whether rehearsal would occur during the free time.

How do teachers learn about what learners already know? How do learners aid teachers by providing them with information about their background knowledge and what they find confusing? We formalize this collaborative reasoning process using a hierarchical Bayesian model of pedagogy. We then evaluate this model in two online behavioral experiments (N = 312 adults). In Experiment 1, we show that teachers select examples that account for learners' background knowledge, and adjust their examples based on learners' feedback. In Experiment 2, we show that learners strategically provide more feedback when teachers' examples deviate from their background knowledge. These findings provide a foundation for extending computational accounts of pedagogy to richer interactive settings.

Research has shown that multiple-indicator multiple-cause (MIMIC) models can result in inflated Type I error rates in detecting differential item functioning (DIF) when the assumption of equal latent variance is violated. This study explains how the violation of the equal variance assumption adversely impacts the detection of nonuniform DIF and how it can be addressed through moderated nonlinear factor analysis (MNLFA) model via Bayesian estimation approach to overcome limitations from the restrictive assumption. The Bayesian MNLFA approach suggested in this study better control Type I errors by freely estimating latent factor variances across different groups. Our experimentation with simulated data demonstrates that the BMNFA models outperform the existing MIMIC models, in terms of Type I error control as well as parameter recovery. The results suggest that the MNLFA models have the potential to be a superior choice to the existing MIMIC models, especially in situations where the assumption of equal latent variance assumption is not likely to hold.

Given the frequent presence of slipping and guessing in item responses, models for the inclusion of their effects are highly important. Unfortunately, the most common model for their inclusion, the four-parameter item response theory model, potentially has severe deficiencies related to its possible unidentifiability. With this issue in mind, the dyad four-parameter normal ogive (Dyad-4PNO) model was developed. This model allows for slipping and guessing effects by including binary augmented variables--each indicated by two items whose probabilities are determined by slipping and guessing parameters--which are subsequently related to a continuous latent trait through a two-parameter model. Furthermore, the Dyad-4PNO assumes uncertainty as to which items are paired on each augmented variable. In this way, the model is inherently exploratory. In the current article, the new model, called the Set-4PNO model, is an extension of the Dyad-4PNO in two ways. First, the new model allows for more than two items per augmented variable. Second, these item sets are assumed to be fixed, that is, the model is confirmatory. This article discusses this extension and introduces a Gibbs sampling algorithm to estimate the model. A Monte Carlo simulation study shows the efficacy of the algorithm at estimating the model parameters. A real data example shows that this extension may be viable in practice, with the data fitting a more general Set-4PNO model (i.e., more than two items per augmented variable) better than the Dyad-4PNO, 2PNO, 3PNO, and 4PNO models.

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's information criterion (DIC), sample size adjusted BIC (SABIC), relative entropy, the integrated classification likelihood criterion (ICL-BIC), the adjusted Lo-Mendell-Rubin (LMR), and Vuong-Lo-Mendell-Rubin (VLMR). The accuracy of the fit indices was assessed for correct detection of the number of latent classes for different simulation conditions including sample size (2,500 and 5,000), test length (15, 30, and 45), mixture proportions (equal and unequal), number of latent classes (2, 3, and 4), and latent class separation (no-separation and small separation). Simulation study results indicated that as the number of examinees or number of items increased, correct identification rates also increased for most of the indices. Correct identification rates by the different fit indices, however, decreased as the number of estimated latent classes or parameters (i.e., model complexity) increased. Results were good for BIC, CAIC, DIC, SABIC, ICL-BIC, LMR, and VLMR, and the relative entropy index tended to select correct models most of the time. Consistent with previous studies, AIC and AICc showed poor performance. Most of these indices had limited utility for three-class and four-class mixture 3PL model conditions.

Causal reasoning is a fundamental cognitive ability that enables individuals to learn about the complex interactions in the world around them. However, the mechanisms that underpin causal reasoning are not well understood. For example, it remains unresolved whether children's causal inferences are best explained by Bayesian inference or associative learning. The two experiments and computational models reported here were designed to examine whether 5- and 6-year-olds will retrospectively reevaluate objects--that is, adjust their beliefs about the causal status of some objects presented at an earlier point in time based on the observed causal status of other objects presented at a later point in time--when asked to reason about 3 and 4 objects and under varying degrees of information processing demands. Additionally, the experiments and models were designed to determine whether children's retrospective reevaluations were best explained by associative learning, Bayesian inference, or some combination of both. The results indicated that participants retrospectively reevaluated causal inferences under minimal information-processing demands (Experiment 1) but failed to do so under greater information processing demands (Experiment 2) and that their performance was better captured by an associative learning mechanism, with less support for descriptions that rely on Bayesian inference.

Longitudinal studies including a time-to-event outcome in social research often use a form of event history analysis to analyse the influence of time-varying endogenous covariates on the time-to-event outcome. Many standard event history models however assume the covariates of interest to be exogenous and inclusion of an endogenous covariate may lead to bias. Although such bias can be dealt with by using joint models for longitudinal and time-to-event outcomes, these types of models are underused in social research. In order to fill this gap in the social science modelling toolkit, we introduce a novel Bayesian joint model in which a multinomial longitudinal outcome is modelled simultaneously with a time-to-event outcome. The methodological novelty of this model is that it concerns a correlated random effects association structure that includes a multinomial longitudinal outcome. We show the use of the joint model on Danish labour market data and compare the joint model to a standard event history model. The joint model has three advantages over a standard survival model. It decreases bias, allows us to explore the relation between exogenous covariates and the longitudinal outcome and can be flexibly extended with multiple time-to-event and longitudinal outcomes.

The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article, we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of [tau], the between-study standard deviation, and the shrunken estimates of the study effects as a function of [tau]. With a small or moderate number of studies, [tau] is not estimated with much precision, and parameter estimates and shrunken study effect estimates can vary widely depending on the correct value of [tau]. The trace plot allows visualization of the sensitivity to [tau] along with a plot that shows which values of [tau] are plausible and which are implausible. A comparable frequentist or empirical Bayes version provides similar results. The concepts are illustrated using examples in meta-analysis and meta-regression; implementation in R is facilitated in a Bayesian or frequentist framework using the bayesmeta and metafor packages, respectively.

One of the most prominent social influences on human decision making is conformity, which is even more prominent when the perceptual information is ambiguous. The Bayes optimal solution to this problem entails weighting the relative reliability of cognitive information and perceptual signals in constructing the percept from self-sourced/endogenous and social sources, respectively. The current study investigated whether humans integrate the statistics (i.e., mean and variance) of endogenous perceptual and social information in a Bayes optimal way while estimating numerosities. Our results demonstrated adjustment of initial estimations toward group means only when group estimations were more reliable (or "certain"), compared to participants' endogenous metric uncertainty. Our results support Bayes optimal social conformity while also pointing to an implicit form of metacognition.

Teacher feedback is the key to online collaborative discussion. To investigate the effects of different forms of teacher feedback intervention on learners' cognitive and emotional interactions in online collaborative discussion, this study collected collaborative discussion text data of online collaborative learners. Based on the framework of Community of Inquiry theory, naive Bayes algorithm for automatic coding of collaborative discussion text data was adopted. A bivariate (with or without emotion/guidance) experiment was designed based on teacher feedback. The participants of this study were college students (N = 109, average age = 20) of normal major participating in Teaching System Design. They were randomly divided into four experimental groups. In each experimental group, 4-5 people work in a group for collaborative learning. This study adopts quasi experimental research method, and the experiment period is 10 class hours. Reliability analysis, automatic text coding and ANOVA of cognitive-affective variables were used to conduct process mining for the collaborative discussion of four groups of learners by using heuristic mining algorithms. It found that different forms of teacher feedback have different effects on learners' cognitive emotion. Teachers' emotional feedback promotes learners' emotional interaction and cognitive interaction, whiccoch is easier to promote learners' cognitive interaction. Different forms of teacher feedback promote four types of cognitive emotion interaction process. This suggests that the multi-branch type of voice prompt feedback group has the best effect on learners' cognitive and emotional impact.

Text is a burgeoning data source for psychological researchers, but little methodological research has focused on adapting popular modeling approaches for text to the context of psychological research. One popular measurement model for text, topic modeling, uses a latent mixture model to represent topics underlying a body of documents. Recently, psychologists have studied relationships between these topics and other psychological measures by using estimates of the topics as regression predictors along with other manifest variables. While similar two-stage approaches involving estimated latent variables are known to yield biased estimates and incorrect standard errors, two-stage topic modeling approaches have received limited statistical study and, as we show, are subject to the same problems. To address these problems, we proposed a novel statistical model-supervised latent Dirichlet allocation with covariates (SLDAX)-that jointly incorporates a latent variable measurement model of text and a structural regression model to allow the latent topics and other manifest variables to serve as predictors of an outcome. Using a simulation study with data characteristics consistent with psychological text data, we found that SLDAX estimates were generally more accurate and more efficient. To illustrate the application of SLDAX and a two-stage approach, we provide an empirical clinical application to compare the application of both the two-stage and SLDAX approaches. Finally, we implemented the SLDAX model in an open-source R package to facilitate its use and further study. Translational Abstract Text data is an increasingly popular data source in psychological research that can be analyzed with a variety of models and algorithms. Topic models are a popular measurement model that use latent variables to represent constructs underlying a set of documents (e.g., clinical interviews, survey open responses, written or spoken educational assessments). Recent applications have used estimates of these "topics" as predictors of other variables in a regression model, but the statistical behavior of this approach has not been well studied. Similar approaches with other latent variable models are known to yield incorrect regression coefficient estimates and incorrect inferences. We showed that the use of topic estimates as regression predictors is also prone to these problems. As a solution, we proposed a model that jointly estimates the topic model and regression model-supervised latent Dirichlet allocation with covariates (SLDAX). Using a simulation study under typical psychological text data conditions, we found that SLDAX estimates were generally more accurate and more precise than the two-stage approach. We illustrate the SLDAX and two-stage approaches in a clinical study of nonsuicidal self-injury and emotional dysregulation with participant interpersonal narratives. To allow researchers to apply the SLDAX model, we developed an open-source R software package.

Student performance is affected by their knowledge which changes dynamically over time. Therefore, employing recurrent neural networks (RNN), which are known to be very good in dynamic time series prediction, can be a suitable approach for student performance prediction. We propose such a neural network architecture containing two modules: (i) a dynamic sub-network including a recurrent Bi-GRU layer used for knowledge state estimation, (ii) a non-dynamic, feed-forward sub-network for predicting answer correctness based on the current question and current student knowledge state. The model modifies our previously proposed architecture and is different from all other existing models because it estimates the student's knowledge state considering only their previous responses. Thus the dynamic sub-network generates more stable knowledge state vector representations since they are independent of the current question. We studied both single-skill and multi-skill question scenarios and employed embeddings to represent questions and responses. In the multi-skill case the initialization of the question embedding matrix with pretrained word-embeddings is found to improve model performance. The experimental results showed that our current KT-Bi-GRU model and the previous one have similar performance while both surpassed the performance of previous state-of-the-art knowledge tracing models for five out of seven datasets where in some cases, the difference is quite noticeable.

Researchers often have hypotheses concerning the state of affairs in the population from which they sampled their data to compare group means. The classical frequentist approach provides one way of carrying out hypothesis testing using ANOVA to state the null hypothesis that there is no difference in the means and proceed with multiple comparisons if the null hypothesis is rejected. As this approach is not able to incorporate order, inequality, and direction into hypothesis testing, and neither does it able to specify multiple hypotheses, this paper introduces the informative hypothesis that allows more flexibility in stating hypothesis testing and is directly targeted to address and state the researcher's study concern. The two new hypothesis terms under the informative hypothesis framework, the unconstrained and complementary hypotheses are introduced, and the approaches to state the level of evidence using the Bayes factor and Generalization AIC are elaborated. As this hypothesis conception is relatively new and the literature was mostly technical, the main aims of the paper are to introduce this conception, offer a general guideline, and provide an easy-to-read approach to the procedure with practical examples of carrying out this hypothesis approach and contrast it to the frequentist, using the R package.

Classroom rules are a fundamental aspect of classroom management and ensuring compliance with established rules is crucial. Previous research has shown that students often pay little attention to the development of classroom rules. This quantitative study aims to investigate the expectations that students have concerning classroom rules. To this end, a 4-point Likert scale questionnaire consisting of 30 items was administered to 356 secondary school students. The Bayesian Search method and expert opinion were used to obtain a Bayesian Network model. The findings of the study indicate that students expect rules to be determined at the beginning of the academic year, wish to be involved in the determination process, and prefer minimal changes to the rules. They also expect a limited number of rules and reinforcement from teachers for displaying desirable behavior. Additionally, the study found that students are more likely to adhere to classroom rules in a clean and uncrowded environment, and prefer that their parents are not informed about these rules. The results also suggest that increased adherence to classroom rules leads to increased class inclusion, while decreased adherence results in decreased class inclusion. Furthermore, the study found that adoption of classroom rules leads to increased in-class cohesion, while non-adoption results in decreased cohesion. These findings contribute to the existing body of knowledge concerning student expectations of classroom rules.

We live in a digital age, not least accelerated by the COVID-19 pandemic. It is all the more important in our society that students learn and master the key competence of algorithmic thinking to understand the informatics concepts behind every digital phenomena and thus is able to actively shape the future. For this to be successful, concepts must be identified that can convey this key competence to all students in such a way that algorithmic thinking is integrated in the subject of informatics -beyond a pure programming course. Furthermore, based on the Legitimation Code Theory, semantic waves provide a way to develop and review lesson plans. Therefore, we planned a workshop, that follow the phases of a semantic wave addressing algorithmic problems using a block-based programming language. Considering this, we suggest the so-called SWAT concept (Semantic Wave Algorithmic Thinking concept), which is carried out and analyzed in a workshop with students. The workshop was carried out in online format in an 8th grade of a high school during a coronavirus lockdown. The level of algorithmic thinking was measured using a pretest and posttest both in the treatment group and in a control group and with the help of the approximate adjusted fractional Bayes factors for testing informative hypotheses statistically and through a reductive, qualitative content analysis of the students' work results (worksheets and created programs) evaluated. The semantic wave concept was measured using several cognitive load ratings of the students during the workshop and also statistically evaluated with the approximate adjusted fractional Bayes factors for testing informative hypotheses, as well as a qualitative content analysis of the worksheets. Results of this pilot study provide first insights, that the SWAT-concept can be used in combination of unplugged and plugged parts.

Bayesian Networks (BNs) are probabilistic graphical statistical models that have been widely used in many fields over the last decade. This method, which can also be used for educational data mining (EDM) purposes, is a fairly new method in education literature. This study models students' science success using the BN approach. Science is one of the core areas in the PISA exam. To this end, we used the data set including the most successful 25% and the least successful 25% students from Turkey based on their scores from Program for International Student Assessment (PISA) survey. We also made the feature selection to determine the most effective variables on success. The accuracy value of the BN model created with the variables determined by the feature selection is 86.2%. We classified effective variables on success into three categories; individual, family-related and school-related. Based on the analysis, we found that family-related variables are very effective in science success, and gender is not a discriminant variable in this success. In addition, this is the first study in the literature on the evaluation of complex data made with the BN model. In this respect, it serves as a guide in the evaluation of international exams and in the use of the data obtained.

Mixed Membership Models (MMMs) are a popular family of latent structure models for complex multivariate data. Instead of forcing each subject to belong to a single cluster, MMMs incorporate a vector of subject-specific weights characterizing partial membership across clusters. With this flexibility come challenges in uniquely identifying, estimating, and interpreting the parameters. In this article, we propose a new class of "Dimension-Grouped" MMMs (Gro-M[superscript 3]s) for multivariate categorical data, which improve parsimony and interpretability. In Gro-M[superscript 3]s, observed variables are partitioned into groups such that the latent membership is constant for variables within a group but can differ across groups. Traditional latent class models are obtained when all variables are in one group, while traditional MMMs are obtained when each variable is in its own group. The new model corresponds to a novel decomposition of probability tensors. Theoretically, we derive transparent identifiability conditions for both the unknown grouping structure and model parameters in general settings. Methodologically, we propose a Bayesian approach for Dirichlet Gro-M[superscript 3]s to inferring the variable grouping structure and estimating model parameters. Simulation results demonstrate good computational performance and empirically confirm the identifiability results. We illustrate the new methodology through applications to a functional disability survey dataset and a personality test dataset.

Correct specifications of hierarchical attribute structures in analyses using diagnostic classification models (DCMs) are pivotal because misspecifications can lead to biased parameter estimations and inaccurate classification profiles. This research is aimed to demonstrate DCM analyses with various hierarchical attribute structures via Bayesian estimation using freely available R packages, including CDM and R2"jags." We illustrated a step-by-step procedure in R with an eighth-grade mathematics test from the 2007 Trends in International Mathematics and Science Study (TIMSS).

We present an innovative modelling technique that simultaneously constrains student performance, course difficulty, and the sensitivity with which a course can differentiate between students by means of grades. Grade lists are the only necessary ingredient. Networks of courses will be constructed where the edges are populations of students that took both connected course nodes. Using idealized experiments and two real-world data sets, we show that the model, even though simple in its set-up, can constrain the properties of courses very well, as long as some basic requirements in the data set are met: (1) significant overlap in student populations, and thus information exchange through the network; (2) non-zero variance in the grades for a given course; and (3) some correlation between grades for different courses. The model can then be used to evaluate a curriculum, a course, or even subsets of students for a very wide variety of applications, ranging from program accreditation to exam fraud detection. We publicly release the code with examples that fully recreate the results presented here.

Building on previous studies investigating the multidimensional nature of lexical use in task-based L2 performance, this study clarified the roles that the distinct lexical features play in predicting vocabulary proficiency in a corpus of L2 Oral Proficiency Interviews (OPI). A total of 85 OPI samples were rated by three separate raters based on a Common European Frame of Reference (CEFR) based rubric in terms of their linguistic range. The interview transcription was analyzed for 56 lexical and phraseological indices using modern natural language processing tools. The result of an exploratory factor analysis (EFA) revealed that the 56 indices tapped into 10 distinct factors of lexical use in OPI: three factors related to content words, three related to n-grams, three lexical collocation factors, and one function-word factor. A subsequent Bayesian mixed-effect ordinal regression indicated that six out of the 10 factors meaningfully predicted the CEFR levels on Range with reasonable accuracy (quadratic kappa coefficient = 0.81 with the human rating). The result highlights the distinct roles that multiple content-word, collocation, and function-word factors play in characterizing the linguistic range in a CEFR-based assessment of OPI. The implication for the assessment of lexical richness, as well as future directions of this research domain, are discussed.