Search

Current interrater reliability (IRR) coefficients ignore the nested structure of multilevel observational data, resulting in biased estimates of both subject- and cluster-level IRR. We used generalizability theory to provide a conceptualization and estimation method for IRR of continuous multilevel observational data. We explain how generalizability theory decomposes the variance of multilevel observational data into subject-, cluster-, and rater-related components, which can be estimated using Markov chain Monte Carlo (MCMC) estimation. We explain how IRR coefficients for each level can be derived from these variance components, and how they can be estimated as intraclass correlation coefficients (ICC). We assessed the quality of MCMC point and interval estimates with a simulation study, and showed that small numbers of raters were the main source of bias and inefficiency of the ICCs. In a follow-up simulation, we showed that a planned missing data design can diminish most estimation difficulties in these conditions, yielding a useful approach to estimating multilevel interrater reliability for most social and behavioral research. We illustrated the method using data on student-teacher relationships. All software code and data used for this article is available on the Open Science Framework: https://osf.io/bwk5t/. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Background: Adolescents with mild-to-borderline intellectual disability (MBID) may be more susceptible to peer influence than typically developing (TD) adolescents.Aims: This preregistered study compares adolescents with MBID and TD adolescents on their susceptibility to peer influence and investigates links with three potential underlying mechanisms: inhibition, Theory of Mind (ToM) and negative interpretation bias.Methods and Procedures: We assessed 163 adolescents (111 MBID, 52 TD; 14-19 years; 63% boys) using experimental tasks and self- and/or teacher-reports.Outcomes and Results: Adolescents with MBID and TD adolescents did not differ in their susceptibility to peer influence, inhibition and negative interpretations. On two of the four ToM instruments, adolescents with MBID showed weaker ToM than TD adolescents. In a structural equation model, tested in the MBID group, the latent factors inhibition, ToM and negative interpretation bias were not related to susceptibility to peer influence.Conclusions and Implications: Using a multimethod approach, a structural equation model, and a large sample, this study revealed new insights in adolescents with MBID: they did not differ from TD adolescents in many domains and their inhibition, ToM and negative interpretation bias were not linked to their susceptibility to peer influence.

Research questionAre unmet needs for psychosocial counselling, peer support and friends/family support in parents directly and/or indirectly related to the mental health of parents and their donor-children?DesignA cross-sectional sample of 214 parents participated in this quantitative study via an online questionnaire. The sample comprised mothers and fathers in a heterosexual relationship (n = 85), mothers in a lesbian relationship (n = 67) and single mothers (n = 62). Parents were recruited via three Dutch fertility clinics and four network organizations. Unmet support needs were measured with an adapted version of the Unmet Needs for Parenting Support questionnaire, changing the original items into items about donor conception. The items were derived from a qualitative study and checked by experts in donor conception. The parents' mental health was measured with the Adult Self Report and the donor-children's mental health with the Child Behaviour Checklist. A multigroup mediation analysis was conducted to explore relationships between parents’ unmet support needs and their child's mental health, with the parents’ mental health as a possible mediator.ResultsThere were no direct relations between parents’ unmet support needs and the mental health of donor-children. Unmet needs for psychosocial counselling, peer support and friends/family support for parents and children's mental health were indirectly related through the mental health of the parents: 0.074 (CI 95% = 0.013–0.136; P = 0.017), 0.085 (CI 95% = 0.018–0.151; P = 0.036) and 0.063 (CI 95% = 0.019–0.106; P = 0.013), respectively.ConclusionsWe recommend that fertility clinics, network organizations and authorities for infertility counsellors make their support available to parents for extended periods after their treatment. Further qualitative studies are necessary to assess how to relieve unmet support needs during donor sperm treatment.

Interaction with unfamiliar partners is a component of social life from infancy onward. Yet little is known about preverbal communication with strangers. This study compared the development of infant communication with strangers to communication with mothers and fathers and examined the contribution of temperament to partner-specific communication patterns. A sample of 58 infants was observed at four and eight months during separate home-based face-to-face interactions with three partners (mother, father, and stranger). Infant visual, facial, and vocal communication behaviors were coded microanalytically. Each parent reported on infant temperament at both ages. Multilevel regression analyses indicated that infants gazed longer at strangers than at fathers, exhibited less smiling to strangers than to mothers, and produced fewer vocalizations with strangers than with either parent. Both age and temperament moderated these differences. Vocal communication with fathers became more frequent at eight months; smiling to mothers was accentuated among infants with higher levels of temperamental surgency. Importantly, levels of communication behaviors with strangers were concurrently and longitudinally associated with those with mothers and fathers. Overall, findings suggest that infant emotional communication patterns are modulated by individual temperamental differences and are reproduced in and over time, though at different levels, when interacting with novel partners.

Background Physicians increasingly show symptoms of burnout due to the high job demands they face, posing a risk for the quality and safety of care. Job and personal resources as well as support interventions may function as protective factors when demands are high, specifically in times of crisis such as the COVID-19 pandemic. Based on the Job Demands-Resources theory, this longitudinal study investigated how monthly fluctuations in job demands and job and personal resources relate to exhaustion and work engagement and how support interventions are associated with these outcomes over time. Methods A longitudinal survey consisting of eight monthly measures in the period 2020–2021, completed by medical specialists and residents in the Netherlands. We used validated questionnaires to assess job demands (i.e., workload), job resources (e.g., job control), personal resources (e.g., psychological capital), emotional exhaustion, and work engagement. Additionally, we measured the use of specific support interventions (e.g., professional support). Multilevel modeling and longitudinal growth curve modeling were used to analyze the data. Results 378 medical specialists and residents were included in the analysis (response rate: 79.08%). Workload was associated with exhaustion (γ = .383, p < .001). All job resources, as well as the personal resources psychological capital and self-judgement were associated with work engagement (γs ranging from -.093 to .345, all ps < .05). Job control and psychological capital attenuated the workload-exhaustion relationship while positive feedback and peer support strengthened it (all ps < .05). The use of professional support interventions (from a mental health expert or coach) was related to higher work engagement (estimate = .168, p = .032) over time. Participation in organized supportive group meetings was associated with higher exhaustion over time (estimate = .274, p = .006). Conclusions Job and personal resources can safeguard work engagement and mitigate the risk of emotional exhaustion. Professional support programs are associated with higher work engagement over time, whereas organized group support meetings are associated with higher exhaustion. Our results stress the importance of professional individual-level interventions to counteract a loss of work engagement in times of crisis.

Abstract. The objective of the present study was to examine whether different methods for dealing with dependency in meta-analytic structural equation modeling (MASEM) lead to different results. Four different methods for dealing with dependent effect sizes in MASEM were applied to empirical data, including: (1) ignoring dependency; (2) aggregation; (3) elimination; and (4) a multilevel approach. Random-effects two-stage structural equation modeling was conducted for each method separately, and potential moderators were examined using subgroup analysis. Results demonstrated that the different methods of dealing with dependency in MASEM lead to different results. Thus, the decision on which approach should be used in MASEM-analysis should be carefully considered. Given that the multilevel approach is the only approach that includes all available information while explicitly modeling dependency, it is currently the theoretically preferred approach for dealing with dependency in MASEM. Future research should evaluate the multilevel approach with simulated data.

Intensive longitudinal data is increasingly used to study state-like processes such as changes in daily stress. Measures aimed at collecting such data require the same level of scrutiny regarding scale reliability as traditional questionnaires. The most prevalent methods used to assess reliability of intensive longitudinal measures are based on the generalizability theory or a multilevel factor analytic approach. However, the application of recent improvements made for the factor analytic approach may not be readily applicable for all researchers. Therefore, this article illustrates a five-step approach for determining reliability of daily data, which is one type of intensive longitudinal data. First, we show how the proposed reliability equations are applied. Next, we illustrate how these equations are used as part of our five-step approach with empirical data, originating from a study investigating changes in daily stress of secondary school teachers. The results are a within-level (ωw), between-level (ωb) reliability score. Mplus syntax for these examples is included and discussed. As such, this paper anticipates on the need for comprehensive guides for the analysis of daily data.

Purpose Mokken scale analysis (MSA) is an attractive scaling procedure for ordinal data. MSA is frequently used in health-related quality of life research. Two of MSA's prime features are the scalability coefficients and the automated item selection procedure (AISP). The AISP partitions a (large) set of items into scales based on the observed item scores; the resulting scales can be used as measurement instruments. There exist two issues in MSA: First, point estimates, standard errors, and test statistics for scalability coefficients are inappropriate for clustered item scores, which are omnipresent in quality of life research data. Second, the AISP insufficiently takes sampling fluctuation of Mokken’s scalability coefficients into account. Methods We solved both issues by providing point estimates and standard errors for the scalability coefficients for clustered data and by implementing a Wald-based significance test in the AISP algorithm, resulting in a test-guided AISP (T-AISP), that is available for both nonclustered and clustered test scores. Results We integrated the T-AISP into a two-step, test-guided MSA for scale construction, to guide the analysis for nonclustered and clustered data. The first step is performing a T-AISP and select the final scale( s ). For clustered data, within-group dependency is investigated on the final scale(s). In the second step, the strength of the scale(s) is determined and further analyses are performed. The procedure was demonstrated on clustered item scores obtained from administering a questionnaire on quality of life in schools to 639 students nested in 30 classrooms. Conclusions We developed a two-step, test-guided MSA for scale construction that takes into account sample fluctuation of all scalability coefficients and that can be applied to item scores obtained by a nonclustered or clustered sampling design.

The central question of this dissertation was how to estimate the interrater reliability (IRR) from incomplete and dependent observational data. IRR expresses the degree to which subjects' attribute scores are independent of raters. Various IRR coefficients have been proposed to estimate the IRR from data, of which the intraclass correlation coefficients and Cohen’s Kappa are probably most famous. Most of these IRR coefficients are defined for a two-way observational design in which multiple independent subjects are each rated by multiple independent raters, and these raters are the same for each subject. However, in social and behavioral observational research, the raters are often not the same for each subject, resulting in incomplete data. Also, the rated subjects are often nested within clusters or relationships, resulting in dependent data. Prior to the developments in this dissertation, none of the conceptualizations of IRR could be readily used for both incomplete and dependent data, resulting in less or non-informative IRR coefficients. Using the framework of Generalizability theory (GT), I extended the different definitions of intraclass correlation coefficients for IRR to incomplete and dependent data. Using simulation studies, I tested various estimation methods to estimate these intraclass correlation coefficients from such data. Throughout the different chapters, I provide guidance in planning observational studies. To guide researchers in using these methods, the chapters contain applications of the estimation methods to empirical data from clinical and developmental domains. Ultimately, these methods reach a wide range of researchers, to eventually lead to more reliable observational research.

This longitudinal study investigated reciprocal associations among various professional identity tensions and Dutch primary student teachers’ teacher identity. Students (N = 201, 82.9% female) completed the professional identity tensions scale and the teacher identity measurement scale across three waves. Random intercept cross-lagged panel models showed that five out of nine investigated professional identity tensions were negatively associated with teacher identity at the inter-individual level. At the intra-individual level, no significant cross-lagged relationships were detected. Our findings imply that the development of professional identity tensions and teacher identity are not automatically interrelated and should, therefore, be both explicitly addressed in teacher education. The development of the relationship between professional identity tensions and teacher identity: A quantitative longitudinal study among Dutch primary student teachers.

Objectives We evaluated the effects of the family mindfulness-based intervention (MBI) “MYmind” for children with ADHD and their parents, and examined child and parent predictors of child outcome. Methods Using a pragmatic quasi-experimental waitlist design, children aged 7–19 years (n = 167), clinically referred with a DSM-IV ADHD diagnosis, and both their parents completed waitlist (average waiting time was 8 weeks), pre-test, post-test, 8-week, and 1-year follow-up measurements. MYmind consisted of eight weekly 1.5-h mindfulness-based group sessions for children and parallel for parents, and a follow-up session. We assessed children’s and both parents’ ADHD symptoms and other psychopathology, child executive function, parental stress, parental overreactivity, and mindful parenting. Results Multilevel analyses revealed medium-to-large effect-sized reduced child ADHD symptoms between pre- and post-test, becoming stronger at follow-ups, while no waitlist effects occurred. Parents above the ADHD threshold improved on adult ADHD symptoms with similar sized effects. Children’s and parents’ other psychopathology, child executive function, parental overreactivity, and mindful parenting improved, whereas parental stress only improved at 1-year follow-up. Child age, child gender, ADHD medication, parental ADHD, and parent participation did not predict child outcome. Parent gender however interacted with parental ADHD to predict child outcome; children of fathers (but not mothers) above the ADHD threshold improved more than children of fathers below the ADHD threshold at post-test and at 8-week follow-up. Reduced paternal ADHD from pre- to post-test mediated this effect. Conclusions Family MBI (MYmind) may reduce childhood ADHD and improve parental functioning. Fathers with ADHD symptoms appear important in helping offspring with ADHD.

The association between two ordinal variables can be expressed with a polychoric correlation coefficient. This coefficient is conventionally based on the assumption that responses to ordinal variables are generated by two underlying continuous latent variables with a bivariate normal distribution. When the underlying bivariate normality assumption is violated, the estimated polychoric correlation coefficient may be biased. In such a case, we may consider other distributions. In this paper, we aimed to provide an illustration of fitting various bivariate distributions to empirical ordinal data and examining how estimates of the polychoric correlation may vary under different distributional assumptions. Results suggested that the bivariate normal and skew-normal distributions rarely hold in the empirical datasets. In contrast, mixtures of bivariate normal distributions were often not rejected.

Conducting a power analysis can be challenging for researchers who plan to analyze their data using structural equation models (SEMs), particularly when Monte Carlo methods are used to obtain power. In this tutorial, we explain how power calculations without Monte Carlo methods for the χ2 test and the RMSEA tests of (not-)close fit can be conducted using the Shiny app “power4SEM”. power4SEM facilitates power calculations for SEM using two methods that are not computationally intensive and that focus on model fit instead of the statistical significance of (functions of) parameters. These are the method proposed by Satorra and Saris (Psychometrika 50(1), 83–90, 1985) for power calculations of the likelihood ratio test, and that described by MacCallum, Browne, and Sugawara (Psychol Methods 1(2) 130–149, 1996) for RMSEA-based power calculations. We illustrate the use of power4SEM with examples of power analyses for path models, factor models, and a latent growth model.

The choice of cortisol sampling times in early childhood studies varies widely. Given that recommendations on sampling protocols are largely based on adults, the present study aimed to broaden current knowledge by examining how reliably cortisol measures obtained at different daytimes would reveal between-individual differences in toddlers’ cortisol levels. Parents were instructed to take 10 saliva samples consecutively (five per day) from their toddler (N = 19; Mage = 15.8 months, SDage = 4.2 months). Intra-class correlations (ICCs) were computed to evaluate cortisol reliability. Cortisol samples taken in the morning between 30 and 80 min after awakening and bedtime samples were most reliable in differentiating between children (ICCs ≥.80). Wake-up cortisol samples taken within the first 30 min after awakening and afternoon samples showed moderate reliabilities (ICCs = .64), whereas the reliability of noon samples was poor (ICC = .43). Therefore, when investigating cortisol in young children while being restricted to a few samples only, assessing cortisol in the morning (at least 30 min after awakening) and at bedtime would be advisable.

Meta‐analytic structural equation modeling (MASEM) refers to fitting structural equation models (SEMs) (such as path models or factor models) to meta‐analytic data. Currently, fitting MASEMs may be challenging for researchers that are not accustomed to working with R software and packages. Therefore, we developed webMASEM; a web application for MASEM. This app implements the one‐stage MASEM approach, and allows users to apply MASEM in a user‐friendly way. The aim of this article is to provide a tutorial on one‐stage MASEM and a practical guide to webMASEM. We will pay specific attention to how the data should be structured and prepared for webMASEM, because mistakes in this step may lead to faulty results without receiving an error message. The use of webMASEM is illustrated with an analysis of a meta‐analytic path model in which the path coefficients are moderated by a study‐level variable, a meta‐analytic factor model in which the factor loadings are moderated by a study‐level variable, and a meta‐analytic panel model in which the effects are moderated by a study‐level variable. All used datafiles and R scripts are available online.

Background: Researchers frequently use the responses of individuals in clusters to measure cluster-level constructs. Examples are the use of student evaluations to measure teaching quality, or the use of employee ratings of organizational climate. In earlier research, Stapleton and Johnson (2019) provided advice for measuring cluster-level constructs based on a simulation study with inadvertently confounded design factors. We extended their simulation study using both Mplus and lavaan to reveal how their conclusions were dependent on their study conditions. Methods: We generated data sets from the so-called configural model and the simultaneous shared-and-configural model, both with and without nonzero residual variances at the cluster level. We fitted models to these data sets using different maximum likelihood estimation algorithms. Results: Stapleton and Johnson’s results were highly contingent on their confounded design factors. Convergence rates could be very different across algorithms, depending on whether between-level residual variances were zero in the population or in the fitted model. We discovered a worrying convergence issue with the default settings in Mplus, resulting in seemingly converged solutions that are actually not. Rejection rates of the normal-theory test statistic were as expected, while rejection rates of the scaled test statistic were seriously inflated in several conditions. Conclusions: The defaults in Mplus carry specific risks that are easily checked but not well advertised. Our results also shine a different light on earlier advice on the use of measurement models for shared factors.

Objective: PROMIS offers computerized adaptive tests (CAT) of patient-reported outcomes, using a single set of US-based IRT item parameters across populations and language-versions. The use of country-specific item parameters has local appeal, but also disadvantages. We illustrate the effects of choosing US or country-specific item parameters on PROMIS CAT T-scores. Study design and setting: Simulations were performed on response data from Dutch chronic pain patients (n = 1110) who completed the PROMIS Pain Behavior item bank. We compared CAT T-scores obtained with (1) US parameters; (2) Dutch item parameters; (3) US item parameters for DIF-free items and Dutch item parameters (rescaled to the US metric) for DIF items; (4) Dutch item parameters for all items (rescaled to the US metric). Results: Without anchoring to a common metric, CAT T-scores cannot be compared. When scores were rescaled to the US metric, mean differences in CAT T-scores based on US vs. Dutch item parameters were negligible. However, 0.9%–4.3% of the T-score differences were larger than 5 points (0.5 SD). Conclusion: The choice of item parameters can be consequential for individual patient scores. We recommend more studies of translated CATs to examine if strategies that allow for country-specific item parameters should be further investigated.

Structural equation modeling (SEM) has been proposed to estimate generalizability theory (GT) variance components, primarily focusing on estimating relative error to calculate generalizability coefficients. Proposals for estimating absolute-error components have given the impression that a separate SEM must be fitted to a transposed data matrix. This paper uses real and simulated data to demonstrate how a single SEM can be specified to estimate absolute error (and thus dependability) by placing appropriate constraints on the mean structure, as well as thresholds (when used for ordinal measures). Using the R packages lavaan and gtheory, different estimators are compared for normal and discrete measurements. Limitations of SEM for GT are demonstrated using multirater data from a planned missing-data design, and an important remaining area for future development is discussed.

This study compared two single-group approaches for assessing measurement invariance across an observed background variable: restricted factor analysis (RFA) and moderated nonlinear factor analysis (MNLFA). In MNLFA models, heteroskedasticity can be accounted for by allowing the common-factor variance and the residual variances to differ as a function of the background variable. In contrast, RFA models assume homoskedasticity of both the common factor and the residuals. We conducted a simulation study to examine the performance of RFA and MNLFA under common-factor and residual homoskedasticity and heteroskedasticity. Results suggest that MNLFA and RFA with product indicators outperform RFA with latent moderated structural equations in conditions with heteroskedastic common-factors, and MNLFA outperforms RFA in conditions with residual heteroskedasticity. We provide an explanation for the robustness of RFA with product indicators to violations of common-factor homoskedasticity.

Objective: Pain interference and pain behavior are highly relevant outcomes in patients with rheumatoid arthritis (RA). The Patient-Reported Outcomes Measurement Information System (PROMIS) is a universally applicable set of item banks measuring patient-reported health, and if applied as computerized adaptive tests (CATs), more efficiently and precisely than current instruments. The objective was to study the psychometric properties of the Dutch-Flemish PROMIS pain interference (PROMIS-PI) and the PROMIS pain behavior (PROMIS-PB) item banks in patients with RA. Methods: A total of 2,029 patients with RA completed the full PROMIS-PI (version 1.1, 40 items), and 1,554 patients completed the full PROMIS-PB (version 1.1, 39 items). The following psychometric properties were studied: unidimensionality, local dependence, monotonicity and graded response model (GRM) fit, cross-cultural validity (differential item functioning [DIF] for language [Dutch versus Flemish]), other forms of measurement invariance, construct validity, reliability, and floor and ceiling effects. Results: The PROMIS-PI and PROMIS-PB banks were sufficiently unidimensional (Omega-hierarchical [Omega-H] 0.99, 0.95, and explained common variance 0.95, 0.78, respectively), had negligible local dependence (0.3–1.4% of item pairs), good monotonicity (H 0.75, 0.46), and a good GRM model fit (no misfitting items). Furthermore, both item banks showed good cross-cultural validity (no DIF for language), measurement invariance (no DIF for age, sex, administration mode, and disease activity), good construct validity (all hypotheses met), high reliability (>0.90 in the range of patients with RA), and an absence of floor and ceiling effects (0% minimum or maximum score, respectively). Conclusion: Both PROMIS-PI and PROMIS-PB banks showed good psychometric properties in patients with RA and can be used as CATs in research and clinical practice.

In psychological assessment of children, it is pivotal to establish from what age on self-reports can complement or replace informant reports. We introduce a psychometric approach to estimate the minimum age for a child to produce self-report data that is of similar quality as informant data. The approach makes use of statistical validity indicators such as person-fit and long-string indices, and can be readily applied to data commonly collected in psychometric studies of child measures. We evaluate and illustrate the approach, using self-report and informant-report data of the PedsQL, a pediatric health-related quality of life measure, from 651 child–mother pairs. To evaluate the approach, we tested various hypotheses about the validity of the self-report data, using the [Formula: see text] person-fit index as the validity indicator and the mother informant-data as a benchmark for validity. Results showed that [Formula: see text] discriminated between self-reports of younger and older children, between self-reports of children that completed the PedsQL alone or with a parent, and between self-reports and informant reports. We conclude that the validity-index approach has good potential for future applications. Future research should further evaluate the approach for different types of questionnaires (e.g., personality inventories) and using different validity indices (e.g., response-bias indices).

Popular belief holds that sexual behavior is evaluated more liberally for males than females. However, the assessment of this “sexual double standard” is controversial. Therefore, we investigated measurement equivalence of commonly used items to assess sexual double standards in previous research. Based on established measurement equivalence, we investigated whether adolescents endorsed a sexual double standard. Using data from 455 adolescents ( Mage = 14.51, SD = 0.64), confirmatory factor analyzes showed that the sexual double standard concept was measurement equivalent across sex, and partly across evaluations of the same and opposite sex. Factor analyzes demonstrated that there was not one, but two sexual double standards. Male adolescents evaluated male sexual behavior more liberally than female sexual behavior, but female adolescents evaluated female sexual behavior more liberally than male sexual behavior. This contradicts the traditional notion of the existence of one sexual double standard that favors male and suppresses female sexuality.

For the construction of tests and questionnaires that require multiple raters (e.g., a child behaviour checklist completed by both parents) a novel ordinal scaling technique is currently being further developed, called two-level Mokken scale analysis. The technique uses within-rater and between-rater coefficients to assess the scalability of the test. These coefficients are generalizations of Mokken's scalability coefficients. In this paper we derived standard errors for the two-level coefficients and for their ratios. The coefficients, the estimates, the estimated standard errors and the software implementation are discussed and illustrated using a real-data example, and a small-scale simulation study demonstrates the accuracy of the estimates.

It is often unrealistic to assume normally distributed latent traits in the measurement of health outcomes. If normality is violated, the item response theory (IRT) models that are used to calibrate questionnaires may yield parameter estimates that are biased. Recently, IRT models were developed for dealing with specific deviations from normality, such as zero-inflation (“excess zeros”) and skewness. However, these models have not yet been evaluated under conditions representative of item bank development for health outcomes, characterized by a large number of polytomous items. A simulation study was performed to compare the bias in parameter estimates of the graded response model (GRM), polytomous extensions of the zero-inflated mixture IRT (ZIM-GRM), and Davidian Curve IRT (DC-GRM). In the case of zero-inflation, the GRM showed high bias overestimating discrimination parameters and yielding estimates of threshold parameters that were too high and too close to one another, while ZIM-GRM showed no bias. In the case of skewness, the GRM and DC-GRM showed little bias with the GRM showing slightly better results. Consequences for the development of health outcome measures are discussed.

Background: Meta-analysis (Glass, 1976) is a commonly used statistical technique to aggregate sample effect sizes of different independent primary studies in order to draw inferences concerning population effects. To extend the range of research questions that can be answered, new meta-analytic models have been developed, such as meta-analytic structural equation modeling (MASEM) (Becker, 1992, 1995; Cheung, 2014, 2015a; Cheung & Chang, 2005; Jak, 2015; Viswesvaran & Ones, 1995). In primary studies, an effect size may represent the strength and direction of the association between any two variables of interest. Such an effect size can be expressed in different ways, for example as Pearson product-moment correlation, Cohens’ d, biserial correlation, and point-biserial correlation. How an effect size is expressed depends on the nature of the variables (e.g., continuous or dichotomous), but also on the way the variables are measured or analyzed. If one of the two continuous variables is artificially dichotomized, one may express the effect size as a point-biserial correlation. However, this typically provides a negatively biased estimate of the true underlying Pearson product-moment correlation (e.g., Cohen, 1983; MacCallum, Zhang, Preacher, & Rucker, 2002). The biserial correlation on the other hand should generally provide an unbiased estimate (Soper, 1914; Tate, 1955). Bias in the effect size of any primary study may affect meta-analytic results in the same direction (Jacobs & Viechtbauer, 2017). Therefore, we may expect that the use of the point-biserial correlation for the relationship between an artificially dichotomized and continuous variable also biases MASEM-parameters. In the current study we will evaluate how using point-biserial correlations versus biserial correlations from primary studies may affects path coefficients, their standard errors, and model fit in MASEM. Based on the results, we expect to be able to inform researchers about which of the two investigated effect sizes is the most appropriate to use in MASEM-applications and under which conditions. Aim: Our aim is to investigate the effects of using (1) the point-biserial correlation and (2) the biserial correlation for the relationship between an artificially dichotomized variable and a continuous variable on MASEM-parameters and model fit. Specifically, our interest lies in path coefficients, standard errors of these coefficients, and model fit. Method: We simulated meta-analytic data according to a full mediation (hence overidentified) population model (see Figure 1), with a continuous predictor variable X, continuous mediator M, and a continuous variable Y as outcome. Depending on the condition, the predictor variable X is artificially dichotomized in all or a given percentage of the primary studies. We chose this population model because in educational research the median number of variables in a ‘typical’ meta-analysis is three (de Jonge & Jak, 2018) and because mediation is a popular research topic. Figure 1. Population model with fixed parameter values. Under this population model, random meta-analytic datasets were generated under different conditions. We systematically varied the following: (1) the size of the (standardized) path coefficient between X and M (.16, .23, .33), (2) the percentage of primary studies in which X was artificially dichotomized (25%, 75%, 100%), and (3) the cut-off point at which X was artificially dichotomized (at the median value, so a proportion of .05, or when groups become unbalance, at a proportion of .01). These choices were mainly based on typical situations in educational research. The size of the path coefficient, reflect the minimum, mean/median, and maximum pooled Pearson product-moment correlations in a ‘typical’ meta-analysis in educational research (de Jonge & Jak, 2018). The 75% primary studies that artificially dichotomize the variable X, is based on a comparable example of a meta-analysis in educational research (Jansen, Elffers, & Jak, 2019). We used between-study variances of .01. The number of primary studies in a meta-analysis was fixed at the median number of a ‘typical’ meta-analysis, which is 44 (de Jonge & Jak, 2018). Because we use a random-effects MASEM-method, the assumption is thus that the population comprises 44 subpopulations from which the 44 samples are drawn, and that the weighted mean of the subpopulation parameters equals the population parameter. Given a specific condition and the fixed number of 44 primary studies, we randomly sampled the within primary study sample sizes from a positively skewed distribution as used in Hafdahl (2007) with a mean of 421.75, yielding ‘typical’ sample sizes (de Jonge & Jak, 2018) for every iteration. We imposed 39% missing correlations (Sheng, Kong, Cortina, & Hou, 2016) by (pseudo) randomly deleting either variable M or Y from 26 of the 44 studies. In each condition, we generated 2000 meta-analytic datasets drawn from the 44 subpopulations, which we analyzed using (1) the point-biserial and (2) the biserial correlation as effect size between the artificially dichotomized predictor X and continuous mediator M. The full mediation model was fitted using random-effects two stage structural equation modeling (TSSEM) (Cheung, 2014) within the R-package ‘metaSEM’ (Cheung, 2015b). As recommended (Becker, 2009; Hafdahl, 2007), we used the weighted mean correlation across the included primary studies to estimate the sampling variances and covariances of the correlation coefficients in the primary studies. Next, over the converged simulated datasets, we (1) estimated the relative percentage bias in both path coefficients (less than 5% bias was considered negligible; Hoogland & Boomsma, 1998), (2) calculated the relative percentage bias of the standard errors of these path coefficients (less than 10% bias was considered acceptable; Hoogland & Boomsma, 1998), (3) calculated the rejection rates of the chi-square statistic of the model of Stage 2 (df = 1,  = .05) and tested whether the rejection rate significantly differed from the nominal -level with the proportion test, and (4) compared the theoretical chi-square distribution (df = 1) with the empirical chi-square distribution (by means of QQplots and the Kolmogorov-Smirnov test). Main Results: When the point-biserial correlation for the relation between an artificially dichotomized predictor and a continuous mediator was used, the path coefficient of this relationship in the population (βMX) seems systematically underestimated. When the biserial correlation was used instead of the point-biserial correlation, this path coefficient could be considered unbiased in each condition. The estimated path coefficient between the two continuous variables (βYM) could also be considered unbiased in all conditions, no matter if the biserial or point-biserial correlation was used. The relative percentage bias in the standard errors of all path coefficients could be considered as not substantial according to the criteria that were applied. However, we noticed that the relative percentage bias in the standard error of the path coefficient between the predictor and mediator (βMX) seems systematically negatively biased when the biserial correlation was used. We also found that the relative percentage bias in the standard error of the path coefficient between the continuous variables Y and M (βYM) seems systematically negative, regardless if the point-biserial or biserial correlation was used. In most conditions, the rejection rate of the chi-square test of model fit at Stage 2 of the random-effects TSSEM was slightly above the nominal -level, no matter if the point-biserial or biserial correlation was used. The results of the Kolmogorov-Smirnov test and QQplots show that when the biserial correlation was used, there was a statistically significant difference between the empirical chi-square distribution and the theoretical chi-square distribution in five of the 18 conditions. When the point-biserial correlation was used, there was a significant difference in the same five conditions plus in three other conditions. There seems to be no clear pattern in which conditions the distributions differed significantly or not. Expected Conclusions and Implications: We advise researchers who want to apply MASEM and want to investigate mediation to convert the effect size between any dichotomized predictor and continuous variable to a biserial correlation, not to a point-biserial correlation. References: Becker, B. J. (1995). Corrections to “using results from replicated studies to estimate linear models”. Journal of Educational and Behavioral Statistics, 20, 100–102. doi:10.2307/1165390 Becker, B. J. (2009). Model-based meta-analysis. In H. Cooper, L. V. Hedges, & J.C. Valentine (Eds.) The handbook of research synthesis and meta-analysis (2nd ed., pp. 377–395). New York: Russell Sage Foundation. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362. doi:10.2307/1165128 Cheung, M. W.-L. (2014). Fixed-and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46, 29–40. doi:10.3758/s13428-013-0361-y Cheung, M. W.-L. (2015a). Meta-analysis: A structural equation modeling approach. Chichester, United Kingdom: John Wiley & Sons. Cheung, M. W.-L. (2015b). metaSEM: An R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5, [1521]. https://doi.org/10.3389/fpsyg.2014.01521 Cheung, M. W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: a two-stage approach. Psychological methods, 10, 40-64. doi:10.1037/1082-989X.10.1.40 Cohen, J. (1983). The cost of dichotomization. Applied psychological measurement, 7, 249-253. https://doi.org/10.1177/014662168300700301 de Jonge, H., & Jak, S. (2018, June). A Meta-Meta-Analysis: Identifying Typical Conditions of Meta-Analyses in Educational Research. Paper presented at the conference Research Synthesis 2018 of Leibniz Institute for Psychology Information, Trier, Germany. http://dx.doi.org/10.23668/psycharchives.853 Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. The Educational Researcher, 10, 3–8. doi:10.3102/0013189X005010003 Hafdahl, A. R. (2007). Combining correlation matrices: Simulation analysis of improved fixed-effects methods. Journal of Educational and Behavioral Statistics, 32, 180–205. doi:10.3102/1076998606298041 Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26, 329–367. doi:10.1177/0049124198026003003 Jacobs, P., & Viechtbauer, W. (2017). Estimation of the biserial correlation and its sampling variance for use in meta‐analysis. Research synthesis methods, 8, 161-180. doi:10.1002/jrsm.1218 Jak, S. (2015). Meta-analytic structural equation modelling. Springer International Publishing. Jansen, D., Elffers, L., & Jak, S. (2019). The functions of shadow education in school careers: a systematic review. Manuscript submitted for publication. MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological methods, 7, 19–40. doi:10.1037//1082-989X.7.1.19 Tate, R. F. (1955). The theory of correlation between two continuous variables when one is dichotomized. Biometrika, 42, 205–216. doi:10.2307/2333437 Sheng, Z., Kong, W., Cortina, J. M., & Hou, S. (2016). Analyzing matrices of meta‐analytic correlations: current practices and recommendations. Research synthesis methods, 7, 187-208. doi:10.1002/jrsm.1206 Soper, H. E. (1914). On the probable error of the bi-serial expression for the correlation coefficient. Biometrika, 10, 384–390. doi:10.2307/2331789 Viswesvaran, C., & Ones, D. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865–885. doi:10.1111/j.1744-6570.1995.tb01784.x

Objective To examine neurocognitive functioning of children exposed prenatally to carbamazepine, lamotrigine, levetiracetam or valproate monotherapy. Methods In a prospective observational study, children aged 6 or 7 years, identified from the European Registry of Antiepileptic Drugs and Pregnancy database in The Netherlands, were assessed using the Wechsler Intelligence Scale for Children and the developmental neuropsychological assessment. Maternal IQ was measured using Wechsler Adult Intelligence Scale. Assessors were blinded to drug exposures. Results One hundred and sixty-one children (one set of twins and 21 sibling pairs) of 139 mothers were included. As a group, children achieved average scores on neurocognitive outcomes. Children exposed to valproate (n = 22) performed lower on all six neurocognitive domains, especially language, than those exposed to carbamazepine (n = 32), lamotrigine (n = 82) or levetiracetam (n = 25). After controlling for maternal IQ and drug dose, the verbal IQ of valproate-exposed children was on average 9.1 points lower than those exposed to carbamazepine (95% confidence interval [CI] 1.3–17.0; p = 0.023), 10.3 lower than lamotrigine-exposed children (CI 3.4–17.3; p = 0.004) and 13.4 lower than levetiracetam-exposed children (CI 5.2–21.6; p = 0.002). No significant dose–effect was found. Virtually no significant differences were found between lamotrigine and levetiracetam or lamotrigine and carbamazepine exposed children. Conclusions Consistent with previous research, valproate-exposed children experienced more problems compared to three other common antiepileptic drugs, while children exposed to lamotrigine, carbamazepine or levetiracetam revealed little to no problems. This illustrates the need for systematic follow-up of prenatally exposed children, to support pre-pregnancy counseling and treatment decisions in women of reproductive age.

Organizations are confronted with a constant need for efficiency, which affects the working atmosphere, often typified by velocity, time pressure, competition, job insecurity, and information overload, which may lead to stress, burnout, work-life disbalance, and lowered work functioning. This study evaluated an 8-week, 1.5 h-per-week group-based standardized mindfulness program (“Finding peace in a frantic world”), applied on-site in a large multinational company. In a naturalistic longitudinal design self-selected employees (n = 150) completed measurements at wait-list, pre-test, post-test, and 2 and 6 months follow-up. Nearly all participants followed at least five out of eight sessions, were highly satisfied with the training (8.3 out 10), and almost 90% intended to continue with mindfulness practices. Primary outcomes were stress and risk for dropout from work. Using multilevel analyses, mean pre-test scores were compared to the other measurement occasions, and the rate of change in the training period (M = 67 days; SD = 12) was compared to the rate of change in the wait-list period (M = 29 days; SD = 8). Direct and long-term positive effects on risk for dropout from work (p p p

Psychological measurement aims at measuring a psychological construct (e.g., an attribute, skill, or ability) by means of observable variables such as items on a test or questionnaire. To relate item scores to the psychological construct, item response theory (IRT) models are used. Nonparametric IRT models fit test data relatively well compared to other IRT models. Various methods exist to evaluate the fit of nonparametric IRT models, also known as Mokken scale analysis (MSA). In multilevel test data respondents are nested in groups, such as students in classrooms or employees in companies. A multilevel data structure can severely affect statistical analyses, possibly leading to the inclusion of items to the test that do not contribute to (or possibly negatively affect) accurate measurement, or the quality of the test may be overestimated. In addition, most methods in MSA provide results on the respondent level only, not on the group level. Hence, for multilevel test data, currently available MSA methods are only of limited value. This thesis contributes to making MSA available for multilevel test data in the following ways. First, by discussing solutions to two computational problems in MSA. Second, by deriving, investigating, and optimizing point and interval estimates of two-level scalability coefficients and of Mokken’s scalability coefficients. Third, by introducing a test-guided automated item selection procedure. Finally, by proposing four two-level nonparametric IRT models, with their implied measurement and data properties. These developments contribute to knowledge on how tests and questionnaires can be investigated in the presence of multilevel test data.

The measurement of psychological constructs like mathematical ability or social anxiety is at the heart of social and behavioral sciences. Psychological constructs are often referred to as latent constructs because they are not directly observable. As a result, these constructs are measured with observable variables such as responses to questionnaire or test items that are assumed to represent the latent construct. Given the dominant role of such tests and questionnaires in social and behavioral sciences and applications, it is important that the items on these measurement instruments function the same way across individuals or groups. This requirement is also referred to as measurement invariance. If measurement invariance does not hold, differences in observed scores across individuals or groups may arise from differences on background variables (such as age or gender) instead of differences on the latent construct. It is therefore important to assess measurement invariance before comparing individuals or groups on latent constructs. This dissertation addresses novel approaches to assess measurement invariance in the framework of structural equation modeling. More specifically, the focus of this dissertation is on new ways to assess measurement invariance using single-group methods like restricted factor analysis and moderated nonlinear factor analysis

Dyadic network data occur when each member in a group provides data about each other member in the group (e.g., how much they like each other person). Such data have a complex nesting structure, such that bivariate responses (e.g., Person A’s liking of B and vice versa) are dependent upon out-going and in-coming random effects that are correlated within individuals. Dyadic network models for such data include the social relations model for normal data and the p2 and j2 models for dichotomous data, but we have seen no application or generalization to accommodate a rarely discussed type of variable from this framework: variables that are constant within a dyad. Dyad-constant variables could include background variables such as whether a dyad is same or opposite sex or how many years two friends have known each other, which require no special modification to use as predictors (Jorgensen et al., Soc Netw 54:26–40, 2018). But they could also be outcomes, such as the difference in a married couple’s relationship satisfaction or the similarity in symptoms of a (set of) psychological disorder(s). We explore how such dyad-constant outcomes can be modeled, demonstrating on a data set from a clinic for patients with eating disorders.

IntroductionThis paper outlines the study protocol for the Dutch Tackle Your Tics study in youth with tic disorders. Tourette syndrome and chronic tic disorders are prevalent neurodevelopmental disorders, placing considerable burden on youth and their families. Behavioural treatment is the first-line, evidence-based intervention for tic disorders, but tic reduction and availability remain relatively low. Patient associations stress the need for more accessible high-quality treatments, also focusing on improving quality of life. Therefore, the brief, intensive group-based treatment Tackle Your Tics was developed.Methods and analysisTackle Your Tics is a 4-day intensive and comprehensive group-based intervention for children and adolescents (9–17 years) with Tourette syndrome or a chronic tic disorder. The programme encompasses exposure and response prevention treatment and additional supporting components (coping strategies, relaxation exercises and parent support). To study the effectiveness of Tackle Your Tics and identify predictors/moderators at baseline, a single-blinded randomised controlled trial (n=104) is conducted, comparing Tackle Your Tics (n=52) with a waiting list condition lasting 3 months (n=52). Assessments are performed at similar time points for both groups: at baseline, after 4 weeks, and at 3 and 6 months of follow-up, on tic severity, quality of life and other psychosocial variables.Ethics and disseminationEthics approval has been obtained from the medical ethical committee of the Amsterdam Medical Centre (METC nr NL66340.018.18, v3 June 2020). Findings will be presented on national and international conferences, peer-reviewed scientific journals, patient organisation meetings and public media. Patient representatives are fully integrated as part of the research team. If Tackle Your Tics proves to be effective, it can expand evidence-based treatment possibilities for children and adolescents with tic disorders. Identifying the psychosocial predictors/moderators for the effectiveness of this intervention can provide personalised treatment advice in the future.Trial registration numberNL8052.

We introduce the special section on nonparametric item response theory (IRT) in Quality of Life Research. Starting from the well-known Rasch model, we provide a brief overview of nonparametric IRT models and discuss the assumptions, the properties, and the investigation of goodness of fit. We provide references to more detailed texts to help readers getting acquainted with nonparametric IRT models. In addition, we show how the rather diverse papers in the special section fit into the nonparametric IRT framework. Finally, we illustrate the application of nonparametric IRT models using data from a questionnaire measuring activity limitations in walking. The real-data example shows the quality of the scale and its constituent items with respect to dimensionality, local independence, monotonicity, and invariant item ordering.

Background Both the presence of psychological problems and the absence of an employment contract are related to long-term sickness absence, prolonged work disability and unemployment. Studies researching the effectiveness of return-to-work interventions on these non-permanent workers, including unemployed and temporary agency workers and workers with an expired fixed-term contract, are lagging behind. Therefore, a return-to-work intervention called “Brainwork” was developed. The aim of this study was to assess the effectiveness of the ‘Brainwork Intervention’ in reducing the duration of sick leave compared to usual care over a 12-month follow-up. Methods In a multicenter controlled clinical trial, using a quasi-randomization procedure, we compared the Brainwork Intervention (n = 164) to usual care (n = 156). The primary outcome was the duration of sick leave. Secondary outcomes were the duration of sick leave starting from Social Security Agency transfer; the proportion of workers returned to work; the number of hours of paid employment during the follow-up period; the degree of worker participation; the level of psychological complaints; and the self-efficacy for return to work. Protocol adherence (Brainwork Intervention) was considered sufficient when at least three of the five protocol steps were followed. Cox regressions, linear and ordinal regression, and Mixed Model analyses were performed. Results All 320 participants were analyzed. The Brainwork Intervention resulted in a non-significant reduction of the duration of sick leave compared to usual care (269 days versus 296 days; HR = 1.29; 95% CI 0.94–1.76; p = 0.11). For those working (46%) during the 12-month follow-up, the mean number of hours of paid employment was non-significantly higher in the usual care group (682 h versus 493 h; p = 0.053). No significant differences were found for other secondary outcomes. Protocol adherence was 10%. Conclusions The Brainwork Intervention as performed with a low protocol adherence did not result in a significant reduction of the duration of sick leave compared to usual care. It remains unclear what the results would have been if the Brainwork Intervention had been executed according to protocol. Trial registration The Netherlands Trial Register (NTR); NTR3976 (old registration number NTR4190). Registered September 27th 2013.

In the wake of the COVID-19 pandemic, physical distancing behavior turned out to be key to mitigating the virus spread. Therefore, it is crucial that we understand how we can successfully alter our behavior and promote physical distancing. We present a framework to systematically assess the effectiveness of behavioral interventions to stimulate physical distancing. In addition, we demonstrate the feasibility of this framework in a large-scale natural experiment (N = 639) conducted during an art fair. In an experimental design, we varied interventions to evaluate the effect of face masks, walking directions, and immediate feedback on visitors’ contacts. We represent visitors as nodes, and their contacts as links in a contact network. Subsequently, we used network modelling to test for differences in these contact networks. We find no evidence that face masks influence physical distancing, while unidirectional walking directions and buzzer feedback do positively impact physical distancing. This study offers a feasible way to optimize physical distancing interventions through scientific research. As such, the presented framework provides society with the means to directly evaluate interventions, so that policy can be based on evidence rather than conjecture.

Interrater reliability (IRR) is often estimated by intraclass correlation coefficients (ICCs). Using Markov chain Monte Carlo (MCMC) estimation of Bayesian hierarchical models to estimate ICCs has several benefits over traditional approaches such as analysis of variance or maximum likelihood estimation. However, estimation of ICCs with small sample sizes and variance parameters close to zero, which are typical conditions in studies for which the IRR should be estimated, remains problematic in this MCMC approach. The estimation of the variance components that are used to estimate ICCs can heavily depend on the hyperprior distributions specified for these random-effect parameters. In this study, we explore the effect of a uniform and half-t hyperprior distribution on bias, coverage, and efficiency of the random-effect parameters and ICCs. The results indicated that a half-t distribution outperforms a uniform distribution but that slightly increasing the number of raters in a study is more influential than the choice of hyperprior distributions. We discuss implications and directions for future research.

Meta-analytic structural equation modeling (MASEM) is a statistical technique to fit hypothesized models on the combined data of multiple independent studies. Lv and Maeda (2019) present a simulation study on the performance of three fixed-effects correlation-based MASEM methods with varying levels of data missing completely at random (MCAR). In this commentary, we discuss several coding errors and other issues that we identified, which demonstrate that Lv and Maeda did not evaluate any of the three intended methods. Furthermore, the authors report very surprising results and offer specific recommendations for the application of the three methods; these actions compel us to express our concerns regarding the validity of the conclusions provided by Lv and Maeda.

Objective: This study investigated how genetic susceptibility may affect children's sensitivity to parenting practices in their development of externalizing behavior. We created a continuous polygenic index composed of 5 dopamine polymorphisms to investigate the moderating role of dopamine-related genes in shaping parent-child gene-by-environment (Gc×E) interactions. Accumulating research supports that differences in children's dopamine neurotransmission make certain children more susceptible to both negative and positive parenting practices, a “for-better and for-worse” effect.Methods: Data from a 3-wave longitudinal study (4 months between waves) on 190 at-risk families with children aged 4 to 8 were used to investigate whether a heightened polygenic index score amplified the associations between negative and positive parenting and later children's externalizing behavior. Parenting practices and child externalizing behavior were assessed by parent-reported questionnaires.Results: Findings were not in line with the expectation that there was a stronger association between positive and negative parenting and later externalizing behavior for children with higher scores on the polygenic susceptibility index. Rather, children with a lower score on the polygenic susceptibility index showed more later externalizing behavior in response to positive parenting behavior, whereas for children with a higher score on the polygenic index, positive parenting was predictive of relatively lower levels of later child externalizing behavior.Conclusion: The results indicate that not only are children with higher but also lower scores on the polygenic index sensitive to parenting, they suggest that different phenotypical characteristics related to reward processing might underlie these genetic susceptibilities to parenting practices.

IntroductionLoss of sensation due to diabetes‐related neuropathy often leads to diabetic foot ulceration. Several test instruments are used to assess sensation, such as static and moving 2‐point discrimination (S2PD, M2PD), monofilaments, and tuning forks.MethodsMokken scale analysis was applied to the Rotterdam Diabetic Foot Study data to select hierarchies of tests to construct measurement scales.ResultsWe developed 39‐item and 31‐item scales to measure loss of sensation for research purposes and a 13‐item scale for clinical practice. All instruments were strongly scalable and reliable. The 39 items can be classified into 5 hierarchically ordered core clusters: S2PD, M2PD, vibration sense, monofilaments, and prior ulcer or amputation.DiscussionGuided by the presented scales, clinicians may better classify the grade of sensory loss in diabetic patients’ feet. Thus, a more personalized approach concerning individual recommendations, intervention strategies, and patient information may be applied.

We used the Dutch–Flemish version of the USA PROMIS adult V1.0 item bank for Anxiety as input for developing a computerized adaptive test (CAT) to measure the entire latent anxiety continuum. First, psychometric analysis of a combined clinical and general population sample ( N = 2,010) showed that the 29-item bank has psychometric properties that are required for a CAT administration. Second, a post hoc CAT simulation showed efficient and highly precise measurement, with an average number of 8.64 items for the clinical sample, and 9.48 items for the general population sample. Furthermore, the accuracy of our CAT version was highly similar to that of the full item bank administration, both in final score estimates and in distinguishing clinical subjects from persons without a mental health disorder. We discuss the future directions and limitations of CAT development with the Dutch–Flemish version of the PROMIS Anxiety item bank.

BACKGROUND: In the clinical field, the use of questionnaires is ubiquitous, and many different methods for constructing them are available. The reason for using a specific method is usually lacking, and a generally accepted classification of methods is not yet available. To guide test developers and users, this article presents a taxonomy for methods of questionnaire design which links the methods to the goal of a test.METHODS: The taxonomy assumes that construction methods are directed towards psychometric aspects. Four stages of test construction are distinguished to describe methods: concept analysis, item production, scale construction, and evaluation; the scale construction stage is used for identifying methods. It distinguishes six different methods: the rational method utilizes expert judgments to ensure face validity. The prototypical method uses prototypicality judgments to ensure process validity. In the internal method, item sets are selected that optimize homogeneity. The external method optimizes criterion validity by selecting items that best predict an external criterion. Under the construct method theoretical considerations are used to optimize construct validity. The facet method is aimed at optimizing content validity through a complete representation of the concept domain.CONCLUSION: The taxonomy is comprehensive, constitutes a useful tool for describing procedures used in questionnaire design, and allows for setting up a test construction plan in which the priorities among psychometric aspects are made explicit.

The foci of visual attention were modeled as a function of perceptual salience, adult fixation locations, and attentional control mechanisms (measured in separate tasks) in infants (N = 45, 3- to 15-month-olds) as they viewed static real-world scenes. After controlling for the center bias, the results showed that low-level perceptual salience predicts where infants look. In addition, high-level factors also played a role: Infants fixated parts of the scenes frequently fixated by adults and this effect was stronger for older than younger infants. In line with this finding, infant fixation durations were longer on regions more frequently fixated by adults, implying longer time taken to process the available information. Fixation durations decreased with age, and this decline interacted with orienting skills such that fixation durations decreased faster with age for infants with high orienting skills, relative to infants with low orienting skills. There was a further interaction between fixation durations and selective attention abilities: Infants with low selective attention skills showed a decrease in fixation durations with age, whereas infants with higher selective attention skills showed a slight increase in fixation durations with age. These findings imply that infants' visual processing of static real-world stimuli develops in accord with attentional control.

We investigated whether early algebra lessons that explicitly aimed to elicit mathematical discussions (shift-problem lessons) invoke more and qualitatively better mathematical discussions and raise students’ mathematical levels more than conventional lessons in a small group setting. A quasi-experimental study (pre- and post-test, control group) was conducted in 6 seventh-grade classes (N = 160). An analysis of the interaction processes of five student groups showed that more mathematical discussions occurred in the shift-problem condition. The quality of the mathematical discussions in the shift-problem condition was better compared to that in the conventional textbook condition, but there is still more room for improvement. A qualitative illustration of two typical mathematical discussions in the shift-problem condition are provided. Although students’ mathematical levels were raised a fair amount in both conditions, no differences between conditions were found. We concluded that shift-problem lessons are powerful for eliciting mathematical discussions in seventh-grade shift-problem early algebra lessons.

Objective: To examine the behavioral functioning of children prenatally exposed to carbamazepine (CBZ), lamotrigine (LTG), levetiracetam (LEV), or valproate (VPA) monotherapy. Methods: In collaboration with the European Registry of Antiepileptic Drugs and Pregnancy (EURAP), the Dutch EURAP & Development study was designed, a prospective observational study. Between January 2015 and March 2018, the Child Behavior Checklist and the Social Emotional Questionnaire were used to examine the nature and severity of behavioral problems. VPA-exposed children were compared to children exposed to CBZ, LTG, or LEV, taking potential confounders into account. A direct comparison was also made between LTG and LEV, as these are first-choice treatments for many women with epilepsy of childbearing potential.Results: Of the 405 invited, 181 children were included; 26 were exposed to VPA, 37 to CBZ, 88 to LTG, and 30 to LEV. For most children, both parents completed the behavioral questionnaires. Across all four antiepileptic drug (AED) exposure groups, high percentages of children with clinically relevant behavior problems were found, with behavioral problems occurring in 32% of VPA-exposed children, 14% of CBZ, 16% of LTG, and 14% of LEV. After controlling for potential confounders, VPA-exposed children had significantly more social problems than those exposed to LTG (−2.8, 95% confidence interval [CI] = −5.2 to −0.4; P = 0.022) or LEV (−3.2, CI: −6.1 to −0.3; P = 0.028), and significantly more attention problems than LEV-exposed children (−3.7, CI: −6.7 to −0.8; P = 0.013). LTG-exposed children had significantly more attention deficit (−9.2, CI: −17.3 to 1.1; P = 0.026), but significantly less anxious behavior when compared to LEV-exposed children (9.0, CI: 0.3-17.6; P = 0.042). Significance: Compared to population norms, a high proportion of children of mothers with epilepsy exposed prenatally to monotherapy with four common AEDs had clinical behavioral problems reported by parents. Different patterns were seen, with some but not all subscales raised for all AED exposure groups. It is important that prenatally AED-exposed children are regularly screened for behavioral problems so that appropriate help can be provided.

Significant attention has been paid in the international literature to the effect of in-service training on the interaction skills of teachers in early childhood education and care. The growth of pre-service teachers during teacher education has received relatively little attention to date, however. In a mixed-methods longitudinal study, we monitored the development of interaction skills among a group of Dutch pre-service teachers with repeated measures for 3years and structured interviews. The results of a linear mixed-effects model revealed an impressive growth of interaction skills during the pre-service training. The qualitative interview data revealed progress of pre-service teachers' professional reflection on their interaction with young children. These outcomes show the effectiveness of pre-service training for the development of interaction skills and professional reflection in early childhood education and care. However, progress is relatively modest for instructional skills and this domain needs further investment in pre-service training.

Two-level Mokken scale analysis is a generalization of Mokken scale analysis for multi-rater data. The bias of estimated scalability coefficients for two-level Mokken scale analysis, the bias of their estimated standard errors, and the coverage of the confidence intervals has been investigated, under various testing conditions. It was found that the estimated scalability coefficients were unbiased in all tested conditions. For estimating standard errors, the delta method and the cluster bootstrap were compared. The cluster bootstrap structurally underestimated the standard errors of the scalability coefficients, with low coverage values. Except for unequal numbers of raters across subjects and small sets of items, the delta method standard error estimates had negligible bias and good coverage. Post hoc simulations showed that the cluster bootstrap does not correctly reproduce the sampling distribution of the scalability coefficients, and an adapted procedure was suggested. In addition, the delta method standard errors can be slightly improved if the harmonic mean is used for unequal numbers of raters per subject rather than the arithmetic mean.

This mixed-methods systematic review and meta-analysis sheds more light on the role parenting practices play in children’s adjustment after war exposure. Specifically, we quantitatively examined how war exposure shapes parenting behavior, and whether parenting behavior explains some of the well-known associations between war exposure and children’s adjustment. In addition, we meta-synthesized the qualitative evidence answering when and why parenting practices might change for war-affected families. We searched nine electronic databases and contacted experts in the field for relevant studies published until March 2018, identifying 4,147 unique publications that were further screened by title and abstract, resulting in 158 publications being fully screened. By running a meta-analytic structural equation model (MASEM) with 38 quantitative studies (N = 54,372, Mage = 12.00, SDage = 3.54), we found that war-exposed parents showed less warmth and more harshness towards their children, which partly mediated the association between war exposure and child adjustment, i.e., more post-traumatic stress symptoms, depression and anxiety, social problems, externalizing behavior, and lower positive outcomes. War exposure was not associated with parents’ exercise of behavioral control. Through meta-synthesizing ten qualitative studies (N = 1,042, age range = 0-18), we found that the nature of war-related trauma affected parenting differently. That is, parents showed harshness, hostility, inconsistency and less warmth in highly dangerous settings, and more warmth and overprotection when only living under threat. We conclude that it is not only how much but also what families have seen that shape parenting in times of war.

Restricted factor analysis (RFA) is a powerful method to test for uniform differential item functioning (DIF), but it may require empirically selecting anchor items to prevent inflated Type I error rates. We conducted a simulation study to compare two empirical anchor-selection strategies: a one-step rank-based strategy and an iterative selection procedure. Unlike the iterative procedure, the rank-based strategy had a low risk and degree of contamination within the empirically selected anchor set, even with small samples. To detect nonuniform DIF, RFA requires an interaction effect with the latent factor. The latent moderated structural equations (LMS) method has been applied to RFA and has revealed inflated Type I error rates. We propose using product indicators (PI) as a more widely available alternative to measure the latent interaction. A simulation study, involving several sample-size conditions and magnitudes of uniform and nonuniform DIF, revealed that PI obtained similar power but lower Type I error rates, as compared to LMS.

Background: Adolescents with Mild Intellectual Disability (MID) or Borderline Intellectual Functioning (BIF) are highly susceptible to negative peer influence. However, research in typically developing adolescents shows that peers can also promote prosocial behavior, which is an opportunity for positive development. Aims: The current study aimed to investigate the effect of peer influence on prosocial behavior in adolescents with MID or BIF. Methods and procedures: In an experimental donation task, 40 adolescents with MID or BIF (Mage =14.0, 40 % boys) were repeatedly asked how many of five coins they would like to donate to the group. The task had four consecutive within-subject conditions: alone, with virtual peers present, with virtual peer feedback, and alone again. Outcomes and results: Results showed that adolescents made larger donations with virtual peers present, and even larger with peer feedback. This increase in donations sustained for subsequent decisions made alone. Finally, adolescents with BIF made larger donations with peer feedback compared to adolescents with MID. Conclusions and implications: Adolescents with MID or BIF are susceptible to peer influence on prosocial behavior, demonstrating the potential effect social context can have on promoting positive development.

Progress feedback is an intervention aimed at enhancing patient outcomes in routine clinical practice. This study reports a comprehensive multilevel meta-analysis on the effectiveness of progress feedback in psychological treatments in curative care. The short- and long-term effects of feedback on symptom reduction were investigated using 58 (randomized and non-randomized) studies, analyzing 110 effect sizes in a total of 21,699 patients. Effects of feedback on dropout rate, percentage of deteriorated cases, and treatment duration were also examined. Moderation analyses were conducted for study and feedback characteristics. A small significant effect of progress feedback on symptom reduction (d = 0.15, 95% CI: [0.10, 0.20]) was found, compared to control groups. This was also true for not-on-track cases (d = 0.17, 95% CI: [0.11, 0.22]). In addition, feedback had a small favorable effect on dropout rates (OR = 1.19, 95% CI: [1.03, 1.38]). The moderation analyses identified several potentially interesting variables for further research, including feedback instrument, outcome instrument, type of feedback, feedback frequency, treatment intensity, and country in which the study was conducted. Future studies should report on these variables more consistently so that we can obtain a better understanding of when and why feedback improves outcomes.

Tourette syndrome (TS) and other chronic tic disorders (CTD) are prevalent neurodevelopmental disorders, which can have a huge burden on families and society. Behavioral treatment is a first-line intervention for tic disorders. Despite demonstrated efficacy, tic reduction and utilization rates of behavioral treatment remain relatively low. Patient associations point to an urgent need for easy-to-undergo treatments that focus both on tic reduction and improvement of quality of life. To enhance treatment outcome and overcome treatment barriers, this pilot study’s aim was to investigate the feasibility and preliminary results of a brief, intensive group-based treatment. Tackle your Tics is a 4-day intensive and comprehensive group-based program for children and adolescents (9–17 years) with a tic disorder, consisting of exposure and response prevention (ERP) treatment and additional supporting components, such as coping strategies, relaxing activities and parent support. Assessments were performed pre- and post-treatment and at 2 months follow-up, to test outcomes on tic severity and quality of life, and explore premonitory urges, emotional and behavioral functioning and treatment satisfaction (N = 14, of whom 13 completed the treatment). Parents and children rated this treatment positive on a treatment satisfaction questionnaire. On tic severity (Yale Global Tic Severity Scale) and quality of life (Gilles de la Tourette Syndrome Quality of Life Scale for children and adolescents), improvements between pre-treatment and follow-up were found. Intensive ERP in group format is promising as a feasible treatment to improve both tic severity as well as quality of life. Larger controlled trials are needed to establish its effectiveness.