Search

Cyberbullying among youth is an emerging public health concern that has a wide array of deleterious outcomes. The current meta-analytic review synthesized school-based cyberbullying prevention programs' impact on promoting cyber-bystander intervention among K-12 students. As a result of exhaustive searches and a thorough screening procedure, a total of 9 studies were identified as eligible. Meta-analytic synthesis of the 9 studies involving 35 effect sizes demonstrated that overall, the treatment effect was not statistically significant (g = 0.29, SE = 0.14, p = .07, 95% CI [-0.03, 0.61]). Findings of the moderator analyses suggest that incorporating an empathy activation component in the prevention program was associated with better program effectiveness in promoting cyber-bystander intervention. Further, older age was found to be associated with better program outcomes. Findings of the current meta-analysis provide important insight for developing cyberbullying prevention programs that promote cyber-bystander intervention.

This methodological guidance article discusses the elements of a high-quality meta-analysis that is conducted within the context of a systematic review. Meta-analysis, a set of statistical techniques for synthesizing the results of multiple studies, is used when the guiding research question focuses on a quantitative summary of study results. In this guidance article, we discuss the systematic review methods that support high-quality meta-analyses and outline best practice meta-analysis methods for describing the distribution of effect sizes in a set of eligible studies. We also provide suggestions for transparently reporting the methods and results of meta-analyses to influence practice and policy. Given the increasing use of meta-analysis for important policy decisions, the methods and results of meta-analysis should be both transparent and reproducible.

Evidence suggests that cyberbullying among school-age children is related to problem behaviors and other adverse school performance constructs. As a result, numerous school-based programs have been developed and implemented to decrease cyberbullying perpetration and victimization. Given the extensive literature and variation in program effectiveness, we conducted a comprehensive systematic review and meta-analysis of programs to decrease cyberbullying perpetration and victimization. Our review included published and unpublished literature, utilized modern, transparent, and reproducible methods, and examined confirmatory and exploratory moderating factors. A total of 50 studies and 320 effect sizes spanning 45,371 participants met the review protocol criteria. Results indicated that programs significantly reduced cyberbullying perpetration (g = -0.18, SE = 0.05, 95% CI [-0.28, -0.09]) and victimization (g = -0.13, SE = 0.04, 95% CI [-0.21, -0.05]). Moderator analyses, however, yielded only a few statistically significant findings. We interpret these findings and provide implications for future cyberbullying prevention policy and practice.

Abstract screening is one important aspect of conducting a high-quality and comprehensive systematic review and meta-analysis. Abstract screening allows the review team to conduct the tedious but vital first step to synthesize the extant literature: winnowing down the overwhelming amalgamation of citations discovered through research databases to the citations that should be "full-text" screened and eventually included in the review. Although it is a critical process, few guidelines have been put forth since the publications of seminal systematic review textbooks. The purpose of this paper, therefore, is to provide a practical set of best practice guidelines to help future review teams and managers. Each of the 10 proposed guidelines is explained using real world examples or illustrations from applications. We also delineate recent experiences where a team of abstract screeners double-screened 14,923 abstracts in 89 days. [This article was published in "Research Synthesis Methods" (EJ1255353).]

A priori power analyses allow researchers to estimate the number of participants needed to detect the effects of an intervention. However, power analyses are only as valid as the parameter estimates used. One such parameter, the expected effect size, can vary greatly depending on several study characteristics, including the nature of the intervention, developer of the outcome measure, and age of the participants. Researchers should understand this variation when designing studies. Our meta-analysis examines the relationship between science education intervention effect sizes and a host of study characteristics, allowing primary researchers to access better estimates of effect sizes for a priori power analyses. The results of this meta-analysis also support programmatic decisions by setting realistic expectations about the typical magnitude of impacts for science education interventions.

Meta-analysis is a statistical technique that allows an analyst to synthesize effect sizes from multiple primary studies. To estimate meta-analysis models, the open-source statistical environment R is quickly becoming a popular choice. The meta-analytic community has contributed to this growth by developing numerous packages specific to meta-analysis. The purpose of this study is to locate all publicly available meta-analytic R packages. We located 63 packages via a comprehensive online search. To help elucidate these functionalities to the field, we describe each of the packages, recommend applications for researchers interested in using R for meta-analyses, provide a brief tutorial of two meta-analysis packages, and make suggestions for future meta-analytic R package creators.

Overviews, or syntheses of research syntheses, have become a popular approach to synthesizing the rapidly expanding body of research and systematic reviews. Despite their popularity, few guidelines exist and the state of the field in education is unclear. The purpose of this study is to describe the prevalence and current state of overviews of education research and to provide further guidance for conducting overviews and advance the evolution of overview methods. A comprehensive search across multiple online databases and gray literature repositories yielded 25 total education-related overviews. Our analysis revealed that many commonly reported aspects of systematic reviews, such as the search, screen, and coding procedures, were regularly unreported. Only a handful of overview authors discussed the synthesis technique and few authors acknowledged the overlap of included systematic reviews. Suggestions and preliminary guidelines for improving the rigor and utility of overviews are provided.

The incidence of violence in dating relationships has a significant impact on young people, including decreased mental and physical health. This review is the first to provide a quantitative synthesis of empirical evaluations of school-based programs implemented in middle and high schools that sought to prevent or reduce incidents of dating violence. After a systematic search and screening procedure, a meta-analysis of 23 studies was used to examine the effects of school-based programs. Results indicated school-based programs influence dating violence knowledge (g-bar = 0.22, 95% confidence interval [0.05, 0.39]) and attitudes (g-bar = 0.14, 95% confidence interval [0.10, 0.19]); however, to date, the results for dating violence perpetration and victimization indicate programs are not affecting these behaviors to a significant extent. The results of this review are encouraging, but they also highlight the need for modifications to dating violence prevention programs including the incorporation of skill-building components and a need to address the role of bystanders.

We present the R package and web app "statcheck" to automatically detect statistical reporting inconsistencies in primary studies and meta-analyses. Previous research has shown a high prevalence of reported p-values that are inconsistent--meaning a re-calculated p-value, based on the reported test statistic and degrees of freedom, does not match the author-reported p-value. Such inconsistencies affect the reproducibility and evidential value of published findings. The tool "statcheck" can help researchers to identify statistical inconsistencies so that they may correct them. In this paper, we provide an overview of the prevalence and consequences of statistical reporting inconsistencies. We also discuss the tool "statcheck" in more detail and give an example of how it can be used in a meta-analysis. We end with some recommendations concerning the use of "statcheck" in meta-analyses and make a case for better reporting standards of statistical results.

This 3-year study evaluated the effectiveness of the Second Step-Student Success Through Prevention (SS-SSTP) social-emotional learning program on increasing prosocial behaviors that could serve as protective factors against peer conflict and bullying among students with disabilities. Participants included 123 students with disabilities across 12 schools in Midwest United States. Students labelled with a disability were selected for inclusion. Students completed self-report measures of school belonging, empathy, caring, and willingness to intervene in bullying situations. Report card grades and standardized test scores were collected from school records. Students with disabilities in the intervention schools reported a statistical and clinical significant increase in willingness to intervene in bullying incidents in comparison with students with disabilities in control schools and an increase of half a grade on their report cards in comparison with the control sample. The current study demonstrates the promise of social-emotional learning programming for students with disabilities.

Abstract screening is one important aspect of conducting a high-quality and comprehensive systematic review and meta-analysis. Abstract screening allows the review team to conduct the tedious but vital first step to synthesize the extant literature: winnowing down the overwhelming amalgamation of citations discovered through research databases to the citations that should be "full-text" screened and eventually included in the review. Although it is a critical process, few guidelines have been put forth since the publications of seminal systematic review textbooks. The purpose of this paper, therefore, is to provide a practical set of best practice guidelines to help future review teams and managers. Each of the 10 proposed guidelines is explained using real-world examples or illustrations from applications. We also delineate recent experiences where a team of abstract screeners double-screened 14,923 abstracts in 89 days.

"Second Step" is a universal social-emotional learning program designed to decrease antisocial behaviors, increase prosocial behaviors, and increase knowledge about content curriculum. Given the international focus of using Second Step to improve social-emotional skills, it is especially important to provide an updated synthesis on the effect of the intervention. A single-program meta-analysis was completed in order to determine effects of participating in Second Step and to explore moderators of program effectiveness on prosocial and antisocial behavior outcomes as well as knowledge of emotions. Five moderators were explored, including (a) program saturation, (b) dependent variable source, (c) grade range, (d) metro area, and (e) geographical location. A combined total of 18,847 participants were included in the analysis of 27 studies. Results of the current study indicate that "Second Step's" impact on students' knowledge and attitudes of violence and violence prevention is much stronger than on increasing prosocial behavior or reducing antisocial behavior.

Studies of interventions' impact on reading self-efficacy have been conducted since the 1980s. The purpose of this project was to conduct a systematic review of these studies because the primary studies often yielded divergent results. Included studies entailed an intervention, addressed reading specifically, and reported explicit pre- and postintervention measures of reading self-efficacy. Subjects were students in elementary grades through college. The results of a systematic search and screening procedure found 30 studies in which 2,300 subjects received treatments of various kinds while 1,957 were in control or comparison groups. A meta-analysis of three subsets of study designs revealed that each subset generated a significant effect size: treatment-control (g = 0.24, 95% confidence interval [CI] [0.10, 0.39]); treatment-comparison (g = 0.44, 95% CI [0.04, 0.84]); pretest-posttest (g = 0.36, 95% CI [0.16, 0.57]). Significant heterogeneity was found and modeled using moderator analyses conducted on several variables. The results indicated that significant moderators of effect sizes included grade level, number of sources shaping reading self-efficacy, a reading self-efficacy measurement index, and journal publication. In studies that measured the impact of the intervention on reading comprehension, its relationship with reading self-efficacy was analyzed revealing a strong correlation between the two constructs. Discussion includes an exploration of the importance of these findings to future policy, practice, and research on the design of reading self-efficacy measurement instruments and on interventions that utilize major sources of experiences shaping reading self-efficacy.

A modification of the first stage of the standard procedure for two-stage meta-analytic structural equation modeling for use with large complex datasets is presented. This modification addresses two common problems that arise in such meta-analyses: (a) primary studies that provide multiple measures of the same construct and (b) the correlation coefficients that exhibit substantial heterogeneity, some of which obscures the relationships between the constructs of interest or undermines the comparability of the correlations across the cells. One component of this approach is a three-level random effects model capable of synthesizing a pooled correlation matrix with dependent correlation coefficients. Another component is a meta-regression that can be used to generate covariate-adjusted correlation coefficients that reduce the influence of selected unevenly distributed moderator variables. A non-technical presentation of these techniques is given, along with an illustration of the procedures with a meta-analytic dataset.

Practitioners and policymakers rely on meta-analyses to inform decision making around the allocation of resources to individuals and organizations. It is therefore paramount to consider the validity of these results. A well-documented threat to the validity of research synthesis results is the presence of publication bias, a phenomenon where studies with large and/or statistically significant effects, relative to studies with small or null effects, are more likely to be published. We investigated this phenomenon empirically by reviewing meta-analyses published in top-tier journals between 1986 and 2013 that quantified the difference between effect sizes from published and unpublished research. We reviewed 383 meta-analyses of which 81 had sufficient information to calculate an effect size. Results indicated that published studies yielded larger effect sizes than those from unpublished studies ( d[superscript bar over d] = 0.18, 95% confidence interval [0.10, 0.25]). Moderator analyses revealed that the difference was larger in meta-analyses that included a wide range of unpublished literature. We conclude that intervention researchers require continued support to publish null findings and that meta-analyses should include unpublished studies to mitigate the potential bias from publication status.

Social-emotional learning programs are increasingly being implemented in U.S. schools to address a wide range of problematic behaviors (e.g., bullying, delinquency) and to promote academic success. The current study examined the direct and indirect impact of the Second Step Middle School Program (Committee for Children, 2008) on bullying, cyberbullying, homophobic name-calling, and sexual harassment perpetration over the course of a 3-year randomized clinical trial. Delinquency was examined as an intervening variable between treatment condition and aggression outcomes. Thirty-six schools in Kansas and Illinois were assigned to either a Second Step condition or a control condition, and 3,651 sixth-grade students completed self-reported surveys at four time points across 3 years. Students in the Second Step condition received a total of 41 lessons across the 3-year study. No direct intervention effects were found for multiple forms of aggression perpetration at the end of 3 years. However, as hypothesized, decreases in self-reported delinquency (intervening variable) over the first 2 years were significantly related to decreases in bullying, cyberbullying, and homophobic name-calling perpetration for Second Step schools across the 3-year study. Indirect effects of the Second Step program on bullying and aggressive behavior were statistically significant through reductions of delinquency.

School bullying and delinquent behaviors are persistent and pervasive problems for schools, and have lasting effects for all individuals involved (Copeland et al., "JAMA Psychiatry" 70:419-426, 2013; Espelage et al., "J Res Adolesc" 24(2):337-349, 2013a). As a result, policymakers and practitioners have attempted to thwart these ill-effects using school-based interventions. Recent meta-analyses have found, however, that these programs produce only moderate effects (Ttofi and Farrington, "J Exp Criminol" 7:27-56, 2011). Consequently, it is important to investigate further the reasons for such findings. One promising analysis is to assess the relation between treatment intensity variables and program outcomes. Unfortunately, few treatment intensity variables have been utilized in the school-based prevention literature, and it is often cumbersome to model the relation between treatment intensity and outcomes. The purpose of this project, therefore, is to explicate novel measures of treatment intensity and delineate a relatively new meta-analytic technique to model the relation between the variables and program effects. The context for this project is a large-scale, multi-site, cluster-randomized trial; 36 schools and 3,616 students participated in three waves of data collection. The results indicated that, for the second wave of data collection, stronger treatment effects were found when teachers and program implementers spent a greater amount of time prepping lessons, provided additional financial resources, and received outside consultation and support.

Meta-analysis multiplicity, the concept of conducting multiple tests of statistical significance within one review, is an underdeveloped literature. We address this issue by considering how Type I errors can impact meta-analytic results, suggest how statistical power may be affected through the use of multiplicity corrections, and propose how meta-analysts should analyze multiple tests of statistical significance. The context for this study is a meta-review of meta-analyses published in two leading review journals in education and psychology. Our review of 130 meta-analyses revealed a strong reliance on statistical significance testing without consideration of Type I errors or the use of multiplicity corrections. In order to provide valid conclusions, meta-analysts must consider these issues prior to conducting the review.

Results of a 3-year randomized clinical trial of Second Step: Student Success Through Prevention (SS-SSTP) Middle School Program on reducing bullying, physical aggression, and peer victimization among students with disabilities are presented. Teachers implemented 41 lessons of a sixth- to eighth-grade curriculum that focused on social-emotional learning (SEL) skills, including empathy, bully prevention, communication skills, and emotion regulation. Two school districts in a larger clinical trial provided disability information. All sixth-grade students (N = 123) with a disability were included in these analyses, including intervention (n = 47) and control (n = 76) conditions. Linear growth models indicated a significant intervention effect for bully perpetration; compared with students in the control condition, intervention students' bullying perpetration scale scores significantly decreased across the 3-year study (d = -0.20, 95% confidence interval = [-0.38, -0.03]). SEL offers promise in reducing bully perpetration among students with disabilities.

Prosocial behaviors or voluntary acts intended to help or promote the wellbeing of others are learned skills that accumulating research suggests have implications not only for constructively navigating interactions, relationships, and school and other group contexts, but also for life in a civil society. Yet, strikingly, there is limited knowledge of the factors that affect the degree to which prosocial behavior is learned and enhanced. The focus of this quantitative review will be to clarify the types of instructional practices and programs that show evidence of effectively prompting children and youth to act in ways that help others. The benefits of prosocial behavior to lives of children and youth have been highlighted in the last few decades, and this quantitative synthesis of the research is designed to provide evidence-based directions for supporting these constructive, caring, and helpful behaviors in interactions, relationships, schools, homes, and communities.

This study examines how teacher and staff perceptions of the school environment correlate with student self-reports of bullying, aggression, victimization, and willingness to intervene in bullying incidents using multi-informant, multilevel modeling. Data were derived from 3,616 6th grade students across 36 middle schools in the Midwest, who completed survey measures of bullying, aggression, victimization, and willingness to intervene in bullying situations. Teachers and staff (n = 1,447) completed a school environment survey. Bivariate associations between school-level and student self-reports indicated that as teacher and staff perceive aggression as a problem in their school, students reported greater bully perpetration, fighting, peer victimization, and less willingness to intervene. Further, as staff and teacher report greater commitment to prevent bullying and viewed positive teacher and student relationships, there was less bullying, fighting, and peer victimization, and greater willingness to intervene. In a model where all school environment scales were entered together, a school commitment to prevent bullying was associated with less bullying, fighting, and peer victimization. Student-reports of bully perpetration and peer victimization were largely explained by staff and teacher commitment to bully prevention, whereas fighting and willingness to intervene were largely explained by student characteristics (e.g., gender). We conclude that efforts to address bullying and victimization should involve support from the school administration. School psychologists should play an active role in the school climate improvement process, by creating a school climate council consisting of students, parents, and teachers; administering school climate measures; identifying specific school improvement targets from these data, and engaging all stakeholders in the ongoing school improvement plan.

Systematic reviews and meta-analyses are important techniques because they synthesize results from multiple primary studies on a similar topic. To influence policy, practice, and research, however, synthesis researchers must translate the results for various audiences. Ideally, the translation drives future research agendas, informs policymaking, or assists in practical decision-making. An Evidence Gap Map (EGM), a graphical or tabular visualization of systematic review and meta-analysis results, is one ideal translation technique because it provides a structured framework to assess contexts for which primary evidence is available or to determine whether the effectiveness of an intervention or a program differs across populations, conditions, and settings. To bolster the field and promote the use of EGMs, we provide an overview of what constitutes an informative EGM, detail multiple examples of EGMs using extant meta-analytic results, and present a free R Shiny application we created to easily generate EGMs from typical meta-analytic datasets. We conclude by reviewing education-based systematic reviews that included an EGM to describe the current state of the field. [ABSTRACT FROM AUTHOR]