Search

The practice of prospectively registering the details of intervention studies in a public database or registry is gaining momentum across disciplines as a strategy for increasing the transparency, credibility, and accessibility of study findings. In this article, we consider five registries that may be relevant for registration of intervention studies in the field of prevention science: ClinicalTrials.gov, the American Economic Association Registry of Randomized Controlled Trials (AEA RCT Registry), the Open Science Framework Preregistration (OSF Preregistration), the Registry for International Development Impact Evaluations (RIDIE), and the Registry of Efficacy and Effectiveness Studies (REES). We examine the five registries in terms of substantive focus, study designs, and contents of registry entries. We consider two paths forward for prospective registration of intervention studies in the field of prevention science: Path A: register all studies in ClinicalTrials.gov and Path B: allow individual researchers to select the registry with the "best fit." Lastly, we consider how the field might begin to establish norms around registration. [This article was published in "Prevention Science" (EJ1341830).]

As a result of the evidence-based decision-making movement, the number of randomized trials evaluating educational programs and curricula has increased dramatically over the past 20 years. Policy makers and practitioners are encouraged to use the results of these trials to inform their decision making in schools and school districts. At the same time, however, little is known about the schools taking part in these randomized trials, both regarding how and why they were recruited and how they compare to populations in need of research. In this article, we report on a study of 37 cluster randomized trials funded by the Institute of Education Sciences between 2011 and 2015. Principal investigators of these grants were interviewed regarding the recruitment process and practices. Additionally, data on the schools included in 34 of these studies were analyzed to determine the general demographics of schools included in funded research, as well as how these samples compare to important policy relevant populations. We show that the types of schools included in research differ in a variety of ways from these populations. Large schools from large school districts in urban areas were overrepresented, whereas schools from small school districts in rural areas and towns are underrepresented. The article concludes with a discussion of how recruitment practices might be improved in order to meet the goals of the evidence-based decision-making movement.

Over the past 15 years, we have seen an increase in the use of cluster randomized trials (CRTs) to test the efficacy of educational interventions. These studies are often designed with the goal of determining whether a program works, or answering the what works question. Recently, the goals of these studies expanded to include for whom and under what conditions an intervention is effective. In this study, we examine the capacity of a set of CRTs to provide rigorous evidence about for whom and under what conditions an intervention is effective. The findings suggest that studies are more likely to be designed with the capacity to detect potentially meaningful individual-level moderator effects, for example, gender, than cluster-level moderator effects, for example, school type.

With the increasing demand for evidence-based research on teacher effectiveness and improving student achievement, more impact studies are being conducted to examine the effectiveness of professional development (PD) interventions. Cluster randomized trials (CRTs) are often carried out to assess PD interventions that aim to improve both teacher and student outcomes. Due to the different design parameters (i.e., intraclass correlation and R[superscript 2]) and benchmark effect sizes associated with the student and teacher outcomes, two power analyses are necessary for planning CRTs that aim to detect both teacher and student effects in one study. These two power analyses are often conducted separately without considering how design choices to power the study to detect student effects may affect design choices to power the study to detect teacher effects and vice versa. In this study, we consider strategies to maximize the efficiency of the study design when both student and teacher effects are of primary interest.

Calls for improving transparency across the social science are increasing. One strategy that is gaining momentum in the quest to increase transparency is the practice of preregistering a study and analysis plan prior to conducting a study. In this article, we examine the potential of preregistration as a strategy for increasing transparency in education studies. We review existing registries within the social sciences and provide a rationale for why we need a registry specifically geared towards education studies. Finally, we introduce the Registry of Efficacy and Effectiveness Studies (REES), developed by the Society for Research on Educational Effectiveness (SREE). The goal of REES is to increase transparency for studies seeking to draw causal conclusions within the education research and evaluation community.

Calls for improving transparency across the social science are increasing. One strategy that is gaining momentum in the quest to increase transparency is the practice of preregistering a study and analysis plan prior to conducting a study. In this article, we examine the potential of preregistration as a strategy for increasing transparency in education studies. We review existing registries within the social sciences and provide a rationale for why we need a registry specifically geared towards education studies. Finally, we introduce the Registry of Efficacy and Effectiveness Studies (REES), developed by the Society for Research on Educational Effectiveness (SREE). The goal of REES is to increase transparency for studies seeking to draw causal conclusions within the education research and evaluation community. [This article was published in "Educational Researcher" (EJ1204535).]

The Common Guidelines for Education Research and Development were created as a joint effort between the Institute of Education Science and the National Science Foundation in an effort to streamline education research and contribute to an accumulation of knowledge that will lead to improved student outcomes. One type of research that emerged in the guidelines is impact research. In order to achieve the level of rigor expected for an impact study, it is common that a research team will employ a cluster randomized trial (CRT). This article provides empirical estimates of design parameters necessary for planning adequately powered CRTs focused on science achievement. Examples of how to use these parameters to improve the design of science impact studies are discussed.

Using nationally representative samples for public school teachers and principals, the authors inquired into whether principal background and school processes are related to teacher job satisfaction. Employing hierarchical linear modeling (HLM), the authors were able to control for background characteristics at both the teacher and school levels. They found that 17% of the total variance in teacher job satisfaction is between schools, a statistically significant amount that indicates schools can make a difference in teacher job satisfaction. The authors found that school processes--particularly career and working conditions, staff collegiality, administrative support, and to a lesser extent, positive student behavior and teacher empowerment--are positively associated with teacher job satisfaction. Although two principal background variables--the experience of being a department head or an athletic coach/director--are statistically significant, the authors found the block of school process variables explains far more variance than the block of principal background variables. Based on the findings, the authors discussed issues such as the complexity of the phenomenon of teacher job satisfaction, the role of school process versus principal background, and monetary versus cultural factors. (Contains 7 notes, 5 tables, and 1 figure.)

This article examines the power analyses for the first wave of group-randomized trials funded by the Institute of Education Sciences. Specifically, it assesses the precision and technical accuracy of the studies. The authors identified the appropriate experimental design and estimated the minimum detectable standardized effect size (MDES) for each study under plausible assumptions about intra-class correlations, covariate-outcome correlations, and explanatory effects of blocking. The MDESs ranged from 0.18 to 0.40 for studies funded by the National Center for Education Evaluation and Regional Assistance (NCEE) and from 0.20 to 1.0 for studies funded by the National Center for Education Research (NCER). They evaluated the technical accuracy by comparing the estimated MDES to the MDES stated in the proposal. Studies funded by the NCEE tended to be very accurate: estimated MDESs and stated MDESs were similar, whereas studies funded by the NCER were less accurate. However, the technical accuracy of the NCER studies improved over time. (Contains 5 notes, 5 tables, and 1 figure.)

Interest has rapidly increased in studies that randomly assign classrooms or schools to interventions. When well implemented, such studies eliminate selection bias, providing strong evidence about the impact of the interventions. However, unless expected impacts are large, the number of units to be randomized needs to be quite large to achieve adequate statistical power, making these studies potentially quite expensive. This article considers when and to what extent matching or covariance adjustment can reduce the number of groups needed to achieve adequate power and when these approaches actually reduce power. The presentation is nontechnical. (Contains 10 figures and 2 notes.)

Context: The most common modality used to address acute inflammation is cryotherapy. Whereas pain decreases with cryotherapy, evidence that changes occur in perfusion of skeletal muscle is limited. We do not know whether ice attenuates the increases in perfusion associated with acute inflammation. Objective: To examine the effects of repeated applications of ice bags on perfusion of the gastrocnemius muscle after an eccentric exercise protocol. Design: Controlled laboratory study. Setting: Laboratory. Patients or Other Participants: Eighteen healthy participants (3 men, 15 women; age = 22.2 ± 2.2 years, height = 166.0 ± 11.9 cm, mass = 69.4 ± 25.0 kg). Intervention(s): To induce eccentric muscle damage, participants performed 100 unilateral heel-lowering exercises off a step to the beat of a metronome. A randomized intervention (cryotherapy, sham, control) was applied to the exercised lower extremity immediately after the protocol and again at 10, 24, and 34 hours after the protocol. Main Outcome Measure(s): Baseline perfusion measurements (blood volume, blood flow, and blood flow velocity) were taken using contrast-enhanced ultrasound of the exercised leg. Perfusion was reassessed after the first intervention and 48 hours after the protocol as percentage change scores. Pain was measured with a visual analog scale at baseline and at 10, 24, 34, and 48 hours after the protocol. Separate repeatedmeasures analyses of variance were used to assess each dependent variable. Results: We found no interactions among interventions for microvascular perfusion. Blood volume and blood flow, however, increased in all conditions at 48 hours after exercise (P < .001), and blood flow velocity decreased postintervention from baseline (P = .041). We found a time-by-intervention interaction for pain (P = .009). Visual analog scale scores were lower for the cryotherapy group than for the control group at 34 and 48 hours after exercise. Conclusions: Whereas eccentric muscle damage resulted in increased blood flow, ice did not decrease muscle perfusion 48 hours after exercise. Therefore, ice does not seem to decrease muscle perfusion when blood flow is elevated, as it would be during inflammation. [ABSTRACT FROM AUTHOR]