Search

Sociodemographic inequalities in student achievement are a persistent concern for education systems and are increasingly recognized to be intersectional. Intersectionality considers the multidimensional nature of disadvantage, appreciating the interlocking social determinants which shape individual experience. Intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) is a new approach developed in population health but with limited application in educational research. In this study, we introduce and apply this approach to study sociodemographic inequalities in student achievement across two cohorts of students in London, England. We define 144 intersectional strata arising from combinations of student age, gender, free school meal status, special educational needs, and ethnicity. We find substantial strata-level variation in achievement composed primarily by additive rather than interactive effects with results stubbornly consistent across the cohorts. We conclude that policymakers should pay greater attention to multiply marginalized students and intersectional MAIHDA provides a useful approach to study their experiences., Comment: 63 pages (main text 39 pages), 10 figures, 10 tables

Ethnic achievement gaps are often explained in terms of student and school factors. The decomposition of these gaps into their within- and between-school components has therefore been applied as a strategy to quantify the overall influence of each set of factors. Three competing approaches have previously been proposed, but each is limited to the study of student-school decompositions of the gap between two ethnic groups (e.g., White and Black). The authors show that these approaches can be reformulated as mediation models facilitating new extensions to allow additional levels in the school system (e.g., classrooms, school districts, geographic areas) and multiple ethnic groups (e.g., White, Black, Hispanic, Asian). The authors illustrate these extensions using administrative data for high school students in Colombia and highlight the increased substantive insights and nuanced policy implications they afford., Comment: Sociological Methodology (2022)

School value-added models are widely applied to study, monitor, and hold schools to account for school differences in student learning. The traditional model is a mixed-effects linear regression of student current achievement on student prior achievement, background characteristics, and a school random intercept effect. The latter is referred to as the school value-added score and measures the mean student covariate-adjusted achievement in each school. In this article, we argue that further insights may be gained by additionally studying the variance in this quantity in each school. These include the ability to identify both individual schools and school types that exhibit unusually high or low variability in student achievement, even after accounting for differences in student intakes. We explore and illustrate how this can be done via fitting mixed-effects location scale versions of the traditional school value-added model. We discuss the implications of our work for research and school accountability systems.

School accountability systems increasingly hold schools to account for their performances using value-added models purporting to measure the effects of schools on student learning. The most common approach is to fit a linear regression of student current achievement on student prior achievement, where the school effects are the school means of the predicted residuals. In the literature, further adjustments are usually made for student sociodemographics and sometimes school composition and 'non-malleable' characteristics. However, accountability systems typically make fewer adjustments: for transparency to end users, because data is unavailable or of insufficient quality, or for ideological reasons. There is therefore considerable interest in understanding the extent to which simpler models give similar school effects to more theoretically justified but complex models. We explore these issues via a case study and empirical analysis of England's 'Progress 8' secondary school accountability system.

School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the Progress 8 secondary school accountability system for state-funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measures; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi-year averages. We then discuss challenges for Progress 8 raised by the COVID-19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England., Comment: 68 pages, 8 figures

This paper proposes an alternative educational accountability system for England that moves away from simplistic comparisons, or league tables, among schools towards a more nuanced reporting at a level of education authorities or other school groupings. Based upon the sampling of pupils within schools, it proposes the use of quantitative pupil assessment data as screening devices that provide evidence within an integrated accountability system that includes inspection. At the same time, it aims to provide a richer set of data than currently available for research as well as accountability purposes. We argue that if carefully implemented within a context of school improvement, such a system has the potential to largely eliminate the deleterious side effects and curriculum distortions of the present system. While being proposed within the context of the current English secondary school system, the proposals will have relevance for other phases of schooling and similar systems in other countries., Comment: 5700 words

Education systems around the world increasingly rely on school value-added models to hold schools to account. These models typically focus on a limited number of academic outcomes, failing to recognise the broader range of non-academic student outcomes, attitudes and behaviours to which schools contribute. We explore how the traditional multilevel modelling approach to school value-added models can be extended to simultaneously analyse multiple academic and non-academic outcomes and thereby can potentially provide a more rounded approach to using student data to inform school accountability. We jointly model student attainment, absence and exclusion data for schools in England. We find different results across the three outcomes, in terms of the size and consistency of school effects, and the importance of adjusting for student and school characteristics. The results suggest the three outcomes are capturing fundamentally distinct aspects of school performance, recommending the consideration of non-academic outcomes in systems of school accountability., Comment: Main Manuscript 38 pages, 4 figures, Supplementary Material 17 pages, 4 figures

In England, Progress 8 is the Conservative government's headline secondary school performance and accountability measure. Progress 8 attempts to measure the average academic progress pupils make in each school between their KS2 tests and their GCSE Attainment 8 examinations. The Labour opposition recently announced they would scrap the KS2 tests were they to be elected. Such a move, however, would preclude the publication of Progress 8 and would leave schools to be compared in terms of their average Attainment 8 scores or, at best, their Attainment 8 scores only adjusted for school differences in pupil demographic and socioeconomic characteristics. In this paper, we argue and illustrate empirically that this best-case scenario of an 'Adjusted Attainment 8' measure would prove less fair and meaningful than Progress 8 and therefore a backwards step, especially when Progress 8 itself has been criticised as biased against schools teaching educationally disadvantaged intakes.

A first step when fitting multilevel models to continuous responses is to explore the degree of clustering in the data. Researchers fit variance-component models and then report the proportion of variation in the response that is due to systematic differences between clusters. Equally they report the response correlation between units within a cluster. These statistics are popularly referred to as variance partition coefficients (VPCs) and intraclass correlation coefficients (ICCs). When fitting multilevel models to categorical (binary, ordinal, or nominal) and count responses, these statistics prove more challenging to calculate. For categorical response models, researchers appeal to their latent response formulations and report VPCs/ICCs in terms of latent continuous responses envisaged to underly the observed categorical responses. For standard count response models, however, there are no corresponding latent response formulations. More generally, there is a paucity of guidance on how to partition the variation. As a result, applied researchers are likely to avoid or inadequately report and discuss the substantive importance of clustering and cluster effects in their studies. A recent article drew attention to a little-known exact algebraic expression for the VPC/ICC for the special case of the two-level random-intercept Poisson model. In this article, we make a substantial new contribution. First, we derive exact VPC/ICC expressions for more flexible negative binomial models that allows for overdispersion, a phenomenon which often occurs in practice. Then we derive exact VPC/ICC expressions for three-level and random-coefficient extensions to these models. We illustrate our work with an application to student absenteeism.

This tutorial demonstrates the estimation and interpretation of the Multilevel Social Relations Model for dyadic data. The Social Relations Model is appropriate for data structures in which individuals appear multiple times as both the source and recipient of dyadic outcomes. Estimated using Stat-JR statistical software, the models are fitted to multiple outcome types: continuous, count, and binary outcomes. In addition, models are demonstrated for dyadic data from a single group and from multiple groups. The modeling approaches are illustrated via a series of case studies, and the data and software to replicate these analyses are available as supplemental files.

Multilevel models (mixed-effect models or hierarchical linear models) are now a standard approach to analysing clustered and longitudinal data in the social, behavioural and medical sciences. This review article focuses on multilevel linear regression models for continuous responses (outcomes or dependent variables). These models can be viewed as an extension of conventional linear regression models to account for and learn from the clustering in the data. Common clustered applications include studies of school effects on student achievement, hospital effects on patient health, and neighbourhood effects on respondent attitudes. In all these examples, multilevel models allow one to study how the regression relationships vary across clusters, to identify those cluster characteristics which predict such variation, to disentangle social processes operating at different levels of analysis, and to make cluster-specific predictions. Common longitudinal applications include studies of growth curves of individual height and weight and developmental trajectories of individual behaviours. In these examples, multilevel models allow one to describe and explain variation in growth rates and to simultaneously explore predictors of both of intra- and inter-individual variation. This article introduces and illustrates this powerful class of model. We start by focusing on the most commonly applied two-level random-intercept and -slope models. We illustrate through two detailed examples how these models can be applied to both clustered and longitudinal data and in both observational and experimental settings. We then review more flexible three-level, cross-classified, multiple membership and multivariate response models. We end by recommending a range of further reading on all these topics.

Multiple membership multilevel models are an extension of standard multilevel models for non-hierarchical data that have multiple membership structures. Traditional multilevel models involve hierarchical data structures whereby lower-level units such as students are nested within higher-level units such as schools and where these higher-level units may in turn be nested within further groupings or clusters such as school districts, regions, and countries. With hierarchical data structures, there is an exact nesting of each lower-level unit in one and only one higher-level unit. For example, each student attends one school, each school is located within one school district, and so on. However, social reality is more complicated than this, and so social and behavioural data often do not follow pure or strict hierarchies. Two types of non-hierarchical data structures which often appear in practice are cross-classified and multiple membership structures. In this article, we describe multiple membership data structures and multiple membership models which can be used to analyse them., Comment: arXiv admin note: substantial text overlap with arXiv:1907.02569

Cross-classified multilevel modelling is an extension of standard multilevel modelling for non-hierarchical data that have cross-classified structures. Traditional multilevel models involve hierarchical data structures whereby lower level units such as students are nested within higher level units such as schools and where these higher level units may in turn be nested within further groupings or clusters such as school districts, regions, and countries. With hierarchical data structures, there is an exact nesting of each lower level unit in one and only one higher level unit. For example, each student attends one school, each school is located within one school district, and so on. However, social reality is more complicated than this, and so social and behavioural data often do not follow pure or strict hierarchies. Two types of non-hierarchical data structures which often appear in practice are cross-classified and multiple membership structures. In this article, we describe cross-classified data structures and cross-classified hierarchical linear modelling which can be used to analyse them.

In the UK, US and elsewhere, school accountability systems increasingly compare schools using value-added measures of school performance derived from pupil scores in high-stakes standardised tests. Rather than naively comparing school average scores, which largely reflect school intake differences in prior attainment, these measures attempt to compare the average progress or improvement pupils make during a year or phase of schooling. Schools, however, also differ in terms of their pupil demographic and socioeconomic characteristics and these also predict why some schools subsequently score higher than others. Many therefore argue that value-added measures unadjusted for pupil background are biased in favour of schools with more 'educationally advantaged' intakes. But, others worry that adjusting for pupil background entrenches socioeconomic inequities and excuses low performing schools. In this article we explore these theoretical arguments and their practical importance in the context of the 'Progress 8' secondary school accountability system in England which has chosen to ignore pupil background. We reveal how the reported low or high performance of many schools changes dramatically once adjustments are made for pupil background and these changes also affect the reported differential performances of region and of different school types. We conclude that accountability systems which choose to ignore pupil background are likely to reward and punish the wrong schools and this will likely have detrimental effects on pupil learning. These findings, especially when coupled with more general concerns surrounding high-stakes testing and school value-added models, raise serious doubts about their use in school accountability systems.