Search

School value-added models are widely applied to study, monitor, and hold schools to account for school differences in student learning. The traditional model is a mixed-effects linear regression of student current achievement on student prior achievement, background characteristics, and a school random intercept effect. The latter is referred to as the school value-added score and measures the mean student covariate-adjusted achievement in each school. In this article, we argue that further insights may be gained by additionally studying the variance in this quantity in each school. These include the ability to identify both individual schools and school types that exhibit unusually high or low variability in student achievement, even after accounting for differences in student intakes. We explore and illustrate how this can be done via fitting mixed-effects location scale versions of the traditional school value-added model. We discuss the implications of our work for research and school accountability systems.

This paper proposes an alternative educational accountability system for England that moves away from simplistic comparisons, or league tables, among schools towards a more nuanced reporting at a level of education authorities or other school groupings. Based upon the sampling of pupils within schools, it proposes the use of quantitative pupil assessment data as screening devices that provide evidence within an integrated accountability system that includes inspection. At the same time, it aims to provide a richer set of data than currently available for research as well as accountability purposes. We argue that if carefully implemented within a context of school improvement, such a system has the potential to largely eliminate the deleterious side effects and curriculum distortions of the present system. While being proposed within the context of the current English secondary school system, the proposals will have relevance for other phases of schooling and similar systems in other countries., Comment: 5700 words

Education systems around the world increasingly rely on school value-added models to hold schools to account. These models typically focus on a limited number of academic outcomes, failing to recognise the broader range of non-academic student outcomes, attitudes and behaviours to which schools contribute. We explore how the traditional multilevel modelling approach to school value-added models can be extended to simultaneously analyse multiple academic and non-academic outcomes and thereby can potentially provide a more rounded approach to using student data to inform school accountability. We jointly model student attainment, absence and exclusion data for schools in England. We find different results across the three outcomes, in terms of the size and consistency of school effects, and the importance of adjusting for student and school characteristics. The results suggest the three outcomes are capturing fundamentally distinct aspects of school performance, recommending the consideration of non-academic outcomes in systems of school accountability., Comment: Main Manuscript 38 pages, 4 figures, Supplementary Material 17 pages, 4 figures

In England, Progress 8 is the Conservative government's headline secondary school performance and accountability measure. Progress 8 attempts to measure the average academic progress pupils make in each school between their KS2 tests and their GCSE Attainment 8 examinations. The Labour opposition recently announced they would scrap the KS2 tests were they to be elected. Such a move, however, would preclude the publication of Progress 8 and would leave schools to be compared in terms of their average Attainment 8 scores or, at best, their Attainment 8 scores only adjusted for school differences in pupil demographic and socioeconomic characteristics. In this paper, we argue and illustrate empirically that this best-case scenario of an 'Adjusted Attainment 8' measure would prove less fair and meaningful than Progress 8 and therefore a backwards step, especially when Progress 8 itself has been criticised as biased against schools teaching educationally disadvantaged intakes.

A first step when fitting multilevel models to continuous responses is to explore the degree of clustering in the data. Researchers fit variance-component models and then report the proportion of variation in the response that is due to systematic differences between clusters. Equally they report the response correlation between units within a cluster. These statistics are popularly referred to as variance partition coefficients (VPCs) and intraclass correlation coefficients (ICCs). When fitting multilevel models to categorical (binary, ordinal, or nominal) and count responses, these statistics prove more challenging to calculate. For categorical response models, researchers appeal to their latent response formulations and report VPCs/ICCs in terms of latent continuous responses envisaged to underly the observed categorical responses. For standard count response models, however, there are no corresponding latent response formulations. More generally, there is a paucity of guidance on how to partition the variation. As a result, applied researchers are likely to avoid or inadequately report and discuss the substantive importance of clustering and cluster effects in their studies. A recent article drew attention to a little-known exact algebraic expression for the VPC/ICC for the special case of the two-level random-intercept Poisson model. In this article, we make a substantial new contribution. First, we derive exact VPC/ICC expressions for more flexible negative binomial models that allows for overdispersion, a phenomenon which often occurs in practice. Then we derive exact VPC/ICC expressions for three-level and random-coefficient extensions to these models. We illustrate our work with an application to student absenteeism.

In the UK, US and elsewhere, school accountability systems increasingly compare schools using value-added measures of school performance derived from pupil scores in high-stakes standardised tests. Rather than naively comparing school average scores, which largely reflect school intake differences in prior attainment, these measures attempt to compare the average progress or improvement pupils make during a year or phase of schooling. Schools, however, also differ in terms of their pupil demographic and socioeconomic characteristics and these also predict why some schools subsequently score higher than others. Many therefore argue that value-added measures unadjusted for pupil background are biased in favour of schools with more 'educationally advantaged' intakes. But, others worry that adjusting for pupil background entrenches socioeconomic inequities and excuses low performing schools. In this article we explore these theoretical arguments and their practical importance in the context of the 'Progress 8' secondary school accountability system in England which has chosen to ignore pupil background. We reveal how the reported low or high performance of many schools changes dramatically once adjustments are made for pupil background and these changes also affect the reported differential performances of region and of different school types. We conclude that accountability systems which choose to ignore pupil background are likely to reward and punish the wrong schools and this will likely have detrimental effects on pupil learning. These findings, especially when coupled with more general concerns surrounding high-stakes testing and school value-added models, raise serious doubts about their use in school accountability systems.

Comment on "Citation Statistics" [arXiv:0910.3529], Comment: Published in at http://dx.doi.org/10.1214/09-STS285C the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)