Search

The three articles that comprise this dissertation use a variety of methods associated with the field of educational measurement to address distinct research questions related to psychometric tools and their implications for practice in education. All three articles involve methods related to instrument design, while drawing on different combinations of methods related to precision and generalization; validity and validation; context for educational measurement; and psychometric modeling. "Supporting the Interpretive Validity of Student-Level Claims in Science Assessment with Tiered Claim Structures": We address two persistent challenges in large-scale assessments of the Next Generation Science Standards: (a) the validity of score interpretations that target the standards broadly and (b) how to structure claims for assessments of this complex domain. The NGSS pose a particular challenge for specifying claims about students that evidence from summative assessments can support. As a solution, we propose tiered claims, which explicitly distinguish between claims about what students have done or can do on test items--which are typically easier to support under current test designs--and claims about what students could do in the broader domain of performances described by the standards, for which novel evidence is likely required. We discuss the positive implications of tiered claims for test construction, validation, and reporting of results. "Empirical Growth Benchmarks and the Role of Scale Choice": The practice of benchmarking causal effect estimates against estimates of years of learning in standard deviation is widespread in education research and policy. Yet, this is an assumption-laden, technically complex endeavor, due in part to the reliance of these methods on vertical achievement scales. This paper draws upon research on scale design and psychometric issues in vertical scaling to motivate the introduction of new benchmarks for annual growth based on 2019 grade 3-8 state achievement tests in math and ELA. These benchmarks are compared to previously-developed benchmarks, including those introduced by Dadey and Briggs (2012) and Bloom et al. (2008). While trends in growth across these three sets of benchmarks are generally consistent, differences do arise that would lead to substantively different conclusions about the practical significance of a causal effect. An exploratory analysis then demonstrates the extent to which differences in growth across different scales appear attributable to psychometric and design differences across the scales. Implications for practice are discussed, with a focus on interpreting the practical significance of an estimated causal effect using growth benchmarks. "The Sensitivity of Growth across Grades to Vertical Scale Common Item Linking Design": Although vertical scales are used specifically to facilitate the absolute measurement of growth, the definition of "growth" in vertical scaling has received relatively little attention in the research literature. Whether growth from one grade to the next represents improvement on material already learned in the earlier grade, learned only in the later grade, or a mix of the two is a key distinction likely to lead to different conclusions about how much students grow. Using a dataset from Curriculum Associates' i-Ready Diagnostic in math in grades 3-7, we demonstrate how different definitions of growth and their corresponding operationalization in a common item, nonequivalent group linking design directly influence estimates of how much students grow from one grade to the next. We then explore the possibility that this phenomenon is due, in part, to violations of invariance in which common items reflect different constructs for students in different grades. We describe implications for several areas of practice including scale construction, item mapping, and downstream research involving vertically scaled tests. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]