Working at scale to initiate ongoing validation of learning trajectory-based classroom assessments for middle grade mathematics.

• Summary of interdisciplinary validation process framed on learning trajectories. • Working at scale in an assessment system allows for better understanding of patterns. • Classroom assessments to diagnose student progress in mathematics learning. • Importance of collaboration among learning sciences and psychometrics. • Using measures of learning trajectories to support learner-centered instruction. The paper reports on the design and validation argument for classroom assessments within a digital diagnostic assessment system built on learning trajectories (LTs). It consists of a learning map of nine big ideas, 25 relational learning clusters, and 62 LTs for grades 6−8 mathematics. Students take cluster assessments, and teachers use the data to adapt instruction. An ongoing validation process is presented with data for an algebra cluster. Validation among practitioners, learning scientists, and psychometricians is conceptualized as examining and adjusting inter-level, intra-level, and construct-irrelevant variation in measures of item difficulty and deploying item response theory modeling followed by sequential regressions. Using data from 37,000 assessments collected over three years at 3 middle schools, 167 potentially non-conforming items of the 676 calibrated items (24 %) were identified and revised. The paper discusses how the trajectories and map were refined through a combination of data analysis and feedback from practitioners. [ABSTRACT FROM AUTHOR]